2024-03-26T03:20:31.517 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-26T03:20:31.517 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/adking-2024-03-26_01:50:53-orch:cephadm-wip-adk-testing-2024-03-25-1628-distro-default-smithi/7622440 --owner scheduled_adking@teuthology --verbose --name adking-2024-03-26_01:50:53-orch:cephadm-wip-adk-testing-2024-03-25-1628-distro-default-smithi --description orch:cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} -- /home/teuthworker/archive/adking-2024-03-26_01:50:53-orch:cephadm-wip-adk-testing-2024-03-25-1628-distro-default-smithi/7622440/orig.config.yaml 2024-03-26T03:20:31.545 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T03:20:31.620 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/adking-2024-03-26_01:50:53-orch:cephadm-wip-adk-testing-2024-03-25-1628-distro-default-smithi/7622440 branch: wip-adk-testing-2024-03-25-1628 description: orch:cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} email: adking@redhat.com first_in_suite: false job_id: '7622440' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: adking-2024-03-26_01:50:53-orch:cephadm-wip-adk-testing-2024-03-25-1628-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-adk-testing-2024-03-25-1628 ceph: conf: client: debug ms: 1 global: mon election default strategy: 1 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd: debug ms: 1 debug osd: 20 osd class default list: '*' osd class load list: '*' osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - MON_DOWN - mons down, quorum - is down \(out of quorum\) - POOL_APP_NOT_ENABLED - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) sha1: e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: cephadm-package install: ceph: flavor: default sha1: e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 extra_packages: - cephadm workunit: branch: wip-adk-testing-2024-03-25-1628 sha1: e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 owner: scheduled_adking@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 9192 sha1: e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 sleep_before_teardown: 0 subset: 1/16 suite: orch:cephadm suite_branch: wip-adk-testing-2024-03-25-1628 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 targets: smithi175.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLS3vMmR6LQz7OTue9eOIL2OPIHfMsTq81cV/Hd9LKHcdbqE0SCDqNH4t5UP7TRj/iqpCxaQcXJBnrU9FdGf+lU= smithi190.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM4zMdKdSS3SArQxMk+GuqerWQeHcu1JcHhWEy+x3TQqdrbK7Bd5P0UyT+uh7l7rHeoPKRAMF2tf8fODpQTstsA= tasks: - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - workunit: clients: client.0: - rados/test.sh - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-26_01:50:53 tube: smithi user: adking verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-26T03:20:31.620 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/qa; will attempt to use it 2024-03-26T03:20:31.621 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/qa/tasks 2024-03-26T03:20:31.621 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-26T03:20:31.622 INFO:teuthology.task.internal:Checking packages... 2024-03-26T03:20:31.646 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003' 2024-03-26T03:20:31.646 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-26T03:20:31.646 INFO:teuthology.packaging:ref: None 2024-03-26T03:20:31.646 INFO:teuthology.packaging:tag: None 2024-03-26T03:20:31.646 INFO:teuthology.packaging:branch: wip-adk-testing-2024-03-25-1628 2024-03-26T03:20:31.646 INFO:teuthology.packaging:sha1: e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 2024-03-26T03:20:31.646 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-adk-testing-2024-03-25-1628 2024-03-26T03:20:31.785 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2475-ge42b8ea3-1jammy 2024-03-26T03:20:31.786 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-26T03:20:31.793 INFO:teuthology.task.internal:no buildpackages task found 2024-03-26T03:20:31.793 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-26T03:20:31.806 INFO:teuthology.task.internal:Saving configuration 2024-03-26T03:20:31.817 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-26T03:20:31.823 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-26T03:20:31.846 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi175.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2024-03-26_01:50:53-orch:cephadm-wip-adk-testing-2024-03-25-1628-distro-default-smithi/7622440', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 03:15:02.068639', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKWUj3IlREHH3QuEzh6rQvX5XdGmWc1Sls3JHdUs31go'} 2024-03-26T03:20:31.866 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2024-03-26_01:50:53-orch:cephadm-wip-adk-testing-2024-03-25-1628-distro-default-smithi/7622440', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 03:15:02.067232', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILYbFphkvk9Bo1viKR4ikCBSqDthgKS0QaIsfRUfaDPN'} 2024-03-26T03:20:31.867 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-26T03:20:31.874 INFO:teuthology.task.internal:roles: ubuntu@smithi175.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-03-26T03:20:31.874 INFO:teuthology.task.internal:roles: ubuntu@smithi190.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-03-26T03:20:31.874 INFO:teuthology.run_tasks:Running task console_log... 2024-03-26T03:20:31.946 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f92a20874c0>, signals=[15]) 2024-03-26T03:20:31.947 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-26T03:20:31.953 INFO:teuthology.task.internal:Opening connections... 2024-03-26T03:20:31.953 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi175.front.sepia.ceph.com 2024-03-26T03:20:31.955 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T03:20:32.033 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi190.front.sepia.ceph.com 2024-03-26T03:20:32.034 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T03:20:32.110 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-26T03:20:32.116 DEBUG:teuthology.orchestra.run.smithi175:> uname -m 2024-03-26T03:20:32.121 INFO:teuthology.orchestra.run.smithi175.stdout:x86_64 2024-03-26T03:20:32.121 DEBUG:teuthology.orchestra.run.smithi175:> cat /etc/os-release 2024-03-26T03:20:32.166 INFO:teuthology.orchestra.run.smithi175.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-03-26T03:20:32.167 INFO:teuthology.orchestra.run.smithi175.stdout:NAME="Ubuntu" 2024-03-26T03:20:32.167 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_ID="22.04" 2024-03-26T03:20:32.167 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-03-26T03:20:32.167 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_CODENAME=jammy 2024-03-26T03:20:32.167 INFO:teuthology.orchestra.run.smithi175.stdout:ID=ubuntu 2024-03-26T03:20:32.167 INFO:teuthology.orchestra.run.smithi175.stdout:ID_LIKE=debian 2024-03-26T03:20:32.167 INFO:teuthology.orchestra.run.smithi175.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-26T03:20:32.167 INFO:teuthology.orchestra.run.smithi175.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-26T03:20:32.167 INFO:teuthology.orchestra.run.smithi175.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-26T03:20:32.167 INFO:teuthology.orchestra.run.smithi175.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-26T03:20:32.167 INFO:teuthology.orchestra.run.smithi175.stdout:UBUNTU_CODENAME=jammy 2024-03-26T03:20:32.168 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server 2024-03-26T03:20:32.191 DEBUG:teuthology.orchestra.run.smithi190:> uname -m 2024-03-26T03:20:32.196 INFO:teuthology.orchestra.run.smithi190.stdout:x86_64 2024-03-26T03:20:32.196 DEBUG:teuthology.orchestra.run.smithi190:> cat /etc/os-release 2024-03-26T03:20:32.245 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-03-26T03:20:32.245 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="Ubuntu" 2024-03-26T03:20:32.245 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="22.04" 2024-03-26T03:20:32.245 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-03-26T03:20:32.245 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_CODENAME=jammy 2024-03-26T03:20:32.245 INFO:teuthology.orchestra.run.smithi190.stdout:ID=ubuntu 2024-03-26T03:20:32.245 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE=debian 2024-03-26T03:20:32.245 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-26T03:20:32.245 INFO:teuthology.orchestra.run.smithi190.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-26T03:20:32.245 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-26T03:20:32.245 INFO:teuthology.orchestra.run.smithi190.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-26T03:20:32.246 INFO:teuthology.orchestra.run.smithi190.stdout:UBUNTU_CODENAME=jammy 2024-03-26T03:20:32.246 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2024-03-26T03:20:32.265 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-26T03:20:32.276 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-26T03:20:32.281 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-26T03:20:32.282 DEBUG:teuthology.orchestra.run.smithi175:> test '!' -e /home/ubuntu/cephtest 2024-03-26T03:20:32.284 DEBUG:teuthology.orchestra.run.smithi190:> test '!' -e /home/ubuntu/cephtest 2024-03-26T03:20:32.296 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-26T03:20:32.301 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-26T03:20:32.301 DEBUG:teuthology.orchestra.run.smithi175:> test -z $(ls -A /var/lib/ceph) 2024-03-26T03:20:32.331 DEBUG:teuthology.orchestra.run.smithi190:> test -z $(ls -A /var/lib/ceph) 2024-03-26T03:20:32.428 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-26T03:20:32.484 INFO:teuthology.run_tasks:Running task kernel... 2024-03-26T03:20:32.499 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-26T03:20:32.499 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-26T03:20:32.499 DEBUG:teuthology.orchestra.run.smithi175:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-26T03:20:32.500 DEBUG:teuthology.orchestra.run.smithi190:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-26T03:20:32.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T03:20:32.504 DEBUG:teuthology.orchestra.run.smithi175:> uname -r 2024-03-26T03:20:32.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T03:20:32.505 DEBUG:teuthology.orchestra.run.smithi190:> uname -r 2024-03-26T03:20:32.551 INFO:teuthology.orchestra.run.smithi175.stdout:5.15.0-56-generic 2024-03-26T03:20:32.551 INFO:teuthology.task.kernel:Running kernel on smithi175: 5.15.0-56-generic 2024-03-26T03:20:32.551 DEBUG:teuthology.orchestra.run.smithi175:> sudo apt-get clean 2024-03-26T03:20:32.553 INFO:teuthology.orchestra.run.smithi190.stdout:5.15.0-56-generic 2024-03-26T03:20:32.553 INFO:teuthology.task.kernel:Running kernel on smithi190: 5.15.0-56-generic 2024-03-26T03:20:32.553 DEBUG:teuthology.orchestra.run.smithi190:> sudo apt-get clean 2024-03-26T03:20:32.679 DEBUG:teuthology.orchestra.run.smithi190:> sudo apt-get update 2024-03-26T03:20:32.683 DEBUG:teuthology.orchestra.run.smithi175:> sudo apt-get update 2024-03-26T03:20:32.815 INFO:teuthology.orchestra.run.smithi175.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-26T03:20:32.817 INFO:teuthology.orchestra.run.smithi175.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-03-26T03:20:32.817 INFO:teuthology.orchestra.run.smithi175.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-03-26T03:20:32.907 INFO:teuthology.orchestra.run.smithi175.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-03-26T03:20:32.948 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-26T03:20:32.948 INFO:teuthology.orchestra.run.smithi190.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-03-26T03:20:33.002 INFO:teuthology.orchestra.run.smithi190.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-03-26T03:20:33.048 INFO:teuthology.orchestra.run.smithi175.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [429 kB] 2024-03-26T03:20:33.127 INFO:teuthology.orchestra.run.smithi175.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [595 kB] 2024-03-26T03:20:33.127 INFO:teuthology.orchestra.run.smithi175.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,290 kB] 2024-03-26T03:20:33.172 INFO:teuthology.orchestra.run.smithi175.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [230 kB] 2024-03-26T03:20:33.172 INFO:teuthology.orchestra.run.smithi175.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-03-26T03:20:33.172 INFO:teuthology.orchestra.run.smithi175.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [33.7 kB] 2024-03-26T03:20:33.172 INFO:teuthology.orchestra.run.smithi175.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,600 kB] 2024-03-26T03:20:33.195 INFO:teuthology.orchestra.run.smithi175.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [268 kB] 2024-03-26T03:20:33.197 INFO:teuthology.orchestra.run.smithi175.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-03-26T03:20:33.197 INFO:teuthology.orchestra.run.smithi175.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [851 kB] 2024-03-26T03:20:33.204 INFO:teuthology.orchestra.run.smithi175.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,505 kB] 2024-03-26T03:20:33.207 INFO:teuthology.orchestra.run.smithi175.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [599 kB] 2024-03-26T03:20:33.214 INFO:teuthology.orchestra.run.smithi175.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [162 kB] 2024-03-26T03:20:33.243 INFO:teuthology.orchestra.run.smithi175.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-03-26T03:20:33.243 INFO:teuthology.orchestra.run.smithi175.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-03-26T03:20:33.243 INFO:teuthology.orchestra.run.smithi175.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [290 kB] 2024-03-26T03:20:33.243 INFO:teuthology.orchestra.run.smithi175.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.1 kB] 2024-03-26T03:20:33.243 INFO:teuthology.orchestra.run.smithi175.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,476 B] 2024-03-26T03:20:33.243 INFO:teuthology.orchestra.run.smithi175.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-03-26T03:20:33.249 INFO:teuthology.orchestra.run.smithi175.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-03-26T03:20:33.249 INFO:teuthology.orchestra.run.smithi175.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.3 kB] 2024-03-26T03:20:33.250 INFO:teuthology.orchestra.run.smithi175.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,628 kB] 2024-03-26T03:20:33.272 INFO:teuthology.orchestra.run.smithi175.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [273 kB] 2024-03-26T03:20:33.275 INFO:teuthology.orchestra.run.smithi175.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-03-26T03:20:33.275 INFO:teuthology.orchestra.run.smithi175.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,059 kB] 2024-03-26T03:20:33.287 INFO:teuthology.orchestra.run.smithi175.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [697 kB] 2024-03-26T03:20:33.295 INFO:teuthology.orchestra.run.smithi175.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [240 kB] 2024-03-26T03:20:33.297 INFO:teuthology.orchestra.run.smithi175.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-03-26T03:20:33.298 INFO:teuthology.orchestra.run.smithi175.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.1 kB] 2024-03-26T03:20:33.299 INFO:teuthology.orchestra.run.smithi175.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,160 B] 2024-03-26T03:20:33.299 INFO:teuthology.orchestra.run.smithi175.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.1 kB] 2024-03-26T03:20:33.317 INFO:teuthology.orchestra.run.smithi175.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-03-26T03:20:33.317 INFO:teuthology.orchestra.run.smithi175.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-03-26T03:20:33.318 INFO:teuthology.orchestra.run.smithi175.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-03-26T03:20:33.319 INFO:teuthology.orchestra.run.smithi175.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-03-26T03:20:33.319 INFO:teuthology.orchestra.run.smithi175.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-03-26T03:20:33.319 INFO:teuthology.orchestra.run.smithi175.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-03-26T03:20:33.319 INFO:teuthology.orchestra.run.smithi175.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-03-26T03:20:33.337 INFO:teuthology.orchestra.run.smithi175.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-03-26T03:20:33.338 INFO:teuthology.orchestra.run.smithi175.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-03-26T03:20:33.359 INFO:teuthology.orchestra.run.smithi190.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,290 kB] 2024-03-26T03:20:33.367 INFO:teuthology.orchestra.run.smithi190.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-03-26T03:20:33.507 INFO:teuthology.orchestra.run.smithi190.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [595 kB] 2024-03-26T03:20:33.700 INFO:teuthology.orchestra.run.smithi190.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,505 kB] 2024-03-26T03:20:33.771 INFO:teuthology.orchestra.run.smithi190.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [429 kB] 2024-03-26T03:20:33.789 INFO:teuthology.orchestra.run.smithi190.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [230 kB] 2024-03-26T03:20:33.799 INFO:teuthology.orchestra.run.smithi190.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-03-26T03:20:33.802 INFO:teuthology.orchestra.run.smithi190.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,600 kB] 2024-03-26T03:20:33.847 INFO:teuthology.orchestra.run.smithi190.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [290 kB] 2024-03-26T03:20:33.862 INFO:teuthology.orchestra.run.smithi190.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-03-26T03:20:33.862 INFO:teuthology.orchestra.run.smithi190.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.3 kB] 2024-03-26T03:20:33.864 INFO:teuthology.orchestra.run.smithi190.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,628 kB] 2024-03-26T03:20:33.891 INFO:teuthology.orchestra.run.smithi190.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [33.7 kB] 2024-03-26T03:20:33.893 INFO:teuthology.orchestra.run.smithi190.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [268 kB] 2024-03-26T03:20:33.908 INFO:teuthology.orchestra.run.smithi190.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-03-26T03:20:33.909 INFO:teuthology.orchestra.run.smithi190.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [851 kB] 2024-03-26T03:20:33.923 INFO:teuthology.orchestra.run.smithi190.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [273 kB] 2024-03-26T03:20:33.930 INFO:teuthology.orchestra.run.smithi190.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-03-26T03:20:33.934 INFO:teuthology.orchestra.run.smithi190.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [599 kB] 2024-03-26T03:20:34.094 INFO:teuthology.orchestra.run.smithi190.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [697 kB] 2024-03-26T03:20:34.111 INFO:teuthology.orchestra.run.smithi190.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [162 kB] 2024-03-26T03:20:34.515 INFO:teuthology.orchestra.run.smithi190.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-03-26T03:20:34.520 INFO:teuthology.orchestra.run.smithi190.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.1 kB] 2024-03-26T03:20:34.535 INFO:teuthology.orchestra.run.smithi190.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-03-26T03:20:34.535 INFO:teuthology.orchestra.run.smithi190.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,476 B] 2024-03-26T03:20:34.537 INFO:teuthology.orchestra.run.smithi190.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-03-26T03:20:34.675 INFO:teuthology.orchestra.run.smithi190.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,059 kB] 2024-03-26T03:20:34.785 INFO:teuthology.orchestra.run.smithi190.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [240 kB] 2024-03-26T03:20:34.805 INFO:teuthology.orchestra.run.smithi190.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-03-26T03:20:34.806 INFO:teuthology.orchestra.run.smithi190.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,160 B] 2024-03-26T03:20:34.807 INFO:teuthology.orchestra.run.smithi190.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.1 kB] 2024-03-26T03:20:34.810 INFO:teuthology.orchestra.run.smithi190.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.1 kB] 2024-03-26T03:20:34.811 INFO:teuthology.orchestra.run.smithi190.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-03-26T03:20:34.811 INFO:teuthology.orchestra.run.smithi190.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-03-26T03:20:34.815 INFO:teuthology.orchestra.run.smithi190.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-03-26T03:20:34.819 INFO:teuthology.orchestra.run.smithi190.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-03-26T03:20:34.984 INFO:teuthology.orchestra.run.smithi190.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-03-26T03:20:35.066 INFO:teuthology.orchestra.run.smithi190.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-03-26T03:20:35.174 INFO:teuthology.orchestra.run.smithi190.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-03-26T03:20:35.232 INFO:teuthology.orchestra.run.smithi190.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-03-26T03:20:35.266 INFO:teuthology.orchestra.run.smithi190.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-03-26T03:20:38.858 INFO:teuthology.orchestra.run.smithi175.stdout:Fetched 12.5 MB in 2s (7,141 kB/s) 2024-03-26T03:20:39.707 INFO:teuthology.orchestra.run.smithi190.stdout:Fetched 12.5 MB in 3s (4,941 kB/s) 2024-03-26T03:20:40.168 INFO:teuthology.orchestra.run.smithi175.stdout:Reading package lists... 2024-03-26T03:20:40.185 DEBUG:teuthology.orchestra.run.smithi175:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-26T03:20:40.246 INFO:teuthology.orchestra.run.smithi175.stdout:Reading package lists... 2024-03-26T03:20:40.440 INFO:teuthology.orchestra.run.smithi175.stdout:Building dependency tree... 2024-03-26T03:20:40.441 INFO:teuthology.orchestra.run.smithi175.stdout:Reading state information... 2024-03-26T03:20:40.610 INFO:teuthology.orchestra.run.smithi175.stdout:The following packages were automatically installed and are no longer required: 2024-03-26T03:20:40.610 INFO:teuthology.orchestra.run.smithi175.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-26T03:20:40.611 INFO:teuthology.orchestra.run.smithi175.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-26T03:20:40.611 INFO:teuthology.orchestra.run.smithi175.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-26T03:20:40.611 INFO:teuthology.orchestra.run.smithi175.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-03-26T03:20:40.611 INFO:teuthology.orchestra.run.smithi175.stdout: python2.7-minimal 2024-03-26T03:20:40.611 INFO:teuthology.orchestra.run.smithi175.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-26T03:20:40.612 INFO:teuthology.orchestra.run.smithi175.stdout:The following additional packages will be installed: 2024-03-26T03:20:40.612 INFO:teuthology.orchestra.run.smithi175.stdout: linux-generic linux-headers-5.15.0-101 linux-headers-5.15.0-101-generic 2024-03-26T03:20:40.612 INFO:teuthology.orchestra.run.smithi175.stdout: linux-headers-generic linux-image-5.15.0-101-generic 2024-03-26T03:20:40.613 INFO:teuthology.orchestra.run.smithi175.stdout: linux-modules-5.15.0-101-generic linux-modules-extra-5.15.0-101-generic 2024-03-26T03:20:40.614 INFO:teuthology.orchestra.run.smithi175.stdout:Suggested packages: 2024-03-26T03:20:40.614 INFO:teuthology.orchestra.run.smithi175.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-03-26T03:20:40.614 INFO:teuthology.orchestra.run.smithi175.stdout:Recommended packages: 2024-03-26T03:20:40.614 INFO:teuthology.orchestra.run.smithi175.stdout: thermald 2024-03-26T03:20:40.671 INFO:teuthology.orchestra.run.smithi175.stdout:The following NEW packages will be installed: 2024-03-26T03:20:40.672 INFO:teuthology.orchestra.run.smithi175.stdout: linux-headers-5.15.0-101 linux-headers-5.15.0-101-generic 2024-03-26T03:20:40.672 INFO:teuthology.orchestra.run.smithi175.stdout: linux-image-5.15.0-101-generic linux-modules-5.15.0-101-generic 2024-03-26T03:20:40.672 INFO:teuthology.orchestra.run.smithi175.stdout: linux-modules-extra-5.15.0-101-generic 2024-03-26T03:20:40.673 INFO:teuthology.orchestra.run.smithi175.stdout:The following packages will be upgraded: 2024-03-26T03:20:40.673 INFO:teuthology.orchestra.run.smithi175.stdout: linux-generic linux-headers-generic linux-image-generic 2024-03-26T03:20:40.731 INFO:teuthology.orchestra.run.smithi175.stdout:3 upgraded, 5 newly installed, 0 to remove and 298 not upgraded. 2024-03-26T03:20:40.731 INFO:teuthology.orchestra.run.smithi175.stdout:Need to get 113 MB of archives. 2024-03-26T03:20:40.731 INFO:teuthology.orchestra.run.smithi175.stdout:After this operation, 582 MB of additional disk space will be used. 2024-03-26T03:20:40.731 INFO:teuthology.orchestra.run.smithi175.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-101-generic amd64 5.15.0-101.111 [22.6 MB] 2024-03-26T03:20:40.953 INFO:teuthology.orchestra.run.smithi190.stdout:Reading package lists... 2024-03-26T03:20:40.970 DEBUG:teuthology.orchestra.run.smithi190:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-26T03:20:41.033 INFO:teuthology.orchestra.run.smithi190.stdout:Reading package lists... 2024-03-26T03:20:41.229 INFO:teuthology.orchestra.run.smithi190.stdout:Building dependency tree... 2024-03-26T03:20:41.230 INFO:teuthology.orchestra.run.smithi190.stdout:Reading state information... 2024-03-26T03:20:41.245 INFO:teuthology.orchestra.run.smithi175.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-101-generic amd64 5.15.0-101.111 [11.5 MB] 2024-03-26T03:20:41.377 INFO:teuthology.orchestra.run.smithi175.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-101-generic amd64 5.15.0-101.111 [63.9 MB] 2024-03-26T03:20:41.400 INFO:teuthology.orchestra.run.smithi190.stdout:The following packages were automatically installed and are no longer required: 2024-03-26T03:20:41.400 INFO:teuthology.orchestra.run.smithi190.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-26T03:20:41.400 INFO:teuthology.orchestra.run.smithi190.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-26T03:20:41.401 INFO:teuthology.orchestra.run.smithi190.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-26T03:20:41.401 INFO:teuthology.orchestra.run.smithi190.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-03-26T03:20:41.401 INFO:teuthology.orchestra.run.smithi190.stdout: python2.7-minimal 2024-03-26T03:20:41.401 INFO:teuthology.orchestra.run.smithi190.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-26T03:20:41.402 INFO:teuthology.orchestra.run.smithi190.stdout:The following additional packages will be installed: 2024-03-26T03:20:41.402 INFO:teuthology.orchestra.run.smithi190.stdout: linux-generic linux-headers-5.15.0-101 linux-headers-5.15.0-101-generic 2024-03-26T03:20:41.402 INFO:teuthology.orchestra.run.smithi190.stdout: linux-headers-generic linux-image-5.15.0-101-generic 2024-03-26T03:20:41.403 INFO:teuthology.orchestra.run.smithi190.stdout: linux-modules-5.15.0-101-generic linux-modules-extra-5.15.0-101-generic 2024-03-26T03:20:41.404 INFO:teuthology.orchestra.run.smithi190.stdout:Suggested packages: 2024-03-26T03:20:41.404 INFO:teuthology.orchestra.run.smithi190.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-03-26T03:20:41.404 INFO:teuthology.orchestra.run.smithi190.stdout:Recommended packages: 2024-03-26T03:20:41.404 INFO:teuthology.orchestra.run.smithi190.stdout: thermald 2024-03-26T03:20:41.455 INFO:teuthology.orchestra.run.smithi190.stdout:The following NEW packages will be installed: 2024-03-26T03:20:41.456 INFO:teuthology.orchestra.run.smithi190.stdout: linux-headers-5.15.0-101 linux-headers-5.15.0-101-generic 2024-03-26T03:20:41.456 INFO:teuthology.orchestra.run.smithi190.stdout: linux-image-5.15.0-101-generic linux-modules-5.15.0-101-generic 2024-03-26T03:20:41.456 INFO:teuthology.orchestra.run.smithi190.stdout: linux-modules-extra-5.15.0-101-generic 2024-03-26T03:20:41.457 INFO:teuthology.orchestra.run.smithi190.stdout:The following packages will be upgraded: 2024-03-26T03:20:41.458 INFO:teuthology.orchestra.run.smithi190.stdout: linux-generic linux-headers-generic linux-image-generic 2024-03-26T03:20:41.637 INFO:teuthology.orchestra.run.smithi190.stdout:3 upgraded, 5 newly installed, 0 to remove and 298 not upgraded. 2024-03-26T03:20:41.637 INFO:teuthology.orchestra.run.smithi190.stdout:Need to get 113 MB of archives. 2024-03-26T03:20:41.637 INFO:teuthology.orchestra.run.smithi190.stdout:After this operation, 582 MB of additional disk space will be used. 2024-03-26T03:20:41.638 INFO:teuthology.orchestra.run.smithi190.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-101-generic amd64 5.15.0-101.111 [22.6 MB] 2024-03-26T03:20:42.581 INFO:teuthology.orchestra.run.smithi175.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.101.98 [1,698 B] 2024-03-26T03:20:42.582 INFO:teuthology.orchestra.run.smithi175.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.101.98 [2,486 B] 2024-03-26T03:20:42.582 INFO:teuthology.orchestra.run.smithi175.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-101 all 5.15.0-101.111 [12.3 MB] 2024-03-26T03:20:42.831 INFO:teuthology.orchestra.run.smithi175.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-101-generic amd64 5.15.0-101.111 [2,876 kB] 2024-03-26T03:20:42.889 INFO:teuthology.orchestra.run.smithi175.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.101.98 [2,332 B] 2024-03-26T03:20:43.056 INFO:teuthology.orchestra.run.smithi190.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-101-generic amd64 5.15.0-101.111 [11.5 MB] 2024-03-26T03:20:43.295 INFO:teuthology.orchestra.run.smithi175.stdout:Fetched 113 MB in 2s (51.2 MB/s) 2024-03-26T03:20:43.403 INFO:teuthology.orchestra.run.smithi190.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-101-generic amd64 5.15.0-101.111 [63.9 MB] 2024-03-26T03:20:43.461 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package linux-modules-5.15.0-101-generic. 2024-03-26T03:20:45.372 INFO:teuthology.orchestra.run.smithi175.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-03-26T03:20:45.377 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../0-linux-modules-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-26T03:20:45.450 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking linux-modules-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-26T03:20:45.587 INFO:teuthology.orchestra.run.smithi190.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.101.98 [1,698 B] 2024-03-26T03:20:45.587 INFO:teuthology.orchestra.run.smithi190.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.101.98 [2,486 B] 2024-03-26T03:20:45.588 INFO:teuthology.orchestra.run.smithi190.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-101 all 5.15.0-101.111 [12.3 MB] 2024-03-26T03:20:46.375 INFO:teuthology.orchestra.run.smithi190.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-101-generic amd64 5.15.0-101.111 [2,876 kB] 2024-03-26T03:20:46.501 INFO:teuthology.orchestra.run.smithi190.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.101.98 [2,332 B] 2024-03-26T03:20:46.895 INFO:teuthology.orchestra.run.smithi190.stdout:Fetched 113 MB in 5s (22.5 MB/s) 2024-03-26T03:20:47.076 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package linux-modules-5.15.0-101-generic. 2024-03-26T03:20:47.525 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package linux-image-5.15.0-101-generic. 2024-03-26T03:20:47.547 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../1-linux-image-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-26T03:20:47.634 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-26T03:20:48.078 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-101-generic. 2024-03-26T03:20:48.103 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-26T03:20:48.136 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking linux-modules-extra-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-26T03:20:49.025 INFO:teuthology.orchestra.run.smithi190.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-03-26T03:20:49.029 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../0-linux-modules-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-26T03:20:49.097 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking linux-modules-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-26T03:20:51.539 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package linux-image-5.15.0-101-generic. 2024-03-26T03:20:51.561 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../1-linux-image-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-26T03:20:51.639 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-26T03:20:52.076 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-101-generic. 2024-03-26T03:20:52.096 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-26T03:20:52.125 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking linux-modules-extra-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-26T03:20:54.569 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../3-linux-generic_5.15.0.101.98_amd64.deb ... 2024-03-26T03:20:54.695 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking linux-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-26T03:20:55.029 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.101.98_amd64.deb ... 2024-03-26T03:20:55.157 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking linux-image-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-26T03:20:55.460 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package linux-headers-5.15.0-101. 2024-03-26T03:20:55.482 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../5-linux-headers-5.15.0-101_5.15.0-101.111_all.deb ... 2024-03-26T03:20:55.526 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking linux-headers-5.15.0-101 (5.15.0-101.111) ... 2024-03-26T03:20:58.350 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../3-linux-generic_5.15.0.101.98_amd64.deb ... 2024-03-26T03:20:58.492 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking linux-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-26T03:20:58.817 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.101.98_amd64.deb ... 2024-03-26T03:20:58.945 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking linux-image-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-26T03:20:59.248 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package linux-headers-5.15.0-101. 2024-03-26T03:20:59.270 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../5-linux-headers-5.15.0-101_5.15.0-101.111_all.deb ... 2024-03-26T03:20:59.323 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking linux-headers-5.15.0-101 (5.15.0-101.111) ... 2024-03-26T03:21:00.743 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package linux-headers-5.15.0-101-generic. 2024-03-26T03:21:00.780 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../6-linux-headers-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-26T03:21:00.818 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking linux-headers-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-26T03:21:03.733 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.101.98_amd64.deb ... 2024-03-26T03:21:03.838 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking linux-headers-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-26T03:21:04.180 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up linux-headers-5.15.0-101 (5.15.0-101.111) ... 2024-03-26T03:21:04.310 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up linux-headers-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-26T03:21:04.436 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up linux-headers-generic (5.15.0.101.98) ... 2024-03-26T03:21:04.565 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-26T03:21:04.973 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package linux-headers-5.15.0-101-generic. 2024-03-26T03:21:05.011 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../6-linux-headers-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-26T03:21:05.048 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking linux-headers-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-26T03:21:05.875 INFO:teuthology.orchestra.run.smithi175.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-03-26T03:21:05.875 INFO:teuthology.orchestra.run.smithi175.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-03-26T03:21:05.876 INFO:teuthology.orchestra.run.smithi175.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-101-generic 2024-03-26T03:21:05.876 INFO:teuthology.orchestra.run.smithi175.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-101-generic 2024-03-26T03:21:06.051 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up linux-modules-extra-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-26T03:21:07.443 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up linux-modules-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-26T03:21:07.637 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.101.98_amd64.deb ... 2024-03-26T03:21:07.760 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking linux-headers-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-26T03:21:08.101 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up linux-headers-5.15.0-101 (5.15.0-101.111) ... 2024-03-26T03:21:08.940 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up linux-headers-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-26T03:21:09.066 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up linux-headers-generic (5.15.0.101.98) ... 2024-03-26T03:21:09.195 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-26T03:21:09.286 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up linux-image-generic (5.15.0.101.98) ... 2024-03-26T03:21:09.412 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up linux-generic (5.15.0.101.98) ... 2024-03-26T03:21:09.539 INFO:teuthology.orchestra.run.smithi175.stdout:Processing triggers for linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-26T03:21:09.607 INFO:teuthology.orchestra.run.smithi175.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-03-26T03:21:09.607 INFO:teuthology.orchestra.run.smithi175.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-101-generic 2024-03-26T03:21:10.516 INFO:teuthology.orchestra.run.smithi190.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-03-26T03:21:10.516 INFO:teuthology.orchestra.run.smithi190.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-03-26T03:21:10.516 INFO:teuthology.orchestra.run.smithi190.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-101-generic 2024-03-26T03:21:10.516 INFO:teuthology.orchestra.run.smithi190.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-101-generic 2024-03-26T03:21:10.673 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up linux-modules-extra-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-26T03:21:12.090 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up linux-modules-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-26T03:21:13.508 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up linux-image-generic (5.15.0.101.98) ... 2024-03-26T03:21:13.634 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up linux-generic (5.15.0.101.98) ... 2024-03-26T03:21:13.752 INFO:teuthology.orchestra.run.smithi190.stdout:Processing triggers for linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-26T03:21:13.823 INFO:teuthology.orchestra.run.smithi190.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-03-26T03:21:13.823 INFO:teuthology.orchestra.run.smithi190.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-101-generic 2024-03-26T03:21:24.037 INFO:teuthology.orchestra.run.smithi175.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-03-26T03:21:24.037 INFO:teuthology.orchestra.run.smithi175.stdout:Sourcing file `/etc/default/grub' 2024-03-26T03:21:24.048 INFO:teuthology.orchestra.run.smithi175.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-26T03:21:24.072 INFO:teuthology.orchestra.run.smithi175.stdout:Generating grub configuration file ... 2024-03-26T03:21:24.306 INFO:teuthology.orchestra.run.smithi175.stdout:Found linux image: /boot/vmlinuz-5.15.0-101-generic 2024-03-26T03:21:24.326 INFO:teuthology.orchestra.run.smithi175.stdout:Found initrd image: /boot/initrd.img-5.15.0-101-generic 2024-03-26T03:21:24.609 INFO:teuthology.orchestra.run.smithi175.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-26T03:21:24.615 INFO:teuthology.orchestra.run.smithi175.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-26T03:21:24.693 INFO:teuthology.orchestra.run.smithi175.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-26T03:21:24.698 INFO:teuthology.orchestra.run.smithi175.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-26T03:21:24.871 INFO:teuthology.orchestra.run.smithi175.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-26T03:21:24.872 INFO:teuthology.orchestra.run.smithi175.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-03-26T03:21:24.872 INFO:teuthology.orchestra.run.smithi175.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-26T03:21:24.922 INFO:teuthology.orchestra.run.smithi175.stdout:done 2024-03-26T03:21:25.435 DEBUG:teuthology.orchestra.run.smithi175:> dpkg -s linux-image-generic 2024-03-26T03:21:25.453 INFO:teuthology.orchestra.run.smithi175.stdout:Package: linux-image-generic 2024-03-26T03:21:25.453 INFO:teuthology.orchestra.run.smithi175.stdout:Status: install ok installed 2024-03-26T03:21:25.454 INFO:teuthology.orchestra.run.smithi175.stdout:Priority: optional 2024-03-26T03:21:25.454 INFO:teuthology.orchestra.run.smithi175.stdout:Section: kernel 2024-03-26T03:21:25.454 INFO:teuthology.orchestra.run.smithi175.stdout:Installed-Size: 21 2024-03-26T03:21:25.454 INFO:teuthology.orchestra.run.smithi175.stdout:Maintainer: Ubuntu Kernel Team 2024-03-26T03:21:25.454 INFO:teuthology.orchestra.run.smithi175.stdout:Architecture: amd64 2024-03-26T03:21:25.454 INFO:teuthology.orchestra.run.smithi175.stdout:Source: linux-meta 2024-03-26T03:21:25.454 INFO:teuthology.orchestra.run.smithi175.stdout:Version: 5.15.0.101.98 2024-03-26T03:21:25.454 INFO:teuthology.orchestra.run.smithi175.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-101), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-03-26T03:21:25.454 INFO:teuthology.orchestra.run.smithi175.stdout:Depends: linux-image-5.15.0-101-generic, linux-modules-extra-5.15.0-101-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-26T03:21:25.454 INFO:teuthology.orchestra.run.smithi175.stdout:Recommends: thermald 2024-03-26T03:21:25.454 INFO:teuthology.orchestra.run.smithi175.stdout:Description: Generic Linux kernel image 2024-03-26T03:21:25.454 INFO:teuthology.orchestra.run.smithi175.stdout: This package will always depend on the latest generic kernel image 2024-03-26T03:21:25.455 INFO:teuthology.orchestra.run.smithi175.stdout: available. 2024-03-26T03:21:25.455 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-101-generic 2024-03-26T03:21:25.455 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-26T03:21:25.455 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-26T03:21:25.455 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-03-26T03:21:25.455 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi175.front.sepia.ceph.com, path=None, version=distro) 2024-03-26T03:21:25.455 DEBUG:teuthology.orchestra.run.smithi175:> sudo apt-get clean 2024-03-26T03:21:25.609 DEBUG:teuthology.orchestra.run.smithi175:> sudo apt-get update 2024-03-26T03:21:25.724 INFO:teuthology.orchestra.run.smithi175.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-26T03:21:25.724 INFO:teuthology.orchestra.run.smithi175.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-26T03:21:25.739 INFO:teuthology.orchestra.run.smithi175.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-26T03:21:25.759 INFO:teuthology.orchestra.run.smithi175.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-26T03:21:27.053 INFO:teuthology.orchestra.run.smithi175.stdout:Reading package lists... 2024-03-26T03:21:27.071 DEBUG:teuthology.orchestra.run.smithi175:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-26T03:21:27.121 INFO:teuthology.orchestra.run.smithi175.stdout:Reading package lists... 2024-03-26T03:21:27.296 INFO:teuthology.orchestra.run.smithi175.stdout:Building dependency tree... 2024-03-26T03:21:27.296 INFO:teuthology.orchestra.run.smithi175.stdout:Reading state information... 2024-03-26T03:21:27.448 INFO:teuthology.orchestra.run.smithi175.stdout:linux-image-generic is already the newest version (5.15.0.101.98). 2024-03-26T03:21:27.448 INFO:teuthology.orchestra.run.smithi175.stdout:The following packages were automatically installed and are no longer required: 2024-03-26T03:21:27.448 INFO:teuthology.orchestra.run.smithi175.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-26T03:21:27.448 INFO:teuthology.orchestra.run.smithi175.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-26T03:21:27.448 INFO:teuthology.orchestra.run.smithi175.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-26T03:21:27.448 INFO:teuthology.orchestra.run.smithi175.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-26T03:21:27.449 INFO:teuthology.orchestra.run.smithi175.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-26T03:21:27.449 INFO:teuthology.orchestra.run.smithi175.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-26T03:21:27.486 INFO:teuthology.orchestra.run.smithi175.stdout:0 upgraded, 0 newly installed, 0 to remove and 298 not upgraded. 2024-03-26T03:21:27.488 DEBUG:teuthology.orchestra.run.smithi175:> dpkg -s linux-image-generic 2024-03-26T03:21:27.540 INFO:teuthology.orchestra.run.smithi175.stdout:Package: linux-image-generic 2024-03-26T03:21:27.540 INFO:teuthology.orchestra.run.smithi175.stdout:Status: install ok installed 2024-03-26T03:21:27.541 INFO:teuthology.orchestra.run.smithi175.stdout:Priority: optional 2024-03-26T03:21:27.541 INFO:teuthology.orchestra.run.smithi175.stdout:Section: kernel 2024-03-26T03:21:27.541 INFO:teuthology.orchestra.run.smithi175.stdout:Installed-Size: 21 2024-03-26T03:21:27.541 INFO:teuthology.orchestra.run.smithi175.stdout:Maintainer: Ubuntu Kernel Team 2024-03-26T03:21:27.541 INFO:teuthology.orchestra.run.smithi175.stdout:Architecture: amd64 2024-03-26T03:21:27.541 INFO:teuthology.orchestra.run.smithi175.stdout:Source: linux-meta 2024-03-26T03:21:27.541 INFO:teuthology.orchestra.run.smithi175.stdout:Version: 5.15.0.101.98 2024-03-26T03:21:27.541 INFO:teuthology.orchestra.run.smithi175.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-101), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-03-26T03:21:27.541 INFO:teuthology.orchestra.run.smithi175.stdout:Depends: linux-image-5.15.0-101-generic, linux-modules-extra-5.15.0-101-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-26T03:21:27.541 INFO:teuthology.orchestra.run.smithi175.stdout:Recommends: thermald 2024-03-26T03:21:27.541 INFO:teuthology.orchestra.run.smithi175.stdout:Description: Generic Linux kernel image 2024-03-26T03:21:27.541 INFO:teuthology.orchestra.run.smithi175.stdout: This package will always depend on the latest generic kernel image 2024-03-26T03:21:27.541 INFO:teuthology.orchestra.run.smithi175.stdout: available. 2024-03-26T03:21:27.542 DEBUG:teuthology.orchestra.run.smithi175:> mktemp 2024-03-26T03:21:27.585 INFO:teuthology.orchestra.run.smithi175.stdout:/tmp/tmp.CV2wdJo60O 2024-03-26T03:21:27.586 DEBUG:teuthology.orchestra.run.smithi175:> sudo cp /boot/grub/grub.cfg /tmp/tmp.CV2wdJo60O 2024-03-26T03:21:27.642 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0666 /tmp/tmp.CV2wdJo60O 2024-03-26T03:21:27.655 INFO:teuthology.orchestra.run.smithi190.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-03-26T03:21:27.655 INFO:teuthology.orchestra.run.smithi190.stdout:Sourcing file `/etc/default/grub' 2024-03-26T03:21:27.674 INFO:teuthology.orchestra.run.smithi190.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-26T03:21:27.701 INFO:teuthology.orchestra.run.smithi190.stdout:Generating grub configuration file ... 2024-03-26T03:21:27.757 DEBUG:teuthology.orchestra.remote:smithi175:/tmp/tmp.CV2wdJo60O is 10KB 2024-03-26T03:21:27.770 DEBUG:teuthology.orchestra.run.smithi175:> rm -fr /tmp/tmp.CV2wdJo60O 2024-03-26T03:21:27.775 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-03-26T03:21:27.830 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-26T03:21:27.830 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-03-26T03:21:27.830 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-03-26T03:21:27.897 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-101-generic 2024-03-26T03:21:27.897 DEBUG:teuthology.orchestra.run.smithi175:> sudo update-grub 2024-03-26T03:21:27.958 INFO:teuthology.orchestra.run.smithi190.stdout:Found linux image: /boot/vmlinuz-5.15.0-101-generic 2024-03-26T03:21:27.979 INFO:teuthology.orchestra.run.smithi190.stdout:Found initrd image: /boot/initrd.img-5.15.0-101-generic 2024-03-26T03:21:28.250 INFO:teuthology.orchestra.run.smithi190.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-26T03:21:28.256 INFO:teuthology.orchestra.run.smithi190.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-26T03:21:28.324 INFO:teuthology.orchestra.run.smithi190.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-26T03:21:28.329 INFO:teuthology.orchestra.run.smithi190.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-26T03:21:28.482 INFO:teuthology.orchestra.run.smithi190.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-26T03:21:28.482 INFO:teuthology.orchestra.run.smithi190.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-03-26T03:21:28.482 INFO:teuthology.orchestra.run.smithi190.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-26T03:21:28.538 INFO:teuthology.orchestra.run.smithi190.stdout:done 2024-03-26T03:21:28.954 DEBUG:teuthology.orchestra.run.smithi190:> dpkg -s linux-image-generic 2024-03-26T03:21:28.969 INFO:teuthology.orchestra.run.smithi190.stdout:Package: linux-image-generic 2024-03-26T03:21:28.969 INFO:teuthology.orchestra.run.smithi190.stdout:Status: install ok installed 2024-03-26T03:21:28.970 INFO:teuthology.orchestra.run.smithi190.stdout:Priority: optional 2024-03-26T03:21:28.970 INFO:teuthology.orchestra.run.smithi190.stdout:Section: kernel 2024-03-26T03:21:28.970 INFO:teuthology.orchestra.run.smithi190.stdout:Installed-Size: 21 2024-03-26T03:21:28.970 INFO:teuthology.orchestra.run.smithi190.stdout:Maintainer: Ubuntu Kernel Team 2024-03-26T03:21:28.970 INFO:teuthology.orchestra.run.smithi190.stdout:Architecture: amd64 2024-03-26T03:21:28.970 INFO:teuthology.orchestra.run.smithi190.stdout:Source: linux-meta 2024-03-26T03:21:28.970 INFO:teuthology.orchestra.run.smithi190.stdout:Version: 5.15.0.101.98 2024-03-26T03:21:28.970 INFO:teuthology.orchestra.run.smithi190.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-101), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-03-26T03:21:28.970 INFO:teuthology.orchestra.run.smithi190.stdout:Depends: linux-image-5.15.0-101-generic, linux-modules-extra-5.15.0-101-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-26T03:21:28.970 INFO:teuthology.orchestra.run.smithi190.stdout:Recommends: thermald 2024-03-26T03:21:28.970 INFO:teuthology.orchestra.run.smithi190.stdout:Description: Generic Linux kernel image 2024-03-26T03:21:28.970 INFO:teuthology.orchestra.run.smithi190.stdout: This package will always depend on the latest generic kernel image 2024-03-26T03:21:28.970 INFO:teuthology.orchestra.run.smithi190.stdout: available. 2024-03-26T03:21:28.970 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-101-generic 2024-03-26T03:21:28.971 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-26T03:21:28.971 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-26T03:21:28.971 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-03-26T03:21:28.971 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi190.front.sepia.ceph.com, path=None, version=distro) 2024-03-26T03:21:28.971 DEBUG:teuthology.orchestra.run.smithi190:> sudo apt-get clean 2024-03-26T03:21:29.057 INFO:teuthology.orchestra.run.smithi175.stderr:Sourcing file `/etc/default/grub' 2024-03-26T03:21:29.065 INFO:teuthology.orchestra.run.smithi175.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-26T03:21:29.078 INFO:teuthology.orchestra.run.smithi175.stderr:Generating grub configuration file ... 2024-03-26T03:21:29.162 DEBUG:teuthology.orchestra.run.smithi190:> sudo apt-get update 2024-03-26T03:21:29.267 INFO:teuthology.orchestra.run.smithi175.stderr:Found linux image: /boot/vmlinuz-5.15.0-101-generic 2024-03-26T03:21:29.273 INFO:teuthology.orchestra.run.smithi175.stderr:Found initrd image: /boot/initrd.img-5.15.0-101-generic 2024-03-26T03:21:29.284 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-26T03:21:29.284 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-26T03:21:29.295 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-26T03:21:29.315 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-26T03:21:29.567 INFO:teuthology.orchestra.run.smithi175.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-26T03:21:29.572 INFO:teuthology.orchestra.run.smithi175.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-26T03:21:29.614 INFO:teuthology.orchestra.run.smithi175.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-26T03:21:29.622 INFO:teuthology.orchestra.run.smithi175.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-26T03:21:29.746 INFO:teuthology.orchestra.run.smithi175.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-26T03:21:29.746 INFO:teuthology.orchestra.run.smithi175.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-26T03:21:29.746 INFO:teuthology.orchestra.run.smithi175.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-26T03:21:29.756 INFO:teuthology.orchestra.run.smithi175.stderr:done 2024-03-26T03:21:29.769 DEBUG:teuthology.orchestra.run.smithi175:> sudo shutdown -r now 2024-03-26T03:21:30.664 INFO:teuthology.orchestra.run.smithi190.stdout:Reading package lists... 2024-03-26T03:21:30.682 DEBUG:teuthology.orchestra.run.smithi190:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-26T03:21:30.733 INFO:teuthology.orchestra.run.smithi190.stdout:Reading package lists... 2024-03-26T03:21:30.907 INFO:teuthology.orchestra.run.smithi190.stdout:Building dependency tree... 2024-03-26T03:21:30.907 INFO:teuthology.orchestra.run.smithi190.stdout:Reading state information... 2024-03-26T03:21:31.062 INFO:teuthology.orchestra.run.smithi190.stdout:linux-image-generic is already the newest version (5.15.0.101.98). 2024-03-26T03:21:31.063 INFO:teuthology.orchestra.run.smithi190.stdout:The following packages were automatically installed and are no longer required: 2024-03-26T03:21:31.063 INFO:teuthology.orchestra.run.smithi190.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-26T03:21:31.063 INFO:teuthology.orchestra.run.smithi190.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-26T03:21:31.063 INFO:teuthology.orchestra.run.smithi190.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-26T03:21:31.063 INFO:teuthology.orchestra.run.smithi190.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-26T03:21:31.063 INFO:teuthology.orchestra.run.smithi190.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-26T03:21:31.063 INFO:teuthology.orchestra.run.smithi190.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-26T03:21:31.101 INFO:teuthology.orchestra.run.smithi190.stdout:0 upgraded, 0 newly installed, 0 to remove and 298 not upgraded. 2024-03-26T03:21:31.103 DEBUG:teuthology.orchestra.run.smithi190:> dpkg -s linux-image-generic 2024-03-26T03:21:31.162 INFO:teuthology.orchestra.run.smithi190.stdout:Package: linux-image-generic 2024-03-26T03:21:31.163 INFO:teuthology.orchestra.run.smithi190.stdout:Status: install ok installed 2024-03-26T03:21:31.163 INFO:teuthology.orchestra.run.smithi190.stdout:Priority: optional 2024-03-26T03:21:31.163 INFO:teuthology.orchestra.run.smithi190.stdout:Section: kernel 2024-03-26T03:21:31.163 INFO:teuthology.orchestra.run.smithi190.stdout:Installed-Size: 21 2024-03-26T03:21:31.163 INFO:teuthology.orchestra.run.smithi190.stdout:Maintainer: Ubuntu Kernel Team 2024-03-26T03:21:31.163 INFO:teuthology.orchestra.run.smithi190.stdout:Architecture: amd64 2024-03-26T03:21:31.163 INFO:teuthology.orchestra.run.smithi190.stdout:Source: linux-meta 2024-03-26T03:21:31.163 INFO:teuthology.orchestra.run.smithi190.stdout:Version: 5.15.0.101.98 2024-03-26T03:21:31.163 INFO:teuthology.orchestra.run.smithi190.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-101), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-03-26T03:21:31.163 INFO:teuthology.orchestra.run.smithi190.stdout:Depends: linux-image-5.15.0-101-generic, linux-modules-extra-5.15.0-101-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-26T03:21:31.163 INFO:teuthology.orchestra.run.smithi190.stdout:Recommends: thermald 2024-03-26T03:21:31.163 INFO:teuthology.orchestra.run.smithi190.stdout:Description: Generic Linux kernel image 2024-03-26T03:21:31.163 INFO:teuthology.orchestra.run.smithi190.stdout: This package will always depend on the latest generic kernel image 2024-03-26T03:21:31.164 INFO:teuthology.orchestra.run.smithi190.stdout: available. 2024-03-26T03:21:31.164 DEBUG:teuthology.orchestra.run.smithi190:> mktemp 2024-03-26T03:21:31.208 INFO:teuthology.orchestra.run.smithi190.stdout:/tmp/tmp.XS7V6AdXnb 2024-03-26T03:21:31.208 DEBUG:teuthology.orchestra.run.smithi190:> sudo cp /boot/grub/grub.cfg /tmp/tmp.XS7V6AdXnb 2024-03-26T03:21:31.264 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0666 /tmp/tmp.XS7V6AdXnb 2024-03-26T03:21:31.395 DEBUG:teuthology.orchestra.remote:smithi190:/tmp/tmp.XS7V6AdXnb is 10KB 2024-03-26T03:21:31.408 DEBUG:teuthology.orchestra.run.smithi190:> rm -fr /tmp/tmp.XS7V6AdXnb 2024-03-26T03:21:31.413 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-03-26T03:21:31.467 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-03-26T03:21:31.467 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-03-26T03:21:31.467 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-03-26T03:21:31.541 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-101-generic 2024-03-26T03:21:31.541 DEBUG:teuthology.orchestra.run.smithi190:> sudo update-grub 2024-03-26T03:21:32.692 INFO:teuthology.orchestra.run.smithi190.stderr:Sourcing file `/etc/default/grub' 2024-03-26T03:21:32.693 INFO:teuthology.orchestra.run.smithi190.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-26T03:21:32.704 INFO:teuthology.orchestra.run.smithi190.stderr:Generating grub configuration file ... 2024-03-26T03:21:32.876 INFO:teuthology.orchestra.run.smithi190.stderr:Found linux image: /boot/vmlinuz-5.15.0-101-generic 2024-03-26T03:21:32.882 INFO:teuthology.orchestra.run.smithi190.stderr:Found initrd image: /boot/initrd.img-5.15.0-101-generic 2024-03-26T03:21:33.104 INFO:teuthology.orchestra.run.smithi190.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-26T03:21:33.109 INFO:teuthology.orchestra.run.smithi190.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-26T03:21:33.145 INFO:teuthology.orchestra.run.smithi190.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-26T03:21:33.149 INFO:teuthology.orchestra.run.smithi190.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-26T03:21:33.272 INFO:teuthology.orchestra.run.smithi190.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-26T03:21:33.272 INFO:teuthology.orchestra.run.smithi190.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-26T03:21:33.272 INFO:teuthology.orchestra.run.smithi190.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-26T03:21:33.291 INFO:teuthology.orchestra.run.smithi190.stderr:done 2024-03-26T03:21:33.293 DEBUG:teuthology.orchestra.run.smithi190:> sudo shutdown -r now 2024-03-26T03:21:59.798 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-26T03:21:59.799 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-03-26T03:21:59.800 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T03:22:03.302 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-26T03:22:03.302 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-03-26T03:22:03.302 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T03:22:18.270 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.175 2024-03-26T03:22:21.858 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.190 2024-03-26T03:22:27.277 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-03-26T03:22:27.278 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T03:22:30.334 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.175 2024-03-26T03:22:30.859 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-03-26T03:22:30.860 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T03:22:33.918 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.190 2024-03-26T03:22:42.342 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-03-26T03:22:42.343 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T03:22:45.922 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-03-26T03:22:45.923 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T03:23:04.350 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.190 2024-03-26T03:23:14.206 DEBUG:teuthology.orchestra.run.smithi175:> true 2024-03-26T03:23:14.747 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-03-26T03:23:14.747 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-101-generic"... 2024-03-26T03:23:14.747 DEBUG:teuthology.orchestra.run.smithi175:> uname -r 2024-03-26T03:23:14.793 INFO:teuthology.orchestra.run.smithi175.stdout:5.15.0-101-generic 2024-03-26T03:23:14.794 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-101-generic vs 5.15.0-101-generic 2024-03-26T03:23:14.794 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-26T03:23:14.794 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-26T03:23:15.795 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-26T03:23:15.795 DEBUG:teuthology.orchestra.run.smithi175:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-26T03:23:15.873 INFO:teuthology.orchestra.run.smithi175.stdout:ttyS1 2024-03-26T03:23:15.895 DEBUG:teuthology.parallel:result is None 2024-03-26T03:23:19.352 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-03-26T03:23:19.353 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T03:23:19.620 DEBUG:teuthology.orchestra.run.smithi190:> true 2024-03-26T03:23:20.193 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-03-26T03:23:20.193 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-101-generic"... 2024-03-26T03:23:20.193 DEBUG:teuthology.orchestra.run.smithi190:> uname -r 2024-03-26T03:23:20.239 INFO:teuthology.orchestra.run.smithi190.stdout:5.15.0-101-generic 2024-03-26T03:23:20.239 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-101-generic vs 5.15.0-101-generic 2024-03-26T03:23:20.239 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-26T03:23:20.240 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-26T03:23:21.241 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-26T03:23:21.241 DEBUG:teuthology.orchestra.run.smithi190:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-26T03:23:21.617 INFO:teuthology.orchestra.run.smithi190.stdout:ttyS1 2024-03-26T03:23:21.641 DEBUG:teuthology.parallel:result is None 2024-03-26T03:23:21.641 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-26T03:23:21.649 INFO:teuthology.task.internal:Creating test directory... 2024-03-26T03:23:21.649 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-26T03:23:21.652 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-26T03:23:21.657 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-26T03:23:21.664 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-26T03:23:21.670 INFO:teuthology.task.internal:Creating archive directory... 2024-03-26T03:23:21.670 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-26T03:23:21.699 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-26T03:23:21.781 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-26T03:23:21.787 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-26T03:23:21.788 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-26T03:23:21.790 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-26T03:23:21.807 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T03:23:21.810 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T03:23:21.815 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T03:23:21.818 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T03:23:21.819 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-26T03:23:21.827 INFO:teuthology.task.internal:Configuring sudo... 2024-03-26T03:23:21.827 DEBUG:teuthology.orchestra.run.smithi175:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-26T03:23:21.863 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-26T03:23:21.890 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-26T03:23:21.899 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-26T03:23:21.899 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-26T03:23:21.935 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-26T03:23:21.940 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T03:23:21.986 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T03:23:22.034 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-26T03:23:22.034 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-26T03:23:22.099 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T03:23:22.105 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T03:23:22.152 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-03-26T03:23:22.152 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-26T03:23:22.218 DEBUG:teuthology.orchestra.run.smithi175:> sudo service rsyslog restart 2024-03-26T03:23:22.221 DEBUG:teuthology.orchestra.run.smithi190:> sudo service rsyslog restart 2024-03-26T03:23:22.297 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-26T03:23:22.304 INFO:teuthology.task.internal:Starting timer... 2024-03-26T03:23:22.304 INFO:teuthology.run_tasks:Running task pcp... 2024-03-26T03:23:22.314 INFO:teuthology.run_tasks:Running task selinux... 2024-03-26T03:23:22.323 DEBUG:teuthology.task.selinux:Excluding smithi175: OS 'ubuntu' does not support SELinux 2024-03-26T03:23:22.323 DEBUG:teuthology.task.selinux:Excluding smithi190: OS 'ubuntu' does not support SELinux 2024-03-26T03:23:22.323 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-26T03:23:22.323 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-03-26T03:23:22.323 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-26T03:23:22.324 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-26T03:23:22.332 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-26T03:23:22.348 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-26T03:23:22.349 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi175.front.sepia.ceph.com,smithi190.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-26T03:30:21.360 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi175.front.sepia.ceph.com'), Remote(name='ubuntu@smithi190.front.sepia.ceph.com')] 2024-03-26T03:30:21.362 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-03-26T03:30:21.363 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T03:30:21.444 DEBUG:teuthology.orchestra.run.smithi175:> true 2024-03-26T03:30:21.518 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-03-26T03:30:21.518 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-03-26T03:30:21.519 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T03:30:21.597 DEBUG:teuthology.orchestra.run.smithi190:> true 2024-03-26T03:30:21.675 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-03-26T03:30:21.676 INFO:teuthology.run_tasks:Running task clock... 2024-03-26T03:30:21.687 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-26T03:30:21.688 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-26T03:30:21.688 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T03:30:21.691 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-26T03:30:21.691 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T03:30:21.715 INFO:teuthology.orchestra.run.smithi175.stdout:26 Mar 03:30:21 ntpd[17178]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-03-26T03:30:21.715 INFO:teuthology.orchestra.run.smithi175.stdout:26 Mar 03:30:21 ntpd[17178]: Command line: ntpd -gq 2024-03-26T03:30:21.716 INFO:teuthology.orchestra.run.smithi175.stdout:26 Mar 03:30:21 ntpd[17178]: ---------------------------------------------------- 2024-03-26T03:30:21.716 INFO:teuthology.orchestra.run.smithi175.stdout:26 Mar 03:30:21 ntpd[17178]: ntp-4 is maintained by Network Time Foundation, 2024-03-26T03:30:21.716 INFO:teuthology.orchestra.run.smithi175.stdout:26 Mar 03:30:21 ntpd[17178]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-03-26T03:30:21.716 INFO:teuthology.orchestra.run.smithi175.stdout:26 Mar 03:30:21 ntpd[17178]: corporation. Support and training for ntp-4 are 2024-03-26T03:30:21.716 INFO:teuthology.orchestra.run.smithi175.stdout:26 Mar 03:30:21 ntpd[17178]: available at https://www.nwtime.org/support 2024-03-26T03:30:21.716 INFO:teuthology.orchestra.run.smithi175.stdout:26 Mar 03:30:21 ntpd[17178]: ---------------------------------------------------- 2024-03-26T03:30:21.717 INFO:teuthology.orchestra.run.smithi175.stdout:26 Mar 03:30:21 ntpd[17178]: proto: precision = 0.059 usec (-24) 2024-03-26T03:30:21.717 INFO:teuthology.orchestra.run.smithi175.stdout:26 Mar 03:30:21 ntpd[17178]: basedate set to 2022-02-04 2024-03-26T03:30:21.717 INFO:teuthology.orchestra.run.smithi175.stdout:26 Mar 03:30:21 ntpd[17178]: gps base set to 2022-02-06 (week 2196) 2024-03-26T03:30:21.717 INFO:teuthology.orchestra.run.smithi175.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-26T03:30:21.718 INFO:teuthology.orchestra.run.smithi175.stdout:26 Mar 03:30:21 ntpd[17178]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-26T03:30:21.718 INFO:teuthology.orchestra.run.smithi175.stdout:26 Mar 03:30:21 ntpd[17178]: restrict ::: KOD does nothing without LIMITED. 2024-03-26T03:30:21.718 INFO:teuthology.orchestra.run.smithi175.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-26T03:30:21.719 INFO:teuthology.orchestra.run.smithi175.stdout:26 Mar 03:30:21 ntpd[17178]: Listen and drop on 0 v6wildcard [::]:123 2024-03-26T03:30:21.719 INFO:teuthology.orchestra.run.smithi175.stdout:26 Mar 03:30:21 ntpd[17178]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-26T03:30:21.719 INFO:teuthology.orchestra.run.smithi175.stdout:26 Mar 03:30:21 ntpd[17178]: Listen normally on 2 lo 127.0.0.1:123 2024-03-26T03:30:21.719 INFO:teuthology.orchestra.run.smithi175.stdout:26 Mar 03:30:21 ntpd[17178]: Listen normally on 3 enp3s0f1 172.21.15.175:123 2024-03-26T03:30:21.719 INFO:teuthology.orchestra.run.smithi175.stdout:26 Mar 03:30:21 ntpd[17178]: Listen normally on 4 lo [::1]:123 2024-03-26T03:30:21.719 INFO:teuthology.orchestra.run.smithi175.stdout:26 Mar 03:30:21 ntpd[17178]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:71f1%5]:123 2024-03-26T03:30:21.719 INFO:teuthology.orchestra.run.smithi175.stdout:26 Mar 03:30:21 ntpd[17178]: Listening on routing socket on fd #22 for interface updates 2024-03-26T03:30:21.744 INFO:teuthology.orchestra.run.smithi190.stdout:26 Mar 03:30:21 ntpd[17193]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-03-26T03:30:21.745 INFO:teuthology.orchestra.run.smithi190.stdout:26 Mar 03:30:21 ntpd[17193]: Command line: ntpd -gq 2024-03-26T03:30:21.745 INFO:teuthology.orchestra.run.smithi190.stdout:26 Mar 03:30:21 ntpd[17193]: ---------------------------------------------------- 2024-03-26T03:30:21.745 INFO:teuthology.orchestra.run.smithi190.stdout:26 Mar 03:30:21 ntpd[17193]: ntp-4 is maintained by Network Time Foundation, 2024-03-26T03:30:21.745 INFO:teuthology.orchestra.run.smithi190.stdout:26 Mar 03:30:21 ntpd[17193]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-03-26T03:30:21.745 INFO:teuthology.orchestra.run.smithi190.stdout:26 Mar 03:30:21 ntpd[17193]: corporation. Support and training for ntp-4 are 2024-03-26T03:30:21.745 INFO:teuthology.orchestra.run.smithi190.stdout:26 Mar 03:30:21 ntpd[17193]: available at https://www.nwtime.org/support 2024-03-26T03:30:21.745 INFO:teuthology.orchestra.run.smithi190.stdout:26 Mar 03:30:21 ntpd[17193]: ---------------------------------------------------- 2024-03-26T03:30:21.745 INFO:teuthology.orchestra.run.smithi190.stdout:26 Mar 03:30:21 ntpd[17193]: proto: precision = 0.046 usec (-24) 2024-03-26T03:30:21.746 INFO:teuthology.orchestra.run.smithi190.stdout:26 Mar 03:30:21 ntpd[17193]: basedate set to 2022-02-04 2024-03-26T03:30:21.746 INFO:teuthology.orchestra.run.smithi190.stdout:26 Mar 03:30:21 ntpd[17193]: gps base set to 2022-02-06 (week 2196) 2024-03-26T03:30:21.746 INFO:teuthology.orchestra.run.smithi190.stdout:26 Mar 03:30:21 ntpd[17193]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-26T03:30:21.746 INFO:teuthology.orchestra.run.smithi190.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-26T03:30:21.746 INFO:teuthology.orchestra.run.smithi190.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-26T03:30:21.747 INFO:teuthology.orchestra.run.smithi190.stdout:26 Mar 03:30:21 ntpd[17193]: restrict ::: KOD does nothing without LIMITED. 2024-03-26T03:30:21.747 INFO:teuthology.orchestra.run.smithi190.stdout:26 Mar 03:30:21 ntpd[17193]: Listen and drop on 0 v6wildcard [::]:123 2024-03-26T03:30:21.747 INFO:teuthology.orchestra.run.smithi190.stdout:26 Mar 03:30:21 ntpd[17193]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-26T03:30:21.747 INFO:teuthology.orchestra.run.smithi190.stdout:26 Mar 03:30:21 ntpd[17193]: Listen normally on 2 lo 127.0.0.1:123 2024-03-26T03:30:21.747 INFO:teuthology.orchestra.run.smithi190.stdout:26 Mar 03:30:21 ntpd[17193]: Listen normally on 3 enp3s0f1 172.21.15.190:123 2024-03-26T03:30:21.748 INFO:teuthology.orchestra.run.smithi190.stdout:26 Mar 03:30:21 ntpd[17193]: Listen normally on 4 lo [::1]:123 2024-03-26T03:30:21.748 INFO:teuthology.orchestra.run.smithi190.stdout:26 Mar 03:30:21 ntpd[17193]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7763%5]:123 2024-03-26T03:30:21.748 INFO:teuthology.orchestra.run.smithi190.stdout:26 Mar 03:30:21 ntpd[17193]: Listening on routing socket on fd #22 for interface updates 2024-03-26T03:30:22.716 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:22 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:22.716 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:22 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:22.716 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:22 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:22.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:22 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:22.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:22 ntpd[17178]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-26T03:30:22.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:22 ntpd[17178]: can't open /var/log/ntpstats/peerstats.20240326: Permission denied 2024-03-26T03:30:24.716 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:24 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:24.716 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:24 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:24.716 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:24 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:24.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:24 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:24.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:24 ntpd[17178]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-26T03:30:24.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:24 ntpd[17178]: can't open /var/log/ntpstats/peerstats.20240326: Permission denied 2024-03-26T03:30:24.745 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:24 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:24.745 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:24 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:24.745 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:24 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:24.745 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:24 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:24.745 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:24 ntpd[17193]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-26T03:30:24.745 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:24 ntpd[17193]: can't open /var/log/ntpstats/peerstats.20240326: Permission denied 2024-03-26T03:30:25.716 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:25 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:25.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:25 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:25.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:25 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:25.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:25 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:25.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:25 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:25.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:25 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:25.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:25 ntpd[17178]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-26T03:30:25.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:25 ntpd[17178]: can't open /var/log/ntpstats/peerstats.20240326: Permission denied 2024-03-26T03:30:25.744 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:25 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:25.745 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:25 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:26.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:26 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:26.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:26 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:26.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:26 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:26.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:26 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:26.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:26 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:26.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:26 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:26.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:26 ntpd[17178]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-26T03:30:26.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:26 ntpd[17178]: can't open /var/log/ntpstats/peerstats.20240326: Permission denied 2024-03-26T03:30:26.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:26 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:26.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:26 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:26.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:26 ntpd[17178]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-26T03:30:26.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:26 ntpd[17178]: can't open /var/log/ntpstats/peerstats.20240326: Permission denied 2024-03-26T03:30:26.746 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:26 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:26.746 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:26 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:26.746 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:26 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:26.746 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:26 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:26.746 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:26 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:26.746 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:26 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:26.746 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:26 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:26.746 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:26 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:26.746 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:26 ntpd[17193]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-26T03:30:26.746 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:26 ntpd[17193]: can't open /var/log/ntpstats/peerstats.20240326: Permission denied 2024-03-26T03:30:26.747 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:26 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:26.747 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:26 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:26.747 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:26 ntpd[17193]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-26T03:30:26.747 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:26 ntpd[17193]: can't open /var/log/ntpstats/peerstats.20240326: Permission denied 2024-03-26T03:30:26.747 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:26 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:26.747 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:26 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:26.747 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:26 ntpd[17193]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-26T03:30:26.747 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:26 ntpd[17193]: can't open /var/log/ntpstats/peerstats.20240326: Permission denied 2024-03-26T03:30:27.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:27 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:27.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:27 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:27.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:27 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:27.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:27 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:27.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:27 ntpd[17178]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-26T03:30:27.718 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:27 ntpd[17178]: can't open /var/log/ntpstats/peerstats.20240326: Permission denied 2024-03-26T03:30:28.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:28 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:28.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:28 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:28.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:28 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:28.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:28 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:28.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:28 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:28.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:28 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:28.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:28 ntpd[17178]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-26T03:30:28.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:28 ntpd[17178]: can't open /var/log/ntpstats/peerstats.20240326: Permission denied 2024-03-26T03:30:28.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:28 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:28.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:28 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:28.717 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:28 ntpd[17178]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-26T03:30:28.718 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:28 ntpd[17178]: can't open /var/log/ntpstats/peerstats.20240326: Permission denied 2024-03-26T03:30:28.718 INFO:teuthology.orchestra.run.smithi175.stdout:26 Mar 03:30:28 ntpd[17178]: ntpd: time slew -0.000462 s 2024-03-26T03:30:28.718 INFO:teuthology.orchestra.run.smithi175.stdout:ntpd: time slew -0.000462s 2024-03-26T03:30:28.719 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:28 ntpd[17178]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-26T03:30:28.719 INFO:teuthology.orchestra.run.smithi175.stderr:26 Mar 03:30:28 ntpd[17178]: can't open /var/log/ntpstats/loopstats.20240326: Permission denied 2024-03-26T03:30:28.747 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:28 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:28.747 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:28 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:28.747 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:28 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:28.747 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:28 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:28.747 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:28 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:28.747 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:28 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:28.747 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:28 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:28.747 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:28 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:28.747 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:28 ntpd[17193]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-26T03:30:28.747 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:28 ntpd[17193]: can't open /var/log/ntpstats/peerstats.20240326: Permission denied 2024-03-26T03:30:28.747 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:28 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:28.747 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:28 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:28.747 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:28 ntpd[17193]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-26T03:30:28.748 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:28 ntpd[17193]: can't open /var/log/ntpstats/peerstats.20240326: Permission denied 2024-03-26T03:30:28.748 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:28 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:28.748 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:28 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:28.748 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:28 ntpd[17193]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-26T03:30:28.748 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:28 ntpd[17193]: can't open /var/log/ntpstats/peerstats.20240326: Permission denied 2024-03-26T03:30:28.843 INFO:teuthology.orchestra.run.smithi175.stdout: remote refid st t when poll reach delay offset jitter 2024-03-26T03:30:28.843 INFO:teuthology.orchestra.run.smithi175.stdout:============================================================================== 2024-03-26T03:30:28.843 INFO:teuthology.orchestra.run.smithi175.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-26T03:30:28.843 INFO:teuthology.orchestra.run.smithi175.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-26T03:30:28.843 INFO:teuthology.orchestra.run.smithi175.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-26T03:30:28.843 INFO:teuthology.orchestra.run.smithi175.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-26T03:30:30.745 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:30 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:30.745 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:30 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:30.745 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:30 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:30.745 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:30 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:30.745 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:30 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:30.746 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:30 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:30.746 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:30 ntpd[17193]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-26T03:30:30.746 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:30 ntpd[17193]: can't open /var/log/ntpstats/rawstats.20240326: Permission denied 2024-03-26T03:30:30.746 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:30 ntpd[17193]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-26T03:30:30.746 INFO:teuthology.orchestra.run.smithi190.stdout:26 Mar 03:30:30 ntpd[17193]: ntpd: time slew -0.001674 s 2024-03-26T03:30:30.746 INFO:teuthology.orchestra.run.smithi190.stdout:ntpd: time slew -0.001674s 2024-03-26T03:30:30.747 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:30 ntpd[17193]: can't open /var/log/ntpstats/peerstats.20240326: Permission denied 2024-03-26T03:30:30.747 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:30 ntpd[17193]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-26T03:30:30.747 INFO:teuthology.orchestra.run.smithi190.stderr:26 Mar 03:30:30 ntpd[17193]: can't open /var/log/ntpstats/loopstats.20240326: Permission denied 2024-03-26T03:30:30.800 INFO:teuthology.orchestra.run.smithi190.stdout: remote refid st t when poll reach delay offset jitter 2024-03-26T03:30:30.800 INFO:teuthology.orchestra.run.smithi190.stdout:============================================================================== 2024-03-26T03:30:30.800 INFO:teuthology.orchestra.run.smithi190.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-26T03:30:30.800 INFO:teuthology.orchestra.run.smithi190.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-26T03:30:30.800 INFO:teuthology.orchestra.run.smithi190.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-26T03:30:30.800 INFO:teuthology.orchestra.run.smithi190.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-26T03:30:30.800 INFO:teuthology.run_tasks:Running task install... 2024-03-26T03:30:30.812 DEBUG:teuthology.task.install:project ceph 2024-03-26T03:30:30.813 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003'}, 'extra_packages': ['cephadm']} 2024-03-26T03:30:30.813 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003'} 2024-03-26T03:30:30.813 INFO:teuthology.task.install:Using flavor: default 2024-03-26T03:30:30.822 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-26T03:30:30.822 INFO:teuthology.task.install:extra packages: [] 2024-03-26T03:30:30.823 DEBUG:teuthology.orchestra.run.smithi175:> sudo apt-key list | grep Ceph 2024-03-26T03:30:30.823 DEBUG:teuthology.orchestra.run.smithi190:> sudo apt-key list | grep Ceph 2024-03-26T03:30:30.886 INFO:teuthology.orchestra.run.smithi175.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-03-26T03:30:30.902 INFO:teuthology.orchestra.run.smithi190.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-03-26T03:30:30.917 INFO:teuthology.orchestra.run.smithi175.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-03-26T03:30:30.917 INFO:teuthology.orchestra.run.smithi175.stdout:uid [ unknown] Ceph.com (release key) 2024-03-26T03:30:30.917 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-03-26T03:30:30.918 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 2024-03-26T03:30:30.932 INFO:teuthology.orchestra.run.smithi190.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-03-26T03:30:30.932 INFO:teuthology.orchestra.run.smithi190.stdout:uid [ unknown] Ceph.com (release key) 2024-03-26T03:30:30.932 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-03-26T03:30:30.932 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 2024-03-26T03:30:31.073 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default/ 2024-03-26T03:30:31.073 INFO:teuthology.task.install.deb:Package version is 19.0.0-2475-ge42b8ea3-1jammy 2024-03-26T03:30:31.164 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default/ 2024-03-26T03:30:31.164 INFO:teuthology.task.install.deb:Package version is 19.0.0-2475-ge42b8ea3-1jammy 2024-03-26T03:30:31.185 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-26T03:30:31.185 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-03-26T03:30:31.199 DEBUG:teuthology.orchestra.run.smithi175:> sudo apt-get update 2024-03-26T03:30:31.272 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-03-26T03:30:31.272 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-03-26T03:30:31.286 DEBUG:teuthology.orchestra.run.smithi190:> sudo apt-get update 2024-03-26T03:30:31.371 INFO:teuthology.orchestra.run.smithi175.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-26T03:30:31.454 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-26T03:30:31.487 INFO:teuthology.orchestra.run.smithi175.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-26T03:30:31.488 INFO:teuthology.orchestra.run.smithi175.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy InRelease 2024-03-26T03:30:31.502 INFO:teuthology.orchestra.run.smithi190.stdout:Ign:2 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy InRelease 2024-03-26T03:30:31.523 INFO:teuthology.orchestra.run.smithi190.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-03-26T03:30:31.524 INFO:teuthology.orchestra.run.smithi175.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-03-26T03:30:31.545 INFO:teuthology.orchestra.run.smithi190.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy Release.gpg 2024-03-26T03:30:31.566 INFO:teuthology.orchestra.run.smithi190.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,422 B] 2024-03-26T03:30:31.570 INFO:teuthology.orchestra.run.smithi175.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-26T03:30:31.571 INFO:teuthology.orchestra.run.smithi175.stdout:Ign:6 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy Release.gpg 2024-03-26T03:30:31.572 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-26T03:30:31.592 INFO:teuthology.orchestra.run.smithi190.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-03-26T03:30:31.607 INFO:teuthology.orchestra.run.smithi175.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-03-26T03:30:31.650 INFO:teuthology.orchestra.run.smithi175.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-26T03:30:31.651 INFO:teuthology.orchestra.run.smithi175.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,422 B] 2024-03-26T03:30:31.651 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-26T03:30:31.732 INFO:teuthology.orchestra.run.smithi190.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-26T03:30:31.784 INFO:teuthology.orchestra.run.smithi175.stdout:Fetched 29.2 kB in 0s (71.9 kB/s) 2024-03-26T03:30:31.869 INFO:teuthology.orchestra.run.smithi190.stdout:Fetched 29.2 kB in 0s (71.4 kB/s) 2024-03-26T03:30:32.750 INFO:teuthology.orchestra.run.smithi175.stdout:Reading package lists... 2024-03-26T03:30:32.762 INFO:teuthology.orchestra.run.smithi190.stdout:Reading package lists... 2024-03-26T03:30:32.764 DEBUG:teuthology.orchestra.run.smithi175:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2475-ge42b8ea3-1jammy cephadm=19.0.0-2475-ge42b8ea3-1jammy ceph-mds=19.0.0-2475-ge42b8ea3-1jammy ceph-mgr=19.0.0-2475-ge42b8ea3-1jammy ceph-common=19.0.0-2475-ge42b8ea3-1jammy ceph-fuse=19.0.0-2475-ge42b8ea3-1jammy ceph-test=19.0.0-2475-ge42b8ea3-1jammy ceph-volume=19.0.0-2475-ge42b8ea3-1jammy radosgw=19.0.0-2475-ge42b8ea3-1jammy python3-rados=19.0.0-2475-ge42b8ea3-1jammy python3-rgw=19.0.0-2475-ge42b8ea3-1jammy python3-cephfs=19.0.0-2475-ge42b8ea3-1jammy python3-rbd=19.0.0-2475-ge42b8ea3-1jammy libcephfs2=19.0.0-2475-ge42b8ea3-1jammy libcephfs-dev=19.0.0-2475-ge42b8ea3-1jammy librados2=19.0.0-2475-ge42b8ea3-1jammy librbd1=19.0.0-2475-ge42b8ea3-1jammy rbd-fuse=19.0.0-2475-ge42b8ea3-1jammy 2024-03-26T03:30:32.778 DEBUG:teuthology.orchestra.run.smithi190:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2475-ge42b8ea3-1jammy cephadm=19.0.0-2475-ge42b8ea3-1jammy ceph-mds=19.0.0-2475-ge42b8ea3-1jammy ceph-mgr=19.0.0-2475-ge42b8ea3-1jammy ceph-common=19.0.0-2475-ge42b8ea3-1jammy ceph-fuse=19.0.0-2475-ge42b8ea3-1jammy ceph-test=19.0.0-2475-ge42b8ea3-1jammy ceph-volume=19.0.0-2475-ge42b8ea3-1jammy radosgw=19.0.0-2475-ge42b8ea3-1jammy python3-rados=19.0.0-2475-ge42b8ea3-1jammy python3-rgw=19.0.0-2475-ge42b8ea3-1jammy python3-cephfs=19.0.0-2475-ge42b8ea3-1jammy python3-rbd=19.0.0-2475-ge42b8ea3-1jammy libcephfs2=19.0.0-2475-ge42b8ea3-1jammy libcephfs-dev=19.0.0-2475-ge42b8ea3-1jammy librados2=19.0.0-2475-ge42b8ea3-1jammy librbd1=19.0.0-2475-ge42b8ea3-1jammy rbd-fuse=19.0.0-2475-ge42b8ea3-1jammy 2024-03-26T03:30:32.824 INFO:teuthology.orchestra.run.smithi175.stdout:Reading package lists... 2024-03-26T03:30:32.841 INFO:teuthology.orchestra.run.smithi190.stdout:Reading package lists... 2024-03-26T03:30:33.018 INFO:teuthology.orchestra.run.smithi175.stdout:Building dependency tree... 2024-03-26T03:30:33.019 INFO:teuthology.orchestra.run.smithi175.stdout:Reading state information... 2024-03-26T03:30:33.036 INFO:teuthology.orchestra.run.smithi190.stdout:Building dependency tree... 2024-03-26T03:30:33.036 INFO:teuthology.orchestra.run.smithi190.stdout:Reading state information... 2024-03-26T03:30:33.199 INFO:teuthology.orchestra.run.smithi175.stdout:The following packages were automatically installed and are no longer required: 2024-03-26T03:30:33.199 INFO:teuthology.orchestra.run.smithi175.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-26T03:30:33.199 INFO:teuthology.orchestra.run.smithi175.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-26T03:30:33.200 INFO:teuthology.orchestra.run.smithi175.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-26T03:30:33.200 INFO:teuthology.orchestra.run.smithi175.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-26T03:30:33.200 INFO:teuthology.orchestra.run.smithi175.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-26T03:30:33.200 INFO:teuthology.orchestra.run.smithi175.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-26T03:30:33.201 INFO:teuthology.orchestra.run.smithi175.stdout:The following additional packages will be installed: 2024-03-26T03:30:33.201 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-03-26T03:30:33.201 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-03-26T03:30:33.202 INFO:teuthology.orchestra.run.smithi175.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-26T03:30:33.202 INFO:teuthology.orchestra.run.smithi175.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-03-26T03:30:33.202 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-03-26T03:30:33.202 INFO:teuthology.orchestra.run.smithi175.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-03-26T03:30:33.203 INFO:teuthology.orchestra.run.smithi175.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-03-26T03:30:33.203 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-03-26T03:30:33.203 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-03-26T03:30:33.203 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-03-26T03:30:33.203 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-26T03:30:33.203 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-26T03:30:33.203 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-26T03:30:33.203 INFO:teuthology.orchestra.run.smithi175.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-26T03:30:33.203 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-26T03:30:33.204 INFO:teuthology.orchestra.run.smithi175.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-26T03:30:33.204 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-03-26T03:30:33.204 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-03-26T03:30:33.204 INFO:teuthology.orchestra.run.smithi175.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-03-26T03:30:33.204 INFO:teuthology.orchestra.run.smithi175.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-03-26T03:30:33.204 INFO:teuthology.orchestra.run.smithi175.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-03-26T03:30:33.204 INFO:teuthology.orchestra.run.smithi175.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-03-26T03:30:33.204 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-03-26T03:30:33.204 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet zip 2024-03-26T03:30:33.204 INFO:teuthology.orchestra.run.smithi175.stdout:Suggested packages: 2024-03-26T03:30:33.205 INFO:teuthology.orchestra.run.smithi175.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-03-26T03:30:33.205 INFO:teuthology.orchestra.run.smithi175.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-03-26T03:30:33.205 INFO:teuthology.orchestra.run.smithi175.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-03-26T03:30:33.205 INFO:teuthology.orchestra.run.smithi175.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-03-26T03:30:33.205 INFO:teuthology.orchestra.run.smithi175.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-03-26T03:30:33.205 INFO:teuthology.orchestra.run.smithi175.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-03-26T03:30:33.205 INFO:teuthology.orchestra.run.smithi175.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-03-26T03:30:33.205 INFO:teuthology.orchestra.run.smithi175.stdout:Recommended packages: 2024-03-26T03:30:33.205 INFO:teuthology.orchestra.run.smithi175.stdout: btrfs-tools 2024-03-26T03:30:33.217 INFO:teuthology.orchestra.run.smithi190.stdout:The following packages were automatically installed and are no longer required: 2024-03-26T03:30:33.217 INFO:teuthology.orchestra.run.smithi190.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-26T03:30:33.217 INFO:teuthology.orchestra.run.smithi190.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-26T03:30:33.217 INFO:teuthology.orchestra.run.smithi190.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-26T03:30:33.217 INFO:teuthology.orchestra.run.smithi190.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-26T03:30:33.217 INFO:teuthology.orchestra.run.smithi190.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-26T03:30:33.218 INFO:teuthology.orchestra.run.smithi190.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-26T03:30:33.218 INFO:teuthology.orchestra.run.smithi190.stdout:The following additional packages will be installed: 2024-03-26T03:30:33.218 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-03-26T03:30:33.218 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-03-26T03:30:33.220 INFO:teuthology.orchestra.run.smithi190.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-26T03:30:33.220 INFO:teuthology.orchestra.run.smithi190.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-03-26T03:30:33.220 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-03-26T03:30:33.220 INFO:teuthology.orchestra.run.smithi190.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-03-26T03:30:33.220 INFO:teuthology.orchestra.run.smithi190.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-03-26T03:30:33.220 INFO:teuthology.orchestra.run.smithi190.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-03-26T03:30:33.220 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-03-26T03:30:33.220 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-03-26T03:30:33.220 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-26T03:30:33.220 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-26T03:30:33.221 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-26T03:30:33.221 INFO:teuthology.orchestra.run.smithi190.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-26T03:30:33.221 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-26T03:30:33.221 INFO:teuthology.orchestra.run.smithi190.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-26T03:30:33.221 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-03-26T03:30:33.221 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-03-26T03:30:33.221 INFO:teuthology.orchestra.run.smithi190.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-03-26T03:30:33.221 INFO:teuthology.orchestra.run.smithi190.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-03-26T03:30:33.221 INFO:teuthology.orchestra.run.smithi190.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-03-26T03:30:33.221 INFO:teuthology.orchestra.run.smithi190.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-03-26T03:30:33.221 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-03-26T03:30:33.221 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet zip 2024-03-26T03:30:33.222 INFO:teuthology.orchestra.run.smithi190.stdout:Suggested packages: 2024-03-26T03:30:33.222 INFO:teuthology.orchestra.run.smithi190.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-03-26T03:30:33.222 INFO:teuthology.orchestra.run.smithi190.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-03-26T03:30:33.222 INFO:teuthology.orchestra.run.smithi190.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-03-26T03:30:33.222 INFO:teuthology.orchestra.run.smithi190.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-03-26T03:30:33.222 INFO:teuthology.orchestra.run.smithi190.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-03-26T03:30:33.222 INFO:teuthology.orchestra.run.smithi190.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-03-26T03:30:33.222 INFO:teuthology.orchestra.run.smithi190.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-03-26T03:30:33.222 INFO:teuthology.orchestra.run.smithi190.stdout:Recommended packages: 2024-03-26T03:30:33.222 INFO:teuthology.orchestra.run.smithi190.stdout: btrfs-tools 2024-03-26T03:30:33.303 INFO:teuthology.orchestra.run.smithi175.stdout:The following NEW packages will be installed: 2024-03-26T03:30:33.304 INFO:teuthology.orchestra.run.smithi175.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-03-26T03:30:33.304 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-03-26T03:30:33.304 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-03-26T03:30:33.304 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-03-26T03:30:33.304 INFO:teuthology.orchestra.run.smithi175.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-03-26T03:30:33.304 INFO:teuthology.orchestra.run.smithi175.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-03-26T03:30:33.305 INFO:teuthology.orchestra.run.smithi175.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-03-26T03:30:33.305 INFO:teuthology.orchestra.run.smithi175.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-03-26T03:30:33.305 INFO:teuthology.orchestra.run.smithi175.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-03-26T03:30:33.305 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-03-26T03:30:33.305 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-03-26T03:30:33.305 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-03-26T03:30:33.306 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-26T03:30:33.306 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-26T03:30:33.306 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-26T03:30:33.306 INFO:teuthology.orchestra.run.smithi175.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-26T03:30:33.306 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-26T03:30:33.306 INFO:teuthology.orchestra.run.smithi175.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-26T03:30:33.306 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-03-26T03:30:33.306 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-03-26T03:30:33.306 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-03-26T03:30:33.306 INFO:teuthology.orchestra.run.smithi175.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-03-26T03:30:33.306 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-03-26T03:30:33.306 INFO:teuthology.orchestra.run.smithi175.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-03-26T03:30:33.306 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-03-26T03:30:33.306 INFO:teuthology.orchestra.run.smithi175.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-03-26T03:30:33.324 INFO:teuthology.orchestra.run.smithi190.stdout:The following NEW packages will be installed: 2024-03-26T03:30:33.324 INFO:teuthology.orchestra.run.smithi190.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-03-26T03:30:33.324 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-03-26T03:30:33.324 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-03-26T03:30:33.324 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-03-26T03:30:33.324 INFO:teuthology.orchestra.run.smithi190.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-03-26T03:30:33.325 INFO:teuthology.orchestra.run.smithi190.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-03-26T03:30:33.326 INFO:teuthology.orchestra.run.smithi190.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-03-26T03:30:33.326 INFO:teuthology.orchestra.run.smithi190.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-03-26T03:30:33.326 INFO:teuthology.orchestra.run.smithi190.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-03-26T03:30:33.326 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-03-26T03:30:33.326 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-03-26T03:30:33.326 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-03-26T03:30:33.326 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-26T03:30:33.326 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-26T03:30:33.326 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-26T03:30:33.326 INFO:teuthology.orchestra.run.smithi190.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-26T03:30:33.327 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-26T03:30:33.327 INFO:teuthology.orchestra.run.smithi190.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-26T03:30:33.327 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-03-26T03:30:33.327 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-03-26T03:30:33.327 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-03-26T03:30:33.327 INFO:teuthology.orchestra.run.smithi190.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-03-26T03:30:33.327 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-03-26T03:30:33.327 INFO:teuthology.orchestra.run.smithi190.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-03-26T03:30:33.327 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-03-26T03:30:33.327 INFO:teuthology.orchestra.run.smithi190.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-03-26T03:30:33.389 INFO:teuthology.orchestra.run.smithi190.stdout:0 upgraded, 118 newly installed, 0 to remove and 285 not upgraded. 2024-03-26T03:30:33.389 INFO:teuthology.orchestra.run.smithi190.stdout:Need to get 178 MB of archives. 2024-03-26T03:30:33.389 INFO:teuthology.orchestra.run.smithi190.stdout:After this operation, 819 MB of additional disk space will be used. 2024-03-26T03:30:33.389 INFO:teuthology.orchestra.run.smithi190.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-03-26T03:30:33.465 INFO:teuthology.orchestra.run.smithi190.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2475-ge42b8ea3-1jammy [3,553 kB] 2024-03-26T03:30:33.490 INFO:teuthology.orchestra.run.smithi190.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-03-26T03:30:33.492 INFO:teuthology.orchestra.run.smithi175.stdout:0 upgraded, 118 newly installed, 0 to remove and 285 not upgraded. 2024-03-26T03:30:33.492 INFO:teuthology.orchestra.run.smithi175.stdout:Need to get 178 MB of archives. 2024-03-26T03:30:33.492 INFO:teuthology.orchestra.run.smithi175.stdout:After this operation, 819 MB of additional disk space will be used. 2024-03-26T03:30:33.492 INFO:teuthology.orchestra.run.smithi175.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-03-26T03:30:33.492 INFO:teuthology.orchestra.run.smithi190.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-03-26T03:30:33.512 INFO:teuthology.orchestra.run.smithi190.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-03-26T03:30:33.522 INFO:teuthology.orchestra.run.smithi175.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2475-ge42b8ea3-1jammy [3,553 kB] 2024-03-26T03:30:33.581 INFO:teuthology.orchestra.run.smithi190.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-03-26T03:30:33.587 INFO:teuthology.orchestra.run.smithi190.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-03-26T03:30:33.602 INFO:teuthology.orchestra.run.smithi190.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-03-26T03:30:33.607 INFO:teuthology.orchestra.run.smithi190.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-03-26T03:30:33.607 INFO:teuthology.orchestra.run.smithi190.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-03-26T03:30:33.608 INFO:teuthology.orchestra.run.smithi190.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-03-26T03:30:33.618 INFO:teuthology.orchestra.run.smithi190.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-03-26T03:30:33.620 INFO:teuthology.orchestra.run.smithi190.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-03-26T03:30:33.623 INFO:teuthology.orchestra.run.smithi190.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-03-26T03:30:33.627 INFO:teuthology.orchestra.run.smithi190.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-03-26T03:30:33.646 INFO:teuthology.orchestra.run.smithi190.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-03-26T03:30:33.648 INFO:teuthology.orchestra.run.smithi190.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-03-26T03:30:33.651 INFO:teuthology.orchestra.run.smithi190.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-03-26T03:30:33.655 INFO:teuthology.orchestra.run.smithi190.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-03-26T03:30:33.657 INFO:teuthology.orchestra.run.smithi190.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-03-26T03:30:33.657 INFO:teuthology.orchestra.run.smithi190.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-03-26T03:30:33.659 INFO:teuthology.orchestra.run.smithi190.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-03-26T03:30:33.660 INFO:teuthology.orchestra.run.smithi190.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-03-26T03:30:33.660 INFO:teuthology.orchestra.run.smithi190.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-03-26T03:30:33.682 INFO:teuthology.orchestra.run.smithi190.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-03-26T03:30:33.682 INFO:teuthology.orchestra.run.smithi190.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-03-26T03:30:33.682 INFO:teuthology.orchestra.run.smithi190.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-03-26T03:30:33.683 INFO:teuthology.orchestra.run.smithi190.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-03-26T03:30:33.686 INFO:teuthology.orchestra.run.smithi190.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-03-26T03:30:33.686 INFO:teuthology.orchestra.run.smithi190.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-03-26T03:30:33.706 INFO:teuthology.orchestra.run.smithi190.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-03-26T03:30:33.706 INFO:teuthology.orchestra.run.smithi190.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-03-26T03:30:33.708 INFO:teuthology.orchestra.run.smithi190.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-03-26T03:30:33.708 INFO:teuthology.orchestra.run.smithi190.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-03-26T03:30:33.732 INFO:teuthology.orchestra.run.smithi190.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-03-26T03:30:33.736 INFO:teuthology.orchestra.run.smithi190.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-03-26T03:30:33.738 INFO:teuthology.orchestra.run.smithi190.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-03-26T03:30:33.738 INFO:teuthology.orchestra.run.smithi190.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-03-26T03:30:33.749 INFO:teuthology.orchestra.run.smithi190.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-03-26T03:30:33.750 INFO:teuthology.orchestra.run.smithi190.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-03-26T03:30:33.763 INFO:teuthology.orchestra.run.smithi190.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-03-26T03:30:33.764 INFO:teuthology.orchestra.run.smithi190.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-03-26T03:30:33.767 INFO:teuthology.orchestra.run.smithi190.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-03-26T03:30:33.770 INFO:teuthology.orchestra.run.smithi190.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-03-26T03:30:33.785 INFO:teuthology.orchestra.run.smithi190.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-03-26T03:30:33.788 INFO:teuthology.orchestra.run.smithi190.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-03-26T03:30:33.789 INFO:teuthology.orchestra.run.smithi190.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-03-26T03:30:33.794 INFO:teuthology.orchestra.run.smithi190.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-03-26T03:30:33.795 INFO:teuthology.orchestra.run.smithi190.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-03-26T03:30:33.797 INFO:teuthology.orchestra.run.smithi190.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-03-26T03:30:33.813 INFO:teuthology.orchestra.run.smithi190.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-03-26T03:30:33.900 INFO:teuthology.orchestra.run.smithi190.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-03-26T03:30:33.901 INFO:teuthology.orchestra.run.smithi190.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-03-26T03:30:33.903 INFO:teuthology.orchestra.run.smithi190.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-03-26T03:30:33.903 INFO:teuthology.orchestra.run.smithi190.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-03-26T03:30:33.905 INFO:teuthology.orchestra.run.smithi190.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-03-26T03:30:33.922 INFO:teuthology.orchestra.run.smithi175.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-03-26T03:30:33.934 INFO:teuthology.orchestra.run.smithi175.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-03-26T03:30:33.942 INFO:teuthology.orchestra.run.smithi190.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-03-26T03:30:33.945 INFO:teuthology.orchestra.run.smithi190.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-03-26T03:30:33.946 INFO:teuthology.orchestra.run.smithi190.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-03-26T03:30:33.975 INFO:teuthology.orchestra.run.smithi190.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-03-26T03:30:33.975 INFO:teuthology.orchestra.run.smithi190.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-03-26T03:30:33.987 INFO:teuthology.orchestra.run.smithi190.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-03-26T03:30:33.988 INFO:teuthology.orchestra.run.smithi190.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-03-26T03:30:33.988 INFO:teuthology.orchestra.run.smithi190.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-03-26T03:30:33.989 INFO:teuthology.orchestra.run.smithi190.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-03-26T03:30:33.989 INFO:teuthology.orchestra.run.smithi190.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-03-26T03:30:33.990 INFO:teuthology.orchestra.run.smithi190.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-03-26T03:30:33.999 INFO:teuthology.orchestra.run.smithi190.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-03-26T03:30:34.014 INFO:teuthology.orchestra.run.smithi190.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-03-26T03:30:34.016 INFO:teuthology.orchestra.run.smithi190.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-03-26T03:30:34.016 INFO:teuthology.orchestra.run.smithi175.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-03-26T03:30:34.017 INFO:teuthology.orchestra.run.smithi190.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-03-26T03:30:34.033 INFO:teuthology.orchestra.run.smithi190.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-03-26T03:30:34.037 INFO:teuthology.orchestra.run.smithi190.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-03-26T03:30:34.037 INFO:teuthology.orchestra.run.smithi190.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-03-26T03:30:34.038 INFO:teuthology.orchestra.run.smithi190.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-03-26T03:30:34.041 INFO:teuthology.orchestra.run.smithi190.stdout:Get:76 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2475-ge42b8ea3-1jammy [3,226 kB] 2024-03-26T03:30:34.044 INFO:teuthology.orchestra.run.smithi190.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-03-26T03:30:34.045 INFO:teuthology.orchestra.run.smithi190.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-03-26T03:30:34.049 INFO:teuthology.orchestra.run.smithi190.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-03-26T03:30:34.053 INFO:teuthology.orchestra.run.smithi190.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-03-26T03:30:34.053 INFO:teuthology.orchestra.run.smithi190.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-03-26T03:30:34.054 INFO:teuthology.orchestra.run.smithi190.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-03-26T03:30:34.072 INFO:teuthology.orchestra.run.smithi190.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-03-26T03:30:34.073 INFO:teuthology.orchestra.run.smithi190.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-03-26T03:30:34.082 INFO:teuthology.orchestra.run.smithi190.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-03-26T03:30:34.083 INFO:teuthology.orchestra.run.smithi190.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-03-26T03:30:34.083 INFO:teuthology.orchestra.run.smithi190.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-03-26T03:30:34.086 INFO:teuthology.orchestra.run.smithi190.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-03-26T03:30:34.087 INFO:teuthology.orchestra.run.smithi190.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-03-26T03:30:34.250 INFO:teuthology.orchestra.run.smithi190.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2475-ge42b8ea3-1jammy [712 kB] 2024-03-26T03:30:34.250 INFO:teuthology.orchestra.run.smithi175.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-03-26T03:30:34.260 INFO:teuthology.orchestra.run.smithi175.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-03-26T03:30:34.288 INFO:teuthology.orchestra.run.smithi190.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2475-ge42b8ea3-1jammy [356 kB] 2024-03-26T03:30:34.290 INFO:teuthology.orchestra.run.smithi175.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-03-26T03:30:34.296 INFO:teuthology.orchestra.run.smithi190.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2475-ge42b8ea3-1jammy [32.8 kB] 2024-03-26T03:30:34.296 INFO:teuthology.orchestra.run.smithi190.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2475-ge42b8ea3-1jammy [180 kB] 2024-03-26T03:30:34.300 INFO:teuthology.orchestra.run.smithi175.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-03-26T03:30:34.301 INFO:teuthology.orchestra.run.smithi175.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-03-26T03:30:34.302 INFO:teuthology.orchestra.run.smithi175.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-03-26T03:30:34.311 INFO:teuthology.orchestra.run.smithi190.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2475-ge42b8ea3-1jammy [66.7 kB] 2024-03-26T03:30:34.313 INFO:teuthology.orchestra.run.smithi190.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2475-ge42b8ea3-1jammy [330 kB] 2024-03-26T03:30:34.321 INFO:teuthology.orchestra.run.smithi175.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-03-26T03:30:34.330 INFO:teuthology.orchestra.run.smithi190.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2475-ge42b8ea3-1jammy [6,660 kB] 2024-03-26T03:30:34.489 INFO:teuthology.orchestra.run.smithi175.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-03-26T03:30:34.586 INFO:teuthology.orchestra.run.smithi190.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2475-ge42b8ea3-1jammy [112 kB] 2024-03-26T03:30:34.589 INFO:teuthology.orchestra.run.smithi190.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2475-ge42b8ea3-1jammy [466 kB] 2024-03-26T03:30:34.609 INFO:teuthology.orchestra.run.smithi190.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2475-ge42b8ea3-1jammy [25.3 MB] 2024-03-26T03:30:34.752 INFO:teuthology.orchestra.run.smithi175.stdout:Get:14 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2475-ge42b8ea3-1jammy [3,226 kB] 2024-03-26T03:30:34.892 INFO:teuthology.orchestra.run.smithi175.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-03-26T03:30:34.927 INFO:teuthology.orchestra.run.smithi175.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-03-26T03:30:34.928 INFO:teuthology.orchestra.run.smithi175.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-03-26T03:30:35.008 INFO:teuthology.orchestra.run.smithi175.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-03-26T03:30:35.059 INFO:teuthology.orchestra.run.smithi175.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-03-26T03:30:35.081 INFO:teuthology.orchestra.run.smithi175.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-03-26T03:30:35.087 INFO:teuthology.orchestra.run.smithi175.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-03-26T03:30:35.089 INFO:teuthology.orchestra.run.smithi175.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-03-26T03:30:35.100 INFO:teuthology.orchestra.run.smithi175.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-03-26T03:30:35.101 INFO:teuthology.orchestra.run.smithi175.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-03-26T03:30:35.266 INFO:teuthology.orchestra.run.smithi175.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-03-26T03:30:35.349 INFO:teuthology.orchestra.run.smithi175.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-03-26T03:30:35.377 INFO:teuthology.orchestra.run.smithi175.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-03-26T03:30:35.394 INFO:teuthology.orchestra.run.smithi175.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-03-26T03:30:35.441 INFO:teuthology.orchestra.run.smithi175.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-03-26T03:30:35.441 INFO:teuthology.orchestra.run.smithi175.stdout:Get:30 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2475-ge42b8ea3-1jammy [712 kB] 2024-03-26T03:30:35.598 INFO:teuthology.orchestra.run.smithi175.stdout:Get:31 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2475-ge42b8ea3-1jammy [356 kB] 2024-03-26T03:30:35.649 INFO:teuthology.orchestra.run.smithi175.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-03-26T03:30:35.663 INFO:teuthology.orchestra.run.smithi175.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-03-26T03:30:35.674 INFO:teuthology.orchestra.run.smithi175.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-03-26T03:30:35.674 INFO:teuthology.orchestra.run.smithi190.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2475-ge42b8ea3-1jammy [5,086 kB] 2024-03-26T03:30:35.677 INFO:teuthology.orchestra.run.smithi175.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-03-26T03:30:35.698 INFO:teuthology.orchestra.run.smithi175.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-03-26T03:30:35.702 INFO:teuthology.orchestra.run.smithi175.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-03-26T03:30:35.734 INFO:teuthology.orchestra.run.smithi175.stdout:Get:38 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2475-ge42b8ea3-1jammy [32.8 kB] 2024-03-26T03:30:35.734 INFO:teuthology.orchestra.run.smithi175.stdout:Get:39 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2475-ge42b8ea3-1jammy [180 kB] 2024-03-26T03:30:35.741 INFO:teuthology.orchestra.run.smithi175.stdout:Get:40 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2475-ge42b8ea3-1jammy [66.7 kB] 2024-03-26T03:30:35.741 INFO:teuthology.orchestra.run.smithi175.stdout:Get:41 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2475-ge42b8ea3-1jammy [330 kB] 2024-03-26T03:30:35.773 INFO:teuthology.orchestra.run.smithi175.stdout:Get:42 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2475-ge42b8ea3-1jammy [6,660 kB] 2024-03-26T03:30:35.791 INFO:teuthology.orchestra.run.smithi190.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2475-ge42b8ea3-1jammy [244 kB] 2024-03-26T03:30:35.797 INFO:teuthology.orchestra.run.smithi190.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2475-ge42b8ea3-1jammy [124 kB] 2024-03-26T03:30:35.801 INFO:teuthology.orchestra.run.smithi190.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2475-ge42b8ea3-1jammy [1,526 kB] 2024-03-26T03:30:35.862 INFO:teuthology.orchestra.run.smithi190.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2475-ge42b8ea3-1jammy [6,259 kB] 2024-03-26T03:30:35.868 INFO:teuthology.orchestra.run.smithi175.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-03-26T03:30:36.119 INFO:teuthology.orchestra.run.smithi190.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2475-ge42b8ea3-1jammy [22.8 MB] 2024-03-26T03:30:36.200 INFO:teuthology.orchestra.run.smithi175.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-03-26T03:30:36.239 INFO:teuthology.orchestra.run.smithi175.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-03-26T03:30:36.252 INFO:teuthology.orchestra.run.smithi175.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-03-26T03:30:36.459 INFO:teuthology.orchestra.run.smithi175.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-03-26T03:30:36.460 INFO:teuthology.orchestra.run.smithi175.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-03-26T03:30:36.466 INFO:teuthology.orchestra.run.smithi175.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-03-26T03:30:36.472 INFO:teuthology.orchestra.run.smithi175.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-03-26T03:30:36.486 INFO:teuthology.orchestra.run.smithi175.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-03-26T03:30:36.507 INFO:teuthology.orchestra.run.smithi175.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-03-26T03:30:36.513 INFO:teuthology.orchestra.run.smithi175.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-03-26T03:30:36.689 INFO:teuthology.orchestra.run.smithi175.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-03-26T03:30:36.922 INFO:teuthology.orchestra.run.smithi175.stdout:Get:55 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2475-ge42b8ea3-1jammy [112 kB] 2024-03-26T03:30:36.942 INFO:teuthology.orchestra.run.smithi175.stdout:Get:56 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2475-ge42b8ea3-1jammy [466 kB] 2024-03-26T03:30:36.953 INFO:teuthology.orchestra.run.smithi175.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-03-26T03:30:37.025 INFO:teuthology.orchestra.run.smithi175.stdout:Get:58 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2475-ge42b8ea3-1jammy [25.3 MB] 2024-03-26T03:30:37.130 INFO:teuthology.orchestra.run.smithi175.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-03-26T03:30:37.139 INFO:teuthology.orchestra.run.smithi190.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2475-ge42b8ea3-1jammy [14.1 kB] 2024-03-26T03:30:37.139 INFO:teuthology.orchestra.run.smithi190.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2475-ge42b8ea3-1jammy [870 kB] 2024-03-26T03:30:37.144 INFO:teuthology.orchestra.run.smithi175.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-03-26T03:30:37.156 INFO:teuthology.orchestra.run.smithi190.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2475-ge42b8ea3-1jammy [2,435 kB] 2024-03-26T03:30:37.163 INFO:teuthology.orchestra.run.smithi175.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-03-26T03:30:37.198 INFO:teuthology.orchestra.run.smithi190.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2475-ge42b8ea3-1jammy [791 kB] 2024-03-26T03:30:37.207 INFO:teuthology.orchestra.run.smithi190.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2475-ge42b8ea3-1jammy [147 kB] 2024-03-26T03:30:37.210 INFO:teuthology.orchestra.run.smithi190.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2475-ge42b8ea3-1jammy [2,259 kB] 2024-03-26T03:30:37.229 INFO:teuthology.orchestra.run.smithi175.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-03-26T03:30:37.305 INFO:teuthology.orchestra.run.smithi190.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2475-ge42b8ea3-1jammy [8,625 kB] 2024-03-26T03:30:37.516 INFO:teuthology.orchestra.run.smithi175.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-03-26T03:30:37.520 INFO:teuthology.orchestra.run.smithi175.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-03-26T03:30:37.523 INFO:teuthology.orchestra.run.smithi175.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-03-26T03:30:37.523 INFO:teuthology.orchestra.run.smithi175.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-03-26T03:30:37.525 INFO:teuthology.orchestra.run.smithi175.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-03-26T03:30:37.652 INFO:teuthology.orchestra.run.smithi190.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2475-ge42b8ea3-1jammy [14.2 kB] 2024-03-26T03:30:37.653 INFO:teuthology.orchestra.run.smithi190.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2475-ge42b8ea3-1jammy [50.3 MB] 2024-03-26T03:30:37.747 INFO:teuthology.orchestra.run.smithi175.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-03-26T03:30:38.183 INFO:teuthology.orchestra.run.smithi175.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-03-26T03:30:38.189 INFO:teuthology.orchestra.run.smithi175.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-03-26T03:30:38.479 INFO:teuthology.orchestra.run.smithi175.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-03-26T03:30:38.480 INFO:teuthology.orchestra.run.smithi175.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-03-26T03:30:38.482 INFO:teuthology.orchestra.run.smithi175.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-03-26T03:30:38.484 INFO:teuthology.orchestra.run.smithi175.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-03-26T03:30:38.485 INFO:teuthology.orchestra.run.smithi175.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-03-26T03:30:38.487 INFO:teuthology.orchestra.run.smithi175.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-03-26T03:30:38.489 INFO:teuthology.orchestra.run.smithi175.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-03-26T03:30:38.491 INFO:teuthology.orchestra.run.smithi175.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-03-26T03:30:38.671 INFO:teuthology.orchestra.run.smithi175.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-03-26T03:30:39.085 INFO:teuthology.orchestra.run.smithi175.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-03-26T03:30:39.131 INFO:teuthology.orchestra.run.smithi175.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-03-26T03:30:39.166 INFO:teuthology.orchestra.run.smithi175.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-03-26T03:30:39.262 INFO:teuthology.orchestra.run.smithi175.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-03-26T03:30:39.303 INFO:teuthology.orchestra.run.smithi175.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-03-26T03:30:39.314 INFO:teuthology.orchestra.run.smithi175.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-03-26T03:30:39.320 INFO:teuthology.orchestra.run.smithi175.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-03-26T03:30:39.367 INFO:teuthology.orchestra.run.smithi175.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-03-26T03:30:39.369 INFO:teuthology.orchestra.run.smithi175.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-03-26T03:30:39.391 INFO:teuthology.orchestra.run.smithi175.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-03-26T03:30:39.556 INFO:teuthology.orchestra.run.smithi175.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-03-26T03:30:39.831 INFO:teuthology.orchestra.run.smithi175.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-03-26T03:30:39.848 INFO:teuthology.orchestra.run.smithi175.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-03-26T03:30:39.926 INFO:teuthology.orchestra.run.smithi190.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2475-ge42b8ea3-1jammy [121 kB] 2024-03-26T03:30:39.928 INFO:teuthology.orchestra.run.smithi190.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2475-ge42b8ea3-1jammy [38.9 kB] 2024-03-26T03:30:39.930 INFO:teuthology.orchestra.run.smithi190.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2475-ge42b8ea3-1jammy [13.2 MB] 2024-03-26T03:30:40.001 INFO:teuthology.orchestra.run.smithi175.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-03-26T03:30:40.031 INFO:teuthology.orchestra.run.smithi175.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-03-26T03:30:40.188 INFO:teuthology.orchestra.run.smithi175.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-03-26T03:30:40.190 INFO:teuthology.orchestra.run.smithi175.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-03-26T03:30:40.193 INFO:teuthology.orchestra.run.smithi175.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-03-26T03:30:40.215 INFO:teuthology.orchestra.run.smithi175.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-03-26T03:30:40.220 INFO:teuthology.orchestra.run.smithi175.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-03-26T03:30:40.311 INFO:teuthology.orchestra.run.smithi190.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2475-ge42b8ea3-1jammy [91.2 kB] 2024-03-26T03:30:40.872 INFO:teuthology.orchestra.run.smithi190.stdout:Fetched 178 MB in 7s (25.6 MB/s) 2024-03-26T03:30:40.916 INFO:teuthology.orchestra.run.smithi175.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2475-ge42b8ea3-1jammy [5,086 kB] 2024-03-26T03:30:41.038 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-03-26T03:30:41.139 INFO:teuthology.orchestra.run.smithi190.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165997 files and directories currently installed.) 2024-03-26T03:30:41.144 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-03-26T03:30:41.228 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-26T03:30:41.280 INFO:teuthology.orchestra.run.smithi175.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2475-ge42b8ea3-1jammy [244 kB] 2024-03-26T03:30:41.286 INFO:teuthology.orchestra.run.smithi175.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2475-ge42b8ea3-1jammy [124 kB] 2024-03-26T03:30:41.313 INFO:teuthology.orchestra.run.smithi175.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2475-ge42b8ea3-1jammy [1,526 kB] 2024-03-26T03:30:41.425 INFO:teuthology.orchestra.run.smithi175.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2475-ge42b8ea3-1jammy [6,259 kB] 2024-03-26T03:30:41.573 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-03-26T03:30:41.599 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-03-26T03:30:41.639 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-26T03:30:41.909 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-03-26T03:30:41.929 INFO:teuthology.orchestra.run.smithi175.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2475-ge42b8ea3-1jammy [22.8 MB] 2024-03-26T03:30:41.935 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-03-26T03:30:41.967 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-26T03:30:42.404 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-03-26T03:30:42.430 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-26T03:30:42.478 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-26T03:30:42.915 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-03-26T03:30:42.942 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-26T03:30:42.985 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-26T03:30:43.527 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-03-26T03:30:43.554 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-26T03:30:43.593 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-26T03:30:43.913 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-03-26T03:30:43.939 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-03-26T03:30:43.943 INFO:teuthology.orchestra.run.smithi175.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2475-ge42b8ea3-1jammy [14.1 kB] 2024-03-26T03:30:43.943 INFO:teuthology.orchestra.run.smithi175.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2475-ge42b8ea3-1jammy [870 kB] 2024-03-26T03:30:43.961 INFO:teuthology.orchestra.run.smithi175.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2475-ge42b8ea3-1jammy [2,435 kB] 2024-03-26T03:30:43.979 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-26T03:30:44.142 INFO:teuthology.orchestra.run.smithi175.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2475-ge42b8ea3-1jammy [791 kB] 2024-03-26T03:30:44.222 INFO:teuthology.orchestra.run.smithi175.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2475-ge42b8ea3-1jammy [147 kB] 2024-03-26T03:30:44.245 INFO:teuthology.orchestra.run.smithi175.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2475-ge42b8ea3-1jammy [2,259 kB] 2024-03-26T03:30:44.419 INFO:teuthology.orchestra.run.smithi175.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2475-ge42b8ea3-1jammy [8,625 kB] 2024-03-26T03:30:44.441 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package librados2. 2024-03-26T03:30:44.467 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../007-librados2_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:30:44.507 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking librados2 (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:30:44.961 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package librbd1. 2024-03-26T03:30:44.987 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../008-librbd1_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:30:45.027 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking librbd1 (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:30:45.076 INFO:teuthology.orchestra.run.smithi175.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2475-ge42b8ea3-1jammy [14.2 kB] 2024-03-26T03:30:45.076 INFO:teuthology.orchestra.run.smithi175.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2475-ge42b8ea3-1jammy [50.3 MB] 2024-03-26T03:30:45.429 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libcephfs2. 2024-03-26T03:30:45.445 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:30:45.479 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libcephfs2 (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:30:45.748 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-rados. 2024-03-26T03:30:45.764 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../010-python3-rados_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:30:45.806 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-rados (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:30:46.126 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-03-26T03:30:46.152 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-2475-ge42b8ea3-1jammy_all.deb ... 2024-03-26T03:30:46.194 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-ceph-argparse (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:30:46.445 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-cephfs. 2024-03-26T03:30:46.460 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:30:46.503 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-cephfs (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:30:46.772 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-ceph-common. 2024-03-26T03:30:46.788 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-2475-ge42b8ea3-1jammy_all.deb ... 2024-03-26T03:30:46.822 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-ceph-common (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:30:47.167 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-wcwidth. 2024-03-26T03:30:47.194 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-03-26T03:30:47.234 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-26T03:30:47.494 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-prettytable. 2024-03-26T03:30:47.517 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-03-26T03:30:47.561 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-03-26T03:30:47.830 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-rbd. 2024-03-26T03:30:47.846 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:30:47.905 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-rbd (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:30:48.283 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-03-26T03:30:48.310 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-03-26T03:30:48.359 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-26T03:30:48.777 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-03-26T03:30:48.793 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-03-26T03:30:48.827 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-03-26T03:30:49.026 INFO:teuthology.orchestra.run.smithi175.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2475-ge42b8ea3-1jammy [121 kB] 2024-03-26T03:30:49.028 INFO:teuthology.orchestra.run.smithi175.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2475-ge42b8ea3-1jammy [38.9 kB] 2024-03-26T03:30:49.029 INFO:teuthology.orchestra.run.smithi175.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2475-ge42b8ea3-1jammy [13.2 MB] 2024-03-26T03:30:49.088 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-03-26T03:30:49.104 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-03-26T03:30:49.138 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-26T03:30:49.469 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package lua5.1. 2024-03-26T03:30:49.493 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-03-26T03:30:49.600 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-03-26T03:30:49.739 INFO:teuthology.orchestra.run.smithi175.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-adk-testing-2024-03-25-1628/e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2475-ge42b8ea3-1jammy [91.2 kB] 2024-03-26T03:30:49.919 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package lua-any. 2024-03-26T03:30:49.935 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-03-26T03:30:49.978 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking lua-any (27ubuntu1) ... 2024-03-26T03:30:50.205 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package zip. 2024-03-26T03:30:50.221 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-03-26T03:30:50.255 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking zip (3.0-12build2) ... 2024-03-26T03:30:50.329 INFO:teuthology.orchestra.run.smithi175.stdout:Fetched 178 MB in 16s (10.9 MB/s) 2024-03-26T03:30:50.538 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-03-26T03:30:50.550 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package unzip. 2024-03-26T03:30:50.576 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-03-26T03:30:50.616 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-03-26T03:30:50.639 INFO:teuthology.orchestra.run.smithi175.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165997 files and directories currently installed.) 2024-03-26T03:30:50.645 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-03-26T03:30:50.719 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-26T03:30:51.012 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package luarocks. 2024-03-26T03:30:51.023 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-03-26T03:30:51.039 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-03-26T03:30:51.049 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-03-26T03:30:51.078 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-03-26T03:30:51.089 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-26T03:30:51.451 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-03-26T03:30:51.477 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-03-26T03:30:51.477 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package librgw2. 2024-03-26T03:30:51.504 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../025-librgw2_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:30:51.517 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-26T03:30:51.544 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking librgw2 (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:30:51.970 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-03-26T03:30:51.997 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-26T03:30:52.046 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-26T03:30:52.054 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-rgw. 2024-03-26T03:30:52.070 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:30:52.105 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-rgw (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:30:52.424 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package liboath0:amd64. 2024-03-26T03:30:52.432 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-03-26T03:30:52.450 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-03-26T03:30:52.458 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-26T03:30:52.490 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-03-26T03:30:52.499 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-26T03:30:52.827 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libradosstriper1. 2024-03-26T03:30:52.854 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:30:52.860 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-03-26T03:30:52.886 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-26T03:30:52.893 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libradosstriper1 (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:30:52.926 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-26T03:30:53.305 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-03-26T03:30:53.330 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-common. 2024-03-26T03:30:53.331 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-03-26T03:30:53.357 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../029-ceph-common_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:30:53.371 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-26T03:30:53.396 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-common (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:30:53.649 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package librados2. 2024-03-26T03:30:53.700 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../007-librados2_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:30:53.707 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking librados2 (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:30:54.126 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package librbd1. 2024-03-26T03:30:54.153 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../008-librbd1_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:30:54.193 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking librbd1 (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:30:54.629 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libcephfs2. 2024-03-26T03:30:54.655 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:30:54.696 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libcephfs2 (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:30:54.735 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-base. 2024-03-26T03:30:54.762 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../030-ceph-base_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:30:54.811 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-base (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:30:54.998 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-rados. 2024-03-26T03:30:55.024 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../010-python3-rados_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:30:55.065 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-rados (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:30:55.317 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-03-26T03:30:55.332 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-2475-ge42b8ea3-1jammy_all.deb ... 2024-03-26T03:30:55.367 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-ceph-argparse (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:30:55.390 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-03-26T03:30:55.418 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-03-26T03:30:55.455 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-03-26T03:30:55.586 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-cephfs. 2024-03-26T03:30:55.612 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:30:55.653 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-cephfs (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:30:55.783 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-cheroot. 2024-03-26T03:30:55.810 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-03-26T03:30:55.850 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-26T03:30:55.972 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-ceph-common. 2024-03-26T03:30:55.999 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-2475-ge42b8ea3-1jammy_all.deb ... 2024-03-26T03:30:56.039 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-ceph-common (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:30:56.178 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-03-26T03:30:56.204 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-03-26T03:30:56.244 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-03-26T03:30:56.383 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-wcwidth. 2024-03-26T03:30:56.409 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-03-26T03:30:56.450 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-26T03:30:56.514 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-jaraco.text. 2024-03-26T03:30:56.540 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-03-26T03:30:56.588 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-03-26T03:30:56.777 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-prettytable. 2024-03-26T03:30:56.804 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-03-26T03:30:56.844 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-03-26T03:30:56.866 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-03-26T03:30:56.893 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-03-26T03:30:56.933 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-03-26T03:30:57.113 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-rbd. 2024-03-26T03:30:57.140 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:30:57.180 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-rbd (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:30:57.236 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-tempora. 2024-03-26T03:30:57.262 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-03-26T03:30:57.302 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-03-26T03:30:57.491 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-03-26T03:30:57.517 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-03-26T03:30:57.558 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-26T03:30:57.605 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-portend. 2024-03-26T03:30:57.632 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-03-26T03:30:57.672 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-03-26T03:30:57.802 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-03-26T03:30:57.817 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-03-26T03:30:57.851 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-03-26T03:30:57.918 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-03-26T03:30:57.942 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-03-26T03:30:57.974 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-03-26T03:30:58.146 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-03-26T03:30:58.172 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-03-26T03:30:58.213 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-26T03:30:58.277 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-cherrypy3. 2024-03-26T03:30:58.304 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-03-26T03:30:58.344 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-03-26T03:30:58.574 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package lua5.1. 2024-03-26T03:30:58.601 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-03-26T03:30:58.641 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-03-26T03:30:58.672 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-natsort. 2024-03-26T03:30:58.688 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-03-26T03:30:58.722 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-03-26T03:30:58.944 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package lua-any. 2024-03-26T03:30:58.966 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-logutils. 2024-03-26T03:30:58.970 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-03-26T03:30:58.982 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-03-26T03:30:59.010 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking lua-any (27ubuntu1) ... 2024-03-26T03:30:59.025 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-03-26T03:30:59.213 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package zip. 2024-03-26T03:30:59.228 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-03-26T03:30:59.271 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking zip (3.0-12build2) ... 2024-03-26T03:30:59.294 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-markupsafe. 2024-03-26T03:30:59.321 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-03-26T03:30:59.352 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-03-26T03:30:59.599 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package unzip. 2024-03-26T03:30:59.625 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-03-26T03:30:59.665 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-03-26T03:30:59.830 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-mako. 2024-03-26T03:30:59.857 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-03-26T03:30:59.897 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-26T03:31:00.061 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package luarocks. 2024-03-26T03:31:00.087 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-03-26T03:31:00.127 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-03-26T03:31:00.250 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-simplegeneric. 2024-03-26T03:31:00.276 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-03-26T03:31:00.316 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-03-26T03:31:00.477 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package librgw2. 2024-03-26T03:31:00.503 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../025-librgw2_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:31:00.543 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking librgw2 (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:00.627 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-singledispatch. 2024-03-26T03:31:00.654 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-03-26T03:31:00.694 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-03-26T03:31:00.988 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-webob. 2024-03-26T03:31:01.015 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-03-26T03:31:01.046 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-03-26T03:31:01.053 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-rgw. 2024-03-26T03:31:01.069 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:31:01.103 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-rgw (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:01.366 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-waitress. 2024-03-26T03:31:01.393 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-03-26T03:31:01.429 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-26T03:31:01.442 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package liboath0:amd64. 2024-03-26T03:31:01.468 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-03-26T03:31:01.508 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-03-26T03:31:01.735 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-tempita. 2024-03-26T03:31:01.763 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-03-26T03:31:01.802 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-03-26T03:31:01.827 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libradosstriper1. 2024-03-26T03:31:01.854 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:31:01.894 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libradosstriper1 (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:02.088 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-paste. 2024-03-26T03:31:02.115 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-03-26T03:31:02.154 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-03-26T03:31:02.272 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-common. 2024-03-26T03:31:02.299 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../029-ceph-common_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:31:02.339 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-common (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:02.491 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-03-26T03:31:02.518 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-03-26T03:31:02.557 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-03-26T03:31:02.844 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pastedeploy. 2024-03-26T03:31:02.870 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-03-26T03:31:02.910 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-03-26T03:31:03.196 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-webtest. 2024-03-26T03:31:03.223 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-03-26T03:31:03.263 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-03-26T03:31:03.574 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pecan. 2024-03-26T03:31:03.601 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-03-26T03:31:03.640 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-03-26T03:31:03.644 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-base. 2024-03-26T03:31:03.670 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../030-ceph-base_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:31:03.720 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-base (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:04.012 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-certifi. 2024-03-26T03:31:04.039 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-03-26T03:31:04.078 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-03-26T03:31:04.290 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-03-26T03:31:04.316 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-03-26T03:31:04.356 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-03-26T03:31:04.439 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-idna. 2024-03-26T03:31:04.466 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-03-26T03:31:04.506 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-idna (3.3-1) ... 2024-03-26T03:31:04.775 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-urllib3. 2024-03-26T03:31:04.791 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-03-26T03:31:04.825 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-26T03:31:05.101 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-cheroot. 2024-03-26T03:31:05.128 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-03-26T03:31:05.212 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-requests. 2024-03-26T03:31:05.239 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-03-26T03:31:05.270 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-26T03:31:05.376 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-26T03:31:05.556 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-werkzeug. 2024-03-26T03:31:05.582 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-03-26T03:31:05.622 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-26T03:31:05.729 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-03-26T03:31:05.755 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-03-26T03:31:05.795 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-03-26T03:31:05.892 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-03-26T03:31:05.908 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-2475-ge42b8ea3-1jammy_all.deb ... 2024-03-26T03:31:05.942 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:06.098 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-jaraco.text. 2024-03-26T03:31:06.125 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-03-26T03:31:06.165 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-03-26T03:31:06.280 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-03-26T03:31:06.307 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:31:06.338 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:06.442 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-03-26T03:31:06.458 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-03-26T03:31:06.492 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-03-26T03:31:06.616 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-bcrypt. 2024-03-26T03:31:06.643 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-03-26T03:31:06.753 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-tempora. 2024-03-26T03:31:06.780 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-03-26T03:31:06.820 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-03-26T03:31:07.091 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-03-26T03:31:07.123 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-portend. 2024-03-26T03:31:07.149 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-03-26T03:31:07.198 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-03-26T03:31:07.500 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-03-26T03:31:07.527 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-03-26T03:31:07.567 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-03-26T03:31:07.761 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-openssl. 2024-03-26T03:31:07.788 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-03-26T03:31:07.827 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-03-26T03:31:07.828 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-cherrypy3. 2024-03-26T03:31:07.843 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-03-26T03:31:07.878 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-03-26T03:31:08.172 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mgr. 2024-03-26T03:31:08.198 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:31:08.215 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-natsort. 2024-03-26T03:31:08.238 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mgr (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:08.241 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-03-26T03:31:08.281 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-03-26T03:31:08.559 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-logutils. 2024-03-26T03:31:08.586 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-03-26T03:31:08.626 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-03-26T03:31:08.642 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mon. 2024-03-26T03:31:08.668 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:31:08.724 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mon (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:08.928 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-markupsafe. 2024-03-26T03:31:08.955 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-03-26T03:31:08.995 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-03-26T03:31:09.297 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-mako. 2024-03-26T03:31:09.324 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-03-26T03:31:09.330 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-osd. 2024-03-26T03:31:09.357 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:31:09.373 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-26T03:31:09.396 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-osd (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:09.692 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-simplegeneric. 2024-03-26T03:31:09.719 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-03-26T03:31:09.759 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-03-26T03:31:10.028 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-singledispatch. 2024-03-26T03:31:10.055 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-03-26T03:31:10.086 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-03-26T03:31:10.372 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-webob. 2024-03-26T03:31:10.399 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-03-26T03:31:10.439 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-03-26T03:31:10.591 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph. 2024-03-26T03:31:10.618 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../066-ceph_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:31:10.658 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:10.792 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-waitress. 2024-03-26T03:31:10.818 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-03-26T03:31:10.862 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-26T03:31:10.935 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-fuse. 2024-03-26T03:31:10.962 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:31:11.002 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-fuse (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:11.153 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-tempita. 2024-03-26T03:31:11.179 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-03-26T03:31:11.219 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-03-26T03:31:11.364 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mds. 2024-03-26T03:31:11.391 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:31:11.430 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mds (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:11.522 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-paste. 2024-03-26T03:31:11.549 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-03-26T03:31:11.589 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-03-26T03:31:11.825 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package cephadm. 2024-03-26T03:31:11.851 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../069-cephadm_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:31:11.891 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking cephadm (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:11.917 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-03-26T03:31:11.932 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-03-26T03:31:11.975 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-03-26T03:31:12.210 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-asyncssh. 2024-03-26T03:31:12.237 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-03-26T03:31:12.276 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-03-26T03:31:12.278 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-pastedeploy. 2024-03-26T03:31:12.304 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-03-26T03:31:12.344 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-03-26T03:31:12.614 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-webtest. 2024-03-26T03:31:12.630 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-03-26T03:31:12.638 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python-babel-localedata. 2024-03-26T03:31:12.654 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-03-26T03:31:12.664 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-03-26T03:31:12.696 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-26T03:31:12.992 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-pecan. 2024-03-26T03:31:13.018 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-03-26T03:31:13.058 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-03-26T03:31:13.404 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-certifi. 2024-03-26T03:31:13.431 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-03-26T03:31:13.471 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-03-26T03:31:13.751 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-babel. 2024-03-26T03:31:13.778 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-03-26T03:31:13.799 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-idna. 2024-03-26T03:31:13.817 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-03-26T03:31:13.825 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-03-26T03:31:13.865 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-idna (3.3-1) ... 2024-03-26T03:31:14.145 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-jinja2. 2024-03-26T03:31:14.174 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-03-26T03:31:14.185 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-urllib3. 2024-03-26T03:31:14.211 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-03-26T03:31:14.220 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-26T03:31:14.251 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-26T03:31:14.556 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-03-26T03:31:14.563 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-requests. 2024-03-26T03:31:14.583 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-2475-ge42b8ea3-1jammy_all.deb ... 2024-03-26T03:31:14.589 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-03-26T03:31:14.622 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:14.629 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-26T03:31:14.907 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-werkzeug. 2024-03-26T03:31:14.922 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-03-26T03:31:14.942 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-repoze.lru. 2024-03-26T03:31:14.957 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-26T03:31:14.969 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-03-26T03:31:15.009 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-03-26T03:31:15.261 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-routes. 2024-03-26T03:31:15.276 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-03-26T03:31:15.277 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-03-26T03:31:15.292 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-2475-ge42b8ea3-1jammy_all.deb ... 2024-03-26T03:31:15.320 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-03-26T03:31:15.335 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:15.698 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-03-26T03:31:15.724 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:31:15.764 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:15.775 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-03-26T03:31:15.802 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-2475-ge42b8ea3-1jammy_all.deb ... 2024-03-26T03:31:15.839 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:16.067 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-bcrypt. 2024-03-26T03:31:16.094 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-03-26T03:31:16.133 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-03-26T03:31:16.453 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-openssl. 2024-03-26T03:31:16.480 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-03-26T03:31:16.522 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-03-26T03:31:16.814 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-mgr. 2024-03-26T03:31:16.840 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:31:16.872 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-mgr (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:17.103 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-03-26T03:31:17.130 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-03-26T03:31:17.169 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-26T03:31:17.267 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-mon. 2024-03-26T03:31:17.293 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:31:17.334 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-mon (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:17.673 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-joblib. 2024-03-26T03:31:17.700 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-03-26T03:31:17.731 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-03-26T03:31:17.937 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-osd. 2024-03-26T03:31:17.963 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:31:18.003 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-osd (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:18.017 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-03-26T03:31:18.033 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-03-26T03:31:18.067 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-03-26T03:31:18.362 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-sklearn. 2024-03-26T03:31:18.389 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-03-26T03:31:18.428 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-26T03:31:19.039 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph. 2024-03-26T03:31:19.066 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../066-ceph_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:31:19.095 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-03-26T03:31:19.106 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:19.123 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-2475-ge42b8ea3-1jammy_all.deb ... 2024-03-26T03:31:19.255 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:19.384 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-fuse. 2024-03-26T03:31:19.410 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:31:19.450 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-fuse (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:19.803 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-mds. 2024-03-26T03:31:19.830 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:31:19.870 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-mds (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:20.142 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-cachetools. 2024-03-26T03:31:20.170 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-03-26T03:31:20.209 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-03-26T03:31:20.298 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package cephadm. 2024-03-26T03:31:20.327 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../069-cephadm_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:31:20.364 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking cephadm (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:20.511 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pyasn1. 2024-03-26T03:31:20.541 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-03-26T03:31:20.577 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-03-26T03:31:20.642 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-asyncssh. 2024-03-26T03:31:20.668 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-03-26T03:31:20.700 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-03-26T03:31:20.915 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-03-26T03:31:20.942 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-03-26T03:31:20.981 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-03-26T03:31:21.044 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python-babel-localedata. 2024-03-26T03:31:21.060 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-03-26T03:31:21.094 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-26T03:31:21.301 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-rsa. 2024-03-26T03:31:21.328 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-03-26T03:31:21.367 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-rsa (4.8-1) ... 2024-03-26T03:31:21.659 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-google-auth. 2024-03-26T03:31:21.682 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-03-26T03:31:21.721 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-03-26T03:31:22.057 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-03-26T03:31:22.084 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-03-26T03:31:22.124 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-26T03:31:22.435 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-websocket. 2024-03-26T03:31:22.441 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-babel. 2024-03-26T03:31:22.463 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-03-26T03:31:22.468 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-03-26T03:31:22.501 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-03-26T03:31:22.507 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-03-26T03:31:22.827 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-jinja2. 2024-03-26T03:31:22.838 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-kubernetes. 2024-03-26T03:31:22.854 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-03-26T03:31:22.866 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-03-26T03:31:22.893 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-26T03:31:22.904 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-26T03:31:23.238 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-03-26T03:31:23.265 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-2475-ge42b8ea3-1jammy_all.deb ... 2024-03-26T03:31:23.304 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:23.610 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-03-26T03:31:23.638 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-2475-ge42b8ea3-1jammy_all.deb ... 2024-03-26T03:31:23.649 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-repoze.lru. 2024-03-26T03:31:23.676 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:23.677 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-03-26T03:31:23.716 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-03-26T03:31:23.979 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libonig5:amd64. 2024-03-26T03:31:24.006 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-03-26T03:31:24.018 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-routes. 2024-03-26T03:31:24.037 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-26T03:31:24.045 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-03-26T03:31:24.086 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-03-26T03:31:24.340 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libjq1:amd64. 2024-03-26T03:31:24.368 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-03-26T03:31:24.406 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-26T03:31:24.413 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-03-26T03:31:24.440 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-2475-ge42b8ea3-1jammy_all.deb ... 2024-03-26T03:31:24.479 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:24.668 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package jq. 2024-03-26T03:31:24.695 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-03-26T03:31:24.734 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-03-26T03:31:24.986 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package socat. 2024-03-26T03:31:25.014 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-03-26T03:31:25.053 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-03-26T03:31:25.348 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package xmlstarlet. 2024-03-26T03:31:25.364 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-03-26T03:31:25.398 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-03-26T03:31:25.684 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-03-26T03:31:25.712 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-03-26T03:31:25.735 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-test. 2024-03-26T03:31:25.751 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-26T03:31:25.751 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../097-ceph-test_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:31:25.785 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-test (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:26.254 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-joblib. 2024-03-26T03:31:26.281 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-03-26T03:31:26.320 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-03-26T03:31:26.691 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-03-26T03:31:26.718 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-03-26T03:31:26.759 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-03-26T03:31:27.051 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-sklearn. 2024-03-26T03:31:27.078 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-03-26T03:31:27.109 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-26T03:31:27.843 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-03-26T03:31:27.871 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-2475-ge42b8ea3-1jammy_all.deb ... 2024-03-26T03:31:27.994 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:28.164 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package ceph-volume. 2024-03-26T03:31:28.193 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-2475-ge42b8ea3-1jammy_all.deb ... 2024-03-26T03:31:28.231 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking ceph-volume (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:28.518 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package libcephfs-dev. 2024-03-26T03:31:28.546 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:31:28.584 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking libcephfs-dev (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:28.878 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package lua-socket:amd64. 2024-03-26T03:31:28.907 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-03-26T03:31:28.937 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-26T03:31:28.965 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-cachetools. 2024-03-26T03:31:28.992 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-03-26T03:31:29.031 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-03-26T03:31:29.250 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package lua-sec:amd64. 2024-03-26T03:31:29.277 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-03-26T03:31:29.316 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-03-26T03:31:29.342 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-pyasn1. 2024-03-26T03:31:29.370 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-03-26T03:31:29.408 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-03-26T03:31:29.695 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package nvme-cli. 2024-03-26T03:31:29.723 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-03-26T03:31:29.737 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-03-26T03:31:29.761 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-03-26T03:31:29.764 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-03-26T03:31:29.803 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-03-26T03:31:30.089 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-rsa. 2024-03-26T03:31:30.105 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-03-26T03:31:30.139 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-rsa (4.8-1) ... 2024-03-26T03:31:30.183 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package pkg-config. 2024-03-26T03:31:30.211 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-03-26T03:31:30.249 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-03-26T03:31:30.468 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-google-auth. 2024-03-26T03:31:30.496 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-03-26T03:31:30.519 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-03-26T03:31:30.534 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-03-26T03:31:30.546 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-03-26T03:31:30.585 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-03-26T03:31:30.879 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-03-26T03:31:30.906 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-03-26T03:31:30.946 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-26T03:31:30.997 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-iniconfig. 2024-03-26T03:31:31.024 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-03-26T03:31:31.066 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-03-26T03:31:31.274 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-websocket. 2024-03-26T03:31:31.301 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-03-26T03:31:31.341 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-03-26T03:31:31.366 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pastescript. 2024-03-26T03:31:31.393 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-03-26T03:31:31.433 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-03-26T03:31:31.627 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-kubernetes. 2024-03-26T03:31:31.654 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-03-26T03:31:31.685 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-26T03:31:31.794 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pluggy. 2024-03-26T03:31:31.822 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-03-26T03:31:31.860 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-03-26T03:31:32.130 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-psutil. 2024-03-26T03:31:32.148 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-03-26T03:31:32.188 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-03-26T03:31:32.299 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-03-26T03:31:32.326 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-2475-ge42b8ea3-1jammy_all.deb ... 2024-03-26T03:31:32.365 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:32.541 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-py. 2024-03-26T03:31:32.569 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-03-26T03:31:32.616 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-py (1.10.0-1) ... 2024-03-26T03:31:32.676 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libonig5:amd64. 2024-03-26T03:31:32.703 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-03-26T03:31:32.742 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-26T03:31:32.919 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pygments. 2024-03-26T03:31:32.947 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-03-26T03:31:32.985 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-03-26T03:31:33.020 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libjq1:amd64. 2024-03-26T03:31:33.048 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-03-26T03:31:33.087 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-26T03:31:33.381 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package jq. 2024-03-26T03:31:33.409 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-03-26T03:31:33.423 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pyinotify. 2024-03-26T03:31:33.439 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-03-26T03:31:33.456 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-03-26T03:31:33.481 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-03-26T03:31:33.734 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package socat. 2024-03-26T03:31:33.749 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-03-26T03:31:33.793 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-03-26T03:31:33.800 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-toml. 2024-03-26T03:31:33.828 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-03-26T03:31:33.866 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-03-26T03:31:34.087 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package xmlstarlet. 2024-03-26T03:31:34.103 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-03-26T03:31:34.145 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-03-26T03:31:34.169 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-pytest. 2024-03-26T03:31:34.198 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-03-26T03:31:34.235 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-03-26T03:31:34.482 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-test. 2024-03-26T03:31:34.498 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../097-ceph-test_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:31:34.532 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-test (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:34.573 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package python3-simplejson. 2024-03-26T03:31:34.601 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-03-26T03:31:34.639 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-03-26T03:31:34.908 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-03-26T03:31:34.924 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-03-26T03:31:34.966 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-03-26T03:31:35.662 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package radosgw. 2024-03-26T03:31:35.690 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../116-radosgw_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:31:35.728 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking radosgw (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:36.498 INFO:teuthology.orchestra.run.smithi190.stdout:Selecting previously unselected package rbd-fuse. 2024-03-26T03:31:36.514 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:31:36.556 INFO:teuthology.orchestra.run.smithi190.stdout:Unpacking rbd-fuse (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:36.819 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package ceph-volume. 2024-03-26T03:31:36.847 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-2475-ge42b8ea3-1jammy_all.deb ... 2024-03-26T03:31:36.886 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking ceph-volume (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:37.102 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-03-26T03:31:37.198 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package libcephfs-dev. 2024-03-26T03:31:37.214 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:31:37.256 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking libcephfs-dev (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:37.352 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-26T03:31:37.479 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-03-26T03:31:37.534 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package lua-socket:amd64. 2024-03-26T03:31:37.550 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-03-26T03:31:37.583 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-26T03:31:37.734 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-03-26T03:31:37.904 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package lua-sec:amd64. 2024-03-26T03:31:37.931 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-03-26T03:31:37.971 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-03-26T03:31:38.093 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-03-26T03:31:38.350 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package nvme-cli. 2024-03-26T03:31:38.377 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-03-26T03:31:38.416 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-03-26T03:31:38.503 INFO:teuthology.orchestra.run.smithi190.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-03-26T03:31:38.564 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up cephadm (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:38.855 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package pkg-config. 2024-03-26T03:31:38.883 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-03-26T03:31:38.929 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-03-26T03:31:39.063 INFO:teuthology.orchestra.run.smithi190.stdout:Adding system user cephadm....done 2024-03-26T03:31:39.233 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-03-26T03:31:39.247 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-26T03:31:39.260 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-03-26T03:31:39.307 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-03-26T03:31:39.538 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-03-26T03:31:39.785 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-iniconfig. 2024-03-26T03:31:39.788 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-03-26T03:31:39.813 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-03-26T03:31:39.860 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-03-26T03:31:39.915 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-03-26T03:31:40.163 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-pastescript. 2024-03-26T03:31:40.165 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-03-26T03:31:40.191 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-03-26T03:31:40.246 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-03-26T03:31:40.417 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-03-26T03:31:40.543 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-py (1.10.0-1) ... 2024-03-26T03:31:40.568 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-pluggy. 2024-03-26T03:31:40.584 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-03-26T03:31:40.626 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-03-26T03:31:40.844 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-03-26T03:31:40.921 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-psutil. 2024-03-26T03:31:40.949 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-03-26T03:31:40.987 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-03-26T03:31:41.197 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-03-26T03:31:41.340 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-py. 2024-03-26T03:31:41.368 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-03-26T03:31:41.406 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-py (1.10.0-1) ... 2024-03-26T03:31:41.456 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-03-26T03:31:41.724 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-03-26T03:31:41.777 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-pygments. 2024-03-26T03:31:41.805 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-03-26T03:31:41.835 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-03-26T03:31:42.017 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-03-26T03:31:42.251 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-03-26T03:31:42.272 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-pyinotify. 2024-03-26T03:31:42.301 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-03-26T03:31:42.339 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-03-26T03:31:42.511 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-03-26T03:31:42.633 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-toml. 2024-03-26T03:31:42.661 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-03-26T03:31:42.699 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-03-26T03:31:42.761 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-ceph-argparse (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:42.969 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-pytest. 2024-03-26T03:31:42.997 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-03-26T03:31:43.036 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-03-26T03:31:43.113 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-26T03:31:43.239 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-26T03:31:43.339 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package python3-simplejson. 2024-03-26T03:31:43.355 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-03-26T03:31:43.365 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-03-26T03:31:43.389 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-03-26T03:31:43.508 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-03-26T03:31:43.641 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-03-26T03:31:43.658 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-03-26T03:31:43.691 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-03-26T03:31:43.758 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-03-26T03:31:44.132 INFO:teuthology.orchestra.run.smithi190.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-03-26T03:31:44.175 INFO:teuthology.orchestra.run.smithi190.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-03-26T03:31:44.228 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-26T03:31:44.354 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-03-26T03:31:44.454 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package radosgw. 2024-03-26T03:31:44.482 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../116-radosgw_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:31:44.520 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking radosgw (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:44.714 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-26T03:31:44.840 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-03-26T03:31:45.107 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-03-26T03:31:45.340 INFO:teuthology.orchestra.run.smithi175.stdout:Selecting previously unselected package rbd-fuse. 2024-03-26T03:31:45.356 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-2475-ge42b8ea3-1jammy_amd64.deb ... 2024-03-26T03:31:45.375 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-03-26T03:31:45.398 INFO:teuthology.orchestra.run.smithi175.stdout:Unpacking rbd-fuse (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:45.668 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up zip (3.0-12build2) ... 2024-03-26T03:31:45.711 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-03-26T03:31:45.768 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-03-26T03:31:45.962 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-26T03:31:46.087 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-03-26T03:31:46.349 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-03-26T03:31:46.445 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-03-26T03:31:46.674 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-03-26T03:31:46.697 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-03-26T03:31:46.822 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-03-26T03:31:46.948 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-26T03:31:47.080 INFO:teuthology.orchestra.run.smithi175.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-03-26T03:31:47.164 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up cephadm (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:47.241 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-03-26T03:31:47.609 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-03-26T03:31:47.656 INFO:teuthology.orchestra.run.smithi175.stdout:Adding system user cephadm....done 2024-03-26T03:31:47.804 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-26T03:31:47.877 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-03-26T03:31:48.072 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-03-26T03:31:48.236 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-26T03:31:48.332 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-03-26T03:31:48.457 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-03-26T03:31:48.555 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-26T03:31:48.725 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-03-26T03:31:48.906 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-03-26T03:31:48.968 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-03-26T03:31:49.069 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-py (1.10.0-1) ... 2024-03-26T03:31:49.157 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-03-26T03:31:49.283 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-ceph-common (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:49.354 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-03-26T03:31:49.567 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-idna (3.3-1) ... 2024-03-26T03:31:49.714 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-03-26T03:31:49.868 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-26T03:31:49.966 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-03-26T03:31:50.258 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-03-26T03:31:50.502 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-03-26T03:31:50.761 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-03-26T03:31:50.982 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-03-26T03:31:51.020 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-03-26T03:31:51.233 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-26T03:31:51.289 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-ceph-argparse (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:51.368 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-toml (0.10.2-1) ... 2024-03-26T03:31:51.606 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-26T03:31:51.627 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-26T03:31:51.770 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-03-26T03:31:51.782 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-26T03:31:51.904 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-26T03:31:51.917 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-03-26T03:31:52.034 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-03-26T03:31:52.248 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-03-26T03:31:52.269 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-03-26T03:31:52.498 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-03-26T03:31:52.679 INFO:teuthology.orchestra.run.smithi175.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-03-26T03:31:52.708 INFO:teuthology.orchestra.run.smithi175.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-03-26T03:31:52.741 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-26T03:31:52.772 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-26T03:31:52.842 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-03-26T03:31:52.898 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-03-26T03:31:53.110 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-03-26T03:31:53.266 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-26T03:31:53.361 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-03-26T03:31:53.392 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-03-26T03:31:53.621 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-03-26T03:31:53.651 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-03-26T03:31:53.872 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-03-26T03:31:53.894 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-03-26T03:31:54.123 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-03-26T03:31:54.212 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up zip (3.0-12build2) ... 2024-03-26T03:31:54.338 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-03-26T03:31:54.382 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-26T03:31:54.483 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-03-26T03:31:54.734 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-26T03:31:54.852 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-26T03:31:55.006 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-03-26T03:31:55.145 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-03-26T03:31:55.266 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-03-26T03:31:55.392 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-03-26T03:31:55.448 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-03-26T03:31:55.493 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-26T03:31:55.699 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-26T03:31:55.769 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-03-26T03:31:55.800 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-03-26T03:31:55.901 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-03-26T03:31:55.982 INFO:teuthology.orchestra.run.smithi190.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-03-26T03:31:56.179 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-03-26T03:31:56.211 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-26T03:31:56.337 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-03-26T03:31:56.422 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-03-26T03:31:56.730 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-03-26T03:31:56.798 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-26T03:31:56.998 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up lua-any (27ubuntu1) ... 2024-03-26T03:31:57.099 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-portend (3.0.0-1) ... 2024-03-26T03:31:57.117 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-26T03:31:57.358 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-26T03:31:57.459 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-26T03:31:57.752 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-03-26T03:31:57.817 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-03-26T03:31:58.051 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-03-26T03:31:58.053 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-26T03:31:58.178 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-ceph-common (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:31:58.337 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-03-26T03:31:58.462 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-idna (3.3-1) ... 2024-03-26T03:31:58.463 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-rsa (4.8-1) ... 2024-03-26T03:31:58.748 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-03-26T03:31:58.771 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-26T03:31:59.007 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-26T03:31:59.327 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-03-26T03:31:59.720 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-03-26T03:31:59.869 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-03-26T03:32:00.005 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-03-26T03:32:00.129 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-26T03:32:00.263 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-toml (0.10.2-1) ... 2024-03-26T03:32:00.349 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-26T03:32:00.450 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up librados2 (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:00.522 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-26T03:32:00.554 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:00.649 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-03-26T03:32:00.677 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-03-26T03:32:00.766 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-26T03:32:01.038 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libcephfs2 (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:01.043 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-03-26T03:32:01.139 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libradosstriper1 (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:01.240 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-03-26T03:32:01.277 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-03-26T03:32:01.534 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up librbd1 (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:01.537 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-26T03:32:01.635 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mgr-modules-core (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:01.663 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-03-26T03:32:01.760 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-fuse (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:01.958 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-03-26T03:32:01.974 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-26T03:32:01.974 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-26T03:32:02.207 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-03-26T03:32:02.466 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-03-26T03:32:02.701 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-03-26T03:32:02.737 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up libcephfs-dev (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:02.921 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-rados (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:02.935 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-03-26T03:32:03.039 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up librgw2 (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:03.140 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-26T03:32:03.228 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-26T03:32:03.354 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-03-26T03:32:03.638 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-26T03:32:03.789 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-26T03:32:04.090 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-03-26T03:32:04.294 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-rbd (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:04.400 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-03-26T03:32:04.419 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up rbd-fuse (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:04.545 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-rgw (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:04.660 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-26T03:32:04.671 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up python3-cephfs (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:04.781 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-common (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:04.786 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-03-26T03:32:04.912 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-03-26T03:32:05.009 INFO:teuthology.orchestra.run.smithi175.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-03-26T03:32:05.238 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-26T03:32:05.307 INFO:teuthology.orchestra.run.smithi190.stdout:Adding group ceph....done 2024-03-26T03:32:05.379 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-03-26T03:32:05.692 INFO:teuthology.orchestra.run.smithi190.stdout:Adding system user ceph....done 2024-03-26T03:32:05.750 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-03-26T03:32:05.834 INFO:teuthology.orchestra.run.smithi190.stdout:Setting system user ceph properties....done 2024-03-26T03:32:05.849 INFO:teuthology.orchestra.run.smithi190.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-03-26T03:32:05.986 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-03-26T03:32:06.034 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up lua-any (27ubuntu1) ... 2024-03-26T03:32:06.135 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-portend (3.0.0-1) ... 2024-03-26T03:32:06.331 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-03-26T03:32:06.386 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-26T03:32:06.512 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-26T03:32:06.872 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-03-26T03:32:06.991 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-test (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:07.173 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-26T03:32:07.258 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up radosgw (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:07.457 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-03-26T03:32:07.566 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-rsa (4.8-1) ... 2024-03-26T03:32:07.752 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-26T03:32:07.752 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-26T03:32:07.826 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-03-26T03:32:08.102 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-26T03:32:08.235 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-base (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:08.480 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-03-26T03:32:08.579 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-03-26T03:32:08.866 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-03-26T03:32:09.045 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mds (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:09.142 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-03-26T03:32:09.268 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-26T03:32:09.268 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-26T03:32:09.470 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-26T03:32:09.596 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up librados2 (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:09.722 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:09.747 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mgr (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:09.848 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-03-26T03:32:09.967 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-26T03:32:09.967 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-26T03:32:10.184 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libcephfs2 (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:10.285 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libradosstriper1 (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:10.411 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-03-26T03:32:10.482 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-osd (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:10.713 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up librbd1 (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:10.743 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-26T03:32:10.743 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-26T03:32:10.839 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-mgr-modules-core (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:10.965 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-fuse (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:11.177 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-26T03:32:11.177 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-26T03:32:11.249 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:11.376 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:11.526 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mon (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:11.650 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up libcephfs-dev (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:11.741 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-26T03:32:11.741 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-26T03:32:11.751 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-rados (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:11.852 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up librgw2 (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:11.969 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-26T03:32:12.236 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mgr-cephadm (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:12.403 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:12.704 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-mgr-dashboard (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:12.864 INFO:teuthology.orchestra.run.smithi190.stdout:Setting up ceph-volume (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:13.190 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-rbd (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:13.324 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up rbd-fuse (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:13.354 INFO:teuthology.orchestra.run.smithi190.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-26T03:32:13.433 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-rgw (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:13.480 INFO:teuthology.orchestra.run.smithi190.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-26T03:32:13.543 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up python3-cephfs (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:13.624 INFO:teuthology.orchestra.run.smithi190.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-26T03:32:13.644 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-common (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:14.203 INFO:teuthology.orchestra.run.smithi175.stdout:Adding group ceph....done 2024-03-26T03:32:14.522 INFO:teuthology.orchestra.run.smithi175.stdout:Adding system user ceph....done 2024-03-26T03:32:14.707 INFO:teuthology.orchestra.run.smithi175.stdout:Setting system user ceph properties....done 2024-03-26T03:32:14.723 INFO:teuthology.orchestra.run.smithi175.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-03-26T03:32:14.766 INFO:teuthology.orchestra.run.smithi190.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-26T03:32:14.859 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-03-26T03:32:15.181 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-03-26T03:32:15.391 INFO:teuthology.orchestra.run.smithi190.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-26T03:32:15.398 DEBUG:teuthology.parallel:result is None 2024-03-26T03:32:15.629 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-test (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:15.780 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up radosgw (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:16.278 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-26T03:32:16.279 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-26T03:32:16.731 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-base (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:17.073 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-03-26T03:32:17.525 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-mds (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:17.747 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-26T03:32:17.747 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-26T03:32:18.194 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-mgr (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:18.404 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-26T03:32:18.404 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-26T03:32:18.828 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-osd (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:19.100 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-26T03:32:19.100 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-26T03:32:19.546 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:19.672 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:19.832 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-mon (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:20.039 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-26T03:32:20.039 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-26T03:32:20.475 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-mgr-cephadm (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:20.626 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:20.761 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-mgr-dashboard (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:20.970 INFO:teuthology.orchestra.run.smithi175.stdout:Setting up ceph-volume (19.0.0-2475-ge42b8ea3-1jammy) ... 2024-03-26T03:32:21.332 INFO:teuthology.orchestra.run.smithi175.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-26T03:32:21.440 INFO:teuthology.orchestra.run.smithi175.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-26T03:32:21.593 INFO:teuthology.orchestra.run.smithi175.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-26T03:32:22.719 INFO:teuthology.orchestra.run.smithi175.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-26T03:32:23.383 INFO:teuthology.orchestra.run.smithi175.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-26T03:32:23.388 DEBUG:teuthology.parallel:result is None 2024-03-26T03:32:23.388 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 2024-03-26T03:32:23.671 DEBUG:teuthology.orchestra.run.smithi175:> dpkg-query -W -f '${Version}' ceph 2024-03-26T03:32:23.690 INFO:teuthology.orchestra.run.smithi175.stdout:19.0.0-2475-ge42b8ea3-1jammy 2024-03-26T03:32:23.691 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2475-ge42b8ea3-1jammy 2024-03-26T03:32:23.691 INFO:teuthology.task.install:The correct ceph version 19.0.0-2475-ge42b8ea3-1jammy is installed. 2024-03-26T03:32:23.693 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 2024-03-26T03:32:23.834 DEBUG:teuthology.orchestra.run.smithi190:> dpkg-query -W -f '${Version}' ceph 2024-03-26T03:32:23.854 INFO:teuthology.orchestra.run.smithi190.stdout:19.0.0-2475-ge42b8ea3-1jammy 2024-03-26T03:32:23.854 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2475-ge42b8ea3-1jammy 2024-03-26T03:32:23.854 INFO:teuthology.task.install:The correct ceph version 19.0.0-2475-ge42b8ea3-1jammy is installed. 2024-03-26T03:32:23.856 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-26T03:32:23.856 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-26T03:32:23.857 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-26T03:32:23.871 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-03-26T03:32:23.871 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-26T03:32:23.908 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-26T03:32:23.909 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-26T03:32:23.909 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/daemon-helper 2024-03-26T03:32:23.925 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-26T03:32:23.977 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-03-26T03:32:23.978 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/daemon-helper 2024-03-26T03:32:23.991 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-26T03:32:24.047 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-26T03:32:24.047 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-26T03:32:24.047 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-26T03:32:24.061 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-26T03:32:24.114 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-03-26T03:32:24.114 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-26T03:32:24.126 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-26T03:32:24.180 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-26T03:32:24.180 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-26T03:32:24.180 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/stdin-killer 2024-03-26T03:32:24.193 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-26T03:32:24.247 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-03-26T03:32:24.247 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/stdin-killer 2024-03-26T03:32:24.260 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-26T03:32:24.316 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-26T03:32:24.427 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client': {'debug ms': 1}, 'global': {'mon election default strategy': 1, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'MON_DOWN', 'mons down, quorum', 'is down \\(out of quorum\\)', 'POOL_APP_NOT_ENABLED', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': 'e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003', 'cephadm_mode': 'cephadm-package'} 2024-03-26T03:32:24.427 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 2024-03-26T03:32:24.427 INFO:tasks.cephadm:Cluster fsid is 762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T03:32:24.427 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-26T03:32:24.428 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.175', 'mon.c': '[v2:172.21.15.175:3301,v1:172.21.15.175:6790]', 'mon.b': '172.21.15.190'} 2024-03-26T03:32:24.428 INFO:tasks.cephadm:First mon is mon.a on smithi175 2024-03-26T03:32:24.428 INFO:tasks.cephadm:First mgr is y 2024-03-26T03:32:24.428 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-26T03:32:24.428 DEBUG:teuthology.orchestra.run.smithi175:> sudo hostname $(hostname -s) 2024-03-26T03:32:24.443 DEBUG:teuthology.orchestra.run.smithi190:> sudo hostname $(hostname -s) 2024-03-26T03:32:24.468 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 on all hosts... 2024-03-26T03:32:24.468 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 pull 2024-03-26T03:32:24.488 DEBUG:teuthology.orchestra.run.smithi190:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 pull 2024-03-26T03:32:24.652 INFO:teuthology.orchestra.run.smithi175.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003... 2024-03-26T03:32:24.671 INFO:teuthology.orchestra.run.smithi190.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003... 2024-03-26T03:33:31.501 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-03-26T03:33:31.502 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_version": "ceph version 19.0.0-2475-ge42b8ea3 (e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003) squid (dev)", 2024-03-26T03:33:31.502 INFO:teuthology.orchestra.run.smithi175.stdout: "image_id": "f18c6672b5a35a9268569822301af4f68c7c1f5d599252bb322be37fdc7da354", 2024-03-26T03:33:31.502 INFO:teuthology.orchestra.run.smithi175.stdout: "repo_digests": [ 2024-03-26T03:33:31.502 INFO:teuthology.orchestra.run.smithi175.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:036ff263f269d03295f88ce388558d8d1cbe14b2ee02cd78a973d9f233151717" 2024-03-26T03:33:31.502 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-03-26T03:33:31.502 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-03-26T03:33:33.475 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2024-03-26T03:33:33.475 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_version": "ceph version 19.0.0-2475-ge42b8ea3 (e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003) squid (dev)", 2024-03-26T03:33:33.475 INFO:teuthology.orchestra.run.smithi190.stdout: "image_id": "f18c6672b5a35a9268569822301af4f68c7c1f5d599252bb322be37fdc7da354", 2024-03-26T03:33:33.475 INFO:teuthology.orchestra.run.smithi190.stdout: "repo_digests": [ 2024-03-26T03:33:33.475 INFO:teuthology.orchestra.run.smithi190.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:036ff263f269d03295f88ce388558d8d1cbe14b2ee02cd78a973d9f233151717" 2024-03-26T03:33:33.475 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2024-03-26T03:33:33.475 INFO:teuthology.orchestra.run.smithi190.stdout:} 2024-03-26T03:33:33.523 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /etc/ceph 2024-03-26T03:33:33.538 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph 2024-03-26T03:33:33.552 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 777 /etc/ceph 2024-03-26T03:33:33.593 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 777 /etc/ceph 2024-03-26T03:33:33.606 INFO:tasks.cephadm:Writing seed config... 2024-03-26T03:33:33.607 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-26T03:33:33.607 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-26T03:33:33.607 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-26T03:33:33.608 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-03-26T03:33:33.608 INFO:tasks.cephadm: override: [global] ms type = async 2024-03-26T03:33:33.608 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-26T03:33:33.608 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-26T03:33:33.608 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-26T03:33:33.608 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-03-26T03:33:33.608 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-26T03:33:33.608 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-26T03:33:33.608 INFO:tasks.cephadm: override: [osd] osd class default list = * 2024-03-26T03:33:33.608 INFO:tasks.cephadm: override: [osd] osd class load list = * 2024-03-26T03:33:33.608 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-26T03:33:33.609 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-26T03:33:33.609 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-26T03:33:33.642 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 762e930c-eb21-11ee-95d0-87774f69a715 mon election default strategy = 1 ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd class default list = * osd class load list = * [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] debug ms = 1 2024-03-26T03:33:33.642 DEBUG:teuthology.orchestra.run.smithi175:mon.a> sudo journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@mon.a.service 2024-03-26T03:33:33.687 DEBUG:teuthology.orchestra.run.smithi175:mgr.y> sudo journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@mgr.y.service 2024-03-26T03:33:33.731 INFO:tasks.cephadm:Bootstrapping... 2024-03-26T03:33:33.731 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 -v bootstrap --fsid 762e930c-eb21-11ee-95d0-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.175 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-26T03:33:33.944 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-03-26T03:33:33.944 INFO:teuthology.orchestra.run.smithi175.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003', '-v', 'bootstrap', '--fsid', '762e930c-eb21-11ee-95d0-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.175', '--skip-admin-label'] 2024-03-26T03:33:33.944 INFO:teuthology.orchestra.run.smithi175.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-26T03:33:33.945 INFO:teuthology.orchestra.run.smithi175.stdout:Verifying podman|docker is present... 2024-03-26T03:33:33.945 INFO:teuthology.orchestra.run.smithi175.stdout:Verifying lvm2 is present... 2024-03-26T03:33:33.945 INFO:teuthology.orchestra.run.smithi175.stdout:Verifying time synchronization is in place... 2024-03-26T03:33:33.949 INFO:teuthology.orchestra.run.smithi175.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-26T03:33:33.949 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-26T03:33:33.954 INFO:teuthology.orchestra.run.smithi175.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-26T03:33:33.954 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stdout inactive 2024-03-26T03:33:33.958 INFO:teuthology.orchestra.run.smithi175.stdout:Non-zero exit code 1 from systemctl is-enabled chronyd.service 2024-03-26T03:33:33.958 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stderr Failed to get unit file state for chronyd.service: No such file or directory 2024-03-26T03:33:33.962 INFO:teuthology.orchestra.run.smithi175.stdout:Non-zero exit code 3 from systemctl is-active chronyd.service 2024-03-26T03:33:33.962 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stdout inactive 2024-03-26T03:33:33.966 INFO:teuthology.orchestra.run.smithi175.stdout:Non-zero exit code 1 from systemctl is-enabled systemd-timesyncd.service 2024-03-26T03:33:33.966 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stdout masked 2024-03-26T03:33:33.969 INFO:teuthology.orchestra.run.smithi175.stdout:Non-zero exit code 3 from systemctl is-active systemd-timesyncd.service 2024-03-26T03:33:33.969 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stdout inactive 2024-03-26T03:33:33.973 INFO:teuthology.orchestra.run.smithi175.stdout:Non-zero exit code 1 from systemctl is-enabled ntpd.service 2024-03-26T03:33:33.973 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stderr Failed to get unit file state for ntpd.service: No such file or directory 2024-03-26T03:33:33.976 INFO:teuthology.orchestra.run.smithi175.stdout:Non-zero exit code 3 from systemctl is-active ntpd.service 2024-03-26T03:33:33.976 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stdout inactive 2024-03-26T03:33:33.980 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stdout enabled 2024-03-26T03:33:33.984 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stdout active 2024-03-26T03:33:33.984 INFO:teuthology.orchestra.run.smithi175.stdout:Unit ntp.service is enabled and running 2024-03-26T03:33:33.984 INFO:teuthology.orchestra.run.smithi175.stdout:Repeating the final host check... 2024-03-26T03:33:33.984 INFO:teuthology.orchestra.run.smithi175.stdout:docker (/usr/bin/docker) is present 2024-03-26T03:33:33.984 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl is present 2024-03-26T03:33:33.984 INFO:teuthology.orchestra.run.smithi175.stdout:lvcreate is present 2024-03-26T03:33:33.987 INFO:teuthology.orchestra.run.smithi175.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-26T03:33:33.987 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-26T03:33:33.991 INFO:teuthology.orchestra.run.smithi175.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-26T03:33:33.991 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stdout inactive 2024-03-26T03:33:33.994 INFO:teuthology.orchestra.run.smithi175.stdout:Non-zero exit code 1 from systemctl is-enabled chronyd.service 2024-03-26T03:33:33.994 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stderr Failed to get unit file state for chronyd.service: No such file or directory 2024-03-26T03:33:33.997 INFO:teuthology.orchestra.run.smithi175.stdout:Non-zero exit code 3 from systemctl is-active chronyd.service 2024-03-26T03:33:33.997 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stdout inactive 2024-03-26T03:33:34.001 INFO:teuthology.orchestra.run.smithi175.stdout:Non-zero exit code 1 from systemctl is-enabled systemd-timesyncd.service 2024-03-26T03:33:34.001 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stdout masked 2024-03-26T03:33:34.004 INFO:teuthology.orchestra.run.smithi175.stdout:Non-zero exit code 3 from systemctl is-active systemd-timesyncd.service 2024-03-26T03:33:34.004 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stdout inactive 2024-03-26T03:33:34.007 INFO:teuthology.orchestra.run.smithi175.stdout:Non-zero exit code 1 from systemctl is-enabled ntpd.service 2024-03-26T03:33:34.007 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stderr Failed to get unit file state for ntpd.service: No such file or directory 2024-03-26T03:33:34.010 INFO:teuthology.orchestra.run.smithi175.stdout:Non-zero exit code 3 from systemctl is-active ntpd.service 2024-03-26T03:33:34.010 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stdout inactive 2024-03-26T03:33:34.014 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stdout enabled 2024-03-26T03:33:34.018 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stdout active 2024-03-26T03:33:34.019 INFO:teuthology.orchestra.run.smithi175.stdout:Unit ntp.service is enabled and running 2024-03-26T03:33:34.019 INFO:teuthology.orchestra.run.smithi175.stdout:Host looks OK 2024-03-26T03:33:34.019 INFO:teuthology.orchestra.run.smithi175.stdout:Cluster fsid: 762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T03:33:34.019 INFO:teuthology.orchestra.run.smithi175.stdout:Acquiring lock 140186448357232 on /run/cephadm/762e930c-eb21-11ee-95d0-87774f69a715.lock 2024-03-26T03:33:34.019 INFO:teuthology.orchestra.run.smithi175.stdout:Lock 140186448357232 acquired on /run/cephadm/762e930c-eb21-11ee-95d0-87774f69a715.lock 2024-03-26T03:33:34.019 INFO:teuthology.orchestra.run.smithi175.stdout:Verifying IP 172.21.15.175 port 3300 ... 2024-03-26T03:33:34.019 INFO:teuthology.orchestra.run.smithi175.stdout:Verifying IP 172.21.15.175 port 6789 ... 2024-03-26T03:33:34.019 INFO:teuthology.orchestra.run.smithi175.stdout:Base mon IP(s) is [172.21.15.175:3300, 172.21.15.175:6789], mon addrv is [v2:172.21.15.175:3300,v1:172.21.15.175:6789] 2024-03-26T03:33:34.021 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 2024-03-26T03:33:34.022 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.175 metric 100 2024-03-26T03:33:34.022 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/sbin/ip: stdout 158.69.67.47 via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.175 metric 100 2024-03-26T03:33:34.022 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/sbin/ip: stdout 172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown 2024-03-26T03:33:34.022 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.175 metric 100 2024-03-26T03:33:34.022 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/sbin/ip: stdout 172.21.0.1 dev enp3s0f1 proto dhcp scope link src 172.21.15.175 metric 100 2024-03-26T03:33:34.022 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/sbin/ip: stdout 172.21.0.2 dev enp3s0f1 proto dhcp scope link src 172.21.15.175 metric 100 2024-03-26T03:33:34.022 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/sbin/ip: stdout 172.21.15.254 dev enp3s0f1 proto dhcp scope link src 172.21.15.175 metric 100 2024-03-26T03:33:34.023 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-26T03:33:34.023 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-26T03:33:34.023 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 expires 1790sec pref medium 2024-03-26T03:33:34.025 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-26T03:33:34.025 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-26T03:33:34.026 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-26T03:33:34.026 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-26T03:33:34.026 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:71f1/64 scope link 2024-03-26T03:33:34.026 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-26T03:33:34.028 INFO:teuthology.orchestra.run.smithi175.stdout:Mon IP `172.21.15.175` is in CIDR network `172.21.0.0/20` 2024-03-26T03:33:34.028 INFO:teuthology.orchestra.run.smithi175.stdout:Mon IP `172.21.15.175` is in CIDR network `172.21.0.0/20` 2024-03-26T03:33:34.028 INFO:teuthology.orchestra.run.smithi175.stdout:Mon IP `172.21.15.175` is in CIDR network `172.21.0.1/32` 2024-03-26T03:33:34.028 INFO:teuthology.orchestra.run.smithi175.stdout:Mon IP `172.21.15.175` is in CIDR network `172.21.0.1/32` 2024-03-26T03:33:34.028 INFO:teuthology.orchestra.run.smithi175.stdout:Mon IP `172.21.15.175` is in CIDR network `172.21.0.2/32` 2024-03-26T03:33:34.028 INFO:teuthology.orchestra.run.smithi175.stdout:Mon IP `172.21.15.175` is in CIDR network `172.21.0.2/32` 2024-03-26T03:33:34.028 INFO:teuthology.orchestra.run.smithi175.stdout:Mon IP `172.21.15.175` is in CIDR network `172.21.15.254/32` 2024-03-26T03:33:34.028 INFO:teuthology.orchestra.run.smithi175.stdout:Mon IP `172.21.15.175` is in CIDR network `172.21.15.254/32` 2024-03-26T03:33:34.028 INFO:teuthology.orchestra.run.smithi175.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20', '172.21.0.1/32', '172.21.0.1/32', '172.21.0.2/32', '172.21.0.2/32', '172.21.15.254/32', '172.21.15.254/32'] 2024-03-26T03:33:34.028 INFO:teuthology.orchestra.run.smithi175.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-26T03:33:34.029 INFO:teuthology.orchestra.run.smithi175.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003... 2024-03-26T03:33:34.352 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/docker: stdout e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003: Pulling from ceph-ci/ceph 2024-03-26T03:33:34.352 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/docker: stdout Digest: sha256:036ff263f269d03295f88ce388558d8d1cbe14b2ee02cd78a973d9f233151717 2024-03-26T03:33:34.353 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/docker: stdout Status: Image is up to date for quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 2024-03-26T03:33:34.353 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/docker: stdout quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 2024-03-26T03:33:35.754 INFO:teuthology.orchestra.run.smithi175.stdout:ceph: stdout ceph version 19.0.0-2475-ge42b8ea3 (e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003) squid (dev) 2024-03-26T03:33:35.755 INFO:teuthology.orchestra.run.smithi175.stdout:Ceph version: ceph version 19.0.0-2475-ge42b8ea3 (e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003) squid (dev) 2024-03-26T03:33:35.755 INFO:teuthology.orchestra.run.smithi175.stdout:Extracting ceph user uid/gid from container image... 2024-03-26T03:33:37.008 INFO:teuthology.orchestra.run.smithi175.stdout:stat: stdout 167 167 2024-03-26T03:33:37.008 INFO:teuthology.orchestra.run.smithi175.stdout:Creating initial keys... 2024-03-26T03:33:38.080 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-authtool: stdout AQARQgJmcaJ8IhAAm1mEuO8f0XFn+7DkO2e/vg== 2024-03-26T03:33:39.300 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-authtool: stdout AQASQgJmLNRjLRAAgw3laNiNMbI+zyweFkHsjw== 2024-03-26T03:33:40.498 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-authtool: stdout AQATQgJmpcc8NxAATnhVSmnj9fvRARASLGwo5Q== 2024-03-26T03:33:40.498 INFO:teuthology.orchestra.run.smithi175.stdout:Creating initial monmap... 2024-03-26T03:33:41.743 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-26T03:33:41.743 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-26T03:33:41.743 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T03:33:41.743 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-26T03:33:41.743 INFO:teuthology.orchestra.run.smithi175.stdout:monmaptool for a [v2:172.21.15.175:3300,v1:172.21.15.175:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-26T03:33:41.743 INFO:teuthology.orchestra.run.smithi175.stdout:setting min_mon_release = quincy 2024-03-26T03:33:41.743 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/monmaptool: set fsid to 762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T03:33:41.743 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-26T03:33:41.744 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-26T03:33:41.744 INFO:teuthology.orchestra.run.smithi175.stdout:Creating mon... 2024-03-26T03:33:43.037 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.439+0000 7f596d901c80 0 set uid:gid to 167:167 (ceph:ceph) 2024-03-26T03:33:43.037 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 1 imported monmap: 2024-03-26T03:33:43.037 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr epoch 0 2024-03-26T03:33:43.037 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr fsid 762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T03:33:43.037 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr last_changed 2024-03-26T03:33:41.181831+0000 2024-03-26T03:33:43.037 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr created 2024-03-26T03:33:41.181831+0000 2024-03-26T03:33:43.038 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr min_mon_release 17 (quincy) 2024-03-26T03:33:43.038 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr election_strategy: 1 2024-03-26T03:33:43.038 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr 0: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.a 2024-03-26T03:33:43.038 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr 2024-03-26T03:33:43.038 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 0 /usr/bin/ceph-mon: set fsid to 762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T03:33:43.038 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: RocksDB version: 7.9.2 2024-03-26T03:33:43.038 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr 2024-03-26T03:33:43.038 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Git sha 0 2024-03-26T03:33:43.038 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Compile date 2024-03-25 23:25:11 2024-03-26T03:33:43.038 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: DB SUMMARY 2024-03-26T03:33:43.038 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr 2024-03-26T03:33:43.038 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: DB Session ID: 8LN1QQKTCZIIZDF3VAQP 2024-03-26T03:33:43.038 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr 2024-03-26T03:33:43.038 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 0, files: 2024-03-26T03:33:43.039 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr 2024-03-26T03:33:43.039 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 2024-03-26T03:33:43.039 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr 2024-03-26T03:33:43.039 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.error_if_exists: 0 2024-03-26T03:33:43.039 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.create_if_missing: 1 2024-03-26T03:33:43.039 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.paranoid_checks: 1 2024-03-26T03:33:43.039 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.flush_verify_memtable_count: 1 2024-03-26T03:33:43.039 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-26T03:33:43.039 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-26T03:33:43.039 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.env: 0x55ef9e3a5c60 2024-03-26T03:33:43.039 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.fs: PosixFileSystem 2024-03-26T03:33:43.039 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.info_log: 0x55ef9fc8ad60 2024-03-26T03:33:43.039 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.max_file_opening_threads: 16 2024-03-26T03:33:43.039 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.statistics: (nil) 2024-03-26T03:33:43.039 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.use_fsync: 0 2024-03-26T03:33:43.040 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.max_log_file_size: 0 2024-03-26T03:33:43.040 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-26T03:33:43.040 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.log_file_time_to_roll: 0 2024-03-26T03:33:43.040 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.keep_log_file_num: 1000 2024-03-26T03:33:43.040 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.recycle_log_file_num: 0 2024-03-26T03:33:43.040 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.allow_fallocate: 1 2024-03-26T03:33:43.040 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.allow_mmap_reads: 0 2024-03-26T03:33:43.040 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.allow_mmap_writes: 0 2024-03-26T03:33:43.040 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.use_direct_reads: 0 2024-03-26T03:33:43.040 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-26T03:33:43.040 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.create_missing_column_families: 0 2024-03-26T03:33:43.040 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.db_log_dir: 2024-03-26T03:33:43.040 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.wal_dir: 2024-03-26T03:33:43.040 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.table_cache_numshardbits: 6 2024-03-26T03:33:43.040 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-03-26T03:33:43.041 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-03-26T03:33:43.041 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-26T03:33:43.041 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-26T03:33:43.041 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-03-26T03:33:43.041 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.advise_random_on_open: 1 2024-03-26T03:33:43.041 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.db_write_buffer_size: 0 2024-03-26T03:33:43.041 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.write_buffer_manager: 0x55ef9fc5d360 2024-03-26T03:33:43.041 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-26T03:33:43.041 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-26T03:33:43.041 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.use_adaptive_mutex: 0 2024-03-26T03:33:43.041 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.rate_limiter: (nil) 2024-03-26T03:33:43.041 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-26T03:33:43.041 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.wal_recovery_mode: 2 2024-03-26T03:33:43.041 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.enable_thread_tracking: 0 2024-03-26T03:33:43.041 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.enable_pipelined_write: 0 2024-03-26T03:33:43.041 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.unordered_write: 0 2024-03-26T03:33:43.041 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-26T03:33:43.042 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-26T03:33:43.042 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-26T03:33:43.042 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-26T03:33:43.042 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.row_cache: None 2024-03-26T03:33:43.042 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.wal_filter: None 2024-03-26T03:33:43.042 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-26T03:33:43.042 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.allow_ingest_behind: 0 2024-03-26T03:33:43.042 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.two_write_queues: 0 2024-03-26T03:33:43.042 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.manual_wal_flush: 0 2024-03-26T03:33:43.042 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.wal_compression: 0 2024-03-26T03:33:43.042 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.atomic_flush: 0 2024-03-26T03:33:43.042 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-26T03:33:43.042 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.persist_stats_to_disk: 0 2024-03-26T03:33:43.042 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-03-26T03:33:43.043 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.log_readahead_size: 0 2024-03-26T03:33:43.043 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-26T03:33:43.043 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.best_efforts_recovery: 0 2024-03-26T03:33:43.043 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-26T03:33:43.043 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-26T03:33:43.043 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.allow_data_in_errors: 0 2024-03-26T03:33:43.043 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.db_host_id: __hostname__ 2024-03-26T03:33:43.043 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.enforce_single_del_contracts: true 2024-03-26T03:33:43.043 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.max_background_jobs: 2 2024-03-26T03:33:43.043 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.max_background_compactions: -1 2024-03-26T03:33:43.043 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.max_subcompactions: 1 2024-03-26T03:33:43.043 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-26T03:33:43.043 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-26T03:33:43.043 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.delayed_write_rate : 16777216 2024-03-26T03:33:43.044 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.max_total_wal_size: 0 2024-03-26T03:33:43.044 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-26T03:33:43.044 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.stats_dump_period_sec: 600 2024-03-26T03:33:43.044 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.stats_persist_period_sec: 600 2024-03-26T03:33:43.044 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-26T03:33:43.044 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.max_open_files: -1 2024-03-26T03:33:43.044 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.bytes_per_sync: 0 2024-03-26T03:33:43.044 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-03-26T03:33:43.044 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-03-26T03:33:43.044 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.compaction_readahead_size: 0 2024-03-26T03:33:43.044 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.max_background_flushes: -1 2024-03-26T03:33:43.044 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Compression algorithms supported: 2024-03-26T03:33:43.044 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-26T03:33:43.044 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: kZSTD supported: 0 2024-03-26T03:33:43.045 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: kXpressCompression supported: 0 2024-03-26T03:33:43.045 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: kLZ4HCCompression supported: 1 2024-03-26T03:33:43.045 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: kZlibCompression supported: 1 2024-03-26T03:33:43.045 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: kSnappyCompression supported: 1 2024-03-26T03:33:43.045 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: kLZ4Compression supported: 1 2024-03-26T03:33:43.045 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: kBZip2Compression supported: 0 2024-03-26T03:33:43.045 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-03-26T03:33:43.045 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: DMutex implementation: pthread_mutex_t 2024-03-26T03:33:43.045 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: [db/db_impl/db_impl_open.cc:317] Creating manifest 1 2024-03-26T03:33:43.045 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr 2024-03-26T03:33:43.045 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000001 2024-03-26T03:33:43.045 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr 2024-03-26T03:33:43.045 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-26T03:33:43.045 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr 2024-03-26T03:33:43.045 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-26T03:33:43.045 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.merge_operator: 2024-03-26T03:33:43.045 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.compaction_filter: None 2024-03-26T03:33:43.046 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.compaction_filter_factory: None 2024-03-26T03:33:43.046 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.sst_partitioner_factory: None 2024-03-26T03:33:43.046 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-03-26T03:33:43.046 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.table_factory: BlockBasedTable 2024-03-26T03:33:43.046 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ef9fc8af20) 2024-03-26T03:33:43.046 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr cache_index_and_filter_blocks: 1 2024-03-26T03:33:43.046 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr cache_index_and_filter_blocks_with_high_priority: 0 2024-03-26T03:33:43.046 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-26T03:33:43.046 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr pin_top_level_index_and_filter: 1 2024-03-26T03:33:43.046 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr index_type: 0 2024-03-26T03:33:43.046 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr data_block_index_type: 0 2024-03-26T03:33:43.046 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr index_shortening: 1 2024-03-26T03:33:43.046 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr data_block_hash_table_util_ratio: 0.750000 2024-03-26T03:33:43.046 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr checksum: 4 2024-03-26T03:33:43.047 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr no_block_cache: 0 2024-03-26T03:33:43.047 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr block_cache: 0x55ef9fc8f090 2024-03-26T03:33:43.047 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr block_cache_name: BinnedLRUCache 2024-03-26T03:33:43.047 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr block_cache_options: 2024-03-26T03:33:43.047 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr capacity : 536870912 2024-03-26T03:33:43.047 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr num_shard_bits : 4 2024-03-26T03:33:43.061 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr strict_capacity_limit : 0 2024-03-26T03:33:43.061 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr high_pri_pool_ratio: 0.000 2024-03-26T03:33:43.061 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr block_cache_compressed: (nil) 2024-03-26T03:33:43.061 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr persistent_cache: (nil) 2024-03-26T03:33:43.061 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr block_size: 4096 2024-03-26T03:33:43.061 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr block_size_deviation: 10 2024-03-26T03:33:43.062 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr block_restart_interval: 16 2024-03-26T03:33:43.062 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr index_block_restart_interval: 1 2024-03-26T03:33:43.062 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr metadata_block_size: 4096 2024-03-26T03:33:43.062 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr partition_filters: 0 2024-03-26T03:33:43.062 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr use_delta_encoding: 1 2024-03-26T03:33:43.062 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr filter_policy: bloomfilter 2024-03-26T03:33:43.062 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr whole_key_filtering: 1 2024-03-26T03:33:43.062 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr verify_compression: 0 2024-03-26T03:33:43.062 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr read_amp_bytes_per_bit: 0 2024-03-26T03:33:43.062 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr format_version: 5 2024-03-26T03:33:43.062 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr enable_index_compression: 1 2024-03-26T03:33:43.062 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr block_align: 0 2024-03-26T03:33:43.062 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr max_auto_readahead_size: 262144 2024-03-26T03:33:43.062 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr prepopulate_block_cache: 0 2024-03-26T03:33:43.063 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr initial_auto_readahead_size: 8192 2024-03-26T03:33:43.063 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr num_file_reads_for_auto_readahead: 2 2024-03-26T03:33:43.063 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr 2024-03-26T03:33:43.063 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.write_buffer_size: 33554432 2024-03-26T03:33:43.063 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.max_write_buffer_number: 2 2024-03-26T03:33:43.063 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.compression: NoCompression 2024-03-26T03:33:43.063 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.bottommost_compression: Disabled 2024-03-26T03:33:43.063 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.prefix_extractor: nullptr 2024-03-26T03:33:43.063 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-26T03:33:43.063 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.num_levels: 7 2024-03-26T03:33:43.063 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-26T03:33:43.063 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-26T03:33:43.063 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-26T03:33:43.063 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-26T03:33:43.063 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-26T03:33:43.063 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-26T03:33:43.064 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-26T03:33:43.064 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-26T03:33:43.064 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-26T03:33:43.064 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-26T03:33:43.064 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-26T03:33:43.064 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-26T03:33:43.064 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.compression_opts.window_bits: -14 2024-03-26T03:33:43.064 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.compression_opts.level: 32767 2024-03-26T03:33:43.064 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.compression_opts.strategy: 0 2024-03-26T03:33:43.064 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-26T03:33:43.064 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-26T03:33:43.064 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-26T03:33:43.064 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-26T03:33:43.064 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.compression_opts.enabled: false 2024-03-26T03:33:43.064 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-26T03:33:43.064 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-26T03:33:43.065 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-26T03:33:43.065 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-26T03:33:43.065 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.target_file_size_base: 67108864 2024-03-26T03:33:43.065 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.target_file_size_multiplier: 1 2024-03-26T03:33:43.065 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-26T03:33:43.065 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-26T03:33:43.065 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-26T03:33:43.065 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-26T03:33:43.065 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-26T03:33:43.065 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-26T03:33:43.065 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-26T03:33:43.065 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-26T03:33:43.065 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-26T03:33:43.065 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-26T03:33:43.065 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-26T03:33:43.065 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-26T03:33:43.065 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-26T03:33:43.066 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.arena_block_size: 1048576 2024-03-26T03:33:43.066 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-26T03:33:43.066 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-26T03:33:43.066 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.disable_auto_compactions: 0 2024-03-26T03:33:43.066 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-26T03:33:43.066 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-26T03:33:43.066 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-26T03:33:43.066 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-26T03:33:43.066 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-26T03:33:43.066 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-26T03:33:43.066 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-26T03:33:43.066 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-26T03:33:43.066 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-26T03:33:43.066 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-26T03:33:43.067 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-03-26T03:33:43.067 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.inplace_update_support: 0 2024-03-26T03:33:43.067 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-03-26T03:33:43.067 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-26T03:33:43.067 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-26T03:33:43.067 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.memtable_huge_page_size: 0 2024-03-26T03:33:43.067 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.bloom_locality: 0 2024-03-26T03:33:43.067 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.max_successive_merges: 0 2024-03-26T03:33:43.067 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-03-26T03:33:43.067 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.paranoid_file_checks: 0 2024-03-26T03:33:43.067 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.force_consistency_checks: 1 2024-03-26T03:33:43.067 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.report_bg_io_stats: 0 2024-03-26T03:33:43.067 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.ttl: 2592000 2024-03-26T03:33:43.067 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-03-26T03:33:43.067 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-26T03:33:43.067 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-26T03:33:43.068 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.enable_blob_files: false 2024-03-26T03:33:43.068 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.min_blob_size: 0 2024-03-26T03:33:43.068 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.blob_file_size: 268435456 2024-03-26T03:33:43.068 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.blob_compression_type: NoCompression 2024-03-26T03:33:43.068 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.enable_blob_garbage_collection: false 2024-03-26T03:33:43.068 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-26T03:33:43.068 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-26T03:33:43.068 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-26T03:33:43.068 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.blob_file_starting_level: 0 2024-03-26T03:33:43.068 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-26T03:33:43.068 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-26T03:33:43.068 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr 2024-03-26T03:33:43.068 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-26T03:33:43.068 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr 2024-03-26T03:33:43.068 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 20d84c71-8864-42cf-9e0a-77a82f9fcf70 2024-03-26T03:33:43.068 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr 2024-03-26T03:33:43.069 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.443+0000 7f596d901c80 4 rocksdb: [db/version_set.cc:5047] Creating manifest 5 2024-03-26T03:33:43.069 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr 2024-03-26T03:33:43.069 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.447+0000 7f596d901c80 4 rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ef9fd58000 2024-03-26T03:33:43.069 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.447+0000 7f596d901c80 4 rocksdb: DB pointer 0x55ef9fd48000 2024-03-26T03:33:43.069 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.447+0000 7f595da64700 4 rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-26T03:33:43.069 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.447+0000 7f595da64700 4 rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-26T03:33:43.069 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr ** DB Stats ** 2024-03-26T03:33:43.069 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T03:33:43.069 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-26T03:33:43.069 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T03:33:43.069 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T03:33:43.069 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-26T03:33:43.069 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T03:33:43.069 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T03:33:43.069 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr 2024-03-26T03:33:43.069 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr ** Compaction Stats [default] ** 2024-03-26T03:33:43.070 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T03:33:43.070 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-26T03:33:43.070 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-26T03:33:43.070 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-26T03:33:43.070 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr 2024-03-26T03:33:43.070 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr ** Compaction Stats [default] ** 2024-03-26T03:33:43.070 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T03:33:43.070 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-26T03:33:43.070 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr 2024-03-26T03:33:43.070 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-26T03:33:43.070 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr 2024-03-26T03:33:43.070 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T03:33:43.070 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr Flush(GB): cumulative 0.000, interval 0.000 2024-03-26T03:33:43.071 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr AddFile(GB): cumulative 0.000, interval 0.000 2024-03-26T03:33:43.071 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr AddFile(Total Files): cumulative 0, interval 0 2024-03-26T03:33:43.071 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr AddFile(L0 Files): cumulative 0, interval 0 2024-03-26T03:33:43.071 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr AddFile(Keys): cumulative 0, interval 0 2024-03-26T03:33:43.071 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T03:33:43.071 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T03:33:43.071 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-26T03:33:43.071 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr Block cache BinnedLRUCache@0x55ef9fc8f090#7 capacity: 512.00 MB usage: 0.00 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-03-26T03:33:43.071 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr Block cache entry stats(count,size,portion): Misc(1,0.00 KB,0%) 2024-03-26T03:33:43.071 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr 2024-03-26T03:33:43.071 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr ** File Read Latency Histogram By Level [default] ** 2024-03-26T03:33:43.071 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr 2024-03-26T03:33:43.071 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.447+0000 7f596d901c80 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2024-03-26T03:33:43.071 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.447+0000 7f596d901c80 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2024-03-26T03:33:43.071 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-26T03:33:42.447+0000 7f596d901c80 0 /usr/bin/ceph-mon: created monfs at /var/lib/ceph/mon/ceph-a for mon.a 2024-03-26T03:33:43.072 INFO:teuthology.orchestra.run.smithi175.stdout:create mon.a on 2024-03-26T03:33:43.291 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-03-26T03:33:43.483 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /etc/systemd/system/ceph.target. 2024-03-26T03:33:43.712 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-762e930c-eb21-11ee-95d0-87774f69a715.target -> /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715.target. 2024-03-26T03:33:43.712 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-762e930c-eb21-11ee-95d0-87774f69a715.target -> /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715.target. 2024-03-26T03:33:43.994 INFO:teuthology.orchestra.run.smithi175.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-762e930c-eb21-11ee-95d0-87774f69a715@mon.a 2024-03-26T03:33:43.994 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stderr Failed to reset failed state of unit ceph-762e930c-eb21-11ee-95d0-87774f69a715@mon.a.service: Unit ceph-762e930c-eb21-11ee-95d0-87774f69a715@mon.a.service not loaded. 2024-03-26T03:33:44.201 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715.target.wants/ceph-762e930c-eb21-11ee-95d0-87774f69a715@mon.a.service -> /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service. 2024-03-26T03:33:44.208 INFO:teuthology.orchestra.run.smithi175.stdout:firewalld does not appear to be present 2024-03-26T03:33:44.208 INFO:teuthology.orchestra.run.smithi175.stdout:Not possible to enable service . firewalld.service is not available 2024-03-26T03:33:44.208 INFO:teuthology.orchestra.run.smithi175.stdout:Waiting for mon to start... 2024-03-26T03:33:44.208 INFO:teuthology.orchestra.run.smithi175.stdout:Waiting for mon... 2024-03-26T03:33:45.670 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:45 smithi175 bash[21045]: cluster 2024-03-26T03:33:45.383897+0000 mon.a (mon.0) 1 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2024-03-26T03:33:46.199 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout cluster: 2024-03-26T03:33:46.200 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout id: 762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T03:33:46.200 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-26T03:33:46.200 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 2024-03-26T03:33:46.200 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout services: 2024-03-26T03:33:46.200 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.270656s) 2024-03-26T03:33:46.200 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-26T03:33:46.200 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-26T03:33:46.200 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 2024-03-26T03:33:46.200 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout data: 2024-03-26T03:33:46.200 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-26T03:33:46.200 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-26T03:33:46.205 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-26T03:33:46.205 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout pgs: 2024-03-26T03:33:46.205 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 2024-03-26T03:33:46.205 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.515+0000 7ff535651700 1 Processor -- start 2024-03-26T03:33:46.206 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.515+0000 7ff535651700 1 -- start start 2024-03-26T03:33:46.206 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.515+0000 7ff535651700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff530108640 0x7ff530108a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:33:46.206 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.515+0000 7ff535651700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff530109030 con 0x7ff530108640 2024-03-26T03:33:46.206 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.515+0000 7ff52effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff530108640 0x7ff530108a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:33:46.206 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.515+0000 7ff52effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff530108640 0x7ff530108a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55316/0 (socket says 172.21.15.175:55316) 2024-03-26T03:33:46.206 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.515+0000 7ff52effd700 1 -- 172.21.15.175:0/2731529402 learned_addr learned my addr 172.21.15.175:0/2731529402 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:33:46.206 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.515+0000 7ff52effd700 1 -- 172.21.15.175:0/2731529402 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5301097f0 con 0x7ff530108640 2024-03-26T03:33:46.206 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.515+0000 7ff52effd700 1 --2- 172.21.15.175:0/2731529402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff530108640 0x7ff530108a60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff5200098a0 tx=0x7ff520009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66453871fbe6581a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:33:46.206 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.515+0000 7ff52dffb700 1 -- 172.21.15.175:0/2731529402 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff520004160 con 0x7ff530108640 2024-03-26T03:33:46.206 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.515+0000 7ff52dffb700 1 -- 172.21.15.175:0/2731529402 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff520004300 con 0x7ff530108640 2024-03-26T03:33:46.206 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.515+0000 7ff52dffb700 1 -- 172.21.15.175:0/2731529402 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5200036a0 con 0x7ff530108640 2024-03-26T03:33:46.206 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.515+0000 7ff535651700 1 -- 172.21.15.175:0/2731529402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff530108640 msgr2=0x7ff530108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:33:46.206 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.515+0000 7ff535651700 1 --2- 172.21.15.175:0/2731529402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff530108640 0x7ff530108a60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff5200098a0 tx=0x7ff520009bb0 comp rx=0 tx=0).stop 2024-03-26T03:33:46.206 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.515+0000 7ff535651700 1 -- 172.21.15.175:0/2731529402 shutdown_connections 2024-03-26T03:33:46.207 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.515+0000 7ff535651700 1 --2- 172.21.15.175:0/2731529402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff530108640 0x7ff530108a60 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:33:46.207 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.515+0000 7ff535651700 1 -- 172.21.15.175:0/2731529402 >> 172.21.15.175:0/2731529402 conn(0x7ff53007bc10 msgr2=0x7ff53007c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:33:46.207 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.515+0000 7ff535651700 1 -- 172.21.15.175:0/2731529402 shutdown_connections 2024-03-26T03:33:46.207 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.515+0000 7ff535651700 1 -- 172.21.15.175:0/2731529402 wait complete. 2024-03-26T03:33:46.207 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.519+0000 7ff535651700 1 Processor -- start 2024-03-26T03:33:46.207 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.519+0000 7ff535651700 1 -- start start 2024-03-26T03:33:46.207 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.519+0000 7ff535651700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff530108640 0x7ff53019bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:33:46.207 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.519+0000 7ff535651700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff530109d20 con 0x7ff530108640 2024-03-26T03:33:46.207 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.519+0000 7ff52effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff530108640 0x7ff53019bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:33:46.207 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.519+0000 7ff52effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff530108640 0x7ff53019bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55328/0 (socket says 172.21.15.175:55328) 2024-03-26T03:33:46.207 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.519+0000 7ff52effd700 1 -- 172.21.15.175:0/1582803177 learned_addr learned my addr 172.21.15.175:0/1582803177 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:33:46.207 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.519+0000 7ff52effd700 1 -- 172.21.15.175:0/1582803177 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff53019c4c0 con 0x7ff530108640 2024-03-26T03:33:46.207 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.519+0000 7ff52effd700 1 --2- 172.21.15.175:0/1582803177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff530108640 0x7ff53019bf80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff520000c00 tx=0x7ff520004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:33:46.207 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.519+0000 7ff517fff700 1 -- 172.21.15.175:0/1582803177 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff520003db0 con 0x7ff530108640 2024-03-26T03:33:46.207 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.519+0000 7ff517fff700 1 -- 172.21.15.175:0/1582803177 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff520011630 con 0x7ff530108640 2024-03-26T03:33:46.207 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.519+0000 7ff517fff700 1 -- 172.21.15.175:0/1582803177 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff520011950 con 0x7ff530108640 2024-03-26T03:33:46.208 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.519+0000 7ff535651700 1 -- 172.21.15.175:0/1582803177 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff53019c750 con 0x7ff530108640 2024-03-26T03:33:46.208 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.519+0000 7ff535651700 1 -- 172.21.15.175:0/1582803177 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff53019f440 con 0x7ff530108640 2024-03-26T03:33:46.208 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.519+0000 7ff517fff700 1 -- 172.21.15.175:0/1582803177 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff52001a460 con 0x7ff530108640 2024-03-26T03:33:46.208 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.519+0000 7ff517fff700 1 -- 172.21.15.175:0/1582803177 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff52001b780 con 0x7ff530108640 2024-03-26T03:33:46.208 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.519+0000 7ff535651700 1 -- 172.21.15.175:0/1582803177 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff530063c70 con 0x7ff530108640 2024-03-26T03:33:46.208 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.523+0000 7ff517fff700 1 -- 172.21.15.175:0/1582803177 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7ff52001b980 con 0x7ff530108640 2024-03-26T03:33:46.208 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.655+0000 7ff535651700 1 -- 172.21.15.175:0/1582803177 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7ff53006bce0 con 0x7ff530108640 2024-03-26T03:33:46.208 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.655+0000 7ff517fff700 1 -- 172.21.15.175:0/1582803177 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7ff52001bb60 con 0x7ff530108640 2024-03-26T03:33:46.208 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.659+0000 7ff535651700 1 -- 172.21.15.175:0/1582803177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff530108640 msgr2=0x7ff53019bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:33:46.208 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.659+0000 7ff535651700 1 --2- 172.21.15.175:0/1582803177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff530108640 0x7ff53019bf80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff520000c00 tx=0x7ff520004750 comp rx=0 tx=0).stop 2024-03-26T03:33:46.208 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.659+0000 7ff535651700 1 -- 172.21.15.175:0/1582803177 shutdown_connections 2024-03-26T03:33:46.208 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.659+0000 7ff535651700 1 --2- 172.21.15.175:0/1582803177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff530108640 0x7ff53019bf80 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:33:46.208 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.659+0000 7ff535651700 1 -- 172.21.15.175:0/1582803177 >> 172.21.15.175:0/1582803177 conn(0x7ff53007bc10 msgr2=0x7ff530192900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:33:46.208 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.659+0000 7ff535651700 1 -- 172.21.15.175:0/1582803177 shutdown_connections 2024-03-26T03:33:46.208 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:45.659+0000 7ff535651700 1 -- 172.21.15.175:0/1582803177 wait complete. 2024-03-26T03:33:46.209 INFO:teuthology.orchestra.run.smithi175.stdout:mon is available 2024-03-26T03:33:46.209 INFO:teuthology.orchestra.run.smithi175.stdout:Assimilating anything we can from ceph.conf... 2024-03-26T03:33:46.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:46 smithi175 bash[21045]: cluster 2024-03-26T03:33:45.387626+0000 mon.a (mon.0) 2 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2024-03-26T03:33:46.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:46 smithi175 bash[21045]: cluster 2024-03-26T03:33:45.387810+0000 mon.a (mon.0) 3 : cluster [DBG] monmap epoch 1 2024-03-26T03:33:46.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:46 smithi175 bash[21045]: cluster 2024-03-26T03:33:45.387823+0000 mon.a (mon.0) 4 : cluster [DBG] fsid 762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T03:33:46.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:46 smithi175 bash[21045]: cluster 2024-03-26T03:33:45.387839+0000 mon.a (mon.0) 5 : cluster [DBG] last_changed 2024-03-26T03:33:41.181831+0000 2024-03-26T03:33:46.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:46 smithi175 bash[21045]: cluster 2024-03-26T03:33:45.387852+0000 mon.a (mon.0) 6 : cluster [DBG] created 2024-03-26T03:33:41.181831+0000 2024-03-26T03:33:46.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:46 smithi175 bash[21045]: cluster 2024-03-26T03:33:45.387863+0000 mon.a (mon.0) 7 : cluster [DBG] min_mon_release 19 (squid) 2024-03-26T03:33:46.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:46 smithi175 bash[21045]: cluster 2024-03-26T03:33:45.387874+0000 mon.a (mon.0) 8 : cluster [DBG] election_strategy: 1 2024-03-26T03:33:46.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:46 smithi175 bash[21045]: cluster 2024-03-26T03:33:45.387887+0000 mon.a (mon.0) 9 : cluster [DBG] 0: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.a 2024-03-26T03:33:46.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:46 smithi175 bash[21045]: cluster 2024-03-26T03:33:45.389869+0000 mon.a (mon.0) 10 : cluster [DBG] fsmap 2024-03-26T03:33:46.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:46 smithi175 bash[21045]: cluster 2024-03-26T03:33:45.392592+0000 mon.a (mon.0) 11 : cluster [DBG] osdmap e1: 0 total, 0 up, 0 in 2024-03-26T03:33:46.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:46 smithi175 bash[21045]: cluster 2024-03-26T03:33:45.394720+0000 mon.a (mon.0) 12 : cluster [DBG] mgrmap e1: no daemons active 2024-03-26T03:33:46.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:46 smithi175 bash[21045]: audit 2024-03-26T03:33:45.658216+0000 mon.a (mon.0) 13 : audit [DBG] from='client.? 172.21.15.175:0/1582803177' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-26T03:33:47.660 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 2024-03-26T03:33:47.660 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout [global] 2024-03-26T03:33:47.660 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout fsid = 762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T03:33:47.661 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.175:3300,v1:172.21.15.175:6789] 2024-03-26T03:33:47.661 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-26T03:33:47.661 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-26T03:33:47.661 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-26T03:33:47.661 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-26T03:33:47.661 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 2024-03-26T03:33:47.661 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-26T03:33:47.661 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-26T03:33:47.661 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 2024-03-26T03:33:47.661 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout [osd] 2024-03-26T03:33:47.661 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-26T03:33:47.661 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-26T03:33:47.661 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f4722458700 1 Processor -- start 2024-03-26T03:33:47.661 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f4722458700 1 -- start start 2024-03-26T03:33:47.662 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f4722458700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f471c106420 0x7f471c106840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:33:47.662 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f4722458700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f471c106e10 con 0x7f471c106420 2024-03-26T03:33:47.662 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f471bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f471c106420 0x7f471c106840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:33:47.662 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f471bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f471c106420 0x7f471c106840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55338/0 (socket says 172.21.15.175:55338) 2024-03-26T03:33:47.662 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f471bfff700 1 -- 172.21.15.175:0/3881312302 learned_addr learned my addr 172.21.15.175:0/3881312302 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:33:47.662 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f471bfff700 1 -- 172.21.15.175:0/3881312302 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f471c1075d0 con 0x7f471c106420 2024-03-26T03:33:47.662 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f471bfff700 1 --2- 172.21.15.175:0/3881312302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f471c106420 0x7f471c106840 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f470c009e80 tx=0x7f470c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a986140a9fd9e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:33:47.662 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f471b7fe700 1 -- 172.21.15.175:0/3881312302 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f470c017070 con 0x7f471c106420 2024-03-26T03:33:47.662 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f471b7fe700 1 -- 172.21.15.175:0/3881312302 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f470c003a20 con 0x7f471c106420 2024-03-26T03:33:47.662 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f471b7fe700 1 -- 172.21.15.175:0/3881312302 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f470c003d20 con 0x7f471c106420 2024-03-26T03:33:47.662 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f4722458700 1 -- 172.21.15.175:0/3881312302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f471c106420 msgr2=0x7f471c106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:33:47.662 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f4722458700 1 --2- 172.21.15.175:0/3881312302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f471c106420 0x7f471c106840 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f470c009e80 tx=0x7f470c00b330 comp rx=0 tx=0).stop 2024-03-26T03:33:47.662 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f4722458700 1 -- 172.21.15.175:0/3881312302 shutdown_connections 2024-03-26T03:33:47.663 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f4722458700 1 --2- 172.21.15.175:0/3881312302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f471c106420 0x7f471c106840 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:33:47.663 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f4722458700 1 -- 172.21.15.175:0/3881312302 >> 172.21.15.175:0/3881312302 conn(0x7f471c101980 msgr2=0x7f471c103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:33:47.663 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f4722458700 1 -- 172.21.15.175:0/3881312302 shutdown_connections 2024-03-26T03:33:47.663 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f4722458700 1 -- 172.21.15.175:0/3881312302 wait complete. 2024-03-26T03:33:47.663 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f4722458700 1 Processor -- start 2024-03-26T03:33:47.663 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f4722458700 1 -- start start 2024-03-26T03:33:47.663 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f4722458700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f471c106420 0x7f471c07c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:33:47.663 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f4722458700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f471c107b00 con 0x7f471c106420 2024-03-26T03:33:47.663 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f471bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f471c106420 0x7f471c07c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:33:47.663 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f471bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f471c106420 0x7f471c07c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55344/0 (socket says 172.21.15.175:55344) 2024-03-26T03:33:47.663 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f471bfff700 1 -- 172.21.15.175:0/2535313976 learned_addr learned my addr 172.21.15.175:0/2535313976 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:33:47.663 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f471bfff700 1 -- 172.21.15.175:0/2535313976 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f471c07c960 con 0x7f471c106420 2024-03-26T03:33:47.663 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f471bfff700 1 --2- 172.21.15.175:0/2535313976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f471c106420 0x7f471c07c420 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f470c004000 tx=0x7f470c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:33:47.663 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f4719ffb700 1 -- 172.21.15.175:0/2535313976 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f470c022020 con 0x7f471c106420 2024-03-26T03:33:47.664 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f4719ffb700 1 -- 172.21.15.175:0/2535313976 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f470c018040 con 0x7f471c106420 2024-03-26T03:33:47.664 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f4722458700 1 -- 172.21.15.175:0/2535313976 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f471c07ab50 con 0x7f471c106420 2024-03-26T03:33:47.664 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f4719ffb700 1 -- 172.21.15.175:0/2535313976 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f470c017070 con 0x7f471c106420 2024-03-26T03:33:47.664 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.966+0000 7f4722458700 1 -- 172.21.15.175:0/2535313976 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f471c07b090 con 0x7f471c106420 2024-03-26T03:33:47.664 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.970+0000 7f4719ffb700 1 -- 172.21.15.175:0/2535313976 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f470c025020 con 0x7f471c106420 2024-03-26T03:33:47.664 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.970+0000 7f4719ffb700 1 -- 172.21.15.175:0/2535313976 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f470c00f050 con 0x7f471c106420 2024-03-26T03:33:47.664 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.970+0000 7f4722458700 1 -- 172.21.15.175:0/2535313976 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f471c063c70 con 0x7f471c106420 2024-03-26T03:33:47.664 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:46.970+0000 7f4719ffb700 1 -- 172.21.15.175:0/2535313976 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f470c02c030 con 0x7f471c106420 2024-03-26T03:33:47.664 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:47.090+0000 7f4722458700 1 -- 172.21.15.175:0/2535313976 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f471c06be20 con 0x7f471c106420 2024-03-26T03:33:47.664 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:47.094+0000 7f4719ffb700 1 -- 172.21.15.175:0/2535313976 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+340 (secure 0 0 0) 0x7f471c06be20 con 0x7f471c106420 2024-03-26T03:33:47.664 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:47.098+0000 7f4722458700 1 -- 172.21.15.175:0/2535313976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f471c106420 msgr2=0x7f471c07c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:33:47.664 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:47.098+0000 7f4722458700 1 --2- 172.21.15.175:0/2535313976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f471c106420 0x7f471c07c420 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f470c004000 tx=0x7f470c004ad0 comp rx=0 tx=0).stop 2024-03-26T03:33:47.664 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:47.098+0000 7f4722458700 1 -- 172.21.15.175:0/2535313976 shutdown_connections 2024-03-26T03:33:47.664 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:47.098+0000 7f4722458700 1 --2- 172.21.15.175:0/2535313976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f471c106420 0x7f471c07c420 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:33:47.664 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:47.098+0000 7f4722458700 1 -- 172.21.15.175:0/2535313976 >> 172.21.15.175:0/2535313976 conn(0x7f471c101980 msgr2=0x7f471c1906e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:33:47.665 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:47.098+0000 7f4722458700 1 -- 172.21.15.175:0/2535313976 shutdown_connections 2024-03-26T03:33:47.665 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:47.098+0000 7f4722458700 1 -- 172.21.15.175:0/2535313976 wait complete. 2024-03-26T03:33:47.665 INFO:teuthology.orchestra.run.smithi175.stdout:Generating new minimal ceph.conf... 2024-03-26T03:33:47.681 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:47 smithi175 bash[21045]: audit 2024-03-26T03:33:47.094735+0000 mon.a (mon.0) 14 : audit [INF] from='client.? 172.21.15.175:0/2535313976' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-26T03:33:47.681 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:47 smithi175 bash[21045]: audit 2024-03-26T03:33:47.098549+0000 mon.a (mon.0) 15 : audit [INF] from='client.? 172.21.15.175:0/2535313976' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-26T03:33:48.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:48 smithi175 bash[21045]: audit 2024-03-26T03:33:48.633129+0000 mon.a (mon.0) 16 : audit [DBG] from='client.? 172.21.15.175:0/1137426974' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:33:49.195 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.490+0000 7fd4b6b87700 1 Processor -- start 2024-03-26T03:33:49.195 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.490+0000 7fd4b6b87700 1 -- start start 2024-03-26T03:33:49.195 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.490+0000 7fd4b6b87700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4b01052f0 0x7fd4b0105710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:33:49.195 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.490+0000 7fd4b6b87700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b0105ce0 con 0x7fd4b01052f0 2024-03-26T03:33:49.195 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.494+0000 7fd4b4923700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4b01052f0 0x7fd4b0105710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:33:49.196 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.494+0000 7fd4b4923700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4b01052f0 0x7fd4b0105710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55360/0 (socket says 172.21.15.175:55360) 2024-03-26T03:33:49.196 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.494+0000 7fd4b4923700 1 -- 172.21.15.175:0/3368997725 learned_addr learned my addr 172.21.15.175:0/3368997725 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:33:49.196 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.494+0000 7fd4b4923700 1 -- 172.21.15.175:0/3368997725 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b01064f0 con 0x7fd4b01052f0 2024-03-26T03:33:49.197 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.494+0000 7fd4b4923700 1 --2- 172.21.15.175:0/3368997725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4b01052f0 0x7fd4b0105710 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd4a40098a0 tx=0x7fd4a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df8069d95173141 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:33:49.197 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.494+0000 7fd4af7fe700 1 -- 172.21.15.175:0/3368997725 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4a4004160 con 0x7fd4b01052f0 2024-03-26T03:33:49.197 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.494+0000 7fd4af7fe700 1 -- 172.21.15.175:0/3368997725 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7fd4a4004300 con 0x7fd4b01052f0 2024-03-26T03:33:49.197 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.494+0000 7fd4af7fe700 1 -- 172.21.15.175:0/3368997725 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4a4011940 con 0x7fd4b01052f0 2024-03-26T03:33:49.197 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.494+0000 7fd4b6b87700 1 -- 172.21.15.175:0/3368997725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4b01052f0 msgr2=0x7fd4b0105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:33:49.197 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.494+0000 7fd4b6b87700 1 --2- 172.21.15.175:0/3368997725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4b01052f0 0x7fd4b0105710 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd4a40098a0 tx=0x7fd4a4009bb0 comp rx=0 tx=0).stop 2024-03-26T03:33:49.197 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.494+0000 7fd4b6b87700 1 -- 172.21.15.175:0/3368997725 shutdown_connections 2024-03-26T03:33:49.202 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.494+0000 7fd4b6b87700 1 --2- 172.21.15.175:0/3368997725 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4b01052f0 0x7fd4b0105710 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:33:49.202 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.494+0000 7fd4b6b87700 1 -- 172.21.15.175:0/3368997725 >> 172.21.15.175:0/3368997725 conn(0x7fd4b0100890 msgr2=0x7fd4b0102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:33:49.202 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.494+0000 7fd4b6b87700 1 -- 172.21.15.175:0/3368997725 shutdown_connections 2024-03-26T03:33:49.202 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.494+0000 7fd4b6b87700 1 -- 172.21.15.175:0/3368997725 wait complete. 2024-03-26T03:33:49.202 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.494+0000 7fd4b6b87700 1 Processor -- start 2024-03-26T03:33:49.202 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.494+0000 7fd4b6b87700 1 -- start start 2024-03-26T03:33:49.202 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.494+0000 7fd4b6b87700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4b01052f0 0x7fd4b019c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:33:49.202 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.494+0000 7fd4b6b87700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b0079880 con 0x7fd4b01052f0 2024-03-26T03:33:49.202 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.494+0000 7fd4b4923700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4b01052f0 0x7fd4b019c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:33:49.202 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.494+0000 7fd4b4923700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4b01052f0 0x7fd4b019c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55370/0 (socket says 172.21.15.175:55370) 2024-03-26T03:33:49.202 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.494+0000 7fd4b4923700 1 -- 172.21.15.175:0/1137426974 learned_addr learned my addr 172.21.15.175:0/1137426974 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:33:49.202 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.494+0000 7fd4b4923700 1 -- 172.21.15.175:0/1137426974 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b019cc60 con 0x7fd4b01052f0 2024-03-26T03:33:49.203 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.494+0000 7fd4b4923700 1 --2- 172.21.15.175:0/1137426974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4b01052f0 0x7fd4b019c720 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4011910 tx=0x7fd4a40119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:33:49.203 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.494+0000 7fd4adffb700 1 -- 172.21.15.175:0/1137426974 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4a4003b20 con 0x7fd4b01052f0 2024-03-26T03:33:49.203 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.494+0000 7fd4adffb700 1 -- 172.21.15.175:0/1137426974 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7fd4a401a890 con 0x7fd4b01052f0 2024-03-26T03:33:49.203 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.494+0000 7fd4b6b87700 1 -- 172.21.15.175:0/1137426974 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4b019cef0 con 0x7fd4b01052f0 2024-03-26T03:33:49.203 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.494+0000 7fd4adffb700 1 -- 172.21.15.175:0/1137426974 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4a40197f0 con 0x7fd4b01052f0 2024-03-26T03:33:49.203 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.494+0000 7fd4b6b87700 1 -- 172.21.15.175:0/1137426974 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4b019fbe0 con 0x7fd4b01052f0 2024-03-26T03:33:49.203 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.494+0000 7fd4adffb700 1 -- 172.21.15.175:0/1137426974 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd4a4022020 con 0x7fd4b01052f0 2024-03-26T03:33:49.203 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.494+0000 7fd4adffb700 1 -- 172.21.15.175:0/1137426974 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd4a402c3a0 con 0x7fd4b01052f0 2024-03-26T03:33:49.203 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.498+0000 7fd4b6b87700 1 -- 172.21.15.175:0/1137426974 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4b0063c70 con 0x7fd4b01052f0 2024-03-26T03:33:49.203 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.498+0000 7fd4adffb700 1 -- 172.21.15.175:0/1137426974 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fd4a401aa40 con 0x7fd4b01052f0 2024-03-26T03:33:49.203 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.630+0000 7fd4b6b87700 1 -- 172.21.15.175:0/1137426974 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fd4b006bd80 con 0x7fd4b01052f0 2024-03-26T03:33:49.203 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.630+0000 7fd4adffb700 1 -- 172.21.15.175:0/1137426974 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7fd4a401ad90 con 0x7fd4b01052f0 2024-03-26T03:33:49.203 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.630+0000 7fd4b6b87700 1 -- 172.21.15.175:0/1137426974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4b01052f0 msgr2=0x7fd4b019c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:33:49.203 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.630+0000 7fd4b6b87700 1 --2- 172.21.15.175:0/1137426974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4b01052f0 0x7fd4b019c720 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4011910 tx=0x7fd4a40119f0 comp rx=0 tx=0).stop 2024-03-26T03:33:49.203 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.630+0000 7fd4b6b87700 1 -- 172.21.15.175:0/1137426974 shutdown_connections 2024-03-26T03:33:49.204 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.630+0000 7fd4b6b87700 1 --2- 172.21.15.175:0/1137426974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4b01052f0 0x7fd4b019c720 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:33:49.204 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.630+0000 7fd4b6b87700 1 -- 172.21.15.175:0/1137426974 >> 172.21.15.175:0/1137426974 conn(0x7fd4b0100890 msgr2=0x7fd4b0109f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:33:49.204 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.630+0000 7fd4b6b87700 1 -- 172.21.15.175:0/1137426974 shutdown_connections 2024-03-26T03:33:49.204 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:48.630+0000 7fd4b6b87700 1 -- 172.21.15.175:0/1137426974 wait complete. 2024-03-26T03:33:49.204 INFO:teuthology.orchestra.run.smithi175.stdout:Restarting the monitor... 2024-03-26T03:33:49.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:49 smithi175 systemd[1]: Stopping Ceph mon.a for 762e930c-eb21-11ee-95d0-87774f69a715... 2024-03-26T03:33:49.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:49 smithi175 bash[21045]: debug 2024-03-26T03:33:49.314+0000 7fa8bb19b700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-03-26T03:33:49.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:49 smithi175 bash[21045]: debug 2024-03-26T03:33:49.314+0000 7fa8bb19b700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-26T03:33:50.069 INFO:teuthology.orchestra.run.smithi175.stdout:Setting public_network to 172.21.15.254/32,172.21.0.2/32,172.21.0.0/20,172.21.0.1/32 in mon config section 2024-03-26T03:33:50.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:49 smithi175 bash[21432]: ceph-762e930c-eb21-11ee-95d0-87774f69a715-mon-a 2024-03-26T03:33:50.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:50 smithi175 bash[21491]: Error response from daemon: No such container: ceph-762e930c-eb21-11ee-95d0-87774f69a715-mon-a 2024-03-26T03:33:50.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:50 smithi175 systemd[1]: ceph-762e930c-eb21-11ee-95d0-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-26T03:33:50.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:50 smithi175 systemd[1]: Stopped Ceph mon.a for 762e930c-eb21-11ee-95d0-87774f69a715. 2024-03-26T03:33:50.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:50 smithi175 systemd[1]: Started Ceph mon.a for 762e930c-eb21-11ee-95d0-87774f69a715. 2024-03-26T03:33:51.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.126+0000 7f555bf74c80 0 set uid:gid to 167:167 (ceph:ceph) 2024-03-26T03:33:51.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.126+0000 7f555bf74c80 0 ceph version 19.0.0-2475-ge42b8ea3 (e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003) squid (dev), process ceph-mon, pid 7 2024-03-26T03:33:51.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.126+0000 7f555bf74c80 0 pidfile_write: ignore empty --pid-file 2024-03-26T03:33:51.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.130+0000 7f555bf74c80 0 load: jerasure load: lrc 2024-03-26T03:33:51.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: RocksDB version: 7.9.2 2024-03-26T03:33:51.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Git sha 0 2024-03-26T03:33:51.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Compile date 2024-03-25 23:25:11 2024-03-26T03:33:51.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: DB SUMMARY 2024-03-26T03:33:51.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: DB Session ID: NWCKYYAPVLD477GSGKXV 2024-03-26T03:33:51.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: CURRENT file: CURRENT 2024-03-26T03:33:51.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: IDENTITY file: IDENTITY 2024-03-26T03:33:51.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-26T03:33:51.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-26T03:33:51.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 91315 ; 2024-03-26T03:33:51.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.error_if_exists: 0 2024-03-26T03:33:51.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.create_if_missing: 0 2024-03-26T03:33:51.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.paranoid_checks: 1 2024-03-26T03:33:51.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.flush_verify_memtable_count: 1 2024-03-26T03:33:51.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-26T03:33:51.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-26T03:33:51.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.env: 0x55910f6f2c60 2024-03-26T03:33:51.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.fs: PosixFileSystem 2024-03-26T03:33:51.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.info_log: 0x55911146bb20 2024-03-26T03:33:51.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.max_file_opening_threads: 16 2024-03-26T03:33:51.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.statistics: (nil) 2024-03-26T03:33:51.246 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.use_fsync: 0 2024-03-26T03:33:51.246 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.max_log_file_size: 0 2024-03-26T03:33:51.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-26T03:33:51.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.log_file_time_to_roll: 0 2024-03-26T03:33:51.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.keep_log_file_num: 1000 2024-03-26T03:33:51.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.recycle_log_file_num: 0 2024-03-26T03:33:51.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.allow_fallocate: 1 2024-03-26T03:33:51.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.allow_mmap_reads: 0 2024-03-26T03:33:51.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.allow_mmap_writes: 0 2024-03-26T03:33:51.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.use_direct_reads: 0 2024-03-26T03:33:51.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-26T03:33:51.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.create_missing_column_families: 0 2024-03-26T03:33:51.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.db_log_dir: 2024-03-26T03:33:51.248 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.wal_dir: 2024-03-26T03:33:51.248 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.table_cache_numshardbits: 6 2024-03-26T03:33:51.248 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-03-26T03:33:51.248 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-03-26T03:33:51.248 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-26T03:33:51.248 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-26T03:33:51.248 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-03-26T03:33:51.248 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.advise_random_on_open: 1 2024-03-26T03:33:51.248 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.db_write_buffer_size: 0 2024-03-26T03:33:51.248 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.write_buffer_manager: 0x55911143c1e0 2024-03-26T03:33:51.248 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-26T03:33:51.248 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-26T03:33:51.248 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.use_adaptive_mutex: 0 2024-03-26T03:33:51.248 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.rate_limiter: (nil) 2024-03-26T03:33:51.249 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-26T03:33:51.249 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.wal_recovery_mode: 2 2024-03-26T03:33:51.249 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.enable_thread_tracking: 0 2024-03-26T03:33:51.249 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.enable_pipelined_write: 0 2024-03-26T03:33:51.249 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.unordered_write: 0 2024-03-26T03:33:51.249 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-26T03:33:51.249 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-26T03:33:51.249 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-26T03:33:51.249 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-26T03:33:51.249 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.row_cache: None 2024-03-26T03:33:51.249 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.wal_filter: None 2024-03-26T03:33:51.249 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-26T03:33:51.249 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.allow_ingest_behind: 0 2024-03-26T03:33:51.249 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.two_write_queues: 0 2024-03-26T03:33:51.249 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.manual_wal_flush: 0 2024-03-26T03:33:51.250 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.wal_compression: 0 2024-03-26T03:33:51.250 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.atomic_flush: 0 2024-03-26T03:33:51.250 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-26T03:33:51.250 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.persist_stats_to_disk: 0 2024-03-26T03:33:51.250 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-03-26T03:33:51.250 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.log_readahead_size: 0 2024-03-26T03:33:51.250 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-26T03:33:51.250 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.best_efforts_recovery: 0 2024-03-26T03:33:51.250 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-26T03:33:51.250 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-26T03:33:51.251 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.allow_data_in_errors: 0 2024-03-26T03:33:51.251 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.db_host_id: __hostname__ 2024-03-26T03:33:51.251 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.enforce_single_del_contracts: true 2024-03-26T03:33:51.251 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.max_background_jobs: 2 2024-03-26T03:33:51.251 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.max_background_compactions: -1 2024-03-26T03:33:51.251 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.max_subcompactions: 1 2024-03-26T03:33:51.251 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-26T03:33:51.251 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-26T03:33:51.251 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.delayed_write_rate : 16777216 2024-03-26T03:33:51.251 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.max_total_wal_size: 0 2024-03-26T03:33:51.251 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-26T03:33:51.251 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.stats_dump_period_sec: 600 2024-03-26T03:33:51.251 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.stats_persist_period_sec: 600 2024-03-26T03:33:51.251 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-26T03:33:51.252 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.max_open_files: -1 2024-03-26T03:33:51.252 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.bytes_per_sync: 0 2024-03-26T03:33:51.252 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-03-26T03:33:51.252 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-03-26T03:33:51.252 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.compaction_readahead_size: 0 2024-03-26T03:33:51.252 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.max_background_flushes: -1 2024-03-26T03:33:51.252 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Compression algorithms supported: 2024-03-26T03:33:51.252 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-26T03:33:51.252 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: kZSTD supported: 0 2024-03-26T03:33:51.252 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: kXpressCompression supported: 0 2024-03-26T03:33:51.252 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: kLZ4HCCompression supported: 1 2024-03-26T03:33:51.252 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: kZlibCompression supported: 1 2024-03-26T03:33:51.252 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: kSnappyCompression supported: 1 2024-03-26T03:33:51.252 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: kLZ4Compression supported: 1 2024-03-26T03:33:51.252 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: kBZip2Compression supported: 0 2024-03-26T03:33:51.253 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-03-26T03:33:51.253 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: DMutex implementation: pthread_mutex_t 2024-03-26T03:33:51.253 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-26T03:33:51.253 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-26T03:33:51.253 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-26T03:33:51.253 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.merge_operator: 2024-03-26T03:33:51.253 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.compaction_filter: None 2024-03-26T03:33:51.253 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.compaction_filter_factory: None 2024-03-26T03:33:51.253 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.sst_partitioner_factory: None 2024-03-26T03:33:51.253 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-03-26T03:33:51.253 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.table_factory: BlockBasedTable 2024-03-26T03:33:51.253 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55911146bd00) 2024-03-26T03:33:51.253 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: cache_index_and_filter_blocks: 1 2024-03-26T03:33:51.253 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-26T03:33:51.253 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-26T03:33:51.254 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: pin_top_level_index_and_filter: 1 2024-03-26T03:33:51.254 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: index_type: 0 2024-03-26T03:33:51.254 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: data_block_index_type: 0 2024-03-26T03:33:51.254 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: index_shortening: 1 2024-03-26T03:33:51.254 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: data_block_hash_table_util_ratio: 0.750000 2024-03-26T03:33:51.254 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: checksum: 4 2024-03-26T03:33:51.254 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: no_block_cache: 0 2024-03-26T03:33:51.254 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: block_cache: 0x55911146d090 2024-03-26T03:33:51.254 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: block_cache_name: BinnedLRUCache 2024-03-26T03:33:51.254 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: block_cache_options: 2024-03-26T03:33:51.254 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: capacity : 536870912 2024-03-26T03:33:51.254 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: num_shard_bits : 4 2024-03-26T03:33:51.254 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: strict_capacity_limit : 0 2024-03-26T03:33:51.254 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: high_pri_pool_ratio: 0.000 2024-03-26T03:33:51.255 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: block_cache_compressed: (nil) 2024-03-26T03:33:51.255 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: persistent_cache: (nil) 2024-03-26T03:33:51.255 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: block_size: 4096 2024-03-26T03:33:51.255 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: block_size_deviation: 10 2024-03-26T03:33:51.255 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: block_restart_interval: 16 2024-03-26T03:33:51.255 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: index_block_restart_interval: 1 2024-03-26T03:33:51.255 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: metadata_block_size: 4096 2024-03-26T03:33:51.255 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: partition_filters: 0 2024-03-26T03:33:51.255 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: use_delta_encoding: 1 2024-03-26T03:33:51.255 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: filter_policy: bloomfilter 2024-03-26T03:33:51.255 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: whole_key_filtering: 1 2024-03-26T03:33:51.255 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: verify_compression: 0 2024-03-26T03:33:51.255 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: read_amp_bytes_per_bit: 0 2024-03-26T03:33:51.255 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: format_version: 5 2024-03-26T03:33:51.255 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: enable_index_compression: 1 2024-03-26T03:33:51.256 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: block_align: 0 2024-03-26T03:33:51.256 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: max_auto_readahead_size: 262144 2024-03-26T03:33:51.256 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: prepopulate_block_cache: 0 2024-03-26T03:33:51.256 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: initial_auto_readahead_size: 8192 2024-03-26T03:33:51.256 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: num_file_reads_for_auto_readahead: 2 2024-03-26T03:33:51.256 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.write_buffer_size: 33554432 2024-03-26T03:33:51.256 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.max_write_buffer_number: 2 2024-03-26T03:33:51.256 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.compression: NoCompression 2024-03-26T03:33:51.256 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.bottommost_compression: Disabled 2024-03-26T03:33:51.256 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.prefix_extractor: nullptr 2024-03-26T03:33:51.256 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-26T03:33:51.256 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.num_levels: 7 2024-03-26T03:33:51.256 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-26T03:33:51.256 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-26T03:33:51.257 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-26T03:33:51.257 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-26T03:33:51.257 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-26T03:33:51.257 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-26T03:33:51.257 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-26T03:33:51.257 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-26T03:33:51.257 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-26T03:33:51.257 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-26T03:33:51.257 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-26T03:33:51.257 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-26T03:33:51.257 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.compression_opts.window_bits: -14 2024-03-26T03:33:51.257 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.compression_opts.level: 32767 2024-03-26T03:33:51.257 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.compression_opts.strategy: 0 2024-03-26T03:33:51.257 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-26T03:33:51.257 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-26T03:33:51.258 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-26T03:33:51.258 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-26T03:33:51.258 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.compression_opts.enabled: false 2024-03-26T03:33:51.258 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-26T03:33:51.259 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-26T03:33:51.259 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-26T03:33:51.259 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-26T03:33:51.259 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.target_file_size_base: 67108864 2024-03-26T03:33:51.259 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.target_file_size_multiplier: 1 2024-03-26T03:33:51.259 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-26T03:33:51.259 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-26T03:33:51.259 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-26T03:33:51.259 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-26T03:33:51.260 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-26T03:33:51.260 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-26T03:33:51.260 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-26T03:33:51.260 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-26T03:33:51.260 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-26T03:33:51.260 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-26T03:33:51.260 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-26T03:33:51.260 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-26T03:33:51.260 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-26T03:33:51.261 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.arena_block_size: 1048576 2024-03-26T03:33:51.261 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-26T03:33:51.261 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-26T03:33:51.261 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.disable_auto_compactions: 0 2024-03-26T03:33:51.261 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-26T03:33:51.261 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-26T03:33:51.261 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-26T03:33:51.261 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-26T03:33:51.261 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-26T03:33:51.261 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-26T03:33:51.261 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-26T03:33:51.261 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-26T03:33:51.261 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-26T03:33:51.261 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-26T03:33:51.262 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-03-26T03:33:51.262 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.inplace_update_support: 0 2024-03-26T03:33:51.262 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-03-26T03:33:51.262 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-26T03:33:51.262 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-26T03:33:51.262 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.memtable_huge_page_size: 0 2024-03-26T03:33:51.262 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.bloom_locality: 0 2024-03-26T03:33:51.262 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.max_successive_merges: 0 2024-03-26T03:33:51.262 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-03-26T03:33:51.262 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.paranoid_file_checks: 0 2024-03-26T03:33:51.262 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.force_consistency_checks: 1 2024-03-26T03:33:51.262 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.report_bg_io_stats: 0 2024-03-26T03:33:51.262 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.ttl: 2592000 2024-03-26T03:33:51.262 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-03-26T03:33:51.263 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-26T03:33:51.263 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-26T03:33:51.263 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.enable_blob_files: false 2024-03-26T03:33:51.263 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.min_blob_size: 0 2024-03-26T03:33:51.263 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.blob_file_size: 268435456 2024-03-26T03:33:51.263 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.blob_compression_type: NoCompression 2024-03-26T03:33:51.263 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.enable_blob_garbage_collection: false 2024-03-26T03:33:51.263 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-26T03:33:51.263 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-26T03:33:51.263 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-26T03:33:51.263 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.blob_file_starting_level: 0 2024-03-26T03:33:51.263 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.134+0000 7f555bf74c80 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-26T03:33:51.263 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.138+0000 7f555bf74c80 4 rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-26T03:33:51.263 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.138+0000 7f555bf74c80 4 rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-26T03:33:51.264 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.138+0000 7f555bf74c80 4 rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 20d84c71-8864-42cf-9e0a-77a82f9fcf70 2024-03-26T03:33:51.264 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.138+0000 7f555bf74c80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1711424031139604, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-26T03:33:51.264 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.138+0000 7f555bf74c80 4 rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-26T03:33:51.264 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.138+0000 7f555bf74c80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1711424031140595, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 87729, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 268, "table_properties": {"data_size": 85861, "index_size": 252, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10766, "raw_average_key_size": 47, "raw_value_size": 79771, "raw_average_value_size": 351, "num_data_blocks": 11, "num_entries": 227, "num_filter_entries": 227, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711424031, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "20d84c71-8864-42cf-9e0a-77a82f9fcf70", "db_session_id": "NWCKYYAPVLD477GSGKXV", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-26T03:33:51.264 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.138+0000 7f555bf74c80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1711424031140694, "job": 1, "event": "recovery_finished"} 2024-03-26T03:33:51.264 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.138+0000 7f555bf74c80 4 rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-26T03:33:51.264 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.138+0000 7f555bf74c80 4 rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-26T03:33:51.264 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.138+0000 7f555bf74c80 4 rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559111560000 2024-03-26T03:33:51.264 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.138+0000 7f555bf74c80 4 rocksdb: DB pointer 0x55911154a000 2024-03-26T03:33:51.264 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.138+0000 7f554a6cf700 4 rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-26T03:33:51.264 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.138+0000 7f554a6cf700 4 rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-26T03:33:51.264 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: ** DB Stats ** 2024-03-26T03:33:51.264 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T03:33:51.264 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-26T03:33:51.264 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T03:33:51.265 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T03:33:51.265 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-26T03:33:51.265 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T03:33:51.265 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T03:33:51.265 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: ** Compaction Stats [default] ** 2024-03-26T03:33:51.265 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T03:33:51.265 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-26T03:33:51.265 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: L0 2/0 87.53 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T03:33:51.265 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: Sum 2/0 87.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T03:33:51.265 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T03:33:51.265 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: ** Compaction Stats [default] ** 2024-03-26T03:33:51.265 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T03:33:51.265 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-26T03:33:51.265 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 145.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T03:33:51.265 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-26T03:33:51.266 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T03:33:51.266 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: Flush(GB): cumulative 0.000, interval 0.000 2024-03-26T03:33:51.266 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-26T03:33:51.266 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: AddFile(Total Files): cumulative 0, interval 0 2024-03-26T03:33:51.266 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: AddFile(L0 Files): cumulative 0, interval 0 2024-03-26T03:33:51.266 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: AddFile(Keys): cumulative 0, interval 0 2024-03-26T03:33:51.266 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: Cumulative compaction: 0.00 GB write, 13.75 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T03:33:51.266 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: Interval compaction: 0.00 GB write, 13.75 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T03:33:51.266 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-26T03:33:51.266 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: Block cache BinnedLRUCache@0x55911146d090#7 capacity: 512.00 MB usage: 1.20 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.9e-05 secs_since: 0 2024-03-26T03:33:51.266 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.44 KB,8.34465e-05%) Misc(1,0.00 KB,0%) 2024-03-26T03:33:51.266 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: ** File Read Latency Histogram By Level [default] ** 2024-03-26T03:33:51.266 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.138+0000 7f555bf74c80 0 starting mon.a rank 0 at public addrs [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] at bind addrs [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T03:33:51.266 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.142+0000 7f555bf74c80 1 mon.a@-1(???) e1 preinit fsid 762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T03:33:51.267 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.142+0000 7f555bf74c80 0 mon.a@-1(???).mds e1 new map 2024-03-26T03:33:51.267 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.142+0000 7f555bf74c80 0 mon.a@-1(???).mds e1 print_map 2024-03-26T03:33:51.267 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: e1 2024-03-26T03:33:51.267 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: enable_multiple, ever_enabled_multiple: 1,1 2024-03-26T03:33:51.267 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-26T03:33:51.267 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: legacy client fscid: -1 2024-03-26T03:33:51.267 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: 2024-03-26T03:33:51.267 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: No filesystems configured 2024-03-26T03:33:51.267 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.142+0000 7f555bf74c80 0 mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-26T03:33:51.267 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.142+0000 7f555bf74c80 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T03:33:51.267 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.142+0000 7f555bf74c80 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T03:33:51.267 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.142+0000 7f555bf74c80 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T03:33:51.267 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: debug 2024-03-26T03:33:51.142+0000 7f555bf74c80 1 mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-26T03:33:51.267 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: cluster 2024-03-26T03:33:51.163870+0000 mon.a (mon.0) 1 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2024-03-26T03:33:51.267 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: cluster 2024-03-26T03:33:51.163954+0000 mon.a (mon.0) 2 : cluster [DBG] monmap epoch 1 2024-03-26T03:33:51.268 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: cluster 2024-03-26T03:33:51.163968+0000 mon.a (mon.0) 3 : cluster [DBG] fsid 762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T03:33:51.268 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: cluster 2024-03-26T03:33:51.163979+0000 mon.a (mon.0) 4 : cluster [DBG] last_changed 2024-03-26T03:33:41.181831+0000 2024-03-26T03:33:51.268 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: cluster 2024-03-26T03:33:51.163986+0000 mon.a (mon.0) 5 : cluster [DBG] created 2024-03-26T03:33:41.181831+0000 2024-03-26T03:33:51.268 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: cluster 2024-03-26T03:33:51.163995+0000 mon.a (mon.0) 6 : cluster [DBG] min_mon_release 19 (squid) 2024-03-26T03:33:51.268 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: cluster 2024-03-26T03:33:51.164002+0000 mon.a (mon.0) 7 : cluster [DBG] election_strategy: 1 2024-03-26T03:33:51.268 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: cluster 2024-03-26T03:33:51.164015+0000 mon.a (mon.0) 8 : cluster [DBG] 0: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.a 2024-03-26T03:33:51.268 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: cluster 2024-03-26T03:33:51.164409+0000 mon.a (mon.0) 9 : cluster [DBG] fsmap 2024-03-26T03:33:51.268 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: cluster 2024-03-26T03:33:51.164433+0000 mon.a (mon.0) 10 : cluster [DBG] osdmap e1: 0 total, 0 up, 0 in 2024-03-26T03:33:51.268 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:51 smithi175 bash[21527]: cluster 2024-03-26T03:33:51.164833+0000 mon.a (mon.0) 11 : cluster [DBG] mgrmap e1: no daemons active 2024-03-26T03:33:51.929 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.178+0000 7f16836e6700 1 Processor -- start 2024-03-26T03:33:51.929 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.178+0000 7f16836e6700 1 -- start start 2024-03-26T03:33:51.929 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.178+0000 7f16836e6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f167c108610 0x7f167c108a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:33:51.929 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.178+0000 7f16836e6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f167c109000 con 0x7f167c108610 2024-03-26T03:33:51.929 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.178+0000 7f1681482700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f167c108610 0x7f167c108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:33:51.930 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.178+0000 7f1681482700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f167c108610 0x7f167c108a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:51804/0 (socket says 172.21.15.175:51804) 2024-03-26T03:33:51.930 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.178+0000 7f1681482700 1 -- 172.21.15.175:0/1115588846 learned_addr learned my addr 172.21.15.175:0/1115588846 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:33:51.931 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.178+0000 7f1681482700 1 -- 172.21.15.175:0/1115588846 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f167c1097c0 con 0x7f167c108610 2024-03-26T03:33:51.931 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.178+0000 7f1681482700 1 --2- 172.21.15.175:0/1115588846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f167c108610 0x7f167c108a30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f16780098a0 tx=0x7f1678009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38bcf1fc9672e5fc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:33:51.931 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.178+0000 7f1673fff700 1 -- 172.21.15.175:0/1115588846 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1678004160 con 0x7f167c108610 2024-03-26T03:33:51.931 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.178+0000 7f1673fff700 1 -- 172.21.15.175:0/1115588846 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f1678004300 con 0x7f167c108610 2024-03-26T03:33:51.931 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.178+0000 7f1673fff700 1 -- 172.21.15.175:0/1115588846 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1678011940 con 0x7f167c108610 2024-03-26T03:33:51.931 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.182+0000 7f16836e6700 1 -- 172.21.15.175:0/1115588846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f167c108610 msgr2=0x7f167c108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:33:51.931 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.182+0000 7f16836e6700 1 --2- 172.21.15.175:0/1115588846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f167c108610 0x7f167c108a30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f16780098a0 tx=0x7f1678009bb0 comp rx=0 tx=0).stop 2024-03-26T03:33:51.931 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.182+0000 7f16836e6700 1 -- 172.21.15.175:0/1115588846 shutdown_connections 2024-03-26T03:33:51.931 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.182+0000 7f16836e6700 1 --2- 172.21.15.175:0/1115588846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f167c108610 0x7f167c108a30 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:33:51.933 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.182+0000 7f16836e6700 1 -- 172.21.15.175:0/1115588846 >> 172.21.15.175:0/1115588846 conn(0x7f167c07ba00 msgr2=0x7f167c07be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:33:51.933 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.182+0000 7f16836e6700 1 -- 172.21.15.175:0/1115588846 shutdown_connections 2024-03-26T03:33:51.933 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.182+0000 7f16836e6700 1 -- 172.21.15.175:0/1115588846 wait complete. 2024-03-26T03:33:51.933 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.182+0000 7f16836e6700 1 Processor -- start 2024-03-26T03:33:51.933 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.182+0000 7f16836e6700 1 -- start start 2024-03-26T03:33:51.933 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.182+0000 7f16836e6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f167c108610 0x7f167c19c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:33:51.933 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.182+0000 7f16836e6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f167c10c9f0 con 0x7f167c108610 2024-03-26T03:33:51.933 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.182+0000 7f1681482700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f167c108610 0x7f167c19c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:33:51.933 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.182+0000 7f1681482700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f167c108610 0x7f167c19c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:51810/0 (socket says 172.21.15.175:51810) 2024-03-26T03:33:51.933 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.182+0000 7f1681482700 1 -- 172.21.15.175:0/2670562384 learned_addr learned my addr 172.21.15.175:0/2670562384 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:33:51.933 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.182+0000 7f1681482700 1 -- 172.21.15.175:0/2670562384 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f167c19cc50 con 0x7f167c108610 2024-03-26T03:33:51.934 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.182+0000 7f1681482700 1 --2- 172.21.15.175:0/2670562384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f167c108610 0x7f167c19c710 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1678006ba0 tx=0x7f16780119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:33:51.934 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.182+0000 7f16727fc700 1 -- 172.21.15.175:0/2670562384 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1678003930 con 0x7f167c108610 2024-03-26T03:33:51.934 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.182+0000 7f16727fc700 1 -- 172.21.15.175:0/2670562384 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f1678003ad0 con 0x7f167c108610 2024-03-26T03:33:51.934 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.182+0000 7f16836e6700 1 -- 172.21.15.175:0/2670562384 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f167c19cee0 con 0x7f167c108610 2024-03-26T03:33:51.935 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.182+0000 7f16727fc700 1 -- 172.21.15.175:0/2670562384 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1678019750 con 0x7f167c108610 2024-03-26T03:33:51.935 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.182+0000 7f16836e6700 1 -- 172.21.15.175:0/2670562384 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f167c19fbd0 con 0x7f167c108610 2024-03-26T03:33:51.936 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.182+0000 7f16727fc700 1 -- 172.21.15.175:0/2670562384 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f167802d440 con 0x7f167c108610 2024-03-26T03:33:51.936 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.182+0000 7f16727fc700 1 -- 172.21.15.175:0/2670562384 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1678024780 con 0x7f167c108610 2024-03-26T03:33:51.936 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.182+0000 7f16836e6700 1 -- 172.21.15.175:0/2670562384 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f167c10cb70 con 0x7f167c108610 2024-03-26T03:33:51.936 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.186+0000 7f16727fc700 1 -- 172.21.15.175:0/2670562384 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f1678043dd0 con 0x7f167c108610 2024-03-26T03:33:51.936 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.318+0000 7f16836e6700 1 -- 172.21.15.175:0/2670562384 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f167c02dd30 con 0x7f167c108610 2024-03-26T03:33:51.936 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.322+0000 7f16727fc700 1 -- 172.21.15.175:0/2670562384 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 169+0+0 (secure 0 0 0) 0x7f1678032020 con 0x7f167c108610 2024-03-26T03:33:51.936 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.322+0000 7f16836e6700 1 -- 172.21.15.175:0/2670562384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f167c108610 msgr2=0x7f167c19c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:33:51.936 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.322+0000 7f16836e6700 1 --2- 172.21.15.175:0/2670562384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f167c108610 0x7f167c19c710 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1678006ba0 tx=0x7f16780119c0 comp rx=0 tx=0).stop 2024-03-26T03:33:51.936 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.322+0000 7f16836e6700 1 -- 172.21.15.175:0/2670562384 shutdown_connections 2024-03-26T03:33:51.936 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.322+0000 7f16836e6700 1 --2- 172.21.15.175:0/2670562384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f167c108610 0x7f167c19c710 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:33:51.937 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.322+0000 7f16836e6700 1 -- 172.21.15.175:0/2670562384 >> 172.21.15.175:0/2670562384 conn(0x7f167c07ba00 msgr2=0x7f167c105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:33:51.937 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.322+0000 7f16836e6700 1 -- 172.21.15.175:0/2670562384 shutdown_connections 2024-03-26T03:33:51.937 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:51.322+0000 7f16836e6700 1 -- 172.21.15.175:0/2670562384 wait complete. 2024-03-26T03:33:51.937 INFO:teuthology.orchestra.run.smithi175.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-26T03:33:51.954 INFO:teuthology.orchestra.run.smithi175.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-26T03:33:51.954 INFO:teuthology.orchestra.run.smithi175.stdout:Creating mgr... 2024-03-26T03:33:51.954 INFO:teuthology.orchestra.run.smithi175.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-26T03:33:51.954 INFO:teuthology.orchestra.run.smithi175.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-26T03:33:52.278 INFO:teuthology.orchestra.run.smithi175.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-762e930c-eb21-11ee-95d0-87774f69a715@mgr.y 2024-03-26T03:33:52.278 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stderr Failed to reset failed state of unit ceph-762e930c-eb21-11ee-95d0-87774f69a715@mgr.y.service: Unit ceph-762e930c-eb21-11ee-95d0-87774f69a715@mgr.y.service not loaded. 2024-03-26T03:33:52.321 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:52 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:33:52.461 INFO:teuthology.orchestra.run.smithi175.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715.target.wants/ceph-762e930c-eb21-11ee-95d0-87774f69a715@mgr.y.service -> /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service. 2024-03-26T03:33:52.471 INFO:teuthology.orchestra.run.smithi175.stdout:firewalld does not appear to be present 2024-03-26T03:33:52.471 INFO:teuthology.orchestra.run.smithi175.stdout:Not possible to enable service . firewalld.service is not available 2024-03-26T03:33:52.471 INFO:teuthology.orchestra.run.smithi175.stdout:firewalld does not appear to be present 2024-03-26T03:33:52.471 INFO:teuthology.orchestra.run.smithi175.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-26T03:33:52.471 INFO:teuthology.orchestra.run.smithi175.stdout:Waiting for mgr to start... 2024-03-26T03:33:52.471 INFO:teuthology.orchestra.run.smithi175.stdout:Waiting for mgr... 2024-03-26T03:33:52.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:52 smithi175 bash[21527]: audit 2024-03-26T03:33:51.323780+0000 mon.a (mon.0) 12 : audit [INF] from='client.? 172.21.15.175:0/2670562384' entity='client.admin' 2024-03-26T03:33:52.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:52 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:33:54.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:53 smithi175 bash[21527]: audit 2024-03-26T03:33:53.824657+0000 mon.a (mon.0) 13 : audit [DBG] from='client.? 172.21.15.175:0/2614931748' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-26T03:33:54.244 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:33:53 smithi175 bash[21783]: debug 2024-03-26T03:33:53.778+0000 7f6327f29200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T03:33:54.395 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 2024-03-26T03:33:54.396 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout { 2024-03-26T03:33:54.396 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "fsid": "762e930c-eb21-11ee-95d0-87774f69a715", 2024-03-26T03:33:54.396 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "health": { 2024-03-26T03:33:54.396 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-26T03:33:54.396 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-26T03:33:54.396 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-26T03:33:54.396 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-03-26T03:33:54.396 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-26T03:33:54.396 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-26T03:33:54.396 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 0 2024-03-26T03:33:54.396 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout ], 2024-03-26T03:33:54.396 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-26T03:33:54.396 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "a" 2024-03-26T03:33:54.397 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout ], 2024-03-26T03:33:54.397 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-26T03:33:54.397 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-26T03:33:54.397 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T03:33:54.397 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-26T03:33:54.397 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-26T03:33:54.397 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-03-26T03:33:54.397 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-26T03:33:54.397 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T03:33:54.397 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-26T03:33:54.397 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-26T03:33:54.397 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-26T03:33:54.397 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-26T03:33:54.397 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-26T03:33:54.397 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-26T03:33:54.398 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-03-26T03:33:54.398 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-26T03:33:54.398 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-26T03:33:54.398 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-26T03:33:54.398 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-26T03:33:54.398 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-26T03:33:54.398 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-26T03:33:54.398 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-26T03:33:54.398 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-26T03:33:54.398 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-26T03:33:54.398 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-03-26T03:33:54.398 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-26T03:33:54.398 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T03:33:54.398 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-26T03:33:54.399 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-26T03:33:54.399 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-03-26T03:33:54.399 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-26T03:33:54.399 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-26T03:33:54.399 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-26T03:33:54.399 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-26T03:33:54.399 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-26T03:33:54.399 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-26T03:33:54.399 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "restful" 2024-03-26T03:33:54.399 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout ], 2024-03-26T03:33:54.399 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T03:33:54.399 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-03-26T03:33:54.399 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-26T03:33:54.399 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T03:33:54.400 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "modified": "2024-03-26T03:33:45.388974+0000", 2024-03-26T03:33:54.400 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T03:33:54.400 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-03-26T03:33:54.400 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-26T03:33:54.400 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout } 2024-03-26T03:33:54.400 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.686+0000 7f93b2e46700 1 Processor -- start 2024-03-26T03:33:54.400 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.686+0000 7f93b2e46700 1 -- start start 2024-03-26T03:33:54.400 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.686+0000 7f93b2e46700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93ac104cc0 0x7f93ac1050e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:33:54.400 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.686+0000 7f93b2e46700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f93ac07a6e0 con 0x7f93ac104cc0 2024-03-26T03:33:54.400 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.686+0000 7f93b0be2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93ac104cc0 0x7f93ac1050e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:33:54.400 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.686+0000 7f93b0be2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93ac104cc0 0x7f93ac1050e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:51836/0 (socket says 172.21.15.175:51836) 2024-03-26T03:33:54.400 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.686+0000 7f93b0be2700 1 -- 172.21.15.175:0/362112186 learned_addr learned my addr 172.21.15.175:0/362112186 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:33:54.400 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.686+0000 7f93b0be2700 1 -- 172.21.15.175:0/362112186 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93ac105620 con 0x7f93ac104cc0 2024-03-26T03:33:54.400 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.686+0000 7f93b0be2700 1 --2- 172.21.15.175:0/362112186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93ac104cc0 0x7f93ac1050e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f93980098a0 tx=0x7f9398009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09d9d45150c76e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:33:54.400 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.686+0000 7f93ab7fe700 1 -- 172.21.15.175:0/362112186 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9398004160 con 0x7f93ac104cc0 2024-03-26T03:33:54.401 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.686+0000 7f93ab7fe700 1 -- 172.21.15.175:0/362112186 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f9398004300 con 0x7f93ac104cc0 2024-03-26T03:33:54.401 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.686+0000 7f93ab7fe700 1 -- 172.21.15.175:0/362112186 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9398011940 con 0x7f93ac104cc0 2024-03-26T03:33:54.401 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.686+0000 7f93b2e46700 1 -- 172.21.15.175:0/362112186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93ac104cc0 msgr2=0x7f93ac1050e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:33:54.401 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.686+0000 7f93b2e46700 1 --2- 172.21.15.175:0/362112186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93ac104cc0 0x7f93ac1050e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f93980098a0 tx=0x7f9398009bb0 comp rx=0 tx=0).stop 2024-03-26T03:33:54.401 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.690+0000 7f93b2e46700 1 -- 172.21.15.175:0/362112186 shutdown_connections 2024-03-26T03:33:54.401 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.690+0000 7f93b2e46700 1 --2- 172.21.15.175:0/362112186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93ac104cc0 0x7f93ac1050e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:33:54.401 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.690+0000 7f93b2e46700 1 -- 172.21.15.175:0/362112186 >> 172.21.15.175:0/362112186 conn(0x7f93ac100860 msgr2=0x7f93ac102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:33:54.401 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.690+0000 7f93b2e46700 1 -- 172.21.15.175:0/362112186 shutdown_connections 2024-03-26T03:33:54.401 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.690+0000 7f93b2e46700 1 -- 172.21.15.175:0/362112186 wait complete. 2024-03-26T03:33:54.401 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.690+0000 7f93b2e46700 1 Processor -- start 2024-03-26T03:33:54.401 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.690+0000 7f93b2e46700 1 -- start start 2024-03-26T03:33:54.401 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.690+0000 7f93b2e46700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93ac104cc0 0x7f93ac19c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:33:54.401 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.690+0000 7f93b2e46700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f93ac106590 con 0x7f93ac104cc0 2024-03-26T03:33:54.401 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.690+0000 7f93b0be2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93ac104cc0 0x7f93ac19c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:33:54.402 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.690+0000 7f93b0be2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93ac104cc0 0x7f93ac19c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:51848/0 (socket says 172.21.15.175:51848) 2024-03-26T03:33:54.402 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.690+0000 7f93b0be2700 1 -- 172.21.15.175:0/2614931748 learned_addr learned my addr 172.21.15.175:0/2614931748 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:33:54.402 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.690+0000 7f93b0be2700 1 -- 172.21.15.175:0/2614931748 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93ac19cbd0 con 0x7f93ac104cc0 2024-03-26T03:33:54.402 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.690+0000 7f93b0be2700 1 --2- 172.21.15.175:0/2614931748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93ac104cc0 0x7f93ac19c690 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9398009870 tx=0x7f93980119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:33:54.402 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.690+0000 7f93a9ffb700 1 -- 172.21.15.175:0/2614931748 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f939801a430 con 0x7f93ac104cc0 2024-03-26T03:33:54.402 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.690+0000 7f93b2e46700 1 -- 172.21.15.175:0/2614931748 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93ac19ce60 con 0x7f93ac104cc0 2024-03-26T03:33:54.402 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.690+0000 7f93a9ffb700 1 -- 172.21.15.175:0/2614931748 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f939801aa30 con 0x7f93ac104cc0 2024-03-26T03:33:54.402 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.690+0000 7f93a9ffb700 1 -- 172.21.15.175:0/2614931748 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93980248d0 con 0x7f93ac104cc0 2024-03-26T03:33:54.402 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.690+0000 7f93b2e46700 1 -- 172.21.15.175:0/2614931748 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93ac19fb50 con 0x7f93ac104cc0 2024-03-26T03:33:54.402 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.690+0000 7f93a9ffb700 1 -- 172.21.15.175:0/2614931748 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9398022020 con 0x7f93ac104cc0 2024-03-26T03:33:54.402 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.690+0000 7f93a9ffb700 1 -- 172.21.15.175:0/2614931748 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f939802d9d0 con 0x7f93ac104cc0 2024-03-26T03:33:54.402 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.690+0000 7f93b2e46700 1 -- 172.21.15.175:0/2614931748 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93ac063c70 con 0x7f93ac104cc0 2024-03-26T03:33:54.402 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.694+0000 7f93a9ffb700 1 -- 172.21.15.175:0/2614931748 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f9398004300 con 0x7f93ac104cc0 2024-03-26T03:33:54.402 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.822+0000 7f93b2e46700 1 -- 172.21.15.175:0/2614931748 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f93ac06bdb0 con 0x7f93ac104cc0 2024-03-26T03:33:54.403 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.822+0000 7f93a9ffb700 1 -- 172.21.15.175:0/2614931748 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f939802d3f0 con 0x7f93ac104cc0 2024-03-26T03:33:54.403 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.822+0000 7f93b2e46700 1 -- 172.21.15.175:0/2614931748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93ac104cc0 msgr2=0x7f93ac19c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:33:54.403 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.822+0000 7f93b2e46700 1 --2- 172.21.15.175:0/2614931748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93ac104cc0 0x7f93ac19c690 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9398009870 tx=0x7f93980119c0 comp rx=0 tx=0).stop 2024-03-26T03:33:54.403 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.822+0000 7f93b2e46700 1 -- 172.21.15.175:0/2614931748 shutdown_connections 2024-03-26T03:33:54.403 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.822+0000 7f93b2e46700 1 --2- 172.21.15.175:0/2614931748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93ac104cc0 0x7f93ac19c690 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:33:54.403 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.822+0000 7f93b2e46700 1 -- 172.21.15.175:0/2614931748 >> 172.21.15.175:0/2614931748 conn(0x7f93ac100860 msgr2=0x7f93ac193080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:33:54.403 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.822+0000 7f93b2e46700 1 -- 172.21.15.175:0/2614931748 shutdown_connections 2024-03-26T03:33:54.403 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:53.822+0000 7f93b2e46700 1 -- 172.21.15.175:0/2614931748 wait complete. 2024-03-26T03:33:54.403 INFO:teuthology.orchestra.run.smithi175.stdout:mgr not available, waiting (1/15)... 2024-03-26T03:33:55.695 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:33:55 smithi175 bash[21783]: debug 2024-03-26T03:33:55.354+0000 7f6327f29200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T03:33:55.696 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:33:55 smithi175 bash[21783]: debug 2024-03-26T03:33:55.446+0000 7f6327f29200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T03:33:55.696 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:33:55 smithi175 bash[21783]: debug 2024-03-26T03:33:55.694+0000 7f6327f29200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T03:33:56.698 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:33:56 smithi175 bash[21783]: debug 2024-03-26T03:33:56.486+0000 7f6327f29200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T03:33:56.698 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:33:56 smithi175 bash[21783]: debug 2024-03-26T03:33:56.594+0000 7f6327f29200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T03:33:56.994 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:33:56 smithi175 bash[21783]: debug 2024-03-26T03:33:56.694+0000 7f6327f29200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T03:33:57.486 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:33:57 smithi175 bash[21783]: debug 2024-03-26T03:33:57.054+0000 7f6327f29200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T03:33:57.487 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:33:57 smithi175 bash[21783]: debug 2024-03-26T03:33:57.170+0000 7f6327f29200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T03:33:57.487 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:33:57 smithi175 bash[21783]: debug 2024-03-26T03:33:57.274+0000 7f6327f29200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T03:33:57.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:33:57 smithi175 bash[21527]: audit 2024-03-26T03:33:57.474449+0000 mon.a (mon.0) 14 : audit [DBG] from='client.? 172.21.15.175:0/720680630' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-26T03:33:57.744 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:33:57 smithi175 bash[21783]: debug 2024-03-26T03:33:57.634+0000 7f6327f29200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T03:33:57.991 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 2024-03-26T03:33:57.991 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout { 2024-03-26T03:33:57.992 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "fsid": "762e930c-eb21-11ee-95d0-87774f69a715", 2024-03-26T03:33:57.992 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "health": { 2024-03-26T03:33:57.992 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-26T03:33:57.992 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-26T03:33:57.992 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-26T03:33:57.992 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-03-26T03:33:57.992 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-26T03:33:57.992 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-26T03:33:57.992 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 0 2024-03-26T03:33:57.992 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout ], 2024-03-26T03:33:57.992 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-26T03:33:57.992 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "a" 2024-03-26T03:33:57.992 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout ], 2024-03-26T03:33:57.992 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-03-26T03:33:57.992 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-26T03:33:57.993 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T03:33:57.993 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-26T03:33:57.993 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-26T03:33:57.993 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-03-26T03:33:57.993 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-26T03:33:57.993 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T03:33:57.993 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-26T03:33:57.993 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-26T03:33:57.993 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-26T03:33:57.993 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-26T03:33:57.993 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-26T03:33:57.993 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-26T03:33:57.993 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-03-26T03:33:57.993 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-26T03:33:57.993 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-26T03:33:57.994 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-26T03:33:57.994 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-26T03:33:57.994 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-26T03:33:57.994 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-26T03:33:57.994 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-26T03:33:57.994 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-26T03:33:57.994 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-26T03:33:57.994 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-03-26T03:33:57.994 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-26T03:33:57.994 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T03:33:57.994 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-26T03:33:57.994 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-26T03:33:57.994 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-03-26T03:33:57.994 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-26T03:33:57.994 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-26T03:33:57.995 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-26T03:33:57.995 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-26T03:33:57.995 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-26T03:33:57.995 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-26T03:33:57.995 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "restful" 2024-03-26T03:33:57.995 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout ], 2024-03-26T03:33:57.995 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T03:33:57.995 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-03-26T03:33:57.995 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-26T03:33:57.995 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T03:33:57.996 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "modified": "2024-03-26T03:33:45.388974+0000", 2024-03-26T03:33:57.996 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T03:33:57.996 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-03-26T03:33:57.996 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-26T03:33:57.996 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout } 2024-03-26T03:33:57.996 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.334+0000 7f29b479c700 1 Processor -- start 2024-03-26T03:33:57.996 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.334+0000 7f29b479c700 1 -- start start 2024-03-26T03:33:57.996 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.334+0000 7f29b479c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29ac104c90 0x7f29ac1050b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:33:57.996 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.334+0000 7f29b479c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f29ac079e50 con 0x7f29ac104c90 2024-03-26T03:33:57.996 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.334+0000 7f29b2538700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29ac104c90 0x7f29ac1050b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:33:57.996 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.334+0000 7f29b2538700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29ac104c90 0x7f29ac1050b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:51860/0 (socket says 172.21.15.175:51860) 2024-03-26T03:33:57.996 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.334+0000 7f29b2538700 1 -- 172.21.15.175:0/2314751939 learned_addr learned my addr 172.21.15.175:0/2314751939 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:33:57.996 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.334+0000 7f29b2538700 1 -- 172.21.15.175:0/2314751939 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29ac1055f0 con 0x7f29ac104c90 2024-03-26T03:33:57.996 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.334+0000 7f29b2538700 1 --2- 172.21.15.175:0/2314751939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29ac104c90 0x7f29ac1050b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f299c0098a0 tx=0x7f299c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26988235b2b7aeda server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:33:57.996 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.334+0000 7f29b1536700 1 -- 172.21.15.175:0/2314751939 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f299c004160 con 0x7f29ac104c90 2024-03-26T03:33:57.997 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.334+0000 7f29b1536700 1 -- 172.21.15.175:0/2314751939 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f299c004300 con 0x7f29ac104c90 2024-03-26T03:33:57.997 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.334+0000 7f29b1536700 1 -- 172.21.15.175:0/2314751939 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f299c011940 con 0x7f29ac104c90 2024-03-26T03:33:57.997 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.334+0000 7f29b479c700 1 -- 172.21.15.175:0/2314751939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29ac104c90 msgr2=0x7f29ac1050b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:33:57.997 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.334+0000 7f29b479c700 1 --2- 172.21.15.175:0/2314751939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29ac104c90 0x7f29ac1050b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f299c0098a0 tx=0x7f299c009bb0 comp rx=0 tx=0).stop 2024-03-26T03:33:57.997 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.334+0000 7f29b479c700 1 -- 172.21.15.175:0/2314751939 shutdown_connections 2024-03-26T03:33:57.997 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.334+0000 7f29b479c700 1 --2- 172.21.15.175:0/2314751939 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29ac104c90 0x7f29ac1050b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:33:57.997 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.334+0000 7f29b479c700 1 -- 172.21.15.175:0/2314751939 >> 172.21.15.175:0/2314751939 conn(0x7f29ac100810 msgr2=0x7f29ac102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:33:57.997 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.334+0000 7f29b479c700 1 -- 172.21.15.175:0/2314751939 shutdown_connections 2024-03-26T03:33:57.997 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.334+0000 7f29b479c700 1 -- 172.21.15.175:0/2314751939 wait complete. 2024-03-26T03:33:57.997 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.334+0000 7f29b479c700 1 Processor -- start 2024-03-26T03:33:57.997 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.334+0000 7f29b479c700 1 -- start start 2024-03-26T03:33:57.997 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.334+0000 7f29b479c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29ac104c90 0x7f29ac19c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:33:57.997 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.334+0000 7f29b479c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f29ac106560 con 0x7f29ac104c90 2024-03-26T03:33:57.998 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.338+0000 7f29b2538700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29ac104c90 0x7f29ac19c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:33:57.998 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.338+0000 7f29b2538700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29ac104c90 0x7f29ac19c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:51876/0 (socket says 172.21.15.175:51876) 2024-03-26T03:33:57.998 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.338+0000 7f29b2538700 1 -- 172.21.15.175:0/720680630 learned_addr learned my addr 172.21.15.175:0/720680630 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:33:57.998 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.338+0000 7f29b2538700 1 -- 172.21.15.175:0/720680630 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29ac19c950 con 0x7f29ac104c90 2024-03-26T03:33:57.998 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.338+0000 7f29b2538700 1 --2- 172.21.15.175:0/720680630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29ac104c90 0x7f29ac19c410 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f299c009290 tx=0x7f299c0119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:33:57.998 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.338+0000 7f29a37fe700 1 -- 172.21.15.175:0/720680630 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f299c003930 con 0x7f29ac104c90 2024-03-26T03:33:57.998 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.338+0000 7f29a37fe700 1 -- 172.21.15.175:0/720680630 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f299c003ad0 con 0x7f29ac104c90 2024-03-26T03:33:57.998 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.338+0000 7f29b479c700 1 -- 172.21.15.175:0/720680630 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29ac19cbe0 con 0x7f29ac104c90 2024-03-26T03:33:57.999 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.338+0000 7f29a37fe700 1 -- 172.21.15.175:0/720680630 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f299c019720 con 0x7f29ac104c90 2024-03-26T03:33:57.999 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.338+0000 7f29b479c700 1 -- 172.21.15.175:0/720680630 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29ac19f8d0 con 0x7f29ac104c90 2024-03-26T03:33:57.999 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.338+0000 7f29a37fe700 1 -- 172.21.15.175:0/720680630 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f299c02d440 con 0x7f29ac104c90 2024-03-26T03:33:57.999 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.338+0000 7f29a37fe700 1 -- 172.21.15.175:0/720680630 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f299c024780 con 0x7f29ac104c90 2024-03-26T03:33:57.999 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.338+0000 7f29b479c700 1 -- 172.21.15.175:0/720680630 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29ac063c70 con 0x7f29ac104c90 2024-03-26T03:33:57.999 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.342+0000 7f29a37fe700 1 -- 172.21.15.175:0/720680630 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f299c043dd0 con 0x7f29ac104c90 2024-03-26T03:33:57.999 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.470+0000 7f29b479c700 1 -- 172.21.15.175:0/720680630 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f29ac02ddd0 con 0x7f29ac104c90 2024-03-26T03:33:57.999 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.470+0000 7f29a37fe700 1 -- 172.21.15.175:0/720680630 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f299c032030 con 0x7f29ac104c90 2024-03-26T03:33:57.999 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.474+0000 7f29b479c700 1 -- 172.21.15.175:0/720680630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29ac104c90 msgr2=0x7f29ac19c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:33:57.999 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.474+0000 7f29b479c700 1 --2- 172.21.15.175:0/720680630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29ac104c90 0x7f29ac19c410 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f299c009290 tx=0x7f299c0119c0 comp rx=0 tx=0).stop 2024-03-26T03:33:57.999 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.474+0000 7f29b479c700 1 -- 172.21.15.175:0/720680630 shutdown_connections 2024-03-26T03:33:57.999 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.474+0000 7f29b479c700 1 --2- 172.21.15.175:0/720680630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29ac104c90 0x7f29ac19c410 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:33:57.999 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.474+0000 7f29b479c700 1 -- 172.21.15.175:0/720680630 >> 172.21.15.175:0/720680630 conn(0x7f29ac100810 msgr2=0x7f29ac1906e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:33:57.999 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.474+0000 7f29b479c700 1 -- 172.21.15.175:0/720680630 shutdown_connections 2024-03-26T03:33:57.999 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:33:57.474+0000 7f29b479c700 1 -- 172.21.15.175:0/720680630 wait complete. 2024-03-26T03:33:58.000 INFO:teuthology.orchestra.run.smithi175.stdout:mgr not available, waiting (2/15)... 2024-03-26T03:33:58.140 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:33:57 smithi175 bash[21783]: debug 2024-03-26T03:33:57.874+0000 7f6327f29200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T03:33:58.494 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:33:58 smithi175 bash[21783]: debug 2024-03-26T03:33:58.138+0000 7f6327f29200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T03:33:58.494 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:33:58 smithi175 bash[21783]: debug 2024-03-26T03:33:58.250+0000 7f6327f29200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T03:33:58.932 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:33:58 smithi175 bash[21783]: debug 2024-03-26T03:33:58.518+0000 7f6327f29200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T03:33:59.244 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:33:58 smithi175 bash[21783]: debug 2024-03-26T03:33:58.930+0000 7f6327f29200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T03:33:59.244 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:33:59 smithi175 bash[21783]: debug 2024-03-26T03:33:59.162+0000 7f6327f29200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T03:33:59.744 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:33:59 smithi175 bash[21783]: debug 2024-03-26T03:33:59.278+0000 7f6327f29200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T03:33:59.744 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:33:59 smithi175 bash[21783]: debug 2024-03-26T03:33:59.522+0000 7f6327f29200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T03:34:00.949 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:00 smithi175 bash[21783]: debug 2024-03-26T03:34:00.526+0000 7f6327f29200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T03:34:00.950 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:00 smithi175 bash[21783]: debug 2024-03-26T03:34:00.642+0000 7f6327f29200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T03:34:01.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:01 smithi175 bash[21527]: audit 2024-03-26T03:34:00.969444+0000 mon.a (mon.0) 15 : audit [DBG] from='client.? 172.21.15.175:0/1330497970' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-26T03:34:01.245 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:00 smithi175 bash[21783]: debug 2024-03-26T03:34:00.946+0000 7f6327f29200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T03:34:01.245 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:01 smithi175 bash[21783]: debug 2024-03-26T03:34:01.062+0000 7f6327f29200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T03:34:01.566 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 2024-03-26T03:34:01.566 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout { 2024-03-26T03:34:01.566 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "fsid": "762e930c-eb21-11ee-95d0-87774f69a715", 2024-03-26T03:34:01.566 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "health": { 2024-03-26T03:34:01.566 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-26T03:34:01.566 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-26T03:34:01.567 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-26T03:34:01.567 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-03-26T03:34:01.567 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-26T03:34:01.567 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-26T03:34:01.567 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 0 2024-03-26T03:34:01.567 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout ], 2024-03-26T03:34:01.567 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-26T03:34:01.567 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "a" 2024-03-26T03:34:01.567 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout ], 2024-03-26T03:34:01.567 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-03-26T03:34:01.567 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-26T03:34:01.567 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T03:34:01.568 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-26T03:34:01.568 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-26T03:34:01.568 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-03-26T03:34:01.568 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-26T03:34:01.568 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T03:34:01.568 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-26T03:34:01.568 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-26T03:34:01.568 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-26T03:34:01.577 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-26T03:34:01.577 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-26T03:34:01.577 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-26T03:34:01.577 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-03-26T03:34:01.578 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-26T03:34:01.578 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-26T03:34:01.578 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-26T03:34:01.578 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-26T03:34:01.578 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-26T03:34:01.578 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-26T03:34:01.578 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-26T03:34:01.578 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-26T03:34:01.578 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-26T03:34:01.578 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-03-26T03:34:01.578 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-26T03:34:01.578 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T03:34:01.579 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-26T03:34:01.579 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-26T03:34:01.579 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-03-26T03:34:01.579 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-26T03:34:01.579 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-26T03:34:01.579 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-26T03:34:01.579 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-26T03:34:01.579 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-26T03:34:01.579 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-26T03:34:01.579 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "restful" 2024-03-26T03:34:01.579 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout ], 2024-03-26T03:34:01.579 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T03:34:01.579 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-03-26T03:34:01.579 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-26T03:34:01.580 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T03:34:01.580 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "modified": "2024-03-26T03:33:45.388974+0000", 2024-03-26T03:34:01.580 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T03:34:01.580 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-03-26T03:34:01.580 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-26T03:34:01.580 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout } 2024-03-26T03:34:01.580 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.830+0000 7f5fa67b4700 1 Processor -- start 2024-03-26T03:34:01.580 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.830+0000 7f5fa67b4700 1 -- start start 2024-03-26T03:34:01.580 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.830+0000 7f5fa67b4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fa01063b0 0x7f5fa01067d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:01.580 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.830+0000 7f5fa67b4700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa0106da0 con 0x7f5fa01063b0 2024-03-26T03:34:01.580 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.830+0000 7f5f9ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fa01063b0 0x7f5fa01067d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:01.580 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.830+0000 7f5f9ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fa01063b0 0x7f5fa01067d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:51886/0 (socket says 172.21.15.175:51886) 2024-03-26T03:34:01.580 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.830+0000 7f5f9ffff700 1 -- 172.21.15.175:0/2063254038 learned_addr learned my addr 172.21.15.175:0/2063254038 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:01.580 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.830+0000 7f5f9ffff700 1 -- 172.21.15.175:0/2063254038 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa01075b0 con 0x7f5fa01063b0 2024-03-26T03:34:01.581 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.830+0000 7f5f9ffff700 1 --2- 172.21.15.175:0/2063254038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fa01063b0 0x7f5fa01067d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5f90009e80 tx=0x7f5f9000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210d71c0ef190858 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:01.581 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.830+0000 7f5f9effd700 1 -- 172.21.15.175:0/2063254038 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f90017070 con 0x7f5fa01063b0 2024-03-26T03:34:01.581 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.830+0000 7f5f9effd700 1 -- 172.21.15.175:0/2063254038 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f5f9000b8c0 con 0x7f5fa01063b0 2024-03-26T03:34:01.581 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.830+0000 7f5f9effd700 1 -- 172.21.15.175:0/2063254038 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f90004370 con 0x7f5fa01063b0 2024-03-26T03:34:01.581 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.834+0000 7f5fa67b4700 1 -- 172.21.15.175:0/2063254038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fa01063b0 msgr2=0x7f5fa01067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:01.581 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.834+0000 7f5fa67b4700 1 --2- 172.21.15.175:0/2063254038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fa01063b0 0x7f5fa01067d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5f90009e80 tx=0x7f5f9000b330 comp rx=0 tx=0).stop 2024-03-26T03:34:01.581 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.834+0000 7f5fa67b4700 1 -- 172.21.15.175:0/2063254038 shutdown_connections 2024-03-26T03:34:01.581 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.834+0000 7f5fa67b4700 1 --2- 172.21.15.175:0/2063254038 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fa01063b0 0x7f5fa01067d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:01.581 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.834+0000 7f5fa67b4700 1 -- 172.21.15.175:0/2063254038 >> 172.21.15.175:0/2063254038 conn(0x7f5fa0101910 msgr2=0x7f5fa0103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:01.581 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.834+0000 7f5fa67b4700 1 -- 172.21.15.175:0/2063254038 shutdown_connections 2024-03-26T03:34:01.581 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.834+0000 7f5fa67b4700 1 -- 172.21.15.175:0/2063254038 wait complete. 2024-03-26T03:34:01.581 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.834+0000 7f5fa67b4700 1 Processor -- start 2024-03-26T03:34:01.581 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.834+0000 7f5fa67b4700 1 -- start start 2024-03-26T03:34:01.581 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.834+0000 7f5fa67b4700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fa01063b0 0x7f5fa007c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:01.582 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.834+0000 7f5fa67b4700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa010a7e0 con 0x7f5fa01063b0 2024-03-26T03:34:01.582 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.834+0000 7f5f9ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fa01063b0 0x7f5fa007c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:01.582 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.834+0000 7f5f9ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fa01063b0 0x7f5fa007c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:51902/0 (socket says 172.21.15.175:51902) 2024-03-26T03:34:01.582 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.834+0000 7f5f9ffff700 1 -- 172.21.15.175:0/1330497970 learned_addr learned my addr 172.21.15.175:0/1330497970 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:01.582 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.834+0000 7f5f9ffff700 1 -- 172.21.15.175:0/1330497970 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa007c910 con 0x7f5fa01063b0 2024-03-26T03:34:01.582 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.834+0000 7f5f9ffff700 1 --2- 172.21.15.175:0/1330497970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fa01063b0 0x7f5fa007c3d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5f900040e0 tx=0x7f5f90003c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:01.582 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.834+0000 7f5f9d7fa700 1 -- 172.21.15.175:0/1330497970 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f90021020 con 0x7f5fa01063b0 2024-03-26T03:34:01.582 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.834+0000 7f5f9d7fa700 1 -- 172.21.15.175:0/1330497970 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f5f9000bde0 con 0x7f5fa01063b0 2024-03-26T03:34:01.582 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.834+0000 7f5fa67b4700 1 -- 172.21.15.175:0/1330497970 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fa007ab50 con 0x7f5fa01063b0 2024-03-26T03:34:01.582 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.834+0000 7f5f9d7fa700 1 -- 172.21.15.175:0/1330497970 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f90017040 con 0x7f5fa01063b0 2024-03-26T03:34:01.582 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.834+0000 7f5fa67b4700 1 -- 172.21.15.175:0/1330497970 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fa007b090 con 0x7f5fa01063b0 2024-03-26T03:34:01.582 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.834+0000 7f5f9d7fa700 1 -- 172.21.15.175:0/1330497970 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5f90024020 con 0x7f5fa01063b0 2024-03-26T03:34:01.582 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.834+0000 7f5f9d7fa700 1 -- 172.21.15.175:0/1330497970 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5f9000f050 con 0x7f5fa01063b0 2024-03-26T03:34:01.582 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.834+0000 7f5f96ffd700 1 -- 172.21.15.175:0/1330497970 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f84005120 con 0x7f5fa01063b0 2024-03-26T03:34:01.583 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.838+0000 7f5f9d7fa700 1 -- 172.21.15.175:0/1330497970 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f5f9002b020 con 0x7f5fa01063b0 2024-03-26T03:34:01.583 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.966+0000 7f5f96ffd700 1 -- 172.21.15.175:0/1330497970 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5f840056e0 con 0x7f5fa01063b0 2024-03-26T03:34:01.583 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.966+0000 7f5f9d7fa700 1 -- 172.21.15.175:0/1330497970 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f5f90026df0 con 0x7f5fa01063b0 2024-03-26T03:34:01.583 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.966+0000 7f5f96ffd700 1 -- 172.21.15.175:0/1330497970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fa01063b0 msgr2=0x7f5fa007c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:01.583 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.966+0000 7f5f96ffd700 1 --2- 172.21.15.175:0/1330497970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fa01063b0 0x7f5fa007c3d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5f900040e0 tx=0x7f5f90003c50 comp rx=0 tx=0).stop 2024-03-26T03:34:01.583 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.966+0000 7f5f96ffd700 1 -- 172.21.15.175:0/1330497970 shutdown_connections 2024-03-26T03:34:01.583 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.966+0000 7f5f96ffd700 1 --2- 172.21.15.175:0/1330497970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fa01063b0 0x7f5fa007c3d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:01.583 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.966+0000 7f5f96ffd700 1 -- 172.21.15.175:0/1330497970 >> 172.21.15.175:0/1330497970 conn(0x7f5fa0101910 msgr2=0x7f5fa0102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:01.583 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.966+0000 7f5f96ffd700 1 -- 172.21.15.175:0/1330497970 shutdown_connections 2024-03-26T03:34:01.583 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:00.966+0000 7f5f96ffd700 1 -- 172.21.15.175:0/1330497970 wait complete. 2024-03-26T03:34:01.583 INFO:teuthology.orchestra.run.smithi175.stdout:mgr not available, waiting (3/15)... 2024-03-26T03:34:01.994 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:01 smithi175 bash[21783]: debug 2024-03-26T03:34:01.510+0000 7f6327f29200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T03:34:01.994 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:01 smithi175 bash[21783]: debug 2024-03-26T03:34:01.630+0000 7f6327f29200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T03:34:02.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:02 smithi175 bash[21527]: cluster 2024-03-26T03:34:01.653942+0000 mon.a (mon.0) 16 : cluster [INF] Activating manager daemon y 2024-03-26T03:34:02.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:02 smithi175 bash[21527]: cluster 2024-03-26T03:34:01.660481+0000 mon.a (mon.0) 17 : cluster [DBG] mgrmap e2: y(active, starting, since 0.00703352s) 2024-03-26T03:34:02.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:02 smithi175 bash[21527]: audit 2024-03-26T03:34:01.663124+0000 mon.a (mon.0) 18 : audit [DBG] from='mgr.14100 172.21.15.175:0/521947513' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T03:34:02.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:02 smithi175 bash[21527]: audit 2024-03-26T03:34:01.663683+0000 mon.a (mon.0) 19 : audit [DBG] from='mgr.14100 172.21.15.175:0/521947513' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T03:34:02.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:02 smithi175 bash[21527]: audit 2024-03-26T03:34:01.664096+0000 mon.a (mon.0) 20 : audit [DBG] from='mgr.14100 172.21.15.175:0/521947513' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T03:34:02.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:02 smithi175 bash[21527]: audit 2024-03-26T03:34:01.664479+0000 mon.a (mon.0) 21 : audit [DBG] from='mgr.14100 172.21.15.175:0/521947513' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T03:34:02.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:02 smithi175 bash[21527]: audit 2024-03-26T03:34:01.664905+0000 mon.a (mon.0) 22 : audit [DBG] from='mgr.14100 172.21.15.175:0/521947513' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T03:34:02.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:02 smithi175 bash[21527]: cluster 2024-03-26T03:34:01.693444+0000 mon.a (mon.0) 23 : cluster [INF] Manager daemon y is now available 2024-03-26T03:34:02.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:02 smithi175 bash[21527]: audit 2024-03-26T03:34:01.705910+0000 mon.a (mon.0) 24 : audit [INF] from='mgr.14100 172.21.15.175:0/521947513' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-26T03:34:02.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:02 smithi175 bash[21527]: audit 2024-03-26T03:34:01.708373+0000 mon.a (mon.0) 25 : audit [INF] from='mgr.14100 172.21.15.175:0/521947513' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-26T03:34:02.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:02 smithi175 bash[21527]: audit 2024-03-26T03:34:01.711238+0000 mon.a (mon.0) 26 : audit [INF] from='mgr.14100 172.21.15.175:0/521947513' entity='mgr.y' 2024-03-26T03:34:02.496 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:02 smithi175 bash[21527]: audit 2024-03-26T03:34:01.716657+0000 mon.a (mon.0) 27 : audit [INF] from='mgr.14100 172.21.15.175:0/521947513' entity='mgr.y' 2024-03-26T03:34:02.496 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:02 smithi175 bash[21527]: audit 2024-03-26T03:34:01.721951+0000 mon.a (mon.0) 28 : audit [INF] from='mgr.14100 172.21.15.175:0/521947513' entity='mgr.y' 2024-03-26T03:34:03.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:03 smithi175 bash[21527]: cluster 2024-03-26T03:34:02.668955+0000 mon.a (mon.0) 29 : cluster [DBG] mgrmap e3: y(active, since 1.0155s) 2024-03-26T03:34:05.623 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 2024-03-26T03:34:05.623 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout { 2024-03-26T03:34:05.623 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "fsid": "762e930c-eb21-11ee-95d0-87774f69a715", 2024-03-26T03:34:05.623 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "health": { 2024-03-26T03:34:05.623 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-26T03:34:05.623 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-26T03:34:05.623 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-26T03:34:05.623 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-03-26T03:34:05.624 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-26T03:34:05.624 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-26T03:34:05.624 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 0 2024-03-26T03:34:05.624 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout ], 2024-03-26T03:34:05.624 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-26T03:34:05.624 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "a" 2024-03-26T03:34:05.624 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout ], 2024-03-26T03:34:05.624 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-03-26T03:34:05.624 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-26T03:34:05.624 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T03:34:05.624 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-26T03:34:05.624 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-26T03:34:05.624 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-03-26T03:34:05.624 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-26T03:34:05.624 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T03:34:05.624 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-26T03:34:05.625 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-26T03:34:05.625 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-26T03:34:05.634 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-26T03:34:05.634 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-26T03:34:05.634 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-26T03:34:05.634 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-03-26T03:34:05.634 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-26T03:34:05.634 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-26T03:34:05.634 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-26T03:34:05.634 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-26T03:34:05.634 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-26T03:34:05.635 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-26T03:34:05.635 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-26T03:34:05.635 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-26T03:34:05.635 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-26T03:34:05.635 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-03-26T03:34:05.635 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-26T03:34:05.635 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T03:34:05.635 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-26T03:34:05.635 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-26T03:34:05.635 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-03-26T03:34:05.635 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-26T03:34:05.636 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-26T03:34:05.636 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-26T03:34:05.636 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-26T03:34:05.636 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-26T03:34:05.636 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-26T03:34:05.636 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "restful" 2024-03-26T03:34:05.636 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout ], 2024-03-26T03:34:05.636 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T03:34:05.636 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-03-26T03:34:05.636 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-26T03:34:05.636 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T03:34:05.636 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "modified": "2024-03-26T03:33:45.388974+0000", 2024-03-26T03:34:05.636 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T03:34:05.636 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout }, 2024-03-26T03:34:05.636 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-26T03:34:05.637 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout } 2024-03-26T03:34:05.637 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.529+0000 7f7f3db12700 1 Processor -- start 2024-03-26T03:34:05.637 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.529+0000 7f7f3db12700 1 -- start start 2024-03-26T03:34:05.637 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.529+0000 7f7f3db12700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f38080bc0 0x7f7f3810f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:05.637 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.529+0000 7f7f3db12700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3810fb60 con 0x7f7f38080bc0 2024-03-26T03:34:05.637 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.529+0000 7f7f377fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f38080bc0 0x7f7f3810f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:05.637 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.529+0000 7f7f377fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f38080bc0 0x7f7f3810f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:42844/0 (socket says 172.21.15.175:42844) 2024-03-26T03:34:05.637 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.529+0000 7f7f377fe700 1 -- 172.21.15.175:0/2272147803 learned_addr learned my addr 172.21.15.175:0/2272147803 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:05.637 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.529+0000 7f7f377fe700 1 -- 172.21.15.175:0/2272147803 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f3810fce0 con 0x7f7f38080bc0 2024-03-26T03:34:05.637 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.529+0000 7f7f377fe700 1 --2- 172.21.15.175:0/2272147803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f38080bc0 0x7f7f3810f590 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7f200098a0 tx=0x7f7f20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69790ad76d8b1d3a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:05.637 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.529+0000 7f7f367fc700 1 -- 172.21.15.175:0/2272147803 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f20004160 con 0x7f7f38080bc0 2024-03-26T03:34:05.637 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.529+0000 7f7f367fc700 1 -- 172.21.15.175:0/2272147803 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f7f20004300 con 0x7f7f38080bc0 2024-03-26T03:34:05.637 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.529+0000 7f7f367fc700 1 -- 172.21.15.175:0/2272147803 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f20011940 con 0x7f7f38080bc0 2024-03-26T03:34:05.637 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.533+0000 7f7f3db12700 1 -- 172.21.15.175:0/2272147803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f38080bc0 msgr2=0x7f7f3810f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:05.638 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.533+0000 7f7f3db12700 1 --2- 172.21.15.175:0/2272147803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f38080bc0 0x7f7f3810f590 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7f200098a0 tx=0x7f7f20009bb0 comp rx=0 tx=0).stop 2024-03-26T03:34:05.638 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.533+0000 7f7f3db12700 1 -- 172.21.15.175:0/2272147803 shutdown_connections 2024-03-26T03:34:05.638 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.533+0000 7f7f3db12700 1 --2- 172.21.15.175:0/2272147803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f38080bc0 0x7f7f3810f590 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:05.638 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.533+0000 7f7f3db12700 1 -- 172.21.15.175:0/2272147803 >> 172.21.15.175:0/2272147803 conn(0x7f7f3807bc10 msgr2=0x7f7f3807e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:05.638 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.533+0000 7f7f3db12700 1 -- 172.21.15.175:0/2272147803 shutdown_connections 2024-03-26T03:34:05.638 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.533+0000 7f7f3db12700 1 -- 172.21.15.175:0/2272147803 wait complete. 2024-03-26T03:34:05.638 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.533+0000 7f7f3db12700 1 Processor -- start 2024-03-26T03:34:05.638 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.533+0000 7f7f3db12700 1 -- start start 2024-03-26T03:34:05.638 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.533+0000 7f7f3db12700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f381a0ae0 0x7f7f381a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:05.638 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.533+0000 7f7f3db12700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7f38110ca0 con 0x7f7f381a0ae0 2024-03-26T03:34:05.638 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.533+0000 7f7f377fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f381a0ae0 0x7f7f381a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:05.638 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.533+0000 7f7f377fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f381a0ae0 0x7f7f381a0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:42856/0 (socket says 172.21.15.175:42856) 2024-03-26T03:34:05.638 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.533+0000 7f7f377fe700 1 -- 172.21.15.175:0/2904213713 learned_addr learned my addr 172.21.15.175:0/2904213713 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:05.638 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.533+0000 7f7f377fe700 1 -- 172.21.15.175:0/2904213713 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f381a1440 con 0x7f7f381a0ae0 2024-03-26T03:34:05.639 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.533+0000 7f7f377fe700 1 --2- 172.21.15.175:0/2904213713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f381a0ae0 0x7f7f381a0f00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7f20000c00 tx=0x7f7f200119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:05.639 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.533+0000 7f7f34ff9700 1 -- 172.21.15.175:0/2904213713 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f20003b20 con 0x7f7f381a0ae0 2024-03-26T03:34:05.639 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.533+0000 7f7f34ff9700 1 -- 172.21.15.175:0/2904213713 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f7f2001a890 con 0x7f7f381a0ae0 2024-03-26T03:34:05.639 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.533+0000 7f7f3db12700 1 -- 172.21.15.175:0/2904213713 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f381a16d0 con 0x7f7f381a0ae0 2024-03-26T03:34:05.639 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.533+0000 7f7f34ff9700 1 -- 172.21.15.175:0/2904213713 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f20024890 con 0x7f7f381a0ae0 2024-03-26T03:34:05.639 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.533+0000 7f7f3db12700 1 -- 172.21.15.175:0/2904213713 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f381a41b0 con 0x7f7f381a0ae0 2024-03-26T03:34:05.639 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.533+0000 7f7f34ff9700 1 -- 172.21.15.175:0/2904213713 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 3) ==== 46349+0+0 (secure 0 0 0) 0x7f7f20022020 con 0x7f7f381a0ae0 2024-03-26T03:34:05.639 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.533+0000 7f7f34ff9700 1 --2- 172.21.15.175:0/2904213713 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7f7f24039370 0x7f7f2403b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:05.639 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.533+0000 7f7f34ff9700 1 -- 172.21.15.175:0/2904213713 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7f2004d510 con 0x7f7f381a0ae0 2024-03-26T03:34:05.639 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.533+0000 7f7f36ffd700 1 --2- 172.21.15.175:0/2904213713 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7f7f24039370 0x7f7f2403b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:05.639 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.533+0000 7f7f3db12700 1 -- 172.21.15.175:0/2904213713 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f38063c70 con 0x7f7f381a0ae0 2024-03-26T03:34:05.639 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.537+0000 7f7f36ffd700 1 --2- 172.21.15.175:0/2904213713 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7f7f24039370 0x7f7f2403b830 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7f28006fd0 tx=0x7f7f28006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:05.639 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.541+0000 7f7f34ff9700 1 -- 172.21.15.175:0/2904213713 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f7f2000b3e0 con 0x7f7f381a0ae0 2024-03-26T03:34:05.639 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:04.661+0000 7f7f34ff9700 1 -- 172.21.15.175:0/2904213713 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mgrmap(e 4) ==== 46455+0+0 (secure 0 0 0) 0x7f7f2002daa0 con 0x7f7f381a0ae0 2024-03-26T03:34:05.640 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:05.057+0000 7f7f3db12700 1 -- 172.21.15.175:0/2904213713 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7f3806bd50 con 0x7f7f381a0ae0 2024-03-26T03:34:05.640 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:05.057+0000 7f7f34ff9700 1 -- 172.21.15.175:0/2904213713 <== mon.0 v2:172.21.15.175:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f7f20019de0 con 0x7f7f381a0ae0 2024-03-26T03:34:05.640 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:05.061+0000 7f7f3db12700 1 -- 172.21.15.175:0/2904213713 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7f7f24039370 msgr2=0x7f7f2403b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:05.640 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:05.061+0000 7f7f3db12700 1 --2- 172.21.15.175:0/2904213713 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7f7f24039370 0x7f7f2403b830 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7f28006fd0 tx=0x7f7f28006e40 comp rx=0 tx=0).stop 2024-03-26T03:34:05.640 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:05.061+0000 7f7f3db12700 1 -- 172.21.15.175:0/2904213713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f381a0ae0 msgr2=0x7f7f381a0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:05.640 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:05.061+0000 7f7f3db12700 1 --2- 172.21.15.175:0/2904213713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f381a0ae0 0x7f7f381a0f00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7f20000c00 tx=0x7f7f200119c0 comp rx=0 tx=0).stop 2024-03-26T03:34:05.640 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:05.061+0000 7f7f3db12700 1 -- 172.21.15.175:0/2904213713 shutdown_connections 2024-03-26T03:34:05.640 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:05.061+0000 7f7f3db12700 1 --2- 172.21.15.175:0/2904213713 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7f7f24039370 0x7f7f2403b830 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:05.640 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:05.061+0000 7f7f3db12700 1 --2- 172.21.15.175:0/2904213713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f381a0ae0 0x7f7f381a0f00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:05.640 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:05.061+0000 7f7f3db12700 1 -- 172.21.15.175:0/2904213713 >> 172.21.15.175:0/2904213713 conn(0x7f7f3807bc10 msgr2=0x7f7f3807d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:05.640 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:05.061+0000 7f7f3db12700 1 -- 172.21.15.175:0/2904213713 shutdown_connections 2024-03-26T03:34:05.640 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:05.061+0000 7f7f3db12700 1 -- 172.21.15.175:0/2904213713 wait complete. 2024-03-26T03:34:05.640 INFO:teuthology.orchestra.run.smithi175.stdout:mgr is available 2024-03-26T03:34:05.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:05 smithi175 bash[21527]: cluster 2024-03-26T03:34:04.667703+0000 mon.a (mon.0) 30 : cluster [DBG] mgrmap e4: y(active, since 3s) 2024-03-26T03:34:05.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:05 smithi175 bash[21527]: audit 2024-03-26T03:34:05.061261+0000 mon.a (mon.0) 31 : audit [DBG] from='client.? 172.21.15.175:0/2904213713' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-26T03:34:07.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:06 smithi175 bash[21527]: audit 2024-03-26T03:34:06.851867+0000 mon.a (mon.0) 32 : audit [INF] from='client.? 172.21.15.175:0/302419382' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-26T03:34:07.450 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 2024-03-26T03:34:07.450 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout [global] 2024-03-26T03:34:07.450 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout fsid = 762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T03:34:07.450 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-26T03:34:07.450 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-26T03:34:07.450 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-26T03:34:07.450 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-26T03:34:07.451 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 2024-03-26T03:34:07.451 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-26T03:34:07.451 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-26T03:34:07.451 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 2024-03-26T03:34:07.451 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout [osd] 2024-03-26T03:34:07.451 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-26T03:34:07.451 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-26T03:34:07.451 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.465+0000 7f0bd9821700 1 Processor -- start 2024-03-26T03:34:07.451 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.465+0000 7f0bd9821700 1 -- start start 2024-03-26T03:34:07.451 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.465+0000 7f0bd9821700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bd4108600 0x7f0bd4108a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:07.451 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.465+0000 7f0bd9821700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd4108ff0 con 0x7f0bd4108600 2024-03-26T03:34:07.451 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.465+0000 7f0bd2ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bd4108600 0x7f0bd4108a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:07.451 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.465+0000 7f0bd2ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bd4108600 0x7f0bd4108a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:42872/0 (socket says 172.21.15.175:42872) 2024-03-26T03:34:07.451 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.465+0000 7f0bd2ffd700 1 -- 172.21.15.175:0/3612795608 learned_addr learned my addr 172.21.15.175:0/3612795608 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:07.452 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.465+0000 7f0bd2ffd700 1 -- 172.21.15.175:0/3612795608 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd41097b0 con 0x7f0bd4108600 2024-03-26T03:34:07.452 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.465+0000 7f0bd2ffd700 1 --2- 172.21.15.175:0/3612795608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bd4108600 0x7f0bd4108a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40098a0 tx=0x7f0bc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66e5489b12b1e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:07.452 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.465+0000 7f0bd1ffb700 1 -- 172.21.15.175:0/3612795608 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bc4004160 con 0x7f0bd4108600 2024-03-26T03:34:07.452 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.465+0000 7f0bd1ffb700 1 -- 172.21.15.175:0/3612795608 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f0bc4004300 con 0x7f0bd4108600 2024-03-26T03:34:07.452 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.465+0000 7f0bd1ffb700 1 -- 172.21.15.175:0/3612795608 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bc4011940 con 0x7f0bd4108600 2024-03-26T03:34:07.452 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.465+0000 7f0bd9821700 1 -- 172.21.15.175:0/3612795608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bd4108600 msgr2=0x7f0bd4108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:07.452 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.465+0000 7f0bd9821700 1 --2- 172.21.15.175:0/3612795608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bd4108600 0x7f0bd4108a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40098a0 tx=0x7f0bc4009bb0 comp rx=0 tx=0).stop 2024-03-26T03:34:07.452 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.465+0000 7f0bd9821700 1 -- 172.21.15.175:0/3612795608 shutdown_connections 2024-03-26T03:34:07.452 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.465+0000 7f0bd9821700 1 --2- 172.21.15.175:0/3612795608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bd4108600 0x7f0bd4108a20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:07.452 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.465+0000 7f0bd9821700 1 -- 172.21.15.175:0/3612795608 >> 172.21.15.175:0/3612795608 conn(0x7f0bd407ba10 msgr2=0x7f0bd407be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:07.452 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.465+0000 7f0bd9821700 1 -- 172.21.15.175:0/3612795608 shutdown_connections 2024-03-26T03:34:07.452 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.465+0000 7f0bd9821700 1 -- 172.21.15.175:0/3612795608 wait complete. 2024-03-26T03:34:07.452 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.465+0000 7f0bd9821700 1 Processor -- start 2024-03-26T03:34:07.452 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.465+0000 7f0bd9821700 1 -- start start 2024-03-26T03:34:07.452 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.465+0000 7f0bd9821700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bd4108600 0x7f0bd419c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:07.453 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.465+0000 7f0bd9821700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd410c9e0 con 0x7f0bd4108600 2024-03-26T03:34:07.453 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.465+0000 7f0bd2ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bd4108600 0x7f0bd419c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:07.453 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.465+0000 7f0bd2ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bd4108600 0x7f0bd419c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:42888/0 (socket says 172.21.15.175:42888) 2024-03-26T03:34:07.453 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.465+0000 7f0bd2ffd700 1 -- 172.21.15.175:0/302419382 learned_addr learned my addr 172.21.15.175:0/302419382 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:07.453 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.465+0000 7f0bd2ffd700 1 -- 172.21.15.175:0/302419382 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd419cc40 con 0x7f0bd4108600 2024-03-26T03:34:07.453 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.469+0000 7f0bd2ffd700 1 --2- 172.21.15.175:0/302419382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bd4108600 0x7f0bd419c700 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4011910 tx=0x7f0bc40119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:07.453 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.469+0000 7f0bd881f700 1 -- 172.21.15.175:0/302419382 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bc4003b20 con 0x7f0bd4108600 2024-03-26T03:34:07.453 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.469+0000 7f0bd881f700 1 -- 172.21.15.175:0/302419382 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f0bc401a890 con 0x7f0bd4108600 2024-03-26T03:34:07.453 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.469+0000 7f0bd9821700 1 -- 172.21.15.175:0/302419382 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd419ced0 con 0x7f0bd4108600 2024-03-26T03:34:07.453 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.469+0000 7f0bd881f700 1 -- 172.21.15.175:0/302419382 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bc40197f0 con 0x7f0bd4108600 2024-03-26T03:34:07.453 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.469+0000 7f0bd9821700 1 -- 172.21.15.175:0/302419382 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd419fbc0 con 0x7f0bd4108600 2024-03-26T03:34:07.453 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.469+0000 7f0bd881f700 1 -- 172.21.15.175:0/302419382 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 4) ==== 46455+0+0 (secure 0 0 0) 0x7f0bc4022020 con 0x7f0bd4108600 2024-03-26T03:34:07.453 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.469+0000 7f0bd881f700 1 --2- 172.21.15.175:0/302419382 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7f0bbc039440 0x7f0bbc03b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:07.454 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.469+0000 7f0bd9821700 1 -- 172.21.15.175:0/302419382 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bd410cb60 con 0x7f0bd4108600 2024-03-26T03:34:07.454 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.469+0000 7f0bd881f700 1 -- 172.21.15.175:0/302419382 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0bc404d620 con 0x7f0bd4108600 2024-03-26T03:34:07.454 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.469+0000 7f0bd27fc700 1 --2- 172.21.15.175:0/302419382 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7f0bbc039440 0x7f0bbc03b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:07.454 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.469+0000 7f0bd27fc700 1 --2- 172.21.15.175:0/302419382 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7f0bbc039440 0x7f0bbc03b900 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8006fd0 tx=0x7f0bc8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:07.454 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.473+0000 7f0bd881f700 1 -- 172.21.15.175:0/302419382 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f0bc40172b0 con 0x7f0bd4108600 2024-03-26T03:34:07.454 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.849+0000 7f0bd9821700 1 -- 172.21.15.175:0/302419382 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f0bd406bd10 con 0x7f0bd4108600 2024-03-26T03:34:07.454 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.849+0000 7f0bd881f700 1 -- 172.21.15.175:0/302419382 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+282 (secure 0 0 0) 0x7f0bc401aa40 con 0x7f0bd4108600 2024-03-26T03:34:07.454 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.849+0000 7f0bd9821700 1 -- 172.21.15.175:0/302419382 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7f0bbc039440 msgr2=0x7f0bbc03b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:07.454 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.849+0000 7f0bd9821700 1 --2- 172.21.15.175:0/302419382 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7f0bbc039440 0x7f0bbc03b900 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8006fd0 tx=0x7f0bc8006e40 comp rx=0 tx=0).stop 2024-03-26T03:34:07.454 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.849+0000 7f0bd9821700 1 -- 172.21.15.175:0/302419382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bd4108600 msgr2=0x7f0bd419c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:07.454 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.849+0000 7f0bd9821700 1 --2- 172.21.15.175:0/302419382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bd4108600 0x7f0bd419c700 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4011910 tx=0x7f0bc40119f0 comp rx=0 tx=0).stop 2024-03-26T03:34:07.454 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.849+0000 7f0bd9821700 1 -- 172.21.15.175:0/302419382 shutdown_connections 2024-03-26T03:34:07.454 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.849+0000 7f0bd9821700 1 --2- 172.21.15.175:0/302419382 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7f0bbc039440 0x7f0bbc03b900 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:07.454 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.849+0000 7f0bd9821700 1 --2- 172.21.15.175:0/302419382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bd4108600 0x7f0bd419c700 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:07.455 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.849+0000 7f0bd9821700 1 -- 172.21.15.175:0/302419382 >> 172.21.15.175:0/302419382 conn(0x7f0bd407ba10 msgr2=0x7f0bd4105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:07.455 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.853+0000 7f0bd9821700 1 -- 172.21.15.175:0/302419382 shutdown_connections 2024-03-26T03:34:07.455 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:06.853+0000 7f0bd9821700 1 -- 172.21.15.175:0/302419382 wait complete. 2024-03-26T03:34:07.455 INFO:teuthology.orchestra.run.smithi175.stdout:Enabling cephadm module... 2024-03-26T03:34:09.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:08 smithi175 bash[21527]: audit 2024-03-26T03:34:08.726984+0000 mon.a (mon.0) 33 : audit [INF] from='client.? 172.21.15.175:0/2996594251' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-26T03:34:09.245 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:08 smithi175 bash[21783]: ignoring --setuser ceph since I am not root 2024-03-26T03:34:09.245 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:08 smithi175 bash[21783]: ignoring --setgroup ceph since I am not root 2024-03-26T03:34:09.245 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:08 smithi175 bash[21783]: debug 2024-03-26T03:34:08.977+0000 7f286e693200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T03:34:09.403 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.245+0000 7ff9dc56d700 1 Processor -- start 2024-03-26T03:34:09.403 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.245+0000 7ff9dc56d700 1 -- start start 2024-03-26T03:34:09.403 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.249+0000 7ff9dc56d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9d4106460 0x7ff9d4106880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:09.403 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.249+0000 7ff9dc56d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d4106e50 con 0x7ff9d4106460 2024-03-26T03:34:09.403 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.249+0000 7ff9da309700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9d4106460 0x7ff9d4106880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:09.403 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.249+0000 7ff9da309700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9d4106460 0x7ff9d4106880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:42902/0 (socket says 172.21.15.175:42902) 2024-03-26T03:34:09.403 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.249+0000 7ff9da309700 1 -- 172.21.15.175:0/1743987925 learned_addr learned my addr 172.21.15.175:0/1743987925 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:09.403 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.249+0000 7ff9da309700 1 -- 172.21.15.175:0/1743987925 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d41075c0 con 0x7ff9d4106460 2024-03-26T03:34:09.403 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.249+0000 7ff9da309700 1 --2- 172.21.15.175:0/1743987925 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9d4106460 0x7ff9d4106880 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff9d00098a0 tx=0x7ff9d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d823298df8711b3d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:09.403 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.249+0000 7ff9d9307700 1 -- 172.21.15.175:0/1743987925 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9d0004160 con 0x7ff9d4106460 2024-03-26T03:34:09.404 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.249+0000 7ff9d9307700 1 -- 172.21.15.175:0/1743987925 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7ff9d0004300 con 0x7ff9d4106460 2024-03-26T03:34:09.404 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.249+0000 7ff9d9307700 1 -- 172.21.15.175:0/1743987925 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9d0011940 con 0x7ff9d4106460 2024-03-26T03:34:09.404 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.249+0000 7ff9dc56d700 1 -- 172.21.15.175:0/1743987925 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9d4106460 msgr2=0x7ff9d4106880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:09.404 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.249+0000 7ff9dc56d700 1 --2- 172.21.15.175:0/1743987925 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9d4106460 0x7ff9d4106880 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff9d00098a0 tx=0x7ff9d0009bb0 comp rx=0 tx=0).stop 2024-03-26T03:34:09.404 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.249+0000 7ff9dc56d700 1 -- 172.21.15.175:0/1743987925 shutdown_connections 2024-03-26T03:34:09.404 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.249+0000 7ff9dc56d700 1 --2- 172.21.15.175:0/1743987925 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9d4106460 0x7ff9d4106880 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:09.404 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.249+0000 7ff9dc56d700 1 -- 172.21.15.175:0/1743987925 >> 172.21.15.175:0/1743987925 conn(0x7ff9d4101a60 msgr2=0x7ff9d4103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:09.404 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.249+0000 7ff9dc56d700 1 -- 172.21.15.175:0/1743987925 shutdown_connections 2024-03-26T03:34:09.404 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.249+0000 7ff9dc56d700 1 -- 172.21.15.175:0/1743987925 wait complete. 2024-03-26T03:34:09.404 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.249+0000 7ff9dc56d700 1 Processor -- start 2024-03-26T03:34:09.404 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.249+0000 7ff9dc56d700 1 -- start start 2024-03-26T03:34:09.404 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.249+0000 7ff9dc56d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9d4106460 0x7ff9d419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:09.404 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.249+0000 7ff9dc56d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d410a7f0 con 0x7ff9d4106460 2024-03-26T03:34:09.404 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.249+0000 7ff9da309700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9d4106460 0x7ff9d419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:09.404 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.249+0000 7ff9da309700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9d4106460 0x7ff9d419a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:42912/0 (socket says 172.21.15.175:42912) 2024-03-26T03:34:09.405 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.249+0000 7ff9da309700 1 -- 172.21.15.175:0/2996594251 learned_addr learned my addr 172.21.15.175:0/2996594251 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:09.405 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.249+0000 7ff9da309700 1 -- 172.21.15.175:0/2996594251 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d419aa90 con 0x7ff9d4106460 2024-03-26T03:34:09.405 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.249+0000 7ff9da309700 1 --2- 172.21.15.175:0/2996594251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9d4106460 0x7ff9d419a550 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0006ba0 tx=0x7ff9d00119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:09.405 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.249+0000 7ff9cb7fe700 1 -- 172.21.15.175:0/2996594251 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9d0003930 con 0x7ff9d4106460 2024-03-26T03:34:09.405 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.249+0000 7ff9cb7fe700 1 -- 172.21.15.175:0/2996594251 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7ff9d0003ad0 con 0x7ff9d4106460 2024-03-26T03:34:09.405 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.249+0000 7ff9cb7fe700 1 -- 172.21.15.175:0/2996594251 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9d0019750 con 0x7ff9d4106460 2024-03-26T03:34:09.405 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.249+0000 7ff9dc56d700 1 -- 172.21.15.175:0/2996594251 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9d419ad20 con 0x7ff9d4106460 2024-03-26T03:34:09.405 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.249+0000 7ff9dc56d700 1 -- 172.21.15.175:0/2996594251 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9d419d9c0 con 0x7ff9d4106460 2024-03-26T03:34:09.405 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.253+0000 7ff9cb7fe700 1 -- 172.21.15.175:0/2996594251 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 4) ==== 46455+0+0 (secure 0 0 0) 0x7ff9d001a430 con 0x7ff9d4106460 2024-03-26T03:34:09.405 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.253+0000 7ff9cb7fe700 1 --2- 172.21.15.175:0/2996594251 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7ff9c0039440 0x7ff9c003b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:09.405 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.253+0000 7ff9dc56d700 1 -- 172.21.15.175:0/2996594251 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9d410a970 con 0x7ff9d4106460 2024-03-26T03:34:09.405 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.253+0000 7ff9cb7fe700 1 -- 172.21.15.175:0/2996594251 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff9d004dae0 con 0x7ff9d4106460 2024-03-26T03:34:09.405 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.253+0000 7ff9d9b08700 1 --2- 172.21.15.175:0/2996594251 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7ff9c0039440 0x7ff9c003b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:09.405 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.253+0000 7ff9d9b08700 1 --2- 172.21.15.175:0/2996594251 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7ff9c0039440 0x7ff9c003b900 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4006fd0 tx=0x7ff9c4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:09.406 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.257+0000 7ff9cb7fe700 1 -- 172.21.15.175:0/2996594251 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7ff9d001abe0 con 0x7ff9d4106460 2024-03-26T03:34:09.406 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.721+0000 7ff9dc56d700 1 -- 172.21.15.175:0/2996594251 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7ff9d406bdb0 con 0x7ff9d4106460 2024-03-26T03:34:09.406 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.773+0000 7ff9cb7fe700 1 -- 172.21.15.175:0/2996594251 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7ff9d0023e00 con 0x7ff9d4106460 2024-03-26T03:34:09.406 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.777+0000 7ff9dc56d700 1 -- 172.21.15.175:0/2996594251 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7ff9c0039440 msgr2=0x7ff9c003b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:09.406 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.777+0000 7ff9dc56d700 1 --2- 172.21.15.175:0/2996594251 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7ff9c0039440 0x7ff9c003b900 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4006fd0 tx=0x7ff9c4006e40 comp rx=0 tx=0).stop 2024-03-26T03:34:09.406 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.777+0000 7ff9dc56d700 1 -- 172.21.15.175:0/2996594251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9d4106460 msgr2=0x7ff9d419a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:09.406 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.777+0000 7ff9cb7fe700 1 -- 172.21.15.175:0/2996594251 <== mon.0 v2:172.21.15.175:3300/0 8 ==== mgrmap(e 5) ==== 46466+0+0 (secure 0 0 0) 0x7ff9d001a720 con 0x7ff9d4106460 2024-03-26T03:34:09.406 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.777+0000 7ff9dc56d700 1 --2- 172.21.15.175:0/2996594251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9d4106460 0x7ff9d419a550 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0006ba0 tx=0x7ff9d00119c0 comp rx=0 tx=0).stop 2024-03-26T03:34:09.406 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.781+0000 7ff9dc56d700 1 -- 172.21.15.175:0/2996594251 shutdown_connections 2024-03-26T03:34:09.406 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.781+0000 7ff9dc56d700 1 --2- 172.21.15.175:0/2996594251 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7ff9c0039440 0x7ff9c003b900 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:09.406 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.781+0000 7ff9dc56d700 1 --2- 172.21.15.175:0/2996594251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9d4106460 0x7ff9d419a550 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:09.406 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.781+0000 7ff9dc56d700 1 -- 172.21.15.175:0/2996594251 >> 172.21.15.175:0/2996594251 conn(0x7ff9d4101a60 msgr2=0x7ff9d4102720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:09.406 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.781+0000 7ff9dc56d700 1 -- 172.21.15.175:0/2996594251 shutdown_connections 2024-03-26T03:34:09.407 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:08.781+0000 7ff9dc56d700 1 -- 172.21.15.175:0/2996594251 wait complete. 2024-03-26T03:34:10.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:09 smithi175 bash[21527]: audit 2024-03-26T03:34:08.778825+0000 mon.a (mon.0) 34 : audit [INF] from='client.? 172.21.15.175:0/2996594251' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-26T03:34:10.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:09 smithi175 bash[21527]: cluster 2024-03-26T03:34:08.784676+0000 mon.a (mon.0) 35 : cluster [DBG] mgrmap e5: y(active, since 7s) 2024-03-26T03:34:10.869 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:10 smithi175 bash[21783]: debug 2024-03-26T03:34:10.573+0000 7f286e693200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T03:34:10.869 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:10 smithi175 bash[21783]: debug 2024-03-26T03:34:10.669+0000 7f286e693200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T03:34:10.870 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:10 smithi175 bash[21527]: audit 2024-03-26T03:34:10.682552+0000 mon.a (mon.0) 36 : audit [DBG] from='client.? 172.21.15.175:0/2501041379' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-26T03:34:11.222 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout { 2024-03-26T03:34:11.223 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-26T03:34:11.223 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-26T03:34:11.223 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-03-26T03:34:11.223 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-26T03:34:11.223 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout } 2024-03-26T03:34:11.223 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.185+0000 7f26aadc8700 1 Processor -- start 2024-03-26T03:34:11.223 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.185+0000 7f26aadc8700 1 -- start start 2024-03-26T03:34:11.223 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.185+0000 7f26aadc8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26a4106440 0x7f26a4106860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:11.223 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.185+0000 7f26aadc8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f26a4106e30 con 0x7f26a4106440 2024-03-26T03:34:11.223 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.185+0000 7f26a8b64700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26a4106440 0x7f26a4106860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:11.223 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.185+0000 7f26a8b64700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26a4106440 0x7f26a4106860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:42948/0 (socket says 172.21.15.175:42948) 2024-03-26T03:34:11.224 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.185+0000 7f26a8b64700 1 -- 172.21.15.175:0/206477643 learned_addr learned my addr 172.21.15.175:0/206477643 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:11.224 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.185+0000 7f26a8b64700 1 -- 172.21.15.175:0/206477643 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a41075a0 con 0x7f26a4106440 2024-03-26T03:34:11.224 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.185+0000 7f26a8b64700 1 --2- 172.21.15.175:0/206477643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26a4106440 0x7f26a4106860 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f26980098a0 tx=0x7f2698009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e1d74eaa75a289a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:11.224 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.185+0000 7f26a37fe700 1 -- 172.21.15.175:0/206477643 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2698004160 con 0x7f26a4106440 2024-03-26T03:34:11.224 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.185+0000 7f26a37fe700 1 -- 172.21.15.175:0/206477643 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f2698004300 con 0x7f26a4106440 2024-03-26T03:34:11.224 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.185+0000 7f26a37fe700 1 -- 172.21.15.175:0/206477643 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2698011940 con 0x7f26a4106440 2024-03-26T03:34:11.224 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.185+0000 7f26aadc8700 1 -- 172.21.15.175:0/206477643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26a4106440 msgr2=0x7f26a4106860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:11.224 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.185+0000 7f26aadc8700 1 --2- 172.21.15.175:0/206477643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26a4106440 0x7f26a4106860 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f26980098a0 tx=0x7f2698009bb0 comp rx=0 tx=0).stop 2024-03-26T03:34:11.224 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.185+0000 7f26aadc8700 1 -- 172.21.15.175:0/206477643 shutdown_connections 2024-03-26T03:34:11.225 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.185+0000 7f26aadc8700 1 --2- 172.21.15.175:0/206477643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26a4106440 0x7f26a4106860 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:11.225 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.185+0000 7f26aadc8700 1 -- 172.21.15.175:0/206477643 >> 172.21.15.175:0/206477643 conn(0x7f26a41019e0 msgr2=0x7f26a4103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:11.225 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.185+0000 7f26aadc8700 1 -- 172.21.15.175:0/206477643 shutdown_connections 2024-03-26T03:34:11.225 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.185+0000 7f26aadc8700 1 -- 172.21.15.175:0/206477643 wait complete. 2024-03-26T03:34:11.225 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.185+0000 7f26aadc8700 1 Processor -- start 2024-03-26T03:34:11.225 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.185+0000 7f26aadc8700 1 -- start start 2024-03-26T03:34:11.225 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.185+0000 7f26aadc8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26a4106440 0x7f26a419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:11.225 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.185+0000 7f26aadc8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f26a410a7d0 con 0x7f26a4106440 2024-03-26T03:34:11.225 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.185+0000 7f26a8b64700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26a4106440 0x7f26a419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:11.225 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.185+0000 7f26a8b64700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26a4106440 0x7f26a419a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:42956/0 (socket says 172.21.15.175:42956) 2024-03-26T03:34:11.225 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.185+0000 7f26a8b64700 1 -- 172.21.15.175:0/2501041379 learned_addr learned my addr 172.21.15.175:0/2501041379 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:11.225 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.185+0000 7f26a8b64700 1 -- 172.21.15.175:0/2501041379 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a419aa10 con 0x7f26a4106440 2024-03-26T03:34:11.225 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.185+0000 7f26a8b64700 1 --2- 172.21.15.175:0/2501041379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26a4106440 0x7f26a419a4d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2698009fb0 tx=0x7f26980119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:11.226 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.185+0000 7f26a1ffb700 1 -- 172.21.15.175:0/2501041379 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f269801a430 con 0x7f26a4106440 2024-03-26T03:34:11.226 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.189+0000 7f26aadc8700 1 -- 172.21.15.175:0/2501041379 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26a419aca0 con 0x7f26a4106440 2024-03-26T03:34:11.226 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.189+0000 7f26a1ffb700 1 -- 172.21.15.175:0/2501041379 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f269801aa30 con 0x7f26a4106440 2024-03-26T03:34:11.226 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.189+0000 7f26a1ffb700 1 -- 172.21.15.175:0/2501041379 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2698024860 con 0x7f26a4106440 2024-03-26T03:34:11.226 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.189+0000 7f26aadc8700 1 -- 172.21.15.175:0/2501041379 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a419d940 con 0x7f26a4106440 2024-03-26T03:34:11.226 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.189+0000 7f26a1ffb700 1 -- 172.21.15.175:0/2501041379 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) ==== 46466+0+0 (secure 0 0 0) 0x7f2698022020 con 0x7f26a4106440 2024-03-26T03:34:11.227 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.189+0000 7f26a1ffb700 1 --2- 172.21.15.175:0/2501041379 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7f2694039490 0x7f269403b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:11.227 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.189+0000 7f26aadc8700 1 -- 172.21.15.175:0/2501041379 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26a410a950 con 0x7f26a4106440 2024-03-26T03:34:11.227 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.189+0000 7f26a1ffb700 1 -- 172.21.15.175:0/2501041379 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f269804e3f0 con 0x7f26a4106440 2024-03-26T03:34:11.227 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.189+0000 7f26a3fff700 1 -- 172.21.15.175:0/2501041379 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7f2694039490 msgr2=0x7f269403b950 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1363563398 2024-03-26T03:34:11.227 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.189+0000 7f26a3fff700 1 --2- 172.21.15.175:0/2501041379 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7f2694039490 0x7f269403b950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T03:34:11.227 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.193+0000 7f26a1ffb700 1 -- 172.21.15.175:0/2501041379 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f26980172b0 con 0x7f26a4106440 2024-03-26T03:34:11.227 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.393+0000 7f26a3fff700 1 -- 172.21.15.175:0/2501041379 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7f2694039490 msgr2=0x7f269403b950 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1363563398 2024-03-26T03:34:11.227 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.393+0000 7f26a3fff700 1 --2- 172.21.15.175:0/2501041379 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7f2694039490 0x7f269403b950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T03:34:11.227 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.677+0000 7f26aadc8700 1 -- 172.21.15.175:0/2501041379 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f26a406bd80 con 0x7f26a4106440 2024-03-26T03:34:11.227 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.677+0000 7f26a1ffb700 1 -- 172.21.15.175:0/2501041379 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f2698019e10 con 0x7f26a4106440 2024-03-26T03:34:11.227 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.681+0000 7f26aadc8700 1 -- 172.21.15.175:0/2501041379 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7f2694039490 msgr2=0x7f269403b950 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T03:34:11.227 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.681+0000 7f26aadc8700 1 --2- 172.21.15.175:0/2501041379 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7f2694039490 0x7f269403b950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:11.227 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.681+0000 7f26aadc8700 1 -- 172.21.15.175:0/2501041379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26a4106440 msgr2=0x7f26a419a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:11.228 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.681+0000 7f26aadc8700 1 --2- 172.21.15.175:0/2501041379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26a4106440 0x7f26a419a4d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2698009fb0 tx=0x7f26980119c0 comp rx=0 tx=0).stop 2024-03-26T03:34:11.228 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.681+0000 7f26aadc8700 1 -- 172.21.15.175:0/2501041379 shutdown_connections 2024-03-26T03:34:11.228 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.681+0000 7f26aadc8700 1 --2- 172.21.15.175:0/2501041379 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7f2694039490 0x7f269403b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:11.228 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.681+0000 7f26aadc8700 1 --2- 172.21.15.175:0/2501041379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26a4106440 0x7f26a419a4d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:11.228 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.681+0000 7f26aadc8700 1 -- 172.21.15.175:0/2501041379 >> 172.21.15.175:0/2501041379 conn(0x7f26a41019e0 msgr2=0x7f26a4102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:11.228 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.681+0000 7f26aadc8700 1 -- 172.21.15.175:0/2501041379 shutdown_connections 2024-03-26T03:34:11.228 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:10.681+0000 7f26aadc8700 1 -- 172.21.15.175:0/2501041379 wait complete. 2024-03-26T03:34:11.228 INFO:teuthology.orchestra.run.smithi175.stdout:Waiting for the mgr to restart... 2024-03-26T03:34:11.228 INFO:teuthology.orchestra.run.smithi175.stdout:Waiting for mgr epoch 5... 2024-03-26T03:34:11.235 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:10 smithi175 bash[21783]: debug 2024-03-26T03:34:10.921+0000 7f286e693200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T03:34:11.994 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:11 smithi175 bash[21783]: debug 2024-03-26T03:34:11.717+0000 7f286e693200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T03:34:11.994 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:11 smithi175 bash[21783]: debug 2024-03-26T03:34:11.829+0000 7f286e693200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T03:34:11.994 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:11 smithi175 bash[21783]: debug 2024-03-26T03:34:11.937+0000 7f286e693200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T03:34:12.744 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:12 smithi175 bash[21783]: debug 2024-03-26T03:34:12.293+0000 7f286e693200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T03:34:12.744 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:12 smithi175 bash[21783]: debug 2024-03-26T03:34:12.405+0000 7f286e693200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T03:34:12.745 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:12 smithi175 bash[21783]: debug 2024-03-26T03:34:12.509+0000 7f286e693200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T03:34:13.244 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:12 smithi175 bash[21783]: debug 2024-03-26T03:34:12.861+0000 7f286e693200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T03:34:13.244 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:13 smithi175 bash[21783]: debug 2024-03-26T03:34:13.101+0000 7f286e693200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T03:34:13.744 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:13 smithi175 bash[21783]: debug 2024-03-26T03:34:13.361+0000 7f286e693200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T03:34:13.745 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:13 smithi175 bash[21783]: debug 2024-03-26T03:34:13.473+0000 7f286e693200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T03:34:14.159 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:13 smithi175 bash[21783]: debug 2024-03-26T03:34:13.745+0000 7f286e693200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T03:34:14.495 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:14 smithi175 bash[21783]: debug 2024-03-26T03:34:14.153+0000 7f286e693200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T03:34:14.495 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:14 smithi175 bash[21783]: debug 2024-03-26T03:34:14.393+0000 7f286e693200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T03:34:14.994 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:14 smithi175 bash[21783]: debug 2024-03-26T03:34:14.509+0000 7f286e693200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T03:34:14.995 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:14 smithi175 bash[21783]: debug 2024-03-26T03:34:14.753+0000 7f286e693200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T03:34:16.192 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:15 smithi175 bash[21783]: debug 2024-03-26T03:34:15.781+0000 7f286e693200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T03:34:16.192 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:15 smithi175 bash[21783]: debug 2024-03-26T03:34:15.897+0000 7f286e693200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T03:34:16.494 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:16 smithi175 bash[21783]: debug 2024-03-26T03:34:16.189+0000 7f286e693200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T03:34:16.494 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:16 smithi175 bash[21783]: debug 2024-03-26T03:34:16.301+0000 7f286e693200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T03:34:17.244 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:16 smithi175 bash[21783]: debug 2024-03-26T03:34:16.745+0000 7f286e693200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T03:34:17.245 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:16 smithi175 bash[21783]: debug 2024-03-26T03:34:16.861+0000 7f286e693200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T03:34:17.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:16 smithi175 bash[21527]: cluster 2024-03-26T03:34:16.885451+0000 mon.a (mon.0) 37 : cluster [INF] Active manager daemon y restarted 2024-03-26T03:34:17.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:16 smithi175 bash[21527]: cluster 2024-03-26T03:34:16.885690+0000 mon.a (mon.0) 38 : cluster [INF] Activating manager daemon y 2024-03-26T03:34:17.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:16 smithi175 bash[21527]: cluster 2024-03-26T03:34:16.891935+0000 mon.a (mon.0) 39 : cluster [DBG] osdmap e2: 0 total, 0 up, 0 in 2024-03-26T03:34:17.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:16 smithi175 bash[21527]: cluster 2024-03-26T03:34:16.892195+0000 mon.a (mon.0) 40 : cluster [DBG] mgrmap e6: y(active, starting, since 0.00659105s) 2024-03-26T03:34:17.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:16 smithi175 bash[21527]: audit 2024-03-26T03:34:16.894216+0000 mon.a (mon.0) 41 : audit [DBG] from='mgr.14120 172.21.15.175:0/3794702729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T03:34:17.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:16 smithi175 bash[21527]: audit 2024-03-26T03:34:16.896512+0000 mon.a (mon.0) 42 : audit [DBG] from='mgr.14120 172.21.15.175:0/3794702729' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T03:34:17.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:16 smithi175 bash[21527]: audit 2024-03-26T03:34:16.897443+0000 mon.a (mon.0) 43 : audit [DBG] from='mgr.14120 172.21.15.175:0/3794702729' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T03:34:17.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:16 smithi175 bash[21527]: audit 2024-03-26T03:34:16.897649+0000 mon.a (mon.0) 44 : audit [DBG] from='mgr.14120 172.21.15.175:0/3794702729' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T03:34:17.246 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:16 smithi175 bash[21527]: audit 2024-03-26T03:34:16.897779+0000 mon.a (mon.0) 45 : audit [DBG] from='mgr.14120 172.21.15.175:0/3794702729' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T03:34:17.246 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:16 smithi175 bash[21527]: cluster 2024-03-26T03:34:16.921515+0000 mon.a (mon.0) 46 : cluster [INF] Manager daemon y is now available 2024-03-26T03:34:18.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:17 smithi175 bash[21527]: cephadm 2024-03-26T03:34:16.948576+0000 mgr.y (mgr.14120) 1 : cephadm [INF] Found migration_current of "None". Setting to last migration. 2024-03-26T03:34:18.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:17 smithi175 bash[21527]: audit 2024-03-26T03:34:16.950809+0000 mon.a (mon.0) 47 : audit [INF] from='mgr.14120 172.21.15.175:0/3794702729' entity='mgr.y' 2024-03-26T03:34:18.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:17 smithi175 bash[21527]: audit 2024-03-26T03:34:16.953866+0000 mon.a (mon.0) 48 : audit [INF] from='mgr.14120 172.21.15.175:0/3794702729' entity='mgr.y' 2024-03-26T03:34:18.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:17 smithi175 bash[21527]: audit 2024-03-26T03:34:16.957608+0000 mon.a (mon.0) 49 : audit [DBG] from='mgr.14120 172.21.15.175:0/3794702729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:34:18.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:17 smithi175 bash[21527]: audit 2024-03-26T03:34:16.961252+0000 mon.a (mon.0) 50 : audit [DBG] from='mgr.14120 172.21.15.175:0/3794702729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:34:18.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:17 smithi175 bash[21527]: audit 2024-03-26T03:34:16.971626+0000 mon.a (mon.0) 51 : audit [INF] from='mgr.14120 172.21.15.175:0/3794702729' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-26T03:34:18.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:17 smithi175 bash[21527]: audit 2024-03-26T03:34:16.974582+0000 mon.a (mon.0) 52 : audit [INF] from='mgr.14120 172.21.15.175:0/3794702729' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-26T03:34:18.246 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:17 smithi175 bash[21527]: audit 2024-03-26T03:34:17.595416+0000 mon.a (mon.0) 53 : audit [INF] from='mgr.14120 172.21.15.175:0/3794702729' entity='mgr.y' 2024-03-26T03:34:18.246 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:17 smithi175 bash[21527]: audit 2024-03-26T03:34:17.600841+0000 mon.a (mon.0) 54 : audit [INF] from='mgr.14120 172.21.15.175:0/3794702729' entity='mgr.y' 2024-03-26T03:34:18.246 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:17 smithi175 bash[21527]: cluster 2024-03-26T03:34:17.899943+0000 mon.a (mon.0) 55 : cluster [DBG] mgrmap e7: y(active, since 1.01433s) 2024-03-26T03:34:18.573 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout { 2024-03-26T03:34:18.573 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-26T03:34:18.573 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-26T03:34:18.573 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout } 2024-03-26T03:34:18.573 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:11.997+0000 7fe98788f700 1 Processor -- start 2024-03-26T03:34:18.573 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:11.997+0000 7fe98788f700 1 -- start start 2024-03-26T03:34:18.573 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:11.997+0000 7fe98788f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe980106ee0 0x7fe9801092d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:18.573 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:11.997+0000 7fe98788f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe980079e10 con 0x7fe980106ee0 2024-03-26T03:34:18.573 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:11.997+0000 7fe98562b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe980106ee0 0x7fe9801092d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:18.573 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:11.997+0000 7fe98562b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe980106ee0 0x7fe9801092d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:57266/0 (socket says 172.21.15.175:57266) 2024-03-26T03:34:18.573 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:11.997+0000 7fe98562b700 1 -- 172.21.15.175:0/499102563 learned_addr learned my addr 172.21.15.175:0/499102563 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:18.574 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:11.997+0000 7fe98562b700 1 -- 172.21.15.175:0/499102563 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe980109810 con 0x7fe980106ee0 2024-03-26T03:34:18.574 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:11.997+0000 7fe98562b700 1 --2- 172.21.15.175:0/499102563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe980106ee0 0x7fe9801092d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe97c0098a0 tx=0x7fe97c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752e00ec69c70a4e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:18.574 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:11.997+0000 7fe973fff700 1 -- 172.21.15.175:0/499102563 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe97c004160 con 0x7fe980106ee0 2024-03-26T03:34:18.574 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:11.997+0000 7fe973fff700 1 -- 172.21.15.175:0/499102563 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7fe97c004300 con 0x7fe980106ee0 2024-03-26T03:34:18.574 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:11.997+0000 7fe973fff700 1 -- 172.21.15.175:0/499102563 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe97c011940 con 0x7fe980106ee0 2024-03-26T03:34:18.574 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:12.001+0000 7fe98788f700 1 -- 172.21.15.175:0/499102563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe980106ee0 msgr2=0x7fe9801092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:18.574 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:12.001+0000 7fe98788f700 1 --2- 172.21.15.175:0/499102563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe980106ee0 0x7fe9801092d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe97c0098a0 tx=0x7fe97c009bb0 comp rx=0 tx=0).stop 2024-03-26T03:34:18.574 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:12.001+0000 7fe98788f700 1 -- 172.21.15.175:0/499102563 shutdown_connections 2024-03-26T03:34:18.574 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:12.001+0000 7fe98788f700 1 --2- 172.21.15.175:0/499102563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe980106ee0 0x7fe9801092d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:18.574 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:12.001+0000 7fe98788f700 1 -- 172.21.15.175:0/499102563 >> 172.21.15.175:0/499102563 conn(0x7fe980100810 msgr2=0x7fe980102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:18.574 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:12.001+0000 7fe98788f700 1 -- 172.21.15.175:0/499102563 shutdown_connections 2024-03-26T03:34:18.574 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:12.001+0000 7fe98788f700 1 -- 172.21.15.175:0/499102563 wait complete. 2024-03-26T03:34:18.574 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:12.001+0000 7fe98788f700 1 Processor -- start 2024-03-26T03:34:18.575 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:12.001+0000 7fe98788f700 1 -- start start 2024-03-26T03:34:18.575 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:12.001+0000 7fe98788f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe98019a3e0 0x7fe98019a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:18.575 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:12.001+0000 7fe98788f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe98010a660 con 0x7fe98019a3e0 2024-03-26T03:34:18.575 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:12.001+0000 7fe98562b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe98019a3e0 0x7fe98019a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:18.575 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:12.001+0000 7fe98562b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe98019a3e0 0x7fe98019a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:57274/0 (socket says 172.21.15.175:57274) 2024-03-26T03:34:18.575 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:12.001+0000 7fe98562b700 1 -- 172.21.15.175:0/886419462 learned_addr learned my addr 172.21.15.175:0/886419462 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:18.575 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:12.001+0000 7fe98562b700 1 -- 172.21.15.175:0/886419462 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98019ad40 con 0x7fe98019a3e0 2024-03-26T03:34:18.575 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:12.001+0000 7fe98562b700 1 --2- 172.21.15.175:0/886419462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe98019a3e0 0x7fe98019a800 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe97c009fb0 tx=0x7fe97c012f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:18.575 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:12.001+0000 7fe9727fc700 1 -- 172.21.15.175:0/886419462 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe97c011970 con 0x7fe98019a3e0 2024-03-26T03:34:18.575 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:12.001+0000 7fe9727fc700 1 -- 172.21.15.175:0/886419462 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7fe97c003b00 con 0x7fe98019a3e0 2024-03-26T03:34:18.575 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:12.001+0000 7fe98788f700 1 -- 172.21.15.175:0/886419462 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe98019afd0 con 0x7fe98019a3e0 2024-03-26T03:34:18.575 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:12.001+0000 7fe9727fc700 1 -- 172.21.15.175:0/886419462 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe97c01ad00 con 0x7fe98019a3e0 2024-03-26T03:34:18.575 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:12.001+0000 7fe98788f700 1 -- 172.21.15.175:0/886419462 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe98019dab0 con 0x7fe98019a3e0 2024-03-26T03:34:18.575 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:12.001+0000 7fe9727fc700 1 -- 172.21.15.175:0/886419462 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) ==== 46466+0+0 (secure 0 0 0) 0x7fe97c02d440 con 0x7fe98019a3e0 2024-03-26T03:34:18.575 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:12.001+0000 7fe9727fc700 1 --2- 172.21.15.175:0/886419462 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7fe96c0394e0 0x7fe96c03b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:18.576 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:12.001+0000 7fe98788f700 1 -- 172.21.15.175:0/886419462 --> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe980063c70 con 0x7fe96c0394e0 2024-03-26T03:34:18.576 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:12.001+0000 7fe9727fc700 1 -- 172.21.15.175:0/886419462 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe97c04e9a0 con 0x7fe98019a3e0 2024-03-26T03:34:18.576 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:12.001+0000 7fe984e2a700 1 -- 172.21.15.175:0/886419462 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7fe96c0394e0 msgr2=0x7fe96c03b9a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1363563398 2024-03-26T03:34:18.576 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:12.001+0000 7fe984e2a700 1 --2- 172.21.15.175:0/886419462 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7fe96c0394e0 0x7fe96c03b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T03:34:18.576 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:12.205+0000 7fe984e2a700 1 -- 172.21.15.175:0/886419462 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7fe96c0394e0 msgr2=0x7fe96c03b9a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1363563398 2024-03-26T03:34:18.576 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:12.205+0000 7fe984e2a700 1 --2- 172.21.15.175:0/886419462 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7fe96c0394e0 0x7fe96c03b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T03:34:18.576 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:12.609+0000 7fe984e2a700 1 -- 172.21.15.175:0/886419462 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7fe96c0394e0 msgr2=0x7fe96c03b9a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1363563398 2024-03-26T03:34:18.576 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:12.609+0000 7fe984e2a700 1 --2- 172.21.15.175:0/886419462 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7fe96c0394e0 0x7fe96c03b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-26T03:34:18.576 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:13.413+0000 7fe984e2a700 1 -- 172.21.15.175:0/886419462 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7fe96c0394e0 msgr2=0x7fe96c03b9a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1363563398 2024-03-26T03:34:18.576 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:13.413+0000 7fe984e2a700 1 --2- 172.21.15.175:0/886419462 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7fe96c0394e0 0x7fe96c03b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-26T03:34:18.576 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:15.017+0000 7fe984e2a700 1 -- 172.21.15.175:0/886419462 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7fe96c0394e0 msgr2=0x7fe96c03b9a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/1363563398 2024-03-26T03:34:18.576 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:15.017+0000 7fe984e2a700 1 --2- 172.21.15.175:0/886419462 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7fe96c0394e0 0x7fe96c03b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-26T03:34:18.576 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:16.885+0000 7fe9727fc700 1 -- 172.21.15.175:0/886419462 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mgrmap(e 6) ==== 46233+0+0 (secure 0 0 0) 0x7fe97c01a5d0 con 0x7fe98019a3e0 2024-03-26T03:34:18.576 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:16.885+0000 7fe9727fc700 1 -- 172.21.15.175:0/886419462 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7fe96c0394e0 msgr2=0x7fe96c03b9a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T03:34:18.577 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:16.885+0000 7fe9727fc700 1 --2- 172.21.15.175:0/886419462 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7fe96c0394e0 0x7fe96c03b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:18.577 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:17.889+0000 7fe9727fc700 1 -- 172.21.15.175:0/886419462 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mgrmap(e 7) ==== 46360+0+0 (secure 0 0 0) 0x7fe97c04f930 con 0x7fe98019a3e0 2024-03-26T03:34:18.577 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:17.889+0000 7fe9727fc700 1 --2- 172.21.15.175:0/886419462 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7fe96c03cbe0 0x7fe96c03efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:18.577 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:17.889+0000 7fe9727fc700 1 -- 172.21.15.175:0/886419462 --> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe97c020df0 con 0x7fe96c03cbe0 2024-03-26T03:34:18.577 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:17.893+0000 7fe984e2a700 1 --2- 172.21.15.175:0/886419462 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7fe96c03cbe0 0x7fe96c03efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:18.577 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:17.893+0000 7fe984e2a700 1 --2- 172.21.15.175:0/886419462 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7fe96c03cbe0 0x7fe96c03efd0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe974003a10 tx=0x7fe9740092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:18.577 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:17.893+0000 7fe9727fc700 1 -- 172.21.15.175:0/886419462 <== mgr.14120 v2:172.21.15.175:6800/712246762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fe97c020df0 con 0x7fe96c03cbe0 2024-03-26T03:34:18.577 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:17.905+0000 7fe98788f700 1 -- 172.21.15.175:0/886419462 --> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fe980101b40 con 0x7fe96c03cbe0 2024-03-26T03:34:18.577 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:17.905+0000 7fe9727fc700 1 -- 172.21.15.175:0/886419462 <== mgr.14120 v2:172.21.15.175:6800/712246762 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fe980101b40 con 0x7fe96c03cbe0 2024-03-26T03:34:18.577 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:17.909+0000 7fe98788f700 1 -- 172.21.15.175:0/886419462 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7fe96c03cbe0 msgr2=0x7fe96c03efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:18.577 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:17.909+0000 7fe98788f700 1 --2- 172.21.15.175:0/886419462 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7fe96c03cbe0 0x7fe96c03efd0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe974003a10 tx=0x7fe9740092b0 comp rx=0 tx=0).stop 2024-03-26T03:34:18.577 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:17.909+0000 7fe98788f700 1 -- 172.21.15.175:0/886419462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe98019a3e0 msgr2=0x7fe98019a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:18.577 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:17.909+0000 7fe98788f700 1 --2- 172.21.15.175:0/886419462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe98019a3e0 0x7fe98019a800 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe97c009fb0 tx=0x7fe97c012f30 comp rx=0 tx=0).stop 2024-03-26T03:34:18.577 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:17.909+0000 7fe98788f700 1 -- 172.21.15.175:0/886419462 shutdown_connections 2024-03-26T03:34:18.578 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:17.909+0000 7fe98788f700 1 --2- 172.21.15.175:0/886419462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe98019a3e0 0x7fe98019a800 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:18.578 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:17.909+0000 7fe98788f700 1 --2- 172.21.15.175:0/886419462 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7fe96c03cbe0 0x7fe96c03efd0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:18.578 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:17.909+0000 7fe98788f700 1 --2- 172.21.15.175:0/886419462 >> [v2:172.21.15.175:6800/1363563398,v1:172.21.15.175:6801/1363563398] conn(0x7fe96c0394e0 0x7fe96c03b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:18.578 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:17.909+0000 7fe98788f700 1 -- 172.21.15.175:0/886419462 >> 172.21.15.175:0/886419462 conn(0x7fe980100810 msgr2=0x7fe980101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:18.578 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:17.909+0000 7fe98788f700 1 -- 172.21.15.175:0/886419462 shutdown_connections 2024-03-26T03:34:18.578 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:17.909+0000 7fe98788f700 1 -- 172.21.15.175:0/886419462 wait complete. 2024-03-26T03:34:18.578 INFO:teuthology.orchestra.run.smithi175.stdout:mgr epoch 5 is available 2024-03-26T03:34:18.578 INFO:teuthology.orchestra.run.smithi175.stdout:Setting orchestrator backend to cephadm... 2024-03-26T03:34:19.217 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:18 smithi175 bash[21527]: audit 2024-03-26T03:34:17.897021+0000 mgr.y (mgr.14120) 2 : audit [DBG] from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-26T03:34:19.217 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:18 smithi175 bash[21527]: audit 2024-03-26T03:34:17.911036+0000 mgr.y (mgr.14120) 3 : audit [DBG] from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-26T03:34:19.217 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:18 smithi175 bash[21527]: cephadm 2024-03-26T03:34:18.030577+0000 mgr.y (mgr.14120) 4 : cephadm [INF] [26/Mar/2024:03:34:18] ENGINE Bus STARTING 2024-03-26T03:34:19.217 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:18 smithi175 bash[21527]: cephadm 2024-03-26T03:34:18.149388+0000 mgr.y (mgr.14120) 5 : cephadm [INF] [26/Mar/2024:03:34:18] ENGINE Serving on https://172.21.15.175:7150 2024-03-26T03:34:19.217 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:18 smithi175 bash[21527]: cephadm 2024-03-26T03:34:18.250616+0000 mgr.y (mgr.14120) 6 : cephadm [INF] [26/Mar/2024:03:34:18] ENGINE Serving on http://172.21.15.175:8765 2024-03-26T03:34:19.217 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:18 smithi175 bash[21527]: cephadm 2024-03-26T03:34:18.250800+0000 mgr.y (mgr.14120) 7 : cephadm [INF] [26/Mar/2024:03:34:18] ENGINE Bus STARTED 2024-03-26T03:34:19.217 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:18 smithi175 bash[21527]: audit 2024-03-26T03:34:18.282200+0000 mon.a (mon.0) 56 : audit [DBG] from='mgr.14120 172.21.15.175:0/3794702729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:34:20.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:19 smithi175 bash[21527]: cluster 2024-03-26T03:34:18.960025+0000 mon.a (mon.0) 57 : cluster [DBG] mgrmap e8: y(active, since 2s) 2024-03-26T03:34:20.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:19 smithi175 bash[21527]: audit 2024-03-26T03:34:19.803441+0000 mgr.y (mgr.14120) 8 : audit [DBG] from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:34:20.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:19 smithi175 bash[21527]: audit 2024-03-26T03:34:19.809438+0000 mon.a (mon.0) 58 : audit [INF] from='mgr.14120 172.21.15.175:0/3794702729' entity='mgr.y' 2024-03-26T03:34:20.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:19 smithi175 bash[21527]: audit 2024-03-26T03:34:19.853309+0000 mon.a (mon.0) 59 : audit [DBG] from='mgr.14120 172.21.15.175:0/3794702729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:34:20.424 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.381+0000 7f1f67f7e700 1 Processor -- start 2024-03-26T03:34:20.425 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.381+0000 7f1f67f7e700 1 -- start start 2024-03-26T03:34:20.425 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.381+0000 7f1f67f7e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f60080bc0 0x7f1f6010f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:20.425 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.381+0000 7f1f67f7e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6010fb60 con 0x7f1f60080bc0 2024-03-26T03:34:20.425 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.381+0000 7f1f65d1a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f60080bc0 0x7f1f6010f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:20.425 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.381+0000 7f1f65d1a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f60080bc0 0x7f1f6010f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:57358/0 (socket says 172.21.15.175:57358) 2024-03-26T03:34:20.425 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.381+0000 7f1f65d1a700 1 -- 172.21.15.175:0/3127533430 learned_addr learned my addr 172.21.15.175:0/3127533430 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:20.425 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.381+0000 7f1f65d1a700 1 -- 172.21.15.175:0/3127533430 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f6010fce0 con 0x7f1f60080bc0 2024-03-26T03:34:20.425 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.385+0000 7f1f65d1a700 1 --2- 172.21.15.175:0/3127533430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f60080bc0 0x7f1f6010f590 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1f500098a0 tx=0x7f1f50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=443294c718c22313 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:20.425 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.385+0000 7f1f64d18700 1 -- 172.21.15.175:0/3127533430 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f50004160 con 0x7f1f60080bc0 2024-03-26T03:34:20.425 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.385+0000 7f1f64d18700 1 -- 172.21.15.175:0/3127533430 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f1f50004300 con 0x7f1f60080bc0 2024-03-26T03:34:20.425 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.385+0000 7f1f64d18700 1 -- 172.21.15.175:0/3127533430 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f50011940 con 0x7f1f60080bc0 2024-03-26T03:34:20.425 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.385+0000 7f1f67f7e700 1 -- 172.21.15.175:0/3127533430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f60080bc0 msgr2=0x7f1f6010f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:20.425 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.385+0000 7f1f67f7e700 1 --2- 172.21.15.175:0/3127533430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f60080bc0 0x7f1f6010f590 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1f500098a0 tx=0x7f1f50009bb0 comp rx=0 tx=0).stop 2024-03-26T03:34:20.425 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.385+0000 7f1f67f7e700 1 -- 172.21.15.175:0/3127533430 shutdown_connections 2024-03-26T03:34:20.426 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.385+0000 7f1f67f7e700 1 --2- 172.21.15.175:0/3127533430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f60080bc0 0x7f1f6010f590 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:20.426 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.385+0000 7f1f67f7e700 1 -- 172.21.15.175:0/3127533430 >> 172.21.15.175:0/3127533430 conn(0x7f1f6007bc10 msgr2=0x7f1f6007e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:20.426 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.385+0000 7f1f67f7e700 1 -- 172.21.15.175:0/3127533430 shutdown_connections 2024-03-26T03:34:20.426 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.385+0000 7f1f67f7e700 1 -- 172.21.15.175:0/3127533430 wait complete. 2024-03-26T03:34:20.426 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.385+0000 7f1f67f7e700 1 Processor -- start 2024-03-26T03:34:20.426 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.385+0000 7f1f67f7e700 1 -- start start 2024-03-26T03:34:20.426 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.385+0000 7f1f67f7e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f601a0aa0 0x7f1f601a0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:20.426 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.385+0000 7f1f67f7e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1f60110ca0 con 0x7f1f601a0aa0 2024-03-26T03:34:20.426 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.385+0000 7f1f65d1a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f601a0aa0 0x7f1f601a0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:20.426 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.385+0000 7f1f65d1a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f601a0aa0 0x7f1f601a0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:57360/0 (socket says 172.21.15.175:57360) 2024-03-26T03:34:20.426 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.385+0000 7f1f65d1a700 1 -- 172.21.15.175:0/12981162 learned_addr learned my addr 172.21.15.175:0/12981162 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:20.426 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.385+0000 7f1f65d1a700 1 -- 172.21.15.175:0/12981162 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f601a1400 con 0x7f1f601a0aa0 2024-03-26T03:34:20.426 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.385+0000 7f1f65d1a700 1 --2- 172.21.15.175:0/12981162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f601a0aa0 0x7f1f601a0ec0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1f50006ba0 tx=0x7f1f500119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:20.426 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.385+0000 7f1f56ffd700 1 -- 172.21.15.175:0/12981162 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f50003930 con 0x7f1f601a0aa0 2024-03-26T03:34:20.427 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.385+0000 7f1f56ffd700 1 -- 172.21.15.175:0/12981162 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f1f50003ad0 con 0x7f1f601a0aa0 2024-03-26T03:34:20.427 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.385+0000 7f1f67f7e700 1 -- 172.21.15.175:0/12981162 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f601a1690 con 0x7f1f601a0aa0 2024-03-26T03:34:20.427 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.385+0000 7f1f56ffd700 1 -- 172.21.15.175:0/12981162 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f50019730 con 0x7f1f601a0aa0 2024-03-26T03:34:20.427 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.385+0000 7f1f67f7e700 1 -- 172.21.15.175:0/12981162 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f601a4170 con 0x7f1f601a0aa0 2024-03-26T03:34:20.427 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.385+0000 7f1f56ffd700 1 -- 172.21.15.175:0/12981162 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 8) ==== 46466+0+0 (secure 0 0 0) 0x7f1f500199e0 con 0x7f1f601a0aa0 2024-03-26T03:34:20.427 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.385+0000 7f1f67f7e700 1 -- 172.21.15.175:0/12981162 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f60063c70 con 0x7f1f601a0aa0 2024-03-26T03:34:20.427 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.385+0000 7f1f56ffd700 1 --2- 172.21.15.175:0/12981162 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f1f4c039490 0x7f1f4c03b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:20.427 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.389+0000 7f1f56ffd700 1 -- 172.21.15.175:0/12981162 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1f5004dcf0 con 0x7f1f601a0aa0 2024-03-26T03:34:20.427 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.389+0000 7f1f65519700 1 --2- 172.21.15.175:0/12981162 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f1f4c039490 0x7f1f4c03b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:20.427 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.389+0000 7f1f65519700 1 --2- 172.21.15.175:0/12981162 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f1f4c039490 0x7f1f4c03b950 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c006fd0 tx=0x7f1f5c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:20.427 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.393+0000 7f1f56ffd700 1 -- 172.21.15.175:0/12981162 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f1f5004d640 con 0x7f1f601a0aa0 2024-03-26T03:34:20.427 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.797+0000 7f1f67f7e700 1 -- 172.21.15.175:0/12981162 --> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f1f60080c30 con 0x7f1f4c039490 2024-03-26T03:34:20.427 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.805+0000 7f1f56ffd700 1 -- 172.21.15.175:0/12981162 <== mgr.14120 v2:172.21.15.175:6800/712246762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1f60080c30 con 0x7f1f4c039490 2024-03-26T03:34:20.428 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.809+0000 7f1f67f7e700 1 -- 172.21.15.175:0/12981162 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f1f4c039490 msgr2=0x7f1f4c03b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:20.428 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.809+0000 7f1f67f7e700 1 --2- 172.21.15.175:0/12981162 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f1f4c039490 0x7f1f4c03b950 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c006fd0 tx=0x7f1f5c006e40 comp rx=0 tx=0).stop 2024-03-26T03:34:20.428 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.809+0000 7f1f67f7e700 1 -- 172.21.15.175:0/12981162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f601a0aa0 msgr2=0x7f1f601a0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:20.428 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.809+0000 7f1f67f7e700 1 --2- 172.21.15.175:0/12981162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f601a0aa0 0x7f1f601a0ec0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1f50006ba0 tx=0x7f1f500119c0 comp rx=0 tx=0).stop 2024-03-26T03:34:20.428 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.809+0000 7f1f67f7e700 1 -- 172.21.15.175:0/12981162 shutdown_connections 2024-03-26T03:34:20.428 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.809+0000 7f1f67f7e700 1 --2- 172.21.15.175:0/12981162 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f1f4c039490 0x7f1f4c03b950 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:20.428 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.809+0000 7f1f67f7e700 1 --2- 172.21.15.175:0/12981162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f601a0aa0 0x7f1f601a0ec0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:20.428 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.809+0000 7f1f67f7e700 1 -- 172.21.15.175:0/12981162 >> 172.21.15.175:0/12981162 conn(0x7f1f6007bc10 msgr2=0x7f1f6007d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:20.428 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.809+0000 7f1f67f7e700 1 -- 172.21.15.175:0/12981162 shutdown_connections 2024-03-26T03:34:20.428 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:19.809+0000 7f1f67f7e700 1 -- 172.21.15.175:0/12981162 wait complete. 2024-03-26T03:34:22.189 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-26T03:34:22.189 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.249+0000 7fca420b9700 1 Processor -- start 2024-03-26T03:34:22.190 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.249+0000 7fca420b9700 1 -- start start 2024-03-26T03:34:22.190 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.249+0000 7fca420b9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca3c106400 0x7fca3c106820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:22.190 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.249+0000 7fca420b9700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c106df0 con 0x7fca3c106400 2024-03-26T03:34:22.190 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.249+0000 7fca3b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca3c106400 0x7fca3c106820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:22.190 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.249+0000 7fca3b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca3c106400 0x7fca3c106820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:44078/0 (socket says 172.21.15.175:44078) 2024-03-26T03:34:22.190 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.249+0000 7fca3b7fe700 1 -- 172.21.15.175:0/3658479760 learned_addr learned my addr 172.21.15.175:0/3658479760 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:22.190 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.249+0000 7fca3b7fe700 1 -- 172.21.15.175:0/3658479760 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3c1075b0 con 0x7fca3c106400 2024-03-26T03:34:22.190 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.249+0000 7fca3b7fe700 1 --2- 172.21.15.175:0/3658479760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca3c106400 0x7fca3c106820 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009ab0 tx=0x7fca2c009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2651181c9462c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:22.190 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.249+0000 7fca3a7fc700 1 -- 172.21.15.175:0/3658479760 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca2c004160 con 0x7fca3c106400 2024-03-26T03:34:22.190 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.249+0000 7fca3a7fc700 1 -- 172.21.15.175:0/3658479760 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7fca2c00bc40 con 0x7fca3c106400 2024-03-26T03:34:22.191 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.249+0000 7fca3a7fc700 1 -- 172.21.15.175:0/3658479760 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca2c003b10 con 0x7fca3c106400 2024-03-26T03:34:22.192 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.249+0000 7fca420b9700 1 -- 172.21.15.175:0/3658479760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca3c106400 msgr2=0x7fca3c106820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:22.192 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.249+0000 7fca420b9700 1 --2- 172.21.15.175:0/3658479760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca3c106400 0x7fca3c106820 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009ab0 tx=0x7fca2c009dc0 comp rx=0 tx=0).stop 2024-03-26T03:34:22.192 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.249+0000 7fca420b9700 1 -- 172.21.15.175:0/3658479760 shutdown_connections 2024-03-26T03:34:22.192 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.249+0000 7fca420b9700 1 --2- 172.21.15.175:0/3658479760 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca3c106400 0x7fca3c106820 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:22.192 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.253+0000 7fca420b9700 1 -- 172.21.15.175:0/3658479760 >> 172.21.15.175:0/3658479760 conn(0x7fca3c101960 msgr2=0x7fca3c103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:22.192 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.253+0000 7fca420b9700 1 -- 172.21.15.175:0/3658479760 shutdown_connections 2024-03-26T03:34:22.192 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.253+0000 7fca420b9700 1 -- 172.21.15.175:0/3658479760 wait complete. 2024-03-26T03:34:22.192 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.253+0000 7fca420b9700 1 Processor -- start 2024-03-26T03:34:22.192 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.253+0000 7fca420b9700 1 -- start start 2024-03-26T03:34:22.192 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.253+0000 7fca420b9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca3c106400 0x7fca3c07c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:22.192 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.253+0000 7fca420b9700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c10a7e0 con 0x7fca3c106400 2024-03-26T03:34:22.192 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.253+0000 7fca3b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca3c106400 0x7fca3c07c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:22.192 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.253+0000 7fca3b7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca3c106400 0x7fca3c07c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:44086/0 (socket says 172.21.15.175:44086) 2024-03-26T03:34:22.193 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.253+0000 7fca3b7fe700 1 -- 172.21.15.175:0/3193628788 learned_addr learned my addr 172.21.15.175:0/3193628788 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:22.193 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.253+0000 7fca3b7fe700 1 -- 172.21.15.175:0/3193628788 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3c07c960 con 0x7fca3c106400 2024-03-26T03:34:22.193 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.253+0000 7fca3b7fe700 1 --2- 172.21.15.175:0/3193628788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca3c106400 0x7fca3c07c420 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fca2c004660 tx=0x7fca2c00bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:22.193 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.253+0000 7fca38ff9700 1 -- 172.21.15.175:0/3193628788 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca2c004400 con 0x7fca3c106400 2024-03-26T03:34:22.193 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.253+0000 7fca38ff9700 1 -- 172.21.15.175:0/3193628788 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7fca2c003fd0 con 0x7fca3c106400 2024-03-26T03:34:22.193 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.253+0000 7fca38ff9700 1 -- 172.21.15.175:0/3193628788 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca2c01a430 con 0x7fca3c106400 2024-03-26T03:34:22.193 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.253+0000 7fca420b9700 1 -- 172.21.15.175:0/3193628788 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca3c07ab50 con 0x7fca3c106400 2024-03-26T03:34:22.193 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.253+0000 7fca420b9700 1 -- 172.21.15.175:0/3193628788 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca3c07b090 con 0x7fca3c106400 2024-03-26T03:34:22.193 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.253+0000 7fca38ff9700 1 -- 172.21.15.175:0/3193628788 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 8) ==== 46466+0+0 (secure 0 0 0) 0x7fca2c024440 con 0x7fca3c106400 2024-03-26T03:34:22.193 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.253+0000 7fca420b9700 1 -- 172.21.15.175:0/3193628788 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca3c10a960 con 0x7fca3c106400 2024-03-26T03:34:22.193 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.253+0000 7fca38ff9700 1 --2- 172.21.15.175:0/3193628788 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7fca280394e0 0x7fca2803b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:22.193 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.253+0000 7fca38ff9700 1 -- 172.21.15.175:0/3193628788 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fca2c04d820 con 0x7fca3c106400 2024-03-26T03:34:22.193 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.253+0000 7fca3affd700 1 --2- 172.21.15.175:0/3193628788 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7fca280394e0 0x7fca2803b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:22.193 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.253+0000 7fca3affd700 1 --2- 172.21.15.175:0/3193628788 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7fca280394e0 0x7fca2803b9a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fca2400ad30 tx=0x7fca240093f0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:22.194 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.261+0000 7fca38ff9700 1 -- 172.21.15.175:0/3193628788 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7fca2c00f2e0 con 0x7fca3c106400 2024-03-26T03:34:22.194 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.629+0000 7fca420b9700 1 -- 172.21.15.175:0/3193628788 --> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fca3c193bd0 con 0x7fca280394e0 2024-03-26T03:34:22.194 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.629+0000 7fca38ff9700 1 -- 172.21.15.175:0/3193628788 <== mgr.14120 v2:172.21.15.175:6800/712246762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fca3c193bd0 con 0x7fca280394e0 2024-03-26T03:34:22.194 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.633+0000 7fca420b9700 1 -- 172.21.15.175:0/3193628788 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7fca280394e0 msgr2=0x7fca2803b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:22.194 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.633+0000 7fca420b9700 1 --2- 172.21.15.175:0/3193628788 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7fca280394e0 0x7fca2803b9a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fca2400ad30 tx=0x7fca240093f0 comp rx=0 tx=0).stop 2024-03-26T03:34:22.194 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.633+0000 7fca420b9700 1 -- 172.21.15.175:0/3193628788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca3c106400 msgr2=0x7fca3c07c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:22.194 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.633+0000 7fca420b9700 1 --2- 172.21.15.175:0/3193628788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca3c106400 0x7fca3c07c420 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fca2c004660 tx=0x7fca2c00bac0 comp rx=0 tx=0).stop 2024-03-26T03:34:22.194 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.633+0000 7fca420b9700 1 -- 172.21.15.175:0/3193628788 shutdown_connections 2024-03-26T03:34:22.194 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.633+0000 7fca420b9700 1 --2- 172.21.15.175:0/3193628788 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7fca280394e0 0x7fca2803b9a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:22.194 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.633+0000 7fca420b9700 1 --2- 172.21.15.175:0/3193628788 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca3c106400 0x7fca3c07c420 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:22.194 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.633+0000 7fca420b9700 1 -- 172.21.15.175:0/3193628788 >> 172.21.15.175:0/3193628788 conn(0x7fca3c101960 msgr2=0x7fca3c1024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:22.194 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.633+0000 7fca420b9700 1 -- 172.21.15.175:0/3193628788 shutdown_connections 2024-03-26T03:34:22.194 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:21.633+0000 7fca420b9700 1 -- 172.21.15.175:0/3193628788 wait complete. 2024-03-26T03:34:22.194 INFO:teuthology.orchestra.run.smithi175.stdout:Generating ssh key... 2024-03-26T03:34:22.209 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:21 smithi175 bash[21527]: audit 2024-03-26T03:34:21.634093+0000 mgr.y (mgr.14120) 9 : audit [DBG] from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:34:23.995 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:23 smithi175 bash[21783]: Generating public/private rsa key pair. 2024-03-26T03:34:23.995 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:23 smithi175 bash[21783]: Your identification has been saved in /tmp/tmp2ohx8s51/key. 2024-03-26T03:34:23.996 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:23 smithi175 bash[21783]: Your public key has been saved in /tmp/tmp2ohx8s51/key.pub. 2024-03-26T03:34:23.996 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:23 smithi175 bash[21783]: The key fingerprint is: 2024-03-26T03:34:23.996 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:23 smithi175 bash[21783]: SHA256:RSb/LEjnXCMDdbyh0RmYvVtCJ+/Lj+0xHds3HG1Z5VU ceph-762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T03:34:23.996 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:23 smithi175 bash[21783]: The key's randomart image is: 2024-03-26T03:34:23.996 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:23 smithi175 bash[21783]: +---[RSA 3072]----+ 2024-03-26T03:34:23.996 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:23 smithi175 bash[21783]: | o.+*oo E| 2024-03-26T03:34:23.996 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:23 smithi175 bash[21783]: | *+.O ..o| 2024-03-26T03:34:23.996 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:23 smithi175 bash[21783]: | . *+oB o| 2024-03-26T03:34:23.996 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:23 smithi175 bash[21783]: | . *.*+.o +| 2024-03-26T03:34:23.996 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:23 smithi175 bash[21783]: | S + o= +o| 2024-03-26T03:34:23.996 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:23 smithi175 bash[21783]: | .. o.*| 2024-03-26T03:34:23.996 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:23 smithi175 bash[21783]: | . B+| 2024-03-26T03:34:23.996 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:23 smithi175 bash[21783]: | oo=| 2024-03-26T03:34:23.996 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:23 smithi175 bash[21783]: | .o+| 2024-03-26T03:34:23.997 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:23 smithi175 bash[21783]: +----[SHA256]-----+ 2024-03-26T03:34:24.240 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.073+0000 7f09fa492700 1 Processor -- start 2024-03-26T03:34:24.240 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.073+0000 7f09fa492700 1 -- start start 2024-03-26T03:34:24.240 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.077+0000 7f09fa492700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09f4106460 0x7f09f4106880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:24.240 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.077+0000 7f09fa492700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f09f4106e50 con 0x7f09f4106460 2024-03-26T03:34:24.240 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.077+0000 7f09f3fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09f4106460 0x7f09f4106880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:24.241 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.077+0000 7f09f3fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09f4106460 0x7f09f4106880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:44096/0 (socket says 172.21.15.175:44096) 2024-03-26T03:34:24.241 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.077+0000 7f09f3fff700 1 -- 172.21.15.175:0/3949895580 learned_addr learned my addr 172.21.15.175:0/3949895580 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:24.245 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.077+0000 7f09f3fff700 1 -- 172.21.15.175:0/3949895580 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f41075c0 con 0x7f09f4106460 2024-03-26T03:34:24.245 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.077+0000 7f09f3fff700 1 --2- 172.21.15.175:0/3949895580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09f4106460 0x7f09f4106880 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f09e80098a0 tx=0x7f09e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf10827bedba396 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:24.245 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.077+0000 7f09f2ffd700 1 -- 172.21.15.175:0/3949895580 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09e8004160 con 0x7f09f4106460 2024-03-26T03:34:24.245 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.077+0000 7f09f2ffd700 1 -- 172.21.15.175:0/3949895580 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f09e8004300 con 0x7f09f4106460 2024-03-26T03:34:24.245 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.077+0000 7f09f2ffd700 1 -- 172.21.15.175:0/3949895580 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09e8011940 con 0x7f09f4106460 2024-03-26T03:34:24.245 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.077+0000 7f09fa492700 1 -- 172.21.15.175:0/3949895580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09f4106460 msgr2=0x7f09f4106880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:24.245 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.077+0000 7f09fa492700 1 --2- 172.21.15.175:0/3949895580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09f4106460 0x7f09f4106880 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f09e80098a0 tx=0x7f09e8009bb0 comp rx=0 tx=0).stop 2024-03-26T03:34:24.245 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.077+0000 7f09fa492700 1 -- 172.21.15.175:0/3949895580 shutdown_connections 2024-03-26T03:34:24.245 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.077+0000 7f09fa492700 1 --2- 172.21.15.175:0/3949895580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09f4106460 0x7f09f4106880 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:24.245 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.077+0000 7f09fa492700 1 -- 172.21.15.175:0/3949895580 >> 172.21.15.175:0/3949895580 conn(0x7f09f4101a20 msgr2=0x7f09f4103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:24.246 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.077+0000 7f09fa492700 1 -- 172.21.15.175:0/3949895580 shutdown_connections 2024-03-26T03:34:24.246 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.077+0000 7f09fa492700 1 -- 172.21.15.175:0/3949895580 wait complete. 2024-03-26T03:34:24.246 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.077+0000 7f09fa492700 1 Processor -- start 2024-03-26T03:34:24.246 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.077+0000 7f09fa492700 1 -- start start 2024-03-26T03:34:24.246 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.077+0000 7f09fa492700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09f4106460 0x7f09f419c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:24.246 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.077+0000 7f09fa492700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f09f410a7f0 con 0x7f09f4106460 2024-03-26T03:34:24.246 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.077+0000 7f09f3fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09f4106460 0x7f09f419c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:24.246 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.077+0000 7f09f3fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09f4106460 0x7f09f419c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:44106/0 (socket says 172.21.15.175:44106) 2024-03-26T03:34:24.246 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.077+0000 7f09f3fff700 1 -- 172.21.15.175:0/2305109779 learned_addr learned my addr 172.21.15.175:0/2305109779 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:24.246 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.077+0000 7f09f3fff700 1 -- 172.21.15.175:0/2305109779 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f419cc50 con 0x7f09f4106460 2024-03-26T03:34:24.246 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.077+0000 7f09f3fff700 1 --2- 172.21.15.175:0/2305109779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09f4106460 0x7f09f419c710 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f09e8006ba0 tx=0x7f09e80119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:24.246 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.077+0000 7f09f17fa700 1 -- 172.21.15.175:0/2305109779 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09e8003930 con 0x7f09f4106460 2024-03-26T03:34:24.246 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.077+0000 7f09f17fa700 1 -- 172.21.15.175:0/2305109779 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f09e8003ad0 con 0x7f09f4106460 2024-03-26T03:34:24.246 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.077+0000 7f09f17fa700 1 -- 172.21.15.175:0/2305109779 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09e8019750 con 0x7f09f4106460 2024-03-26T03:34:24.247 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.077+0000 7f09fa492700 1 -- 172.21.15.175:0/2305109779 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09f419cee0 con 0x7f09f4106460 2024-03-26T03:34:24.247 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.077+0000 7f09fa492700 1 -- 172.21.15.175:0/2305109779 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09f419fbd0 con 0x7f09f4106460 2024-03-26T03:34:24.247 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.081+0000 7f09f17fa700 1 -- 172.21.15.175:0/2305109779 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 8) ==== 46466+0+0 (secure 0 0 0) 0x7f09e80199c0 con 0x7f09f4106460 2024-03-26T03:34:24.247 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.081+0000 7f09f17fa700 1 --2- 172.21.15.175:0/2305109779 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f09dc039490 0x7f09dc03b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:24.247 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.081+0000 7f09fa492700 1 -- 172.21.15.175:0/2305109779 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09f410a970 con 0x7f09f4106460 2024-03-26T03:34:24.247 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.081+0000 7f09f17fa700 1 -- 172.21.15.175:0/2305109779 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f09e804dc10 con 0x7f09f4106460 2024-03-26T03:34:24.247 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.081+0000 7f09f37fe700 1 --2- 172.21.15.175:0/2305109779 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f09dc039490 0x7f09dc03b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:24.249 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.081+0000 7f09f37fe700 1 --2- 172.21.15.175:0/2305109779 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f09dc039490 0x7f09dc03b950 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f09e4006fd0 tx=0x7f09e4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:24.249 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.085+0000 7f09f17fa700 1 -- 172.21.15.175:0/2305109779 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f09e8023e00 con 0x7f09f4106460 2024-03-26T03:34:24.249 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.453+0000 7f09fa492700 1 -- 172.21.15.175:0/2305109779 --> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f09f406bdf0 con 0x7f09dc039490 2024-03-26T03:34:24.249 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.625+0000 7f09f17fa700 1 -- 172.21.15.175:0/2305109779 <== mgr.14120 v2:172.21.15.175:6800/712246762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f09f406bdf0 con 0x7f09dc039490 2024-03-26T03:34:24.249 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.629+0000 7f09fa492700 1 -- 172.21.15.175:0/2305109779 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f09dc039490 msgr2=0x7f09dc03b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:24.249 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.629+0000 7f09fa492700 1 --2- 172.21.15.175:0/2305109779 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f09dc039490 0x7f09dc03b950 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f09e4006fd0 tx=0x7f09e4006e40 comp rx=0 tx=0).stop 2024-03-26T03:34:24.249 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.629+0000 7f09fa492700 1 -- 172.21.15.175:0/2305109779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09f4106460 msgr2=0x7f09f419c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:24.249 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.629+0000 7f09fa492700 1 --2- 172.21.15.175:0/2305109779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09f4106460 0x7f09f419c710 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f09e8006ba0 tx=0x7f09e80119c0 comp rx=0 tx=0).stop 2024-03-26T03:34:24.249 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.629+0000 7f09fa492700 1 -- 172.21.15.175:0/2305109779 shutdown_connections 2024-03-26T03:34:24.249 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.629+0000 7f09fa492700 1 --2- 172.21.15.175:0/2305109779 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f09dc039490 0x7f09dc03b950 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:24.249 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.629+0000 7f09fa492700 1 --2- 172.21.15.175:0/2305109779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09f4106460 0x7f09f419c710 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:24.249 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.629+0000 7f09fa492700 1 -- 172.21.15.175:0/2305109779 >> 172.21.15.175:0/2305109779 conn(0x7f09f4101a20 msgr2=0x7f09f4102600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:24.250 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.629+0000 7f09fa492700 1 -- 172.21.15.175:0/2305109779 shutdown_connections 2024-03-26T03:34:24.250 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:23.629+0000 7f09fa492700 1 -- 172.21.15.175:0/2305109779 wait complete. 2024-03-26T03:34:24.940 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:24 smithi175 bash[21527]: audit 2024-03-26T03:34:23.459939+0000 mgr.y (mgr.14120) 10 : audit [DBG] from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:34:24.940 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:24 smithi175 bash[21527]: cephadm 2024-03-26T03:34:23.460229+0000 mgr.y (mgr.14120) 11 : cephadm [INF] Generating ssh key... 2024-03-26T03:34:24.940 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:24 smithi175 bash[21527]: audit 2024-03-26T03:34:23.623571+0000 mon.a (mon.0) 60 : audit [INF] from='mgr.14120 172.21.15.175:0/3794702729' entity='mgr.y' 2024-03-26T03:34:24.940 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:24 smithi175 bash[21527]: audit 2024-03-26T03:34:23.628074+0000 mon.a (mon.0) 61 : audit [INF] from='mgr.14120 172.21.15.175:0/3794702729' entity='mgr.y' 2024-03-26T03:34:26.033 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIxbSJkqZCSs/lOfW0OK6oNIT2bcL4u25oaSQElk3cx0GxdAPDGMr23NqmfnCSvPKt3/GWsQRLo2qhZjMU3yCAErTt4mWgxPrgZIoIQY9fFlro7xHNoztqNYMeMDjrLLdVpL+Dc8Gw3heu2Pyh0YPsVgKVqlEwkcFRKhedWno8s1cEGDof8iXaaVsQM2FRV+KtnU//ZDapHDferKYmyb54R/x9PVwfrbOlR9h3EYvASpkmB+anwS6gfqkoI0NC4Y7ZJpwCfIZTiiZl4TYD8XbYXddepHTNt/lbdHwkBOBEdOU8H3iKWB1T0XiMFSXyW5QaWorLnSvzEA5qxtFdG/RzBxLB7SjTwIUDleCvEFsSjtIBQWPWec8m/7TKy6JB9vxUZqGGH9iyD++1nGL25wFU5ybdGNu1XnxP/yYywvcdXaBQydMB3Wf8Bf/hCA3nsxg0KgkN03r1BeqtTnidvoSjBnHtaqQdr0esYo2gBPlAw2Fe974XEJBxuAz5dRHWnoc= ceph-762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T03:34:26.033 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.120+0000 7efebd3c2700 1 Processor -- start 2024-03-26T03:34:26.034 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.120+0000 7efebd3c2700 1 -- start start 2024-03-26T03:34:26.034 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.120+0000 7efebd3c2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efeb807c740 0x7efeb807abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:26.034 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.120+0000 7efebd3c2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efeb807b1c0 con 0x7efeb807c740 2024-03-26T03:34:26.034 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.120+0000 7efeb6ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efeb807c740 0x7efeb807abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:26.034 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.120+0000 7efeb6ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efeb807c740 0x7efeb807abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:44108/0 (socket says 172.21.15.175:44108) 2024-03-26T03:34:26.034 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.120+0000 7efeb6ffd700 1 -- 172.21.15.175:0/509460444 learned_addr learned my addr 172.21.15.175:0/509460444 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:26.034 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.120+0000 7efeb6ffd700 1 -- 172.21.15.175:0/509460444 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb807b340 con 0x7efeb807c740 2024-03-26T03:34:26.034 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.120+0000 7efeb6ffd700 1 --2- 172.21.15.175:0/509460444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efeb807c740 0x7efeb807abf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efea80098a0 tx=0x7efea8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7bd3300cdef992 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:26.034 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.120+0000 7efeb67fc700 1 -- 172.21.15.175:0/509460444 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efea8004160 con 0x7efeb807c740 2024-03-26T03:34:26.034 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.120+0000 7efeb67fc700 1 -- 172.21.15.175:0/509460444 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7efea8004300 con 0x7efeb807c740 2024-03-26T03:34:26.034 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.120+0000 7efeb67fc700 1 -- 172.21.15.175:0/509460444 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efea8011940 con 0x7efeb807c740 2024-03-26T03:34:26.035 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.120+0000 7efebd3c2700 1 -- 172.21.15.175:0/509460444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efeb807c740 msgr2=0x7efeb807abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:26.035 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.120+0000 7efebd3c2700 1 --2- 172.21.15.175:0/509460444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efeb807c740 0x7efeb807abf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efea80098a0 tx=0x7efea8009bb0 comp rx=0 tx=0).stop 2024-03-26T03:34:26.035 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.120+0000 7efebd3c2700 1 -- 172.21.15.175:0/509460444 shutdown_connections 2024-03-26T03:34:26.035 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.120+0000 7efebd3c2700 1 --2- 172.21.15.175:0/509460444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efeb807c740 0x7efeb807abf0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:26.035 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.120+0000 7efebd3c2700 1 -- 172.21.15.175:0/509460444 >> 172.21.15.175:0/509460444 conn(0x7efeb8101980 msgr2=0x7efeb8103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:26.035 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.120+0000 7efebd3c2700 1 -- 172.21.15.175:0/509460444 shutdown_connections 2024-03-26T03:34:26.035 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.120+0000 7efebd3c2700 1 -- 172.21.15.175:0/509460444 wait complete. 2024-03-26T03:34:26.035 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.120+0000 7efebd3c2700 1 Processor -- start 2024-03-26T03:34:26.035 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.120+0000 7efebd3c2700 1 -- start start 2024-03-26T03:34:26.035 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.120+0000 7efebd3c2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efeb807c740 0x7efeb819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:26.035 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.120+0000 7efebd3c2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efeb810c8d0 con 0x7efeb807c740 2024-03-26T03:34:26.035 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.120+0000 7efeb6ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efeb807c740 0x7efeb819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:26.035 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.120+0000 7efeb6ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efeb807c740 0x7efeb819c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:44122/0 (socket says 172.21.15.175:44122) 2024-03-26T03:34:26.036 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.120+0000 7efeb6ffd700 1 -- 172.21.15.175:0/2531108756 learned_addr learned my addr 172.21.15.175:0/2531108756 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:26.036 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.120+0000 7efeb6ffd700 1 -- 172.21.15.175:0/2531108756 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb819cbe0 con 0x7efeb807c740 2024-03-26T03:34:26.036 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.124+0000 7efeb6ffd700 1 --2- 172.21.15.175:0/2531108756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efeb807c740 0x7efeb819c6a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efea8011910 tx=0x7efea80119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:26.036 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.124+0000 7efeb4ff9700 1 -- 172.21.15.175:0/2531108756 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efea8003ac0 con 0x7efeb807c740 2024-03-26T03:34:26.036 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.124+0000 7efeb4ff9700 1 -- 172.21.15.175:0/2531108756 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7efea801a890 con 0x7efeb807c740 2024-03-26T03:34:26.036 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.124+0000 7efebd3c2700 1 -- 172.21.15.175:0/2531108756 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb819ce70 con 0x7efeb807c740 2024-03-26T03:34:26.036 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.124+0000 7efeb4ff9700 1 -- 172.21.15.175:0/2531108756 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efea80197a0 con 0x7efeb807c740 2024-03-26T03:34:26.036 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.124+0000 7efebd3c2700 1 -- 172.21.15.175:0/2531108756 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb819fb60 con 0x7efeb807c740 2024-03-26T03:34:26.036 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.124+0000 7efeb4ff9700 1 -- 172.21.15.175:0/2531108756 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 8) ==== 46466+0+0 (secure 0 0 0) 0x7efea801a430 con 0x7efeb807c740 2024-03-26T03:34:26.036 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.124+0000 7efebd3c2700 1 -- 172.21.15.175:0/2531108756 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeb8063c70 con 0x7efeb807c740 2024-03-26T03:34:26.036 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.124+0000 7efeb4ff9700 1 --2- 172.21.15.175:0/2531108756 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7efea40394e0 0x7efea403b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:26.037 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.124+0000 7efeb4ff9700 1 -- 172.21.15.175:0/2531108756 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efea804d940 con 0x7efeb807c740 2024-03-26T03:34:26.037 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.124+0000 7efeaffff700 1 --2- 172.21.15.175:0/2531108756 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7efea40394e0 0x7efea403b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:26.037 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.124+0000 7efeaffff700 1 --2- 172.21.15.175:0/2531108756 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7efea40394e0 0x7efea403b9a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efea0006fd0 tx=0x7efea0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:26.037 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.128+0000 7efeb4ff9700 1 -- 172.21.15.175:0/2531108756 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7efea801ad90 con 0x7efeb807c740 2024-03-26T03:34:26.037 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.492+0000 7efebd3c2700 1 -- 172.21.15.175:0/2531108756 --> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7efeb806be50 con 0x7efea40394e0 2024-03-26T03:34:26.037 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.492+0000 7efeb4ff9700 1 -- 172.21.15.175:0/2531108756 <== mgr.14120 v2:172.21.15.175:6800/712246762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7efeb806be50 con 0x7efea40394e0 2024-03-26T03:34:26.037 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.496+0000 7efebd3c2700 1 -- 172.21.15.175:0/2531108756 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7efea40394e0 msgr2=0x7efea403b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:26.037 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.496+0000 7efebd3c2700 1 --2- 172.21.15.175:0/2531108756 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7efea40394e0 0x7efea403b9a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efea0006fd0 tx=0x7efea0006e40 comp rx=0 tx=0).stop 2024-03-26T03:34:26.037 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.496+0000 7efebd3c2700 1 -- 172.21.15.175:0/2531108756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efeb807c740 msgr2=0x7efeb819c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:26.037 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.496+0000 7efebd3c2700 1 --2- 172.21.15.175:0/2531108756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efeb807c740 0x7efeb819c6a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efea8011910 tx=0x7efea80119f0 comp rx=0 tx=0).stop 2024-03-26T03:34:26.037 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.496+0000 7efebd3c2700 1 -- 172.21.15.175:0/2531108756 shutdown_connections 2024-03-26T03:34:26.037 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.496+0000 7efebd3c2700 1 --2- 172.21.15.175:0/2531108756 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7efea40394e0 0x7efea403b9a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:26.038 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.496+0000 7efebd3c2700 1 --2- 172.21.15.175:0/2531108756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efeb807c740 0x7efeb819c6a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:26.038 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.496+0000 7efebd3c2700 1 -- 172.21.15.175:0/2531108756 >> 172.21.15.175:0/2531108756 conn(0x7efeb8101980 msgr2=0x7efeb8102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:26.038 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.496+0000 7efebd3c2700 1 -- 172.21.15.175:0/2531108756 shutdown_connections 2024-03-26T03:34:26.038 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:25.496+0000 7efebd3c2700 1 -- 172.21.15.175:0/2531108756 wait complete. 2024-03-26T03:34:26.038 INFO:teuthology.orchestra.run.smithi175.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-26T03:34:26.038 INFO:teuthology.orchestra.run.smithi175.stdout:Adding key to root@localhost authorized_keys... 2024-03-26T03:34:26.038 INFO:teuthology.orchestra.run.smithi175.stdout:Adding host smithi175... 2024-03-26T03:34:26.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:25 smithi175 bash[21527]: audit 2024-03-26T03:34:25.496435+0000 mgr.y (mgr.14120) 12 : audit [DBG] from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:34:28.229 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:27 smithi175 bash[21527]: audit 2024-03-26T03:34:27.343609+0000 mgr.y (mgr.14120) 13 : audit [DBG] from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi175", "addr": "172.21.15.175", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:34:29.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:28 smithi175 bash[21527]: cephadm 2024-03-26T03:34:27.920963+0000 mgr.y (mgr.14120) 14 : cephadm [INF] Deploying cephadm binary to smithi175 2024-03-26T03:34:33.579 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout Added host 'smithi175' with addr '172.21.15.175' 2024-03-26T03:34:33.579 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.952+0000 7f35dab86700 1 Processor -- start 2024-03-26T03:34:33.579 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.952+0000 7f35dab86700 1 -- start start 2024-03-26T03:34:33.579 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.952+0000 7f35dab86700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35d4106460 0x7f35d4106880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:33.579 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.952+0000 7f35dab86700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f35d4106e50 con 0x7f35d4106460 2024-03-26T03:34:33.579 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.952+0000 7f35d8922700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35d4106460 0x7f35d4106880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:33.579 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.952+0000 7f35d8922700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35d4106460 0x7f35d4106880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:44124/0 (socket says 172.21.15.175:44124) 2024-03-26T03:34:33.579 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.952+0000 7f35d8922700 1 -- 172.21.15.175:0/1055065718 learned_addr learned my addr 172.21.15.175:0/1055065718 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:33.579 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.952+0000 7f35d8922700 1 -- 172.21.15.175:0/1055065718 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d41075c0 con 0x7f35d4106460 2024-03-26T03:34:33.579 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.952+0000 7f35d8922700 1 --2- 172.21.15.175:0/1055065718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35d4106460 0x7f35d4106880 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f35c40098a0 tx=0x7f35c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2ffeab4931c0b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:33.579 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.952+0000 7f35d37fe700 1 -- 172.21.15.175:0/1055065718 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35c4004160 con 0x7f35d4106460 2024-03-26T03:34:33.579 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.952+0000 7f35d37fe700 1 -- 172.21.15.175:0/1055065718 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f35c4004300 con 0x7f35d4106460 2024-03-26T03:34:33.580 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.952+0000 7f35d37fe700 1 -- 172.21.15.175:0/1055065718 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35c4011940 con 0x7f35d4106460 2024-03-26T03:34:33.580 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.952+0000 7f35dab86700 1 -- 172.21.15.175:0/1055065718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35d4106460 msgr2=0x7f35d4106880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:33.580 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.952+0000 7f35dab86700 1 --2- 172.21.15.175:0/1055065718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35d4106460 0x7f35d4106880 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f35c40098a0 tx=0x7f35c4009bb0 comp rx=0 tx=0).stop 2024-03-26T03:34:33.580 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.952+0000 7f35dab86700 1 -- 172.21.15.175:0/1055065718 shutdown_connections 2024-03-26T03:34:33.580 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.952+0000 7f35dab86700 1 --2- 172.21.15.175:0/1055065718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35d4106460 0x7f35d4106880 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:33.580 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.952+0000 7f35dab86700 1 -- 172.21.15.175:0/1055065718 >> 172.21.15.175:0/1055065718 conn(0x7f35d4101a20 msgr2=0x7f35d4103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:33.580 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.952+0000 7f35dab86700 1 -- 172.21.15.175:0/1055065718 shutdown_connections 2024-03-26T03:34:33.580 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.952+0000 7f35dab86700 1 -- 172.21.15.175:0/1055065718 wait complete. 2024-03-26T03:34:33.580 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.956+0000 7f35dab86700 1 Processor -- start 2024-03-26T03:34:33.580 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.956+0000 7f35dab86700 1 -- start start 2024-03-26T03:34:33.580 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.956+0000 7f35dab86700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35d4106460 0x7f35d419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:33.580 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.956+0000 7f35dab86700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f35d410a7f0 con 0x7f35d4106460 2024-03-26T03:34:33.580 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.956+0000 7f35d8922700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35d4106460 0x7f35d419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:33.580 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.956+0000 7f35d8922700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35d4106460 0x7f35d419a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:44132/0 (socket says 172.21.15.175:44132) 2024-03-26T03:34:33.580 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.956+0000 7f35d8922700 1 -- 172.21.15.175:0/2593837186 learned_addr learned my addr 172.21.15.175:0/2593837186 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:33.580 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.956+0000 7f35d8922700 1 -- 172.21.15.175:0/2593837186 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d419aae0 con 0x7f35d4106460 2024-03-26T03:34:33.581 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.956+0000 7f35d8922700 1 --2- 172.21.15.175:0/2593837186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35d4106460 0x7f35d419a5a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f35c4011910 tx=0x7f35c40119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:33.581 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.956+0000 7f35d1ffb700 1 -- 172.21.15.175:0/2593837186 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35c4003b20 con 0x7f35d4106460 2024-03-26T03:34:33.581 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.956+0000 7f35d1ffb700 1 -- 172.21.15.175:0/2593837186 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f35c401a890 con 0x7f35d4106460 2024-03-26T03:34:33.581 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.956+0000 7f35d1ffb700 1 -- 172.21.15.175:0/2593837186 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35c40197f0 con 0x7f35d4106460 2024-03-26T03:34:33.581 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.956+0000 7f35dab86700 1 -- 172.21.15.175:0/2593837186 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d419ad70 con 0x7f35d4106460 2024-03-26T03:34:33.581 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.956+0000 7f35dab86700 1 -- 172.21.15.175:0/2593837186 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d419d9c0 con 0x7f35d4106460 2024-03-26T03:34:33.581 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.956+0000 7f35d1ffb700 1 -- 172.21.15.175:0/2593837186 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 8) ==== 46466+0+0 (secure 0 0 0) 0x7f35c4022020 con 0x7f35d4106460 2024-03-26T03:34:33.581 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.956+0000 7f35dab86700 1 -- 172.21.15.175:0/2593837186 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35d410a970 con 0x7f35d4106460 2024-03-26T03:34:33.581 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.956+0000 7f35d1ffb700 1 --2- 172.21.15.175:0/2593837186 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f35b4039490 0x7f35b403b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:33.581 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.956+0000 7f35d1ffb700 1 -- 172.21.15.175:0/2593837186 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f35c404d680 con 0x7f35d4106460 2024-03-26T03:34:33.581 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.956+0000 7f35d3fff700 1 --2- 172.21.15.175:0/2593837186 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f35b4039490 0x7f35b403b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:33.581 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.956+0000 7f35d3fff700 1 --2- 172.21.15.175:0/2593837186 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f35b4039490 0x7f35b403b950 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f35bc006fd0 tx=0x7f35bc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:33.581 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:26.964+0000 7f35d1ffb700 1 -- 172.21.15.175:0/2593837186 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f35c40172b0 con 0x7f35d4106460 2024-03-26T03:34:33.581 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:27.336+0000 7f35dab86700 1 -- 172.21.15.175:0/2593837186 --> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi175", "addr": "172.21.15.175", "target": ["mon-mgr", ""]}) -- 0x7f35d406be20 con 0x7f35b4039490 2024-03-26T03:34:33.583 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:32.764+0000 7f35d1ffb700 1 -- 172.21.15.175:0/2593837186 <== mgr.14120 v2:172.21.15.175:6800/712246762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f35d406be20 con 0x7f35b4039490 2024-03-26T03:34:33.583 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:32.768+0000 7f35dab86700 1 -- 172.21.15.175:0/2593837186 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f35b4039490 msgr2=0x7f35b403b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:33.583 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:32.768+0000 7f35dab86700 1 --2- 172.21.15.175:0/2593837186 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f35b4039490 0x7f35b403b950 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f35bc006fd0 tx=0x7f35bc006e40 comp rx=0 tx=0).stop 2024-03-26T03:34:33.583 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:32.768+0000 7f35dab86700 1 -- 172.21.15.175:0/2593837186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35d4106460 msgr2=0x7f35d419a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:33.583 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:32.768+0000 7f35dab86700 1 --2- 172.21.15.175:0/2593837186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35d4106460 0x7f35d419a5a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f35c4011910 tx=0x7f35c40119f0 comp rx=0 tx=0).stop 2024-03-26T03:34:33.583 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:32.768+0000 7f35dab86700 1 -- 172.21.15.175:0/2593837186 shutdown_connections 2024-03-26T03:34:33.583 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:32.768+0000 7f35dab86700 1 --2- 172.21.15.175:0/2593837186 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f35b4039490 0x7f35b403b950 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:33.583 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:32.768+0000 7f35dab86700 1 --2- 172.21.15.175:0/2593837186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35d4106460 0x7f35d419a5a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:33.583 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:32.768+0000 7f35dab86700 1 -- 172.21.15.175:0/2593837186 >> 172.21.15.175:0/2593837186 conn(0x7f35d4101a20 msgr2=0x7f35d4102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:33.583 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:32.768+0000 7f35dab86700 1 -- 172.21.15.175:0/2593837186 shutdown_connections 2024-03-26T03:34:33.584 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:32.768+0000 7f35dab86700 1 -- 172.21.15.175:0/2593837186 wait complete. 2024-03-26T03:34:33.584 INFO:teuthology.orchestra.run.smithi175.stdout:Deploying unmanaged mon service... 2024-03-26T03:34:33.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:33 smithi175 bash[21527]: audit 2024-03-26T03:34:32.769586+0000 mon.a (mon.0) 62 : audit [INF] from='mgr.14120 172.21.15.175:0/3794702729' entity='mgr.y' 2024-03-26T03:34:33.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:33 smithi175 bash[21527]: cephadm 2024-03-26T03:34:32.770024+0000 mgr.y (mgr.14120) 15 : cephadm [INF] Added host smithi175 2024-03-26T03:34:33.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:33 smithi175 bash[21527]: audit 2024-03-26T03:34:32.817477+0000 mon.a (mon.0) 63 : audit [DBG] from='mgr.14120 172.21.15.175:0/3794702729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:34:36.184 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-26T03:34:36.185 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.944+0000 7f40bec2d700 1 Processor -- start 2024-03-26T03:34:36.185 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.944+0000 7f40bec2d700 1 -- start start 2024-03-26T03:34:36.186 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.944+0000 7f40bec2d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40b807c6f0 0x7f40b807abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:36.186 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.944+0000 7f40bec2d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f40b807b130 con 0x7f40b807c6f0 2024-03-26T03:34:36.186 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.948+0000 7f40bc9c9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40b807c6f0 0x7f40b807abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:36.186 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.948+0000 7f40bc9c9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40b807c6f0 0x7f40b807abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45786/0 (socket says 172.21.15.175:45786) 2024-03-26T03:34:36.186 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.948+0000 7f40bc9c9700 1 -- 172.21.15.175:0/3711374825 learned_addr learned my addr 172.21.15.175:0/3711374825 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:36.186 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.948+0000 7f40bc9c9700 1 -- 172.21.15.175:0/3711374825 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b807b2b0 con 0x7f40b807c6f0 2024-03-26T03:34:36.186 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.948+0000 7f40bc9c9700 1 --2- 172.21.15.175:0/3711374825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40b807c6f0 0x7f40b807abf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0098a0 tx=0x7f40ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d165c4a61b0f3143 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:36.186 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.948+0000 7f40b77fe700 1 -- 172.21.15.175:0/3711374825 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40ac004160 con 0x7f40b807c6f0 2024-03-26T03:34:36.186 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.948+0000 7f40b77fe700 1 -- 172.21.15.175:0/3711374825 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f40ac004300 con 0x7f40b807c6f0 2024-03-26T03:34:36.186 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.948+0000 7f40b77fe700 1 -- 172.21.15.175:0/3711374825 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40ac011940 con 0x7f40b807c6f0 2024-03-26T03:34:36.187 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.948+0000 7f40bec2d700 1 -- 172.21.15.175:0/3711374825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40b807c6f0 msgr2=0x7f40b807abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:36.187 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.948+0000 7f40bec2d700 1 --2- 172.21.15.175:0/3711374825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40b807c6f0 0x7f40b807abf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0098a0 tx=0x7f40ac009bb0 comp rx=0 tx=0).stop 2024-03-26T03:34:36.187 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.948+0000 7f40bec2d700 1 -- 172.21.15.175:0/3711374825 shutdown_connections 2024-03-26T03:34:36.187 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.948+0000 7f40bec2d700 1 --2- 172.21.15.175:0/3711374825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40b807c6f0 0x7f40b807abf0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:36.187 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.948+0000 7f40bec2d700 1 -- 172.21.15.175:0/3711374825 >> 172.21.15.175:0/3711374825 conn(0x7f40b8101a60 msgr2=0x7f40b8103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:36.187 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.948+0000 7f40bec2d700 1 -- 172.21.15.175:0/3711374825 shutdown_connections 2024-03-26T03:34:36.187 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.948+0000 7f40bec2d700 1 -- 172.21.15.175:0/3711374825 wait complete. 2024-03-26T03:34:36.187 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.948+0000 7f40bec2d700 1 Processor -- start 2024-03-26T03:34:36.187 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.948+0000 7f40bec2d700 1 -- start start 2024-03-26T03:34:36.187 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.948+0000 7f40bec2d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40b807c6f0 0x7f40b81a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:36.187 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.948+0000 7f40bec2d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f40b81084a0 con 0x7f40b807c6f0 2024-03-26T03:34:36.187 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.948+0000 7f40bc9c9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40b807c6f0 0x7f40b81a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:36.188 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.948+0000 7f40bc9c9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40b807c6f0 0x7f40b81a0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45788/0 (socket says 172.21.15.175:45788) 2024-03-26T03:34:36.188 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.948+0000 7f40bc9c9700 1 -- 172.21.15.175:0/3024169746 learned_addr learned my addr 172.21.15.175:0/3024169746 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:36.188 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.948+0000 7f40bc9c9700 1 -- 172.21.15.175:0/3024169746 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b81a0fd0 con 0x7f40b807c6f0 2024-03-26T03:34:36.188 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.948+0000 7f40bc9c9700 1 --2- 172.21.15.175:0/3024169746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40b807c6f0 0x7f40b81a0a90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009290 tx=0x7f40ac0119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:36.188 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.948+0000 7f40b5ffb700 1 -- 172.21.15.175:0/3024169746 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40ac003930 con 0x7f40b807c6f0 2024-03-26T03:34:36.188 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.948+0000 7f40b5ffb700 1 -- 172.21.15.175:0/3024169746 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1011+0+0 (secure 0 0 0) 0x7f40ac003ad0 con 0x7f40b807c6f0 2024-03-26T03:34:36.188 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.948+0000 7f40bec2d700 1 -- 172.21.15.175:0/3024169746 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b81a1260 con 0x7f40b807c6f0 2024-03-26T03:34:36.188 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.948+0000 7f40b5ffb700 1 -- 172.21.15.175:0/3024169746 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40ac019720 con 0x7f40b807c6f0 2024-03-26T03:34:36.188 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.948+0000 7f40bec2d700 1 -- 172.21.15.175:0/3024169746 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b81a3f50 con 0x7f40b807c6f0 2024-03-26T03:34:36.188 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.952+0000 7f40b5ffb700 1 -- 172.21.15.175:0/3024169746 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 8) ==== 46466+0+0 (secure 0 0 0) 0x7f40ac01a430 con 0x7f40b807c6f0 2024-03-26T03:34:36.188 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.952+0000 7f40bec2d700 1 -- 172.21.15.175:0/3024169746 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40b8063c70 con 0x7f40b807c6f0 2024-03-26T03:34:36.188 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.952+0000 7f40b5ffb700 1 --2- 172.21.15.175:0/3024169746 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f40a0039490 0x7f40a003b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:36.189 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.952+0000 7f40b5ffb700 1 -- 172.21.15.175:0/3024169746 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f40ac04dc00 con 0x7f40b807c6f0 2024-03-26T03:34:36.189 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.952+0000 7f40b7fff700 1 --2- 172.21.15.175:0/3024169746 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f40a0039490 0x7f40a003b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:36.189 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.952+0000 7f40b7fff700 1 --2- 172.21.15.175:0/3024169746 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f40a0039490 0x7f40a003b950 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f40a8006fd0 tx=0x7f40a8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:36.189 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:34.956+0000 7f40b5ffb700 1 -- 172.21.15.175:0/3024169746 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f40ac019d50 con 0x7f40b807c6f0 2024-03-26T03:34:36.189 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:35.356+0000 7f40bec2d700 1 -- 172.21.15.175:0/3024169746 --> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f40b806bdf0 con 0x7f40a0039490 2024-03-26T03:34:36.189 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:35.364+0000 7f40b5ffb700 1 -- 172.21.15.175:0/3024169746 <== mgr.14120 v2:172.21.15.175:6800/712246762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f40b806bdf0 con 0x7f40a0039490 2024-03-26T03:34:36.189 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:35.368+0000 7f40bec2d700 1 -- 172.21.15.175:0/3024169746 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f40a0039490 msgr2=0x7f40a003b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:36.189 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:35.368+0000 7f40bec2d700 1 --2- 172.21.15.175:0/3024169746 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f40a0039490 0x7f40a003b950 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f40a8006fd0 tx=0x7f40a8006e40 comp rx=0 tx=0).stop 2024-03-26T03:34:36.189 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:35.368+0000 7f40bec2d700 1 -- 172.21.15.175:0/3024169746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40b807c6f0 msgr2=0x7f40b81a0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:36.189 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:35.368+0000 7f40bec2d700 1 --2- 172.21.15.175:0/3024169746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40b807c6f0 0x7f40b81a0a90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009290 tx=0x7f40ac0119c0 comp rx=0 tx=0).stop 2024-03-26T03:34:36.190 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:35.368+0000 7f40bec2d700 1 -- 172.21.15.175:0/3024169746 shutdown_connections 2024-03-26T03:34:36.190 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:35.368+0000 7f40bec2d700 1 --2- 172.21.15.175:0/3024169746 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f40a0039490 0x7f40a003b950 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:36.190 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:35.368+0000 7f40bec2d700 1 --2- 172.21.15.175:0/3024169746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40b807c6f0 0x7f40b81a0a90 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:36.190 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:35.368+0000 7f40bec2d700 1 -- 172.21.15.175:0/3024169746 >> 172.21.15.175:0/3024169746 conn(0x7f40b8101a60 msgr2=0x7f40b8197750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:36.190 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:35.368+0000 7f40bec2d700 1 -- 172.21.15.175:0/3024169746 shutdown_connections 2024-03-26T03:34:36.190 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:35.368+0000 7f40bec2d700 1 -- 172.21.15.175:0/3024169746 wait complete. 2024-03-26T03:34:36.190 INFO:teuthology.orchestra.run.smithi175.stdout:Deploying unmanaged mgr service... 2024-03-26T03:34:36.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:36 smithi175 bash[21527]: audit 2024-03-26T03:34:35.362790+0000 mgr.y (mgr.14120) 16 : audit [DBG] from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:34:36.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:36 smithi175 bash[21527]: cephadm 2024-03-26T03:34:35.364755+0000 mgr.y (mgr.14120) 17 : cephadm [INF] Saving service mon spec with placement count:5 2024-03-26T03:34:36.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:36 smithi175 bash[21527]: audit 2024-03-26T03:34:35.369080+0000 mon.a (mon.0) 64 : audit [INF] from='mgr.14120 172.21.15.175:0/3794702729' entity='mgr.y' 2024-03-26T03:34:36.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:36 smithi175 bash[21527]: audit 2024-03-26T03:34:35.893765+0000 mon.a (mon.0) 65 : audit [INF] from='mgr.14120 172.21.15.175:0/3794702729' entity='mgr.y' 2024-03-26T03:34:37.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:37 smithi175 bash[21527]: audit 2024-03-26T03:34:36.436159+0000 mon.a (mon.0) 66 : audit [INF] from='mgr.14120 172.21.15.175:0/3794702729' entity='mgr.y' 2024-03-26T03:34:37.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:37 smithi175 bash[21527]: audit 2024-03-26T03:34:37.404664+0000 mon.a (mon.0) 67 : audit [INF] from='mgr.14120 172.21.15.175:0/3794702729' entity='mgr.y' 2024-03-26T03:34:37.988 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-26T03:34:37.988 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.992+0000 7f0095845700 1 Processor -- start 2024-03-26T03:34:37.988 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.992+0000 7f0095845700 1 -- start start 2024-03-26T03:34:37.988 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.996+0000 7f0095845700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0090108680 0x7f0090108aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:37.988 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.996+0000 7f0095845700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0090109070 con 0x7f0090108680 2024-03-26T03:34:37.988 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.996+0000 7f008effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0090108680 0x7f0090108aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:37.988 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.996+0000 7f008effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0090108680 0x7f0090108aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45794/0 (socket says 172.21.15.175:45794) 2024-03-26T03:34:37.988 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.996+0000 7f008effd700 1 -- 172.21.15.175:0/2520673692 learned_addr learned my addr 172.21.15.175:0/2520673692 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:37.988 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.996+0000 7f008effd700 1 -- 172.21.15.175:0/2520673692 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00901097e0 con 0x7f0090108680 2024-03-26T03:34:37.988 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.996+0000 7f008effd700 1 --2- 172.21.15.175:0/2520673692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0090108680 0x7f0090108aa0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f00800098a0 tx=0x7f0080009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c030700b575959ce server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:37.988 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.996+0000 7f008e7fc700 1 -- 172.21.15.175:0/2520673692 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0080004160 con 0x7f0090108680 2024-03-26T03:34:37.989 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.996+0000 7f008e7fc700 1 -- 172.21.15.175:0/2520673692 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f0080004300 con 0x7f0090108680 2024-03-26T03:34:37.989 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.996+0000 7f008e7fc700 1 -- 172.21.15.175:0/2520673692 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00800118f0 con 0x7f0090108680 2024-03-26T03:34:37.989 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.996+0000 7f0095845700 1 -- 172.21.15.175:0/2520673692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0090108680 msgr2=0x7f0090108aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:37.989 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.996+0000 7f0095845700 1 --2- 172.21.15.175:0/2520673692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0090108680 0x7f0090108aa0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f00800098a0 tx=0x7f0080009bb0 comp rx=0 tx=0).stop 2024-03-26T03:34:37.989 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.996+0000 7f0095845700 1 -- 172.21.15.175:0/2520673692 shutdown_connections 2024-03-26T03:34:37.989 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.996+0000 7f0095845700 1 --2- 172.21.15.175:0/2520673692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0090108680 0x7f0090108aa0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:37.989 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.996+0000 7f0095845700 1 -- 172.21.15.175:0/2520673692 >> 172.21.15.175:0/2520673692 conn(0x7f009007bc10 msgr2=0x7f009007c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:37.989 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.996+0000 7f0095845700 1 -- 172.21.15.175:0/2520673692 shutdown_connections 2024-03-26T03:34:37.989 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.996+0000 7f0095845700 1 -- 172.21.15.175:0/2520673692 wait complete. 2024-03-26T03:34:37.989 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.996+0000 7f0095845700 1 Processor -- start 2024-03-26T03:34:37.989 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.996+0000 7f0095845700 1 -- start start 2024-03-26T03:34:37.989 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.996+0000 7f0095845700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0090108680 0x7f009019c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:37.989 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.996+0000 7f0095845700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f009010ca10 con 0x7f0090108680 2024-03-26T03:34:37.989 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.996+0000 7f008effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0090108680 0x7f009019c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:37.989 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.996+0000 7f008effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0090108680 0x7f009019c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45806/0 (socket says 172.21.15.175:45806) 2024-03-26T03:34:37.990 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.996+0000 7f008effd700 1 -- 172.21.15.175:0/1842807513 learned_addr learned my addr 172.21.15.175:0/1842807513 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:37.990 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.996+0000 7f008effd700 1 -- 172.21.15.175:0/1842807513 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f009019cbf0 con 0x7f0090108680 2024-03-26T03:34:37.990 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.996+0000 7f008effd700 1 --2- 172.21.15.175:0/1842807513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0090108680 0x7f009019c6b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0080009870 tx=0x7f0080011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:37.990 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.996+0000 7f0094843700 1 -- 172.21.15.175:0/1842807513 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00800038b0 con 0x7f0090108680 2024-03-26T03:34:37.990 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.996+0000 7f0094843700 1 -- 172.21.15.175:0/1842807513 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f0080003a50 con 0x7f0090108680 2024-03-26T03:34:37.990 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.996+0000 7f0095845700 1 -- 172.21.15.175:0/1842807513 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f009019ce80 con 0x7f0090108680 2024-03-26T03:34:37.990 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.996+0000 7f0094843700 1 -- 172.21.15.175:0/1842807513 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0080024410 con 0x7f0090108680 2024-03-26T03:34:37.990 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:36.996+0000 7f0095845700 1 -- 172.21.15.175:0/1842807513 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f009019fb70 con 0x7f0090108680 2024-03-26T03:34:37.990 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:37.000+0000 7f0094843700 1 -- 172.21.15.175:0/1842807513 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 8) ==== 46466+0+0 (secure 0 0 0) 0x7f0080022020 con 0x7f0090108680 2024-03-26T03:34:37.990 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:37.000+0000 7f0095845700 1 -- 172.21.15.175:0/1842807513 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f009010cb90 con 0x7f0090108680 2024-03-26T03:34:37.990 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:37.000+0000 7f0094843700 1 --2- 172.21.15.175:0/1842807513 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f007c0394e0 0x7f007c03b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:37.990 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:37.000+0000 7f0094843700 1 -- 172.21.15.175:0/1842807513 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f008004d5a0 con 0x7f0090108680 2024-03-26T03:34:37.990 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:37.000+0000 7f0087fff700 1 --2- 172.21.15.175:0/1842807513 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f007c0394e0 0x7f007c03b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:37.990 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:37.000+0000 7f0087fff700 1 --2- 172.21.15.175:0/1842807513 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f007c0394e0 0x7f007c03b9a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0078006fd0 tx=0x7f0078006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:37.991 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:37.004+0000 7f0094843700 1 -- 172.21.15.175:0/1842807513 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f00800212c0 con 0x7f0090108680 2024-03-26T03:34:37.991 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:37.388+0000 7f0095845700 1 -- 172.21.15.175:0/1842807513 --> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f009006bdf0 con 0x7f007c0394e0 2024-03-26T03:34:37.991 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:37.400+0000 7f0094843700 1 -- 172.21.15.175:0/1842807513 <== mgr.14120 v2:172.21.15.175:6800/712246762 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f009006bdf0 con 0x7f007c0394e0 2024-03-26T03:34:37.991 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:37.404+0000 7f0095845700 1 -- 172.21.15.175:0/1842807513 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f007c0394e0 msgr2=0x7f007c03b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:37.991 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:37.404+0000 7f0095845700 1 --2- 172.21.15.175:0/1842807513 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f007c0394e0 0x7f007c03b9a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0078006fd0 tx=0x7f0078006e40 comp rx=0 tx=0).stop 2024-03-26T03:34:37.991 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:37.404+0000 7f0095845700 1 -- 172.21.15.175:0/1842807513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0090108680 msgr2=0x7f009019c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:37.991 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:37.404+0000 7f0095845700 1 --2- 172.21.15.175:0/1842807513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0090108680 0x7f009019c6b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0080009870 tx=0x7f0080011ef0 comp rx=0 tx=0).stop 2024-03-26T03:34:37.991 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:37.404+0000 7f0095845700 1 -- 172.21.15.175:0/1842807513 shutdown_connections 2024-03-26T03:34:37.991 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:37.404+0000 7f0095845700 1 --2- 172.21.15.175:0/1842807513 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f007c0394e0 0x7f007c03b9a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:37.991 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:37.404+0000 7f0095845700 1 --2- 172.21.15.175:0/1842807513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0090108680 0x7f009019c6b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:37.991 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:37.404+0000 7f0095845700 1 -- 172.21.15.175:0/1842807513 >> 172.21.15.175:0/1842807513 conn(0x7f009007bc10 msgr2=0x7f0090105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:37.991 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:37.404+0000 7f0095845700 1 -- 172.21.15.175:0/1842807513 shutdown_connections 2024-03-26T03:34:37.991 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:37.404+0000 7f0095845700 1 -- 172.21.15.175:0/1842807513 wait complete. 2024-03-26T03:34:38.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:38 smithi175 bash[21527]: cluster 2024-03-26T03:34:36.898893+0000 mgr.y (mgr.14120) 18 : cluster [DBG] pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:34:38.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:38 smithi175 bash[21527]: audit 2024-03-26T03:34:37.397303+0000 mgr.y (mgr.14120) 19 : audit [DBG] from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:34:38.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:38 smithi175 bash[21527]: cephadm 2024-03-26T03:34:37.399873+0000 mgr.y (mgr.14120) 20 : cephadm [INF] Saving service mgr spec with placement count:2 2024-03-26T03:34:39.765 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.788+0000 7fd87ae90700 1 Processor -- start 2024-03-26T03:34:39.765 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.788+0000 7fd87ae90700 1 -- start start 2024-03-26T03:34:39.765 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.788+0000 7fd87ae90700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd874106400 0x7fd874106820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:39.765 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.788+0000 7fd87ae90700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd874106df0 con 0x7fd874106400 2024-03-26T03:34:39.766 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.792+0000 7fd878c2c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd874106400 0x7fd874106820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:39.766 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.792+0000 7fd878c2c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd874106400 0x7fd874106820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45810/0 (socket says 172.21.15.175:45810) 2024-03-26T03:34:39.766 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.792+0000 7fd878c2c700 1 -- 172.21.15.175:0/1848466076 learned_addr learned my addr 172.21.15.175:0/1848466076 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:39.766 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.792+0000 7fd878c2c700 1 -- 172.21.15.175:0/1848466076 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8741075b0 con 0x7fd874106400 2024-03-26T03:34:39.766 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.792+0000 7fd878c2c700 1 --2- 172.21.15.175:0/1848466076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd874106400 0x7fd874106820 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd864009ab0 tx=0x7fd864009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41bf6d8af664d392 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:39.766 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.792+0000 7fd8737fe700 1 -- 172.21.15.175:0/1848466076 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd864004160 con 0x7fd874106400 2024-03-26T03:34:39.766 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.792+0000 7fd8737fe700 1 -- 172.21.15.175:0/1848466076 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fd86400bc60 con 0x7fd874106400 2024-03-26T03:34:39.766 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.792+0000 7fd8737fe700 1 -- 172.21.15.175:0/1848466076 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd864003ac0 con 0x7fd874106400 2024-03-26T03:34:39.766 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.792+0000 7fd87ae90700 1 -- 172.21.15.175:0/1848466076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd874106400 msgr2=0x7fd874106820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:39.766 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.792+0000 7fd87ae90700 1 --2- 172.21.15.175:0/1848466076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd874106400 0x7fd874106820 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd864009ab0 tx=0x7fd864009dc0 comp rx=0 tx=0).stop 2024-03-26T03:34:39.766 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.792+0000 7fd87ae90700 1 -- 172.21.15.175:0/1848466076 shutdown_connections 2024-03-26T03:34:39.771 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.792+0000 7fd87ae90700 1 --2- 172.21.15.175:0/1848466076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd874106400 0x7fd874106820 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:39.771 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.792+0000 7fd87ae90700 1 -- 172.21.15.175:0/1848466076 >> 172.21.15.175:0/1848466076 conn(0x7fd874101960 msgr2=0x7fd874103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:39.771 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.792+0000 7fd87ae90700 1 -- 172.21.15.175:0/1848466076 shutdown_connections 2024-03-26T03:34:39.771 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.792+0000 7fd87ae90700 1 -- 172.21.15.175:0/1848466076 wait complete. 2024-03-26T03:34:39.771 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.792+0000 7fd87ae90700 1 Processor -- start 2024-03-26T03:34:39.771 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.792+0000 7fd87ae90700 1 -- start start 2024-03-26T03:34:39.771 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.792+0000 7fd87ae90700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd874106400 0x7fd87419c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:39.771 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.792+0000 7fd87ae90700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd87410a7e0 con 0x7fd874106400 2024-03-26T03:34:39.771 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.792+0000 7fd878c2c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd874106400 0x7fd87419c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:39.771 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.792+0000 7fd878c2c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd874106400 0x7fd87419c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45820/0 (socket says 172.21.15.175:45820) 2024-03-26T03:34:39.772 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.792+0000 7fd878c2c700 1 -- 172.21.15.175:0/2982719172 learned_addr learned my addr 172.21.15.175:0/2982719172 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:39.772 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.792+0000 7fd878c2c700 1 -- 172.21.15.175:0/2982719172 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd87419cc20 con 0x7fd874106400 2024-03-26T03:34:39.772 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.792+0000 7fd878c2c700 1 --2- 172.21.15.175:0/2982719172 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd874106400 0x7fd87419c6e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd864004630 tx=0x7fd86400bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:39.772 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.792+0000 7fd871ffb700 1 -- 172.21.15.175:0/2982719172 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd864003ef0 con 0x7fd874106400 2024-03-26T03:34:39.772 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.792+0000 7fd871ffb700 1 -- 172.21.15.175:0/2982719172 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fd864004090 con 0x7fd874106400 2024-03-26T03:34:39.772 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.792+0000 7fd87ae90700 1 -- 172.21.15.175:0/2982719172 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd87419ceb0 con 0x7fd874106400 2024-03-26T03:34:39.772 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.792+0000 7fd871ffb700 1 -- 172.21.15.175:0/2982719172 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8640115b0 con 0x7fd874106400 2024-03-26T03:34:39.772 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.796+0000 7fd87ae90700 1 -- 172.21.15.175:0/2982719172 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd87419fba0 con 0x7fd874106400 2024-03-26T03:34:39.772 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.796+0000 7fd871ffb700 1 -- 172.21.15.175:0/2982719172 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 8) ==== 46466+0+0 (secure 0 0 0) 0x7fd864022020 con 0x7fd874106400 2024-03-26T03:34:39.772 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.796+0000 7fd871ffb700 1 --2- 172.21.15.175:0/2982719172 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7fd860039440 0x7fd86003b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:39.772 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.796+0000 7fd871ffb700 1 -- 172.21.15.175:0/2982719172 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd864052580 con 0x7fd874106400 2024-03-26T03:34:39.772 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.796+0000 7fd873fff700 1 --2- 172.21.15.175:0/2982719172 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7fd860039440 0x7fd86003b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:39.773 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.796+0000 7fd87ae90700 1 -- 172.21.15.175:0/2982719172 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd87410a960 con 0x7fd874106400 2024-03-26T03:34:39.773 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.796+0000 7fd873fff700 1 --2- 172.21.15.175:0/2982719172 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7fd860039440 0x7fd86003b900 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd85c006fd0 tx=0x7fd85c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:39.773 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:38.800+0000 7fd871ffb700 1 -- 172.21.15.175:0/2982719172 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7fd864029210 con 0x7fd874106400 2024-03-26T03:34:39.773 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:39.156+0000 7fd87ae90700 1 -- 172.21.15.175:0/2982719172 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fd87406bce0 con 0x7fd874106400 2024-03-26T03:34:39.773 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:39.160+0000 7fd871ffb700 1 -- 172.21.15.175:0/2982719172 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fd864018be0 con 0x7fd874106400 2024-03-26T03:34:39.773 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:39.164+0000 7fd87ae90700 1 -- 172.21.15.175:0/2982719172 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7fd860039440 msgr2=0x7fd86003b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:39.773 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:39.164+0000 7fd87ae90700 1 --2- 172.21.15.175:0/2982719172 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7fd860039440 0x7fd86003b900 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd85c006fd0 tx=0x7fd85c006e40 comp rx=0 tx=0).stop 2024-03-26T03:34:39.773 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:39.164+0000 7fd87ae90700 1 -- 172.21.15.175:0/2982719172 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd874106400 msgr2=0x7fd87419c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:39.773 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:39.164+0000 7fd87ae90700 1 --2- 172.21.15.175:0/2982719172 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd874106400 0x7fd87419c6e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd864004630 tx=0x7fd86400bac0 comp rx=0 tx=0).stop 2024-03-26T03:34:39.773 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:39.164+0000 7fd87ae90700 1 -- 172.21.15.175:0/2982719172 shutdown_connections 2024-03-26T03:34:39.773 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:39.164+0000 7fd87ae90700 1 --2- 172.21.15.175:0/2982719172 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7fd860039440 0x7fd86003b900 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:39.773 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:39.164+0000 7fd87ae90700 1 --2- 172.21.15.175:0/2982719172 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd874106400 0x7fd87419c6e0 secure :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd864004630 tx=0x7fd86400bac0 comp rx=0 tx=0).stop 2024-03-26T03:34:39.774 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:39.164+0000 7fd87ae90700 1 -- 172.21.15.175:0/2982719172 >> 172.21.15.175:0/2982719172 conn(0x7fd874101960 msgr2=0x7fd8741024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:39.774 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:39.164+0000 7fd87ae90700 1 -- 172.21.15.175:0/2982719172 shutdown_connections 2024-03-26T03:34:39.774 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:39.164+0000 7fd87ae90700 1 -- 172.21.15.175:0/2982719172 wait complete. 2024-03-26T03:34:40.453 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:40 smithi175 bash[21527]: cluster 2024-03-26T03:34:38.899248+0000 mgr.y (mgr.14120) 21 : cluster [DBG] pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:34:40.453 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:40 smithi175 bash[21527]: audit 2024-03-26T03:34:39.165936+0000 mon.a (mon.0) 68 : audit [INF] from='client.? 172.21.15.175:0/2982719172' entity='client.admin' 2024-03-26T03:34:41.698 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.652+0000 7f6059d06700 1 Processor -- start 2024-03-26T03:34:41.699 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.652+0000 7f6059d06700 1 -- start start 2024-03-26T03:34:41.699 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.652+0000 7f6059d06700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6054106460 0x7f6054106880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:41.699 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.652+0000 7f6059d06700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6054106e50 con 0x7f6054106460 2024-03-26T03:34:41.699 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.652+0000 7f60537fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6054106460 0x7f6054106880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:41.699 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.652+0000 7f60537fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6054106460 0x7f6054106880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45822/0 (socket says 172.21.15.175:45822) 2024-03-26T03:34:41.699 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.652+0000 7f60537fe700 1 -- 172.21.15.175:0/1264361295 learned_addr learned my addr 172.21.15.175:0/1264361295 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:41.699 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.652+0000 7f60537fe700 1 -- 172.21.15.175:0/1264361295 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60541075c0 con 0x7f6054106460 2024-03-26T03:34:41.699 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.652+0000 7f60537fe700 1 --2- 172.21.15.175:0/1264361295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6054106460 0x7f6054106880 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f60440098a0 tx=0x7f6044009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d272f689a96df52 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:41.699 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.652+0000 7f6052ffd700 1 -- 172.21.15.175:0/1264361295 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6044004160 con 0x7f6054106460 2024-03-26T03:34:41.699 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.652+0000 7f6052ffd700 1 -- 172.21.15.175:0/1264361295 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f6044004300 con 0x7f6054106460 2024-03-26T03:34:41.699 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.652+0000 7f6052ffd700 1 -- 172.21.15.175:0/1264361295 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60440118f0 con 0x7f6054106460 2024-03-26T03:34:41.700 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.656+0000 7f6059d06700 1 -- 172.21.15.175:0/1264361295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6054106460 msgr2=0x7f6054106880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:41.700 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.656+0000 7f6059d06700 1 --2- 172.21.15.175:0/1264361295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6054106460 0x7f6054106880 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f60440098a0 tx=0x7f6044009bb0 comp rx=0 tx=0).stop 2024-03-26T03:34:41.700 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.656+0000 7f6059d06700 1 -- 172.21.15.175:0/1264361295 shutdown_connections 2024-03-26T03:34:41.700 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.656+0000 7f6059d06700 1 --2- 172.21.15.175:0/1264361295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6054106460 0x7f6054106880 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:41.700 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.656+0000 7f6059d06700 1 -- 172.21.15.175:0/1264361295 >> 172.21.15.175:0/1264361295 conn(0x7f6054101a20 msgr2=0x7f6054103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:41.700 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.656+0000 7f6059d06700 1 -- 172.21.15.175:0/1264361295 shutdown_connections 2024-03-26T03:34:41.700 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.656+0000 7f6059d06700 1 -- 172.21.15.175:0/1264361295 wait complete. 2024-03-26T03:34:41.700 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.656+0000 7f6059d06700 1 Processor -- start 2024-03-26T03:34:41.700 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.656+0000 7f6059d06700 1 -- start start 2024-03-26T03:34:41.700 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.656+0000 7f6059d06700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6054106460 0x7f605407c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:41.700 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.656+0000 7f6059d06700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f605410a7f0 con 0x7f6054106460 2024-03-26T03:34:41.700 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.656+0000 7f60537fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6054106460 0x7f605407c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:41.700 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.656+0000 7f60537fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6054106460 0x7f605407c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45830/0 (socket says 172.21.15.175:45830) 2024-03-26T03:34:41.700 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.656+0000 7f60537fe700 1 -- 172.21.15.175:0/2412378403 learned_addr learned my addr 172.21.15.175:0/2412378403 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:41.701 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.656+0000 7f60537fe700 1 -- 172.21.15.175:0/2412378403 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f605407c910 con 0x7f6054106460 2024-03-26T03:34:41.701 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.656+0000 7f60537fe700 1 --2- 172.21.15.175:0/2412378403 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6054106460 0x7f605407c3d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6044009fb0 tx=0x7f6044011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:41.701 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.656+0000 7f60517fa700 1 -- 172.21.15.175:0/2412378403 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f604401a430 con 0x7f6054106460 2024-03-26T03:34:41.701 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.656+0000 7f60517fa700 1 -- 172.21.15.175:0/2412378403 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f604401aa50 con 0x7f6054106460 2024-03-26T03:34:41.701 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.656+0000 7f60517fa700 1 -- 172.21.15.175:0/2412378403 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6044019800 con 0x7f6054106460 2024-03-26T03:34:41.701 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.656+0000 7f6059d06700 1 -- 172.21.15.175:0/2412378403 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f605407ab50 con 0x7f6054106460 2024-03-26T03:34:41.701 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.656+0000 7f6059d06700 1 -- 172.21.15.175:0/2412378403 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f605407b030 con 0x7f6054106460 2024-03-26T03:34:41.701 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.656+0000 7f60517fa700 1 -- 172.21.15.175:0/2412378403 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 8) ==== 46466+0+0 (secure 0 0 0) 0x7f6044022020 con 0x7f6054106460 2024-03-26T03:34:41.701 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.656+0000 7f60517fa700 1 --2- 172.21.15.175:0/2412378403 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f6040039490 0x7f604003b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:41.701 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.656+0000 7f6059d06700 1 -- 172.21.15.175:0/2412378403 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f605410a970 con 0x7f6054106460 2024-03-26T03:34:41.701 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.656+0000 7f60517fa700 1 -- 172.21.15.175:0/2412378403 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f604404d5e0 con 0x7f6054106460 2024-03-26T03:34:41.701 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.656+0000 7f604bfff700 1 --2- 172.21.15.175:0/2412378403 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f6040039490 0x7f604003b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:41.701 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.656+0000 7f604bfff700 1 --2- 172.21.15.175:0/2412378403 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f6040039490 0x7f604003b950 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f603c006fd0 tx=0x7f603c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:41.701 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:40.664+0000 7f60517fa700 1 -- 172.21.15.175:0/2412378403 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f604404d340 con 0x7f6054106460 2024-03-26T03:34:41.702 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:41.016+0000 7f6059d06700 1 -- 172.21.15.175:0/2412378403 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f6054063c70 con 0x7f6054106460 2024-03-26T03:34:41.702 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:41.020+0000 7f60517fa700 1 -- 172.21.15.175:0/2412378403 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f604402bbe0 con 0x7f6054106460 2024-03-26T03:34:41.702 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:41.024+0000 7f6059d06700 1 -- 172.21.15.175:0/2412378403 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f6040039490 msgr2=0x7f604003b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:41.702 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:41.024+0000 7f6059d06700 1 --2- 172.21.15.175:0/2412378403 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f6040039490 0x7f604003b950 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f603c006fd0 tx=0x7f603c006e40 comp rx=0 tx=0).stop 2024-03-26T03:34:41.702 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:41.024+0000 7f6059d06700 1 -- 172.21.15.175:0/2412378403 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6054106460 msgr2=0x7f605407c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:41.702 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:41.024+0000 7f6059d06700 1 --2- 172.21.15.175:0/2412378403 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6054106460 0x7f605407c3d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6044009fb0 tx=0x7f6044011a20 comp rx=0 tx=0).stop 2024-03-26T03:34:41.702 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:41.024+0000 7f6059d06700 1 -- 172.21.15.175:0/2412378403 shutdown_connections 2024-03-26T03:34:41.702 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:41.024+0000 7f6059d06700 1 --2- 172.21.15.175:0/2412378403 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f6040039490 0x7f604003b950 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:41.702 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:41.024+0000 7f6059d06700 1 --2- 172.21.15.175:0/2412378403 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6054106460 0x7f605407c3d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:41.702 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:41.024+0000 7f6059d06700 1 -- 172.21.15.175:0/2412378403 >> 172.21.15.175:0/2412378403 conn(0x7f6054101a20 msgr2=0x7f60541025d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:41.702 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:41.024+0000 7f6059d06700 1 -- 172.21.15.175:0/2412378403 shutdown_connections 2024-03-26T03:34:41.702 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:41.024+0000 7f6059d06700 1 -- 172.21.15.175:0/2412378403 wait complete. 2024-03-26T03:34:41.702 INFO:teuthology.orchestra.run.smithi175.stdout:Enabling the dashboard module... 2024-03-26T03:34:42.380 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:42 smithi175 bash[21527]: cluster 2024-03-26T03:34:40.899659+0000 mgr.y (mgr.14120) 22 : cluster [DBG] pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:34:42.380 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:42 smithi175 bash[21527]: audit 2024-03-26T03:34:41.026425+0000 mon.a (mon.0) 69 : audit [INF] from='client.? 172.21.15.175:0/2412378403' entity='client.admin' 2024-03-26T03:34:44.422 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:44 smithi175 bash[21527]: cluster 2024-03-26T03:34:42.900092+0000 mgr.y (mgr.14120) 23 : cluster [DBG] pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:34:44.422 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:44 smithi175 bash[21527]: audit 2024-03-26T03:34:43.049415+0000 mon.a (mon.0) 70 : audit [INF] from='client.? 172.21.15.175:0/2943990315' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-26T03:34:44.422 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:44 smithi175 bash[21783]: ignoring --setuser ceph since I am not root 2024-03-26T03:34:44.422 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:44 smithi175 bash[21783]: ignoring --setgroup ceph since I am not root 2024-03-26T03:34:44.422 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:44 smithi175 bash[21783]: debug 2024-03-26T03:34:44.240+0000 7f61f8e00200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T03:34:44.737 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.564+0000 7f411e323700 1 Processor -- start 2024-03-26T03:34:44.737 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.564+0000 7f411e323700 1 -- start start 2024-03-26T03:34:44.737 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.564+0000 7f411e323700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4118104a20 0x7f4118104e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:44.737 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.564+0000 7f411e323700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4118105410 con 0x7f4118104a20 2024-03-26T03:34:44.738 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.564+0000 7f4117fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4118104a20 0x7f4118104e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:44.738 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.564+0000 7f4117fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4118104a20 0x7f4118104e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:56958/0 (socket says 172.21.15.175:56958) 2024-03-26T03:34:44.738 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.564+0000 7f4117fff700 1 -- 172.21.15.175:0/149673979 learned_addr learned my addr 172.21.15.175:0/149673979 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:44.738 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.568+0000 7f4117fff700 1 -- 172.21.15.175:0/149673979 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4118105c20 con 0x7f4118104a20 2024-03-26T03:34:44.738 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.568+0000 7f4117fff700 1 --2- 172.21.15.175:0/149673979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4118104a20 0x7f4118104e40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f410800aa40 tx=0x7f410800ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d02f1e644a8d37 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:44.738 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.568+0000 7f4116ffd700 1 -- 172.21.15.175:0/149673979 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f410800b040 con 0x7f4118104a20 2024-03-26T03:34:44.738 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.568+0000 7f4116ffd700 1 -- 172.21.15.175:0/149673979 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f410800c8f0 con 0x7f4118104a20 2024-03-26T03:34:44.738 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.568+0000 7f4116ffd700 1 -- 172.21.15.175:0/149673979 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4108003a00 con 0x7f4118104a20 2024-03-26T03:34:44.738 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.568+0000 7f411e323700 1 -- 172.21.15.175:0/149673979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4118104a20 msgr2=0x7f4118104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:44.738 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.568+0000 7f411e323700 1 --2- 172.21.15.175:0/149673979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4118104a20 0x7f4118104e40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f410800aa40 tx=0x7f410800ad50 comp rx=0 tx=0).stop 2024-03-26T03:34:44.738 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.568+0000 7f411e323700 1 -- 172.21.15.175:0/149673979 shutdown_connections 2024-03-26T03:34:44.738 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.568+0000 7f411e323700 1 --2- 172.21.15.175:0/149673979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4118104a20 0x7f4118104e40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:44.739 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.568+0000 7f411e323700 1 -- 172.21.15.175:0/149673979 >> 172.21.15.175:0/149673979 conn(0x7f41180fff80 msgr2=0x7f4118102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:44.739 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.568+0000 7f411e323700 1 -- 172.21.15.175:0/149673979 shutdown_connections 2024-03-26T03:34:44.739 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.568+0000 7f411e323700 1 -- 172.21.15.175:0/149673979 wait complete. 2024-03-26T03:34:44.739 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.568+0000 7f411e323700 1 Processor -- start 2024-03-26T03:34:44.739 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.568+0000 7f411e323700 1 -- start start 2024-03-26T03:34:44.739 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.568+0000 7f411e323700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4118104a20 0x7f411819a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:44.739 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.568+0000 7f411e323700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f41181062d0 con 0x7f4118104a20 2024-03-26T03:34:44.739 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.568+0000 7f4117fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4118104a20 0x7f411819a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:44.739 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.568+0000 7f4117fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4118104a20 0x7f411819a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:56964/0 (socket says 172.21.15.175:56964) 2024-03-26T03:34:44.739 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.568+0000 7f4117fff700 1 -- 172.21.15.175:0/2943990315 learned_addr learned my addr 172.21.15.175:0/2943990315 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:44.739 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.568+0000 7f4117fff700 1 -- 172.21.15.175:0/2943990315 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411819a740 con 0x7f4118104a20 2024-03-26T03:34:44.739 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.568+0000 7f4117fff700 1 --2- 172.21.15.175:0/2943990315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4118104a20 0x7f411819a200 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4108003750 tx=0x7f4108004350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:44.739 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.568+0000 7f41157fa700 1 -- 172.21.15.175:0/2943990315 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f410800cd10 con 0x7f4118104a20 2024-03-26T03:34:44.739 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.568+0000 7f41157fa700 1 -- 172.21.15.175:0/2943990315 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f4108010a80 con 0x7f4118104a20 2024-03-26T03:34:44.740 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.568+0000 7f41157fa700 1 -- 172.21.15.175:0/2943990315 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4108018890 con 0x7f4118104a20 2024-03-26T03:34:44.740 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.568+0000 7f411e323700 1 -- 172.21.15.175:0/2943990315 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f411819a9d0 con 0x7f4118104a20 2024-03-26T03:34:44.740 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.568+0000 7f411e323700 1 -- 172.21.15.175:0/2943990315 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f411819d6c0 con 0x7f4118104a20 2024-03-26T03:34:44.740 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.572+0000 7f41157fa700 1 -- 172.21.15.175:0/2943990315 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 8) ==== 46466+0+0 (secure 0 0 0) 0x7f410801f070 con 0x7f4118104a20 2024-03-26T03:34:44.740 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.572+0000 7f41157fa700 1 --2- 172.21.15.175:0/2943990315 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f4104039490 0x7f410403b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:44.740 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.572+0000 7f411e323700 1 -- 172.21.15.175:0/2943990315 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4118063c70 con 0x7f4118104a20 2024-03-26T03:34:44.740 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.572+0000 7f41157fa700 1 -- 172.21.15.175:0/2943990315 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f410804d960 con 0x7f4118104a20 2024-03-26T03:34:44.740 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.572+0000 7f41177fe700 1 --2- 172.21.15.175:0/2943990315 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f4104039490 0x7f410403b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:44.740 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.572+0000 7f41177fe700 1 --2- 172.21.15.175:0/2943990315 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f4104039490 0x7f410403b950 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4100006fd0 tx=0x7f4100006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:44.740 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:42.576+0000 7f41157fa700 1 -- 172.21.15.175:0/2943990315 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f4108014070 con 0x7f4118104a20 2024-03-26T03:34:44.740 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:43.044+0000 7f411e323700 1 -- 172.21.15.175:0/2943990315 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f411802ddd0 con 0x7f4118104a20 2024-03-26T03:34:44.740 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:44.028+0000 7f41157fa700 1 -- 172.21.15.175:0/2943990315 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f410802ae00 con 0x7f4118104a20 2024-03-26T03:34:44.740 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:44.032+0000 7f41157fa700 1 -- 172.21.15.175:0/2943990315 <== mon.0 v2:172.21.15.175:3300/0 8 ==== mgrmap(e 9) ==== 46479+0+0 (secure 0 0 0) 0x7f4108018b00 con 0x7f4118104a20 2024-03-26T03:34:44.740 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:44.032+0000 7f411e323700 1 -- 172.21.15.175:0/2943990315 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f4104039490 msgr2=0x7f410403b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:44.741 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:44.032+0000 7f411e323700 1 --2- 172.21.15.175:0/2943990315 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f4104039490 0x7f410403b950 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4100006fd0 tx=0x7f4100006e40 comp rx=0 tx=0).stop 2024-03-26T03:34:44.741 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:44.032+0000 7f411e323700 1 -- 172.21.15.175:0/2943990315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4118104a20 msgr2=0x7f411819a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:44.741 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:44.032+0000 7f411e323700 1 --2- 172.21.15.175:0/2943990315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4118104a20 0x7f411819a200 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4108003750 tx=0x7f4108004350 comp rx=0 tx=0).stop 2024-03-26T03:34:44.741 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:44.032+0000 7f411e323700 1 -- 172.21.15.175:0/2943990315 shutdown_connections 2024-03-26T03:34:44.741 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:44.032+0000 7f411e323700 1 --2- 172.21.15.175:0/2943990315 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f4104039490 0x7f410403b950 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:44.741 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:44.032+0000 7f411e323700 1 --2- 172.21.15.175:0/2943990315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4118104a20 0x7f411819a200 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:44.741 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:44.032+0000 7f411e323700 1 -- 172.21.15.175:0/2943990315 >> 172.21.15.175:0/2943990315 conn(0x7f41180fff80 msgr2=0x7f411818e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:44.741 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:44.032+0000 7f411e323700 1 -- 172.21.15.175:0/2943990315 shutdown_connections 2024-03-26T03:34:44.741 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:44.032+0000 7f411e323700 1 -- 172.21.15.175:0/2943990315 wait complete. 2024-03-26T03:34:45.459 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:45 smithi175 bash[21527]: audit 2024-03-26T03:34:44.033469+0000 mon.a (mon.0) 71 : audit [INF] from='client.? 172.21.15.175:0/2943990315' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-26T03:34:45.460 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:45 smithi175 bash[21527]: cluster 2024-03-26T03:34:44.039731+0000 mon.a (mon.0) 72 : cluster [DBG] mgrmap e9: y(active, since 27s) 2024-03-26T03:34:46.152 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:45 smithi175 bash[21783]: debug 2024-03-26T03:34:45.824+0000 7f61f8e00200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T03:34:46.152 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:45 smithi175 bash[21783]: debug 2024-03-26T03:34:45.920+0000 7f61f8e00200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T03:34:46.495 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:46 smithi175 bash[21783]: debug 2024-03-26T03:34:46.172+0000 7f61f8e00200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T03:34:46.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:46 smithi175 bash[21527]: audit 2024-03-26T03:34:46.137194+0000 mon.a (mon.0) 73 : audit [DBG] from='client.? 172.21.15.175:0/1790283013' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-26T03:34:46.755 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout { 2024-03-26T03:34:46.755 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-26T03:34:46.755 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-26T03:34:46.755 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-03-26T03:34:46.755 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-26T03:34:46.755 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout } 2024-03-26T03:34:46.757 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.636+0000 7f6d0f574700 1 Processor -- start 2024-03-26T03:34:46.757 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.636+0000 7f6d0f574700 1 -- start start 2024-03-26T03:34:46.757 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.636+0000 7f6d0f574700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d08106eb0 0x7f6d081092a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:46.758 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.636+0000 7f6d0f574700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0807a700 con 0x7f6d08106eb0 2024-03-26T03:34:46.758 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.640+0000 7f6d0d310700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d08106eb0 0x7f6d081092a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:46.758 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.640+0000 7f6d0d310700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d08106eb0 0x7f6d081092a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:56982/0 (socket says 172.21.15.175:56982) 2024-03-26T03:34:46.758 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.640+0000 7f6d0d310700 1 -- 172.21.15.175:0/1998652873 learned_addr learned my addr 172.21.15.175:0/1998652873 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:46.758 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.640+0000 7f6d0d310700 1 -- 172.21.15.175:0/1998652873 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d081097e0 con 0x7f6d08106eb0 2024-03-26T03:34:46.758 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.640+0000 7f6d0d310700 1 --2- 172.21.15.175:0/1998652873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d08106eb0 0x7f6d081092a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6d040098a0 tx=0x7f6d04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102c56975d20e089 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:46.758 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.640+0000 7f6cfffff700 1 -- 172.21.15.175:0/1998652873 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d04004160 con 0x7f6d08106eb0 2024-03-26T03:34:46.758 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.640+0000 7f6cfffff700 1 -- 172.21.15.175:0/1998652873 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f6d04004300 con 0x7f6d08106eb0 2024-03-26T03:34:46.758 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.640+0000 7f6cfffff700 1 -- 172.21.15.175:0/1998652873 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d040118f0 con 0x7f6d08106eb0 2024-03-26T03:34:46.758 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.640+0000 7f6d0f574700 1 -- 172.21.15.175:0/1998652873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d08106eb0 msgr2=0x7f6d081092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:46.758 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.640+0000 7f6d0f574700 1 --2- 172.21.15.175:0/1998652873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d08106eb0 0x7f6d081092a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6d040098a0 tx=0x7f6d04009bb0 comp rx=0 tx=0).stop 2024-03-26T03:34:46.758 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.640+0000 7f6d0f574700 1 -- 172.21.15.175:0/1998652873 shutdown_connections 2024-03-26T03:34:46.758 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.640+0000 7f6d0f574700 1 --2- 172.21.15.175:0/1998652873 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d08106eb0 0x7f6d081092a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:46.758 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.640+0000 7f6d0f574700 1 -- 172.21.15.175:0/1998652873 >> 172.21.15.175:0/1998652873 conn(0x7f6d08100860 msgr2=0x7f6d08102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:46.759 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.640+0000 7f6d0f574700 1 -- 172.21.15.175:0/1998652873 shutdown_connections 2024-03-26T03:34:46.759 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.640+0000 7f6d0f574700 1 -- 172.21.15.175:0/1998652873 wait complete. 2024-03-26T03:34:46.759 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.640+0000 7f6d0f574700 1 Processor -- start 2024-03-26T03:34:46.762 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.640+0000 7f6d0f574700 1 -- start start 2024-03-26T03:34:46.762 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.640+0000 7f6d0f574700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d08106eb0 0x7f6d0819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:46.762 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.640+0000 7f6d0f574700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0810a6d0 con 0x7f6d08106eb0 2024-03-26T03:34:46.763 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.640+0000 7f6d0d310700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d08106eb0 0x7f6d0819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:46.763 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.640+0000 7f6d0d310700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d08106eb0 0x7f6d0819a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:56990/0 (socket says 172.21.15.175:56990) 2024-03-26T03:34:46.763 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.640+0000 7f6d0d310700 1 -- 172.21.15.175:0/1790283013 learned_addr learned my addr 172.21.15.175:0/1790283013 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:46.763 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.640+0000 7f6d0d310700 1 -- 172.21.15.175:0/1790283013 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d0819a990 con 0x7f6d08106eb0 2024-03-26T03:34:46.763 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.640+0000 7f6d0d310700 1 --2- 172.21.15.175:0/1790283013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d08106eb0 0x7f6d0819a450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6d04003f60 tx=0x7f6d04011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:46.763 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.640+0000 7f6cfe7fc700 1 -- 172.21.15.175:0/1790283013 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d040038b0 con 0x7f6d08106eb0 2024-03-26T03:34:46.763 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.640+0000 7f6d0f574700 1 -- 172.21.15.175:0/1790283013 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d0819ac20 con 0x7f6d08106eb0 2024-03-26T03:34:46.763 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.640+0000 7f6cfe7fc700 1 -- 172.21.15.175:0/1790283013 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f6d04003a50 con 0x7f6d08106eb0 2024-03-26T03:34:46.763 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.640+0000 7f6cfe7fc700 1 -- 172.21.15.175:0/1790283013 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d04024410 con 0x7f6d08106eb0 2024-03-26T03:34:46.763 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.640+0000 7f6d0f574700 1 -- 172.21.15.175:0/1790283013 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d0819d910 con 0x7f6d08106eb0 2024-03-26T03:34:46.763 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.644+0000 7f6cfe7fc700 1 -- 172.21.15.175:0/1790283013 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 9) ==== 46479+0+0 (secure 0 0 0) 0x7f6d04022020 con 0x7f6d08106eb0 2024-03-26T03:34:46.763 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.644+0000 7f6cf3fff700 1 -- 172.21.15.175:0/1790283013 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ce8005120 con 0x7f6d08106eb0 2024-03-26T03:34:46.763 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.644+0000 7f6cfe7fc700 1 --2- 172.21.15.175:0/1790283013 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f6cf40394e0 0x7f6cf403b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:46.763 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.644+0000 7f6cfe7fc700 1 -- 172.21.15.175:0/1790283013 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6d0404d9c0 con 0x7f6d08106eb0 2024-03-26T03:34:46.763 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.644+0000 7f6d0cb0f700 1 -- 172.21.15.175:0/1790283013 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f6cf40394e0 msgr2=0x7f6cf403b9a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/712246762 2024-03-26T03:34:46.764 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.644+0000 7f6d0cb0f700 1 --2- 172.21.15.175:0/1790283013 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f6cf40394e0 0x7f6cf403b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T03:34:46.764 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.648+0000 7f6cfe7fc700 1 -- 172.21.15.175:0/1790283013 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f6d0400e260 con 0x7f6d08106eb0 2024-03-26T03:34:46.764 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.848+0000 7f6d0cb0f700 1 -- 172.21.15.175:0/1790283013 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f6cf40394e0 msgr2=0x7f6cf403b9a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/712246762 2024-03-26T03:34:46.764 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:45.848+0000 7f6d0cb0f700 1 --2- 172.21.15.175:0/1790283013 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f6cf40394e0 0x7f6cf403b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T03:34:46.764 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:46.132+0000 7f6cf3fff700 1 -- 172.21.15.175:0/1790283013 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f6ce8005c20 con 0x7f6d08106eb0 2024-03-26T03:34:46.764 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:46.132+0000 7f6cfe7fc700 1 -- 172.21.15.175:0/1790283013 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f6d0402d3f0 con 0x7f6d08106eb0 2024-03-26T03:34:46.764 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:46.136+0000 7f6cf3fff700 1 -- 172.21.15.175:0/1790283013 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f6cf40394e0 msgr2=0x7f6cf403b9a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T03:34:46.764 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:46.136+0000 7f6cf3fff700 1 --2- 172.21.15.175:0/1790283013 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f6cf40394e0 0x7f6cf403b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:46.764 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:46.136+0000 7f6cf3fff700 1 -- 172.21.15.175:0/1790283013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d08106eb0 msgr2=0x7f6d0819a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:46.764 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:46.136+0000 7f6cf3fff700 1 --2- 172.21.15.175:0/1790283013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d08106eb0 0x7f6d0819a450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6d04003f60 tx=0x7f6d04011ef0 comp rx=0 tx=0).stop 2024-03-26T03:34:46.764 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:46.136+0000 7f6cf3fff700 1 -- 172.21.15.175:0/1790283013 shutdown_connections 2024-03-26T03:34:46.764 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:46.136+0000 7f6cf3fff700 1 --2- 172.21.15.175:0/1790283013 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7f6cf40394e0 0x7f6cf403b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:46.764 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:46.136+0000 7f6cf3fff700 1 --2- 172.21.15.175:0/1790283013 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d08106eb0 0x7f6d0819a450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:46.764 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:46.136+0000 7f6cf3fff700 1 -- 172.21.15.175:0/1790283013 >> 172.21.15.175:0/1790283013 conn(0x7f6d08100860 msgr2=0x7f6d08101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:46.765 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:46.136+0000 7f6cf3fff700 1 -- 172.21.15.175:0/1790283013 shutdown_connections 2024-03-26T03:34:46.765 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:46.136+0000 7f6cf3fff700 1 -- 172.21.15.175:0/1790283013 wait complete. 2024-03-26T03:34:46.765 INFO:teuthology.orchestra.run.smithi175.stdout:Waiting for the mgr to restart... 2024-03-26T03:34:46.765 INFO:teuthology.orchestra.run.smithi175.stdout:Waiting for mgr epoch 9... 2024-03-26T03:34:47.094 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:46 smithi175 bash[21783]: debug 2024-03-26T03:34:46.976+0000 7f61f8e00200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T03:34:47.384 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:47 smithi175 bash[21783]: debug 2024-03-26T03:34:47.088+0000 7f61f8e00200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T03:34:47.384 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:47 smithi175 bash[21783]: debug 2024-03-26T03:34:47.196+0000 7f61f8e00200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T03:34:47.680 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:47 smithi175 bash[21783]: debug 2024-03-26T03:34:47.560+0000 7f61f8e00200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T03:34:47.680 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:47 smithi175 bash[21783]: debug 2024-03-26T03:34:47.676+0000 7f61f8e00200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T03:34:47.995 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:47 smithi175 bash[21783]: debug 2024-03-26T03:34:47.776+0000 7f61f8e00200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T03:34:48.495 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:48 smithi175 bash[21783]: debug 2024-03-26T03:34:48.136+0000 7f61f8e00200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T03:34:48.495 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:48 smithi175 bash[21783]: debug 2024-03-26T03:34:48.380+0000 7f61f8e00200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T03:34:48.995 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:48 smithi175 bash[21783]: debug 2024-03-26T03:34:48.640+0000 7f61f8e00200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T03:34:48.995 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:48 smithi175 bash[21783]: debug 2024-03-26T03:34:48.752+0000 7f61f8e00200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T03:34:49.444 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:49 smithi175 bash[21783]: debug 2024-03-26T03:34:49.024+0000 7f61f8e00200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T03:34:49.745 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:49 smithi175 bash[21783]: debug 2024-03-26T03:34:49.440+0000 7f61f8e00200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T03:34:49.745 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:49 smithi175 bash[21783]: debug 2024-03-26T03:34:49.676+0000 7f61f8e00200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T03:34:50.245 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:49 smithi175 bash[21783]: debug 2024-03-26T03:34:49.792+0000 7f61f8e00200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T03:34:50.245 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:50 smithi175 bash[21783]: debug 2024-03-26T03:34:50.044+0000 7f61f8e00200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T03:34:51.464 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:51 smithi175 bash[21783]: debug 2024-03-26T03:34:51.048+0000 7f61f8e00200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T03:34:51.464 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:51 smithi175 bash[21783]: debug 2024-03-26T03:34:51.164+0000 7f61f8e00200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T03:34:51.745 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:51 smithi175 bash[21783]: debug 2024-03-26T03:34:51.456+0000 7f61f8e00200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T03:34:51.745 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:51 smithi175 bash[21783]: debug 2024-03-26T03:34:51.572+0000 7f61f8e00200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T03:34:52.298 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:52 smithi175 bash[21783]: debug 2024-03-26T03:34:52.020+0000 7f61f8e00200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T03:34:52.298 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:34:52 smithi175 bash[21783]: debug 2024-03-26T03:34:52.140+0000 7f61f8e00200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T03:34:52.298 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:52 smithi175 bash[21527]: cluster 2024-03-26T03:34:52.164905+0000 mon.a (mon.0) 74 : cluster [INF] Active manager daemon y restarted 2024-03-26T03:34:52.298 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:52 smithi175 bash[21527]: cluster 2024-03-26T03:34:52.165362+0000 mon.a (mon.0) 75 : cluster [INF] Activating manager daemon y 2024-03-26T03:34:52.298 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:52 smithi175 bash[21527]: cluster 2024-03-26T03:34:52.169034+0000 mon.a (mon.0) 76 : cluster [DBG] osdmap e3: 0 total, 0 up, 0 in 2024-03-26T03:34:52.298 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:52 smithi175 bash[21527]: cluster 2024-03-26T03:34:52.169259+0000 mon.a (mon.0) 77 : cluster [DBG] mgrmap e10: y(active, starting, since 0.00408659s) 2024-03-26T03:34:52.298 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:52 smithi175 bash[21527]: audit 2024-03-26T03:34:52.170720+0000 mon.a (mon.0) 78 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T03:34:52.298 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:52 smithi175 bash[21527]: audit 2024-03-26T03:34:52.172245+0000 mon.a (mon.0) 79 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T03:34:52.298 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:52 smithi175 bash[21527]: audit 2024-03-26T03:34:52.173126+0000 mon.a (mon.0) 80 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T03:34:52.299 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:52 smithi175 bash[21527]: audit 2024-03-26T03:34:52.173279+0000 mon.a (mon.0) 81 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T03:34:52.299 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:52 smithi175 bash[21527]: audit 2024-03-26T03:34:52.173409+0000 mon.a (mon.0) 82 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T03:34:52.299 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:52 smithi175 bash[21527]: cluster 2024-03-26T03:34:52.197552+0000 mon.a (mon.0) 83 : cluster [INF] Manager daemon y is now available 2024-03-26T03:34:53.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:53 smithi175 bash[21527]: audit 2024-03-26T03:34:52.222677+0000 mon.a (mon.0) 84 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:34:53.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:53 smithi175 bash[21527]: audit 2024-03-26T03:34:52.241224+0000 mon.a (mon.0) 85 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-26T03:34:53.496 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:53 smithi175 bash[21527]: audit 2024-03-26T03:34:52.244352+0000 mon.a (mon.0) 86 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-26T03:34:53.496 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:53 smithi175 bash[21527]: cephadm 2024-03-26T03:34:52.596790+0000 mgr.y (mgr.14152) 1 : cephadm [INF] [26/Mar/2024:03:34:52] ENGINE Bus STARTING 2024-03-26T03:34:53.496 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:53 smithi175 bash[21527]: cephadm 2024-03-26T03:34:52.716280+0000 mgr.y (mgr.14152) 2 : cephadm [INF] [26/Mar/2024:03:34:52] ENGINE Serving on https://172.21.15.175:7150 2024-03-26T03:34:53.496 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:53 smithi175 bash[21527]: cephadm 2024-03-26T03:34:52.817840+0000 mgr.y (mgr.14152) 3 : cephadm [INF] [26/Mar/2024:03:34:52] ENGINE Serving on http://172.21.15.175:8765 2024-03-26T03:34:53.496 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:53 smithi175 bash[21527]: cephadm 2024-03-26T03:34:52.817994+0000 mgr.y (mgr.14152) 4 : cephadm [INF] [26/Mar/2024:03:34:52] ENGINE Bus STARTED 2024-03-26T03:34:53.496 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:53 smithi175 bash[21527]: cluster 2024-03-26T03:34:53.179421+0000 mon.a (mon.0) 87 : cluster [DBG] mgrmap e11: y(active, since 1.01422s) 2024-03-26T03:34:53.900 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout { 2024-03-26T03:34:53.900 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-26T03:34:53.900 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-26T03:34:53.900 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout } 2024-03-26T03:34:53.900 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.572+0000 7fa959ea3700 1 Processor -- start 2024-03-26T03:34:53.900 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.572+0000 7fa959ea3700 1 -- start start 2024-03-26T03:34:53.900 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.572+0000 7fa959ea3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9541069d0 0x7fa954106df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:53.900 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.572+0000 7fa959ea3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa954104f10 con 0x7fa9541069d0 2024-03-26T03:34:53.900 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.572+0000 7fa9537fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9541069d0 0x7fa954106df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:53.900 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.572+0000 7fa9537fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9541069d0 0x7fa954106df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:56998/0 (socket says 172.21.15.175:56998) 2024-03-26T03:34:53.900 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.572+0000 7fa9537fe700 1 -- 172.21.15.175:0/353139415 learned_addr learned my addr 172.21.15.175:0/353139415 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:53.900 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.572+0000 7fa9537fe700 1 -- 172.21.15.175:0/353139415 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa954105090 con 0x7fa9541069d0 2024-03-26T03:34:53.901 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.572+0000 7fa9537fe700 1 --2- 172.21.15.175:0/353139415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9541069d0 0x7fa954106df0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa94400bb40 tx=0x7fa94400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8cd91027caf3bb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:53.901 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.572+0000 7fa9527fc700 1 -- 172.21.15.175:0/353139415 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa94400dbe0 con 0x7fa9541069d0 2024-03-26T03:34:53.901 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.572+0000 7fa9527fc700 1 -- 172.21.15.175:0/353139415 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fa9440044b0 con 0x7fa9541069d0 2024-03-26T03:34:53.901 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.572+0000 7fa9527fc700 1 -- 172.21.15.175:0/353139415 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa944003d90 con 0x7fa9541069d0 2024-03-26T03:34:53.901 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.572+0000 7fa959ea3700 1 -- 172.21.15.175:0/353139415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9541069d0 msgr2=0x7fa954106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:53.901 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.572+0000 7fa959ea3700 1 --2- 172.21.15.175:0/353139415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9541069d0 0x7fa954106df0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa94400bb40 tx=0x7fa94400be50 comp rx=0 tx=0).stop 2024-03-26T03:34:53.901 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.572+0000 7fa959ea3700 1 -- 172.21.15.175:0/353139415 shutdown_connections 2024-03-26T03:34:53.901 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.572+0000 7fa959ea3700 1 --2- 172.21.15.175:0/353139415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9541069d0 0x7fa954106df0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:53.901 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.572+0000 7fa959ea3700 1 -- 172.21.15.175:0/353139415 >> 172.21.15.175:0/353139415 conn(0x7fa954100850 msgr2=0x7fa954102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:53.901 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.572+0000 7fa959ea3700 1 -- 172.21.15.175:0/353139415 shutdown_connections 2024-03-26T03:34:53.901 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.572+0000 7fa959ea3700 1 -- 172.21.15.175:0/353139415 wait complete. 2024-03-26T03:34:53.901 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.572+0000 7fa959ea3700 1 Processor -- start 2024-03-26T03:34:53.901 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.572+0000 7fa959ea3700 1 -- start start 2024-03-26T03:34:53.901 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.572+0000 7fa959ea3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9541069d0 0x7fa9541983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:53.902 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.572+0000 7fa959ea3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa954079a00 con 0x7fa9541069d0 2024-03-26T03:34:53.902 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.572+0000 7fa9537fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9541069d0 0x7fa9541983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:53.902 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.572+0000 7fa9537fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9541069d0 0x7fa9541983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:57006/0 (socket says 172.21.15.175:57006) 2024-03-26T03:34:53.902 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.572+0000 7fa9537fe700 1 -- 172.21.15.175:0/4020875457 learned_addr learned my addr 172.21.15.175:0/4020875457 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:53.902 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.576+0000 7fa9537fe700 1 -- 172.21.15.175:0/4020875457 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa954198900 con 0x7fa9541069d0 2024-03-26T03:34:53.902 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.576+0000 7fa9537fe700 1 --2- 172.21.15.175:0/4020875457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9541069d0 0x7fa9541983c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa94400b340 tx=0x7fa944004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:53.902 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.576+0000 7fa950ff9700 1 -- 172.21.15.175:0/4020875457 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa944027070 con 0x7fa9541069d0 2024-03-26T03:34:53.902 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.576+0000 7fa950ff9700 1 -- 172.21.15.175:0/4020875457 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fa944010b50 con 0x7fa9541069d0 2024-03-26T03:34:53.902 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.576+0000 7fa959ea3700 1 -- 172.21.15.175:0/4020875457 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa954198b90 con 0x7fa9541069d0 2024-03-26T03:34:53.902 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.576+0000 7fa950ff9700 1 -- 172.21.15.175:0/4020875457 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa944022b20 con 0x7fa9541069d0 2024-03-26T03:34:53.902 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.576+0000 7fa959ea3700 1 -- 172.21.15.175:0/4020875457 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa95419b880 con 0x7fa9541069d0 2024-03-26T03:34:53.902 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.576+0000 7fa950ff9700 1 -- 172.21.15.175:0/4020875457 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 9) ==== 46479+0+0 (secure 0 0 0) 0x7fa94402a070 con 0x7fa9541069d0 2024-03-26T03:34:53.902 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.576+0000 7fa950ff9700 1 --2- 172.21.15.175:0/4020875457 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7fa93c0390f0 0x7fa93c03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:53.902 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.576+0000 7fa950ff9700 1 -- 172.21.15.175:0/4020875457 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa94401f070 con 0x7fa9541069d0 2024-03-26T03:34:53.903 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.576+0000 7fa952ffd700 1 -- 172.21.15.175:0/4020875457 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7fa93c0390f0 msgr2=0x7fa93c03b5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/712246762 2024-03-26T03:34:53.903 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.576+0000 7fa952ffd700 1 --2- 172.21.15.175:0/4020875457 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7fa93c0390f0 0x7fa93c03b5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T03:34:53.903 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.576+0000 7fa959ea3700 1 -- 172.21.15.175:0/4020875457 --> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa954063c70 con 0x7fa93c0390f0 2024-03-26T03:34:53.903 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.780+0000 7fa952ffd700 1 -- 172.21.15.175:0/4020875457 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7fa93c0390f0 msgr2=0x7fa93c03b5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/712246762 2024-03-26T03:34:53.903 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:47.780+0000 7fa952ffd700 1 --2- 172.21.15.175:0/4020875457 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7fa93c0390f0 0x7fa93c03b5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T03:34:53.903 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:48.184+0000 7fa952ffd700 1 -- 172.21.15.175:0/4020875457 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7fa93c0390f0 msgr2=0x7fa93c03b5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/712246762 2024-03-26T03:34:53.903 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:48.184+0000 7fa952ffd700 1 --2- 172.21.15.175:0/4020875457 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7fa93c0390f0 0x7fa93c03b5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-26T03:34:53.903 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:48.988+0000 7fa952ffd700 1 -- 172.21.15.175:0/4020875457 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7fa93c0390f0 msgr2=0x7fa93c03b5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/712246762 2024-03-26T03:34:53.903 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:48.988+0000 7fa952ffd700 1 --2- 172.21.15.175:0/4020875457 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7fa93c0390f0 0x7fa93c03b5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-26T03:34:53.903 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:50.592+0000 7fa952ffd700 1 -- 172.21.15.175:0/4020875457 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7fa93c0390f0 msgr2=0x7fa93c03b5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/712246762 2024-03-26T03:34:53.903 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:50.592+0000 7fa952ffd700 1 --2- 172.21.15.175:0/4020875457 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7fa93c0390f0 0x7fa93c03b5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-26T03:34:53.903 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:52.164+0000 7fa950ff9700 1 -- 172.21.15.175:0/4020875457 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mgrmap(e 10) ==== 46246+0+0 (secure 0 0 0) 0x7fa94404a050 con 0x7fa9541069d0 2024-03-26T03:34:53.903 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:52.164+0000 7fa950ff9700 1 -- 172.21.15.175:0/4020875457 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7fa93c0390f0 msgr2=0x7fa93c03b5b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T03:34:53.904 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:52.164+0000 7fa950ff9700 1 --2- 172.21.15.175:0/4020875457 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7fa93c0390f0 0x7fa93c03b5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:53.904 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:53.168+0000 7fa950ff9700 1 -- 172.21.15.175:0/4020875457 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mgrmap(e 11) ==== 46373+0+0 (secure 0 0 0) 0x7fa944051ca0 con 0x7fa9541069d0 2024-03-26T03:34:53.904 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:53.168+0000 7fa950ff9700 1 --2- 172.21.15.175:0/4020875457 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fa93c03ca30 0x7fa93c03ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:53.904 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:53.168+0000 7fa950ff9700 1 -- 172.21.15.175:0/4020875457 --> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa9440490a0 con 0x7fa93c03ca30 2024-03-26T03:34:53.904 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:53.168+0000 7fa952ffd700 1 --2- 172.21.15.175:0/4020875457 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fa93c03ca30 0x7fa93c03ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:53.904 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:53.172+0000 7fa952ffd700 1 --2- 172.21.15.175:0/4020875457 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fa93c03ca30 0x7fa93c03ee20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa948003a10 tx=0x7fa948009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:53.904 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:53.172+0000 7fa950ff9700 1 -- 172.21.15.175:0/4020875457 <== mgr.14152 v2:172.21.15.175:6800/2826039805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fa9440490a0 con 0x7fa93c03ca30 2024-03-26T03:34:53.904 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:53.184+0000 7fa959ea3700 1 -- 172.21.15.175:0/4020875457 --> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fa954199070 con 0x7fa93c03ca30 2024-03-26T03:34:53.904 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:53.184+0000 7fa950ff9700 1 -- 172.21.15.175:0/4020875457 <== mgr.14152 v2:172.21.15.175:6800/2826039805 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fa954199070 con 0x7fa93c03ca30 2024-03-26T03:34:53.904 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:53.188+0000 7fa959ea3700 1 -- 172.21.15.175:0/4020875457 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fa93c03ca30 msgr2=0x7fa93c03ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:53.904 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:53.188+0000 7fa959ea3700 1 --2- 172.21.15.175:0/4020875457 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fa93c03ca30 0x7fa93c03ee20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa948003a10 tx=0x7fa948009270 comp rx=0 tx=0).stop 2024-03-26T03:34:53.904 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:53.188+0000 7fa959ea3700 1 -- 172.21.15.175:0/4020875457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9541069d0 msgr2=0x7fa9541983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:53.904 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:53.188+0000 7fa959ea3700 1 --2- 172.21.15.175:0/4020875457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9541069d0 0x7fa9541983c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa94400b340 tx=0x7fa944004200 comp rx=0 tx=0).stop 2024-03-26T03:34:53.904 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:53.188+0000 7fa959ea3700 1 -- 172.21.15.175:0/4020875457 shutdown_connections 2024-03-26T03:34:53.904 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:53.188+0000 7fa959ea3700 1 --2- 172.21.15.175:0/4020875457 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fa93c03ca30 0x7fa93c03ee20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:53.905 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:53.188+0000 7fa959ea3700 1 --2- 172.21.15.175:0/4020875457 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9541069d0 0x7fa9541983c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:53.905 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:53.188+0000 7fa959ea3700 1 --2- 172.21.15.175:0/4020875457 >> [v2:172.21.15.175:6800/712246762,v1:172.21.15.175:6801/712246762] conn(0x7fa93c0390f0 0x7fa93c03b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:53.905 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:53.188+0000 7fa959ea3700 1 -- 172.21.15.175:0/4020875457 >> 172.21.15.175:0/4020875457 conn(0x7fa954100850 msgr2=0x7fa95418e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:53.905 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:53.188+0000 7fa959ea3700 1 -- 172.21.15.175:0/4020875457 shutdown_connections 2024-03-26T03:34:53.905 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:53.188+0000 7fa959ea3700 1 -- 172.21.15.175:0/4020875457 wait complete. 2024-03-26T03:34:53.905 INFO:teuthology.orchestra.run.smithi175.stdout:mgr epoch 9 is available 2024-03-26T03:34:53.905 INFO:teuthology.orchestra.run.smithi175.stdout:Generating a dashboard self-signed certificate... 2024-03-26T03:34:54.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:54 smithi175 bash[21527]: audit 2024-03-26T03:34:53.177638+0000 mgr.y (mgr.14152) 5 : audit [DBG] from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-26T03:34:54.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:54 smithi175 bash[21527]: audit 2024-03-26T03:34:53.191124+0000 mgr.y (mgr.14152) 6 : audit [DBG] from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-26T03:34:55.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:55 smithi175 bash[21527]: cluster 2024-03-26T03:34:54.226360+0000 mon.a (mon.0) 88 : cluster [DBG] mgrmap e12: y(active, since 2s) 2024-03-26T03:34:55.912 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-26T03:34:55.912 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.736+0000 7fd8eb262700 1 Processor -- start 2024-03-26T03:34:55.912 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.740+0000 7fd8eb262700 1 -- start start 2024-03-26T03:34:55.912 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.740+0000 7fd8eb262700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8e4108640 0x7fd8e4108a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:55.912 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.740+0000 7fd8eb262700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e4109030 con 0x7fd8e4108640 2024-03-26T03:34:55.912 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.740+0000 7fd8e8ffe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8e4108640 0x7fd8e4108a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:55.913 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.740+0000 7fd8e8ffe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8e4108640 0x7fd8e4108a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:39030/0 (socket says 172.21.15.175:39030) 2024-03-26T03:34:55.913 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.740+0000 7fd8e8ffe700 1 -- 172.21.15.175:0/2924855179 learned_addr learned my addr 172.21.15.175:0/2924855179 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:55.913 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.740+0000 7fd8e8ffe700 1 -- 172.21.15.175:0/2924855179 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e41097f0 con 0x7fd8e4108640 2024-03-26T03:34:55.913 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.740+0000 7fd8e8ffe700 1 --2- 172.21.15.175:0/2924855179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8e4108640 0x7fd8e4108a60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd8d40098a0 tx=0x7fd8d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959f4c73c42dda8d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:55.913 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.740+0000 7fd8e37fe700 1 -- 172.21.15.175:0/2924855179 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8d4004160 con 0x7fd8e4108640 2024-03-26T03:34:55.913 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.740+0000 7fd8e37fe700 1 -- 172.21.15.175:0/2924855179 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fd8d4004300 con 0x7fd8e4108640 2024-03-26T03:34:55.913 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.740+0000 7fd8e37fe700 1 -- 172.21.15.175:0/2924855179 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8d40118f0 con 0x7fd8e4108640 2024-03-26T03:34:55.913 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.740+0000 7fd8eb262700 1 -- 172.21.15.175:0/2924855179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8e4108640 msgr2=0x7fd8e4108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:55.913 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.740+0000 7fd8eb262700 1 --2- 172.21.15.175:0/2924855179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8e4108640 0x7fd8e4108a60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd8d40098a0 tx=0x7fd8d4009bb0 comp rx=0 tx=0).stop 2024-03-26T03:34:55.913 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.740+0000 7fd8eb262700 1 -- 172.21.15.175:0/2924855179 shutdown_connections 2024-03-26T03:34:55.913 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.740+0000 7fd8eb262700 1 --2- 172.21.15.175:0/2924855179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8e4108640 0x7fd8e4108a60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:55.913 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.740+0000 7fd8eb262700 1 -- 172.21.15.175:0/2924855179 >> 172.21.15.175:0/2924855179 conn(0x7fd8e407bc10 msgr2=0x7fd8e407c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:55.913 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.740+0000 7fd8eb262700 1 -- 172.21.15.175:0/2924855179 shutdown_connections 2024-03-26T03:34:55.913 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.740+0000 7fd8eb262700 1 -- 172.21.15.175:0/2924855179 wait complete. 2024-03-26T03:34:55.914 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.740+0000 7fd8eb262700 1 Processor -- start 2024-03-26T03:34:55.914 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.740+0000 7fd8eb262700 1 -- start start 2024-03-26T03:34:55.914 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.740+0000 7fd8eb262700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8e4108640 0x7fd8e419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:55.914 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.740+0000 7fd8eb262700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e410ca20 con 0x7fd8e4108640 2024-03-26T03:34:55.914 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.740+0000 7fd8e8ffe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8e4108640 0x7fd8e419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:55.914 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.740+0000 7fd8e8ffe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8e4108640 0x7fd8e419c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:39032/0 (socket says 172.21.15.175:39032) 2024-03-26T03:34:55.914 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.740+0000 7fd8e8ffe700 1 -- 172.21.15.175:0/3511759867 learned_addr learned my addr 172.21.15.175:0/3511759867 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:55.914 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.740+0000 7fd8e8ffe700 1 -- 172.21.15.175:0/3511759867 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e419cc00 con 0x7fd8e4108640 2024-03-26T03:34:55.914 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.740+0000 7fd8e8ffe700 1 --2- 172.21.15.175:0/3511759867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8e4108640 0x7fd8e419c6c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009870 tx=0x7fd8d4011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:55.914 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.740+0000 7fd8e1ffb700 1 -- 172.21.15.175:0/3511759867 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8d40038b0 con 0x7fd8e4108640 2024-03-26T03:34:55.914 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.740+0000 7fd8e1ffb700 1 -- 172.21.15.175:0/3511759867 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fd8d4003a50 con 0x7fd8e4108640 2024-03-26T03:34:55.914 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.740+0000 7fd8eb262700 1 -- 172.21.15.175:0/3511759867 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8e419ce90 con 0x7fd8e4108640 2024-03-26T03:34:55.914 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.740+0000 7fd8e1ffb700 1 -- 172.21.15.175:0/3511759867 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8d4024410 con 0x7fd8e4108640 2024-03-26T03:34:55.914 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.740+0000 7fd8eb262700 1 -- 172.21.15.175:0/3511759867 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8e419fb80 con 0x7fd8e4108640 2024-03-26T03:34:55.915 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.744+0000 7fd8e1ffb700 1 -- 172.21.15.175:0/3511759867 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 12) ==== 46479+0+0 (secure 0 0 0) 0x7fd8d4022020 con 0x7fd8e4108640 2024-03-26T03:34:55.915 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.744+0000 7fd8e1ffb700 1 --2- 172.21.15.175:0/3511759867 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fd8d00394e0 0x7fd8d003b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:55.915 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.744+0000 7fd8e1ffb700 1 -- 172.21.15.175:0/3511759867 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd8d404d860 con 0x7fd8e4108640 2024-03-26T03:34:55.915 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.744+0000 7fd8eb262700 1 -- 172.21.15.175:0/3511759867 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8e410cba0 con 0x7fd8e4108640 2024-03-26T03:34:55.915 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.744+0000 7fd8e3fff700 1 --2- 172.21.15.175:0/3511759867 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fd8d00394e0 0x7fd8d003b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:55.915 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.744+0000 7fd8e3fff700 1 --2- 172.21.15.175:0/3511759867 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fd8d00394e0 0x7fd8d003b9a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc006fd0 tx=0x7fd8cc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:55.915 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:54.748+0000 7fd8e1ffb700 1 -- 172.21.15.175:0/3511759867 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7fd8d402cdf0 con 0x7fd8e4108640 2024-03-26T03:34:55.915 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:55.116+0000 7fd8eb262700 1 -- 172.21.15.175:0/3511759867 --> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fd8e406bce0 con 0x7fd8d00394e0 2024-03-26T03:34:55.915 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:55.236+0000 7fd8e1ffb700 1 -- 172.21.15.175:0/3511759867 <== mgr.14152 v2:172.21.15.175:6800/2826039805 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fd8e406bce0 con 0x7fd8d00394e0 2024-03-26T03:34:55.915 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:55.240+0000 7fd8eb262700 1 -- 172.21.15.175:0/3511759867 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fd8d00394e0 msgr2=0x7fd8d003b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:55.915 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:55.240+0000 7fd8eb262700 1 --2- 172.21.15.175:0/3511759867 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fd8d00394e0 0x7fd8d003b9a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc006fd0 tx=0x7fd8cc006e40 comp rx=0 tx=0).stop 2024-03-26T03:34:55.915 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:55.240+0000 7fd8eb262700 1 -- 172.21.15.175:0/3511759867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8e4108640 msgr2=0x7fd8e419c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:55.915 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:55.240+0000 7fd8eb262700 1 --2- 172.21.15.175:0/3511759867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8e4108640 0x7fd8e419c6c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009870 tx=0x7fd8d4011ef0 comp rx=0 tx=0).stop 2024-03-26T03:34:55.915 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:55.240+0000 7fd8eb262700 1 -- 172.21.15.175:0/3511759867 shutdown_connections 2024-03-26T03:34:55.915 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:55.240+0000 7fd8eb262700 1 --2- 172.21.15.175:0/3511759867 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fd8d00394e0 0x7fd8d003b9a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:55.916 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:55.240+0000 7fd8eb262700 1 --2- 172.21.15.175:0/3511759867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8e4108640 0x7fd8e419c6c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:55.916 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:55.240+0000 7fd8eb262700 1 -- 172.21.15.175:0/3511759867 >> 172.21.15.175:0/3511759867 conn(0x7fd8e407bc10 msgr2=0x7fd8e4105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:55.916 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:55.240+0000 7fd8eb262700 1 -- 172.21.15.175:0/3511759867 shutdown_connections 2024-03-26T03:34:55.916 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:55.240+0000 7fd8eb262700 1 -- 172.21.15.175:0/3511759867 wait complete. 2024-03-26T03:34:55.916 INFO:teuthology.orchestra.run.smithi175.stdout:Creating initial admin user... 2024-03-26T03:34:56.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:56 smithi175 bash[21527]: audit 2024-03-26T03:34:55.123668+0000 mgr.y (mgr.14152) 7 : audit [DBG] from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:34:56.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:56 smithi175 bash[21527]: audit 2024-03-26T03:34:55.236182+0000 mon.a (mon.0) 89 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:34:56.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:56 smithi175 bash[21527]: audit 2024-03-26T03:34:55.240989+0000 mon.a (mon.0) 90 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:34:58.035 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$2rT1koDwGSAm1KcRmv2gxeRrW9zmceqHOIgEbifKA.c8RW4VOlKHC", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711424097, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-26T03:34:58.035 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.764+0000 7f824937c700 1 Processor -- start 2024-03-26T03:34:58.035 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.764+0000 7f824937c700 1 -- start start 2024-03-26T03:34:58.035 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.764+0000 7f824937c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8244108600 0x7f8244108a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:58.035 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.764+0000 7f824937c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8244108ff0 con 0x7f8244108600 2024-03-26T03:34:58.035 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.764+0000 7f8242ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8244108600 0x7f8244108a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:58.035 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.764+0000 7f8242ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8244108600 0x7f8244108a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:39038/0 (socket says 172.21.15.175:39038) 2024-03-26T03:34:58.036 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.764+0000 7f8242ffd700 1 -- 172.21.15.175:0/2067882275 learned_addr learned my addr 172.21.15.175:0/2067882275 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:58.036 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.764+0000 7f8242ffd700 1 -- 172.21.15.175:0/2067882275 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82441097b0 con 0x7f8244108600 2024-03-26T03:34:58.036 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.764+0000 7f8242ffd700 1 --2- 172.21.15.175:0/2067882275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8244108600 0x7f8244108a20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f82340098a0 tx=0x7f8234009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b6aebd643ce3e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:58.036 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.764+0000 7f82427fc700 1 -- 172.21.15.175:0/2067882275 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8234004160 con 0x7f8244108600 2024-03-26T03:34:58.036 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.764+0000 7f82427fc700 1 -- 172.21.15.175:0/2067882275 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f8234004300 con 0x7f8244108600 2024-03-26T03:34:58.036 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.764+0000 7f82427fc700 1 -- 172.21.15.175:0/2067882275 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82340118f0 con 0x7f8244108600 2024-03-26T03:34:58.036 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.768+0000 7f824937c700 1 -- 172.21.15.175:0/2067882275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8244108600 msgr2=0x7f8244108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:58.036 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.768+0000 7f824937c700 1 --2- 172.21.15.175:0/2067882275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8244108600 0x7f8244108a20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f82340098a0 tx=0x7f8234009bb0 comp rx=0 tx=0).stop 2024-03-26T03:34:58.036 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.768+0000 7f824937c700 1 -- 172.21.15.175:0/2067882275 shutdown_connections 2024-03-26T03:34:58.036 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.768+0000 7f824937c700 1 --2- 172.21.15.175:0/2067882275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8244108600 0x7f8244108a20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:58.036 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.768+0000 7f824937c700 1 -- 172.21.15.175:0/2067882275 >> 172.21.15.175:0/2067882275 conn(0x7f824407ba10 msgr2=0x7f824407be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:58.036 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.768+0000 7f824937c700 1 -- 172.21.15.175:0/2067882275 shutdown_connections 2024-03-26T03:34:58.036 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.768+0000 7f824937c700 1 -- 172.21.15.175:0/2067882275 wait complete. 2024-03-26T03:34:58.036 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.768+0000 7f824937c700 1 Processor -- start 2024-03-26T03:34:58.037 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.768+0000 7f824937c700 1 -- start start 2024-03-26T03:34:58.037 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.768+0000 7f824937c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8244108600 0x7f824419c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:58.037 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.768+0000 7f824937c700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f824410c9e0 con 0x7f8244108600 2024-03-26T03:34:58.037 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.768+0000 7f8242ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8244108600 0x7f824419c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:58.037 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.768+0000 7f8242ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8244108600 0x7f824419c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:39054/0 (socket says 172.21.15.175:39054) 2024-03-26T03:34:58.037 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.768+0000 7f8242ffd700 1 -- 172.21.15.175:0/2853347511 learned_addr learned my addr 172.21.15.175:0/2853347511 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:58.037 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.768+0000 7f8242ffd700 1 -- 172.21.15.175:0/2853347511 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824419cbe0 con 0x7f8244108600 2024-03-26T03:34:58.037 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.768+0000 7f8242ffd700 1 --2- 172.21.15.175:0/2853347511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8244108600 0x7f824419c6a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8234009870 tx=0x7f8234011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:58.037 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.768+0000 7f8240ff9700 1 -- 172.21.15.175:0/2853347511 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82340038b0 con 0x7f8244108600 2024-03-26T03:34:58.037 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.768+0000 7f824937c700 1 -- 172.21.15.175:0/2853347511 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f824419ce70 con 0x7f8244108600 2024-03-26T03:34:58.037 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.768+0000 7f8240ff9700 1 -- 172.21.15.175:0/2853347511 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f8234003a50 con 0x7f8244108600 2024-03-26T03:34:58.037 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.768+0000 7f8240ff9700 1 -- 172.21.15.175:0/2853347511 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8234024410 con 0x7f8244108600 2024-03-26T03:34:58.037 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.768+0000 7f824937c700 1 -- 172.21.15.175:0/2853347511 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f824419fb60 con 0x7f8244108600 2024-03-26T03:34:58.037 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.768+0000 7f8240ff9700 1 -- 172.21.15.175:0/2853347511 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 12) ==== 46479+0+0 (secure 0 0 0) 0x7f8234022020 con 0x7f8244108600 2024-03-26T03:34:58.038 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.768+0000 7f8240ff9700 1 --2- 172.21.15.175:0/2853347511 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f8224039540 0x7f822403ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:58.038 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.768+0000 7f824937c700 1 -- 172.21.15.175:0/2853347511 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f824410cb60 con 0x7f8244108600 2024-03-26T03:34:58.038 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.772+0000 7f8240ff9700 1 -- 172.21.15.175:0/2853347511 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f823404d800 con 0x7f8244108600 2024-03-26T03:34:58.038 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.772+0000 7f823bfff700 1 --2- 172.21.15.175:0/2853347511 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f8224039540 0x7f822403ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:58.038 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.772+0000 7f823bfff700 1 --2- 172.21.15.175:0/2853347511 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f8224039540 0x7f822403ba00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f822c006fd0 tx=0x7f822c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:58.038 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:56.776+0000 7f8240ff9700 1 -- 172.21.15.175:0/2853347511 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f823402d910 con 0x7f8244108600 2024-03-26T03:34:58.038 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:57.164+0000 7f824937c700 1 -- 172.21.15.175:0/2853347511 --> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f824406bd80 con 0x7f8224039540 2024-03-26T03:34:58.038 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:57.408+0000 7f8240ff9700 1 -- 172.21.15.175:0/2853347511 <== mgr.14152 v2:172.21.15.175:6800/2826039805 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f824406bd80 con 0x7f8224039540 2024-03-26T03:34:58.038 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:57.412+0000 7f824937c700 1 -- 172.21.15.175:0/2853347511 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f8224039540 msgr2=0x7f822403ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:58.038 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:57.412+0000 7f824937c700 1 --2- 172.21.15.175:0/2853347511 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f8224039540 0x7f822403ba00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f822c006fd0 tx=0x7f822c006e40 comp rx=0 tx=0).stop 2024-03-26T03:34:58.038 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:57.412+0000 7f824937c700 1 -- 172.21.15.175:0/2853347511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8244108600 msgr2=0x7f824419c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:58.038 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:57.412+0000 7f824937c700 1 --2- 172.21.15.175:0/2853347511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8244108600 0x7f824419c6a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8234009870 tx=0x7f8234011ef0 comp rx=0 tx=0).stop 2024-03-26T03:34:58.038 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:57.412+0000 7f824937c700 1 -- 172.21.15.175:0/2853347511 shutdown_connections 2024-03-26T03:34:58.039 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:57.412+0000 7f824937c700 1 --2- 172.21.15.175:0/2853347511 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f8224039540 0x7f822403ba00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:58.039 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:57.412+0000 7f824937c700 1 --2- 172.21.15.175:0/2853347511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8244108600 0x7f824419c6a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:58.039 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:57.412+0000 7f824937c700 1 -- 172.21.15.175:0/2853347511 >> 172.21.15.175:0/2853347511 conn(0x7f824407ba10 msgr2=0x7f8244105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:58.039 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:57.412+0000 7f824937c700 1 -- 172.21.15.175:0/2853347511 shutdown_connections 2024-03-26T03:34:58.039 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:57.412+0000 7f824937c700 1 -- 172.21.15.175:0/2853347511 wait complete. 2024-03-26T03:34:58.039 INFO:teuthology.orchestra.run.smithi175.stdout:Fetching dashboard port number... 2024-03-26T03:34:58.676 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:58 smithi175 bash[21527]: audit 2024-03-26T03:34:57.172412+0000 mgr.y (mgr.14152) 8 : audit [DBG] from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:34:58.676 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:58 smithi175 bash[21527]: audit 2024-03-26T03:34:57.409643+0000 mon.a (mon.0) 91 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:34:59.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:59 smithi175 bash[21527]: cluster 2024-03-26T03:34:58.417315+0000 mon.a (mon.0) 92 : cluster [DBG] mgrmap e13: y(active, since 6s) 2024-03-26T03:34:59.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:59 smithi175 bash[21527]: audit 2024-03-26T03:34:59.130179+0000 mon.a (mon.0) 93 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:34:59.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:34:59 smithi175 bash[21527]: audit 2024-03-26T03:34:59.273167+0000 mon.a (mon.0) 94 : audit [DBG] from='client.? 172.21.15.175:0/2096401890' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-26T03:34:59.913 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stdout 8443 2024-03-26T03:34:59.917 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.876+0000 7fc39c772700 1 Processor -- start 2024-03-26T03:34:59.917 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.876+0000 7fc39c772700 1 -- start start 2024-03-26T03:34:59.917 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.876+0000 7fc39c772700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc394106460 0x7fc394106880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:59.917 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.876+0000 7fc39c772700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc394106e50 con 0x7fc394106460 2024-03-26T03:34:59.917 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.876+0000 7fc39a50e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc394106460 0x7fc394106880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:59.917 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.876+0000 7fc39a50e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc394106460 0x7fc394106880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:39070/0 (socket says 172.21.15.175:39070) 2024-03-26T03:34:59.918 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.876+0000 7fc39a50e700 1 -- 172.21.15.175:0/286818179 learned_addr learned my addr 172.21.15.175:0/286818179 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:59.918 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.876+0000 7fc39a50e700 1 -- 172.21.15.175:0/286818179 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3941075c0 con 0x7fc394106460 2024-03-26T03:34:59.918 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.876+0000 7fc39a50e700 1 --2- 172.21.15.175:0/286818179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc394106460 0x7fc394106880 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc3900098a0 tx=0x7fc390009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf33ceaec7c01ce7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:59.918 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.876+0000 7fc39950c700 1 -- 172.21.15.175:0/286818179 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc390004160 con 0x7fc394106460 2024-03-26T03:34:59.918 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.876+0000 7fc39950c700 1 -- 172.21.15.175:0/286818179 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fc390004300 con 0x7fc394106460 2024-03-26T03:34:59.918 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.876+0000 7fc39950c700 1 -- 172.21.15.175:0/286818179 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3900118f0 con 0x7fc394106460 2024-03-26T03:34:59.918 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.876+0000 7fc39c772700 1 -- 172.21.15.175:0/286818179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc394106460 msgr2=0x7fc394106880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:59.918 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.876+0000 7fc39c772700 1 --2- 172.21.15.175:0/286818179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc394106460 0x7fc394106880 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc3900098a0 tx=0x7fc390009bb0 comp rx=0 tx=0).stop 2024-03-26T03:34:59.918 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.876+0000 7fc39c772700 1 -- 172.21.15.175:0/286818179 shutdown_connections 2024-03-26T03:34:59.918 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.876+0000 7fc39c772700 1 --2- 172.21.15.175:0/286818179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc394106460 0x7fc394106880 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:59.918 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.876+0000 7fc39c772700 1 -- 172.21.15.175:0/286818179 >> 172.21.15.175:0/286818179 conn(0x7fc394101a20 msgr2=0x7fc394103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:59.918 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.876+0000 7fc39c772700 1 -- 172.21.15.175:0/286818179 shutdown_connections 2024-03-26T03:34:59.919 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.876+0000 7fc39c772700 1 -- 172.21.15.175:0/286818179 wait complete. 2024-03-26T03:34:59.919 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.876+0000 7fc39c772700 1 Processor -- start 2024-03-26T03:34:59.919 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.876+0000 7fc39c772700 1 -- start start 2024-03-26T03:34:59.919 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.880+0000 7fc39c772700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc394106460 0x7fc39407c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:59.919 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.880+0000 7fc39c772700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc39410a7f0 con 0x7fc394106460 2024-03-26T03:34:59.919 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.880+0000 7fc39a50e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc394106460 0x7fc39407c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:59.919 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.880+0000 7fc39a50e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc394106460 0x7fc39407c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:39086/0 (socket says 172.21.15.175:39086) 2024-03-26T03:34:59.919 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.880+0000 7fc39a50e700 1 -- 172.21.15.175:0/2096401890 learned_addr learned my addr 172.21.15.175:0/2096401890 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:34:59.919 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.880+0000 7fc39a50e700 1 -- 172.21.15.175:0/2096401890 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc39407c960 con 0x7fc394106460 2024-03-26T03:34:59.919 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.880+0000 7fc39a50e700 1 --2- 172.21.15.175:0/2096401890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc394106460 0x7fc39407c420 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc390003f60 tx=0x7fc390011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:59.919 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.880+0000 7fc38b7fe700 1 -- 172.21.15.175:0/2096401890 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3900038b0 con 0x7fc394106460 2024-03-26T03:34:59.919 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.880+0000 7fc39c772700 1 -- 172.21.15.175:0/2096401890 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc39407ab50 con 0x7fc394106460 2024-03-26T03:34:59.919 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.880+0000 7fc38b7fe700 1 -- 172.21.15.175:0/2096401890 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fc390003a50 con 0x7fc394106460 2024-03-26T03:34:59.919 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.880+0000 7fc38b7fe700 1 -- 172.21.15.175:0/2096401890 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc390024410 con 0x7fc394106460 2024-03-26T03:34:59.919 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.880+0000 7fc39c772700 1 -- 172.21.15.175:0/2096401890 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc39407b090 con 0x7fc394106460 2024-03-26T03:34:59.919 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.880+0000 7fc39c772700 1 -- 172.21.15.175:0/2096401890 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc39410a970 con 0x7fc394106460 2024-03-26T03:34:59.920 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.880+0000 7fc38b7fe700 1 -- 172.21.15.175:0/2096401890 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 13) ==== 46523+0+0 (secure 0 0 0) 0x7fc390022020 con 0x7fc394106460 2024-03-26T03:34:59.920 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.880+0000 7fc38b7fe700 1 --2- 172.21.15.175:0/2096401890 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fc3800395e0 0x7fc38003baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:34:59.920 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.880+0000 7fc38b7fe700 1 -- 172.21.15.175:0/2096401890 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc39004d7e0 con 0x7fc394106460 2024-03-26T03:34:59.920 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.880+0000 7fc399d0d700 1 --2- 172.21.15.175:0/2096401890 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fc3800395e0 0x7fc38003baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:34:59.920 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.884+0000 7fc399d0d700 1 --2- 172.21.15.175:0/2096401890 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fc3800395e0 0x7fc38003baa0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc384006fd0 tx=0x7fc384006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:34:59.920 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:58.888+0000 7fc38b7fe700 1 -- 172.21.15.175:0/2096401890 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7fc39004c0f0 con 0x7fc394106460 2024-03-26T03:34:59.920 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:59.268+0000 7fc39c772700 1 -- 172.21.15.175:0/2096401890 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fc394063c70 con 0x7fc394106460 2024-03-26T03:34:59.920 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:59.268+0000 7fc38b7fe700 1 -- 172.21.15.175:0/2096401890 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fc390031020 con 0x7fc394106460 2024-03-26T03:34:59.920 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:59.272+0000 7fc39c772700 1 -- 172.21.15.175:0/2096401890 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fc3800395e0 msgr2=0x7fc38003baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:59.920 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:59.272+0000 7fc39c772700 1 --2- 172.21.15.175:0/2096401890 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fc3800395e0 0x7fc38003baa0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc384006fd0 tx=0x7fc384006e40 comp rx=0 tx=0).stop 2024-03-26T03:34:59.920 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:59.272+0000 7fc39c772700 1 -- 172.21.15.175:0/2096401890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc394106460 msgr2=0x7fc39407c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:34:59.920 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:59.272+0000 7fc39c772700 1 --2- 172.21.15.175:0/2096401890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc394106460 0x7fc39407c420 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc390003f60 tx=0x7fc390011ef0 comp rx=0 tx=0).stop 2024-03-26T03:34:59.920 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:59.272+0000 7fc39c772700 1 -- 172.21.15.175:0/2096401890 shutdown_connections 2024-03-26T03:34:59.920 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:59.272+0000 7fc39c772700 1 --2- 172.21.15.175:0/2096401890 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fc3800395e0 0x7fc38003baa0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:59.921 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:59.272+0000 7fc39c772700 1 --2- 172.21.15.175:0/2096401890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc394106460 0x7fc39407c420 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:34:59.921 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:59.272+0000 7fc39c772700 1 -- 172.21.15.175:0/2096401890 >> 172.21.15.175:0/2096401890 conn(0x7fc394101a20 msgr2=0x7fc394102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:34:59.921 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:59.272+0000 7fc39c772700 1 -- 172.21.15.175:0/2096401890 shutdown_connections 2024-03-26T03:34:59.921 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:34:59.272+0000 7fc39c772700 1 -- 172.21.15.175:0/2096401890 wait complete. 2024-03-26T03:34:59.921 INFO:teuthology.orchestra.run.smithi175.stdout:firewalld does not appear to be present 2024-03-26T03:34:59.921 INFO:teuthology.orchestra.run.smithi175.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-26T03:34:59.921 INFO:teuthology.orchestra.run.smithi175.stdout:Ceph Dashboard is now available at: 2024-03-26T03:34:59.921 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-26T03:34:59.921 INFO:teuthology.orchestra.run.smithi175.stdout: URL: https://smithi175.front.sepia.ceph.com:8443/ 2024-03-26T03:34:59.921 INFO:teuthology.orchestra.run.smithi175.stdout: User: admin 2024-03-26T03:34:59.921 INFO:teuthology.orchestra.run.smithi175.stdout: Password: gxnhsqkw1m 2024-03-26T03:34:59.921 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-26T03:34:59.921 INFO:teuthology.orchestra.run.smithi175.stdout:Saving cluster configuration to /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/config directory 2024-03-26T03:34:59.921 INFO:teuthology.orchestra.run.smithi175.stdout:Enabling autotune for osd_memory_target 2024-03-26T03:35:01.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:01 smithi175 bash[21527]: audit 2024-03-26T03:35:00.170026+0000 mon.a (mon.0) 95 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:35:01.744 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.740+0000 7f20df094700 1 Processor -- start 2024-03-26T03:35:01.745 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.740+0000 7f20df094700 1 -- start start 2024-03-26T03:35:01.745 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.740+0000 7f20df094700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20d807c770 0x7f20d807abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:35:01.745 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.740+0000 7f20df094700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f20d807b1a0 con 0x7f20d807c770 2024-03-26T03:35:01.745 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.740+0000 7f20dce30700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20d807c770 0x7f20d807abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:35:01.745 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.740+0000 7f20dce30700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20d807c770 0x7f20d807abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:39096/0 (socket says 172.21.15.175:39096) 2024-03-26T03:35:01.745 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.740+0000 7f20dce30700 1 -- 172.21.15.175:0/154081856 learned_addr learned my addr 172.21.15.175:0/154081856 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:35:01.745 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.740+0000 7f20dce30700 1 -- 172.21.15.175:0/154081856 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d807b320 con 0x7f20d807c770 2024-03-26T03:35:01.745 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.740+0000 7f20dce30700 1 --2- 172.21.15.175:0/154081856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20d807c770 0x7f20d807abd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f20d0009e80 tx=0x7f20d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8889bf39f4d68f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:35:01.745 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.740+0000 7f20d77fe700 1 -- 172.21.15.175:0/154081856 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20d0017070 con 0x7f20d807c770 2024-03-26T03:35:01.745 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.740+0000 7f20d77fe700 1 -- 172.21.15.175:0/154081856 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f20d000b8c0 con 0x7f20d807c770 2024-03-26T03:35:01.745 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.740+0000 7f20d77fe700 1 -- 172.21.15.175:0/154081856 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20d0004370 con 0x7f20d807c770 2024-03-26T03:35:01.745 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.740+0000 7f20df094700 1 -- 172.21.15.175:0/154081856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20d807c770 msgr2=0x7f20d807abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:35:01.746 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.740+0000 7f20df094700 1 --2- 172.21.15.175:0/154081856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20d807c770 0x7f20d807abd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f20d0009e80 tx=0x7f20d000b330 comp rx=0 tx=0).stop 2024-03-26T03:35:01.746 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.740+0000 7f20df094700 1 -- 172.21.15.175:0/154081856 shutdown_connections 2024-03-26T03:35:01.746 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.740+0000 7f20df094700 1 --2- 172.21.15.175:0/154081856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20d807c770 0x7f20d807abd0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:35:01.746 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.740+0000 7f20df094700 1 -- 172.21.15.175:0/154081856 >> 172.21.15.175:0/154081856 conn(0x7f20d81019a0 msgr2=0x7f20d8103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:35:01.746 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.740+0000 7f20df094700 1 -- 172.21.15.175:0/154081856 shutdown_connections 2024-03-26T03:35:01.746 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.740+0000 7f20df094700 1 -- 172.21.15.175:0/154081856 wait complete. 2024-03-26T03:35:01.746 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.740+0000 7f20df094700 1 Processor -- start 2024-03-26T03:35:01.746 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.740+0000 7f20df094700 1 -- start start 2024-03-26T03:35:01.746 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.740+0000 7f20df094700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20d807c770 0x7f20d819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:35:01.746 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.740+0000 7f20df094700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f20d810c940 con 0x7f20d807c770 2024-03-26T03:35:01.747 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.740+0000 7f20dce30700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20d807c770 0x7f20d819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:35:01.747 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.740+0000 7f20dce30700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20d807c770 0x7f20d819c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:39106/0 (socket says 172.21.15.175:39106) 2024-03-26T03:35:01.747 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.740+0000 7f20dce30700 1 -- 172.21.15.175:0/920813708 learned_addr learned my addr 172.21.15.175:0/920813708 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:35:01.747 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.740+0000 7f20dce30700 1 -- 172.21.15.175:0/920813708 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d819cbe0 con 0x7f20d807c770 2024-03-26T03:35:01.747 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.740+0000 7f20dce30700 1 --2- 172.21.15.175:0/920813708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20d807c770 0x7f20d819c6a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f20d000bfa0 tx=0x7f20d0003c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:35:01.747 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.744+0000 7f20d5ffb700 1 -- 172.21.15.175:0/920813708 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20d0021020 con 0x7f20d807c770 2024-03-26T03:35:01.747 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.744+0000 7f20d5ffb700 1 -- 172.21.15.175:0/920813708 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f20d000ba70 con 0x7f20d807c770 2024-03-26T03:35:01.747 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.744+0000 7f20df094700 1 -- 172.21.15.175:0/920813708 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d819ce70 con 0x7f20d807c770 2024-03-26T03:35:01.747 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.744+0000 7f20d5ffb700 1 -- 172.21.15.175:0/920813708 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20d0017040 con 0x7f20d807c770 2024-03-26T03:35:01.747 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.744+0000 7f20df094700 1 -- 172.21.15.175:0/920813708 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d819fb10 con 0x7f20d807c770 2024-03-26T03:35:01.747 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.744+0000 7f20d5ffb700 1 -- 172.21.15.175:0/920813708 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 13) ==== 46523+0+0 (secure 0 0 0) 0x7f20d0024020 con 0x7f20d807c770 2024-03-26T03:35:01.747 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.744+0000 7f20d5ffb700 1 --2- 172.21.15.175:0/920813708 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f20c00395e0 0x7f20c003baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:35:01.747 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.744+0000 7f20d5ffb700 1 -- 172.21.15.175:0/920813708 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f20d000f050 con 0x7f20d807c770 2024-03-26T03:35:01.747 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.744+0000 7f20df094700 1 -- 172.21.15.175:0/920813708 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20d8063c70 con 0x7f20d807c770 2024-03-26T03:35:01.748 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.744+0000 7f20d7fff700 1 --2- 172.21.15.175:0/920813708 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f20c00395e0 0x7f20c003baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:35:01.748 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.744+0000 7f20d7fff700 1 --2- 172.21.15.175:0/920813708 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f20c00395e0 0x7f20c003baa0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f20cc006fd0 tx=0x7f20cc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:35:01.748 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:00.748+0000 7f20d5ffb700 1 -- 172.21.15.175:0/920813708 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f20d002b020 con 0x7f20d807c770 2024-03-26T03:35:01.748 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:01.104+0000 7f20df094700 1 -- 172.21.15.175:0/920813708 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f20d806bd50 con 0x7f20d807c770 2024-03-26T03:35:01.748 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:01.104+0000 7f20d5ffb700 1 -- 172.21.15.175:0/920813708 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f20d000c000 con 0x7f20d807c770 2024-03-26T03:35:01.748 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:01.108+0000 7f20df094700 1 -- 172.21.15.175:0/920813708 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f20c00395e0 msgr2=0x7f20c003baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:35:01.748 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:01.108+0000 7f20df094700 1 --2- 172.21.15.175:0/920813708 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f20c00395e0 0x7f20c003baa0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f20cc006fd0 tx=0x7f20cc006e40 comp rx=0 tx=0).stop 2024-03-26T03:35:01.748 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:01.108+0000 7f20df094700 1 -- 172.21.15.175:0/920813708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20d807c770 msgr2=0x7f20d819c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:35:01.748 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:01.108+0000 7f20df094700 1 --2- 172.21.15.175:0/920813708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20d807c770 0x7f20d819c6a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f20d000bfa0 tx=0x7f20d0003c50 comp rx=0 tx=0).stop 2024-03-26T03:35:01.748 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:01.108+0000 7f20df094700 1 -- 172.21.15.175:0/920813708 shutdown_connections 2024-03-26T03:35:01.748 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:01.108+0000 7f20df094700 1 --2- 172.21.15.175:0/920813708 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f20c00395e0 0x7f20c003baa0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:35:01.748 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:01.108+0000 7f20df094700 1 --2- 172.21.15.175:0/920813708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20d807c770 0x7f20d819c6a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:35:01.748 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:01.108+0000 7f20df094700 1 -- 172.21.15.175:0/920813708 >> 172.21.15.175:0/920813708 conn(0x7f20d81019a0 msgr2=0x7f20d8102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:35:01.748 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:01.108+0000 7f20df094700 1 -- 172.21.15.175:0/920813708 shutdown_connections 2024-03-26T03:35:01.749 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:01.108+0000 7f20df094700 1 -- 172.21.15.175:0/920813708 wait complete. 2024-03-26T03:35:03.708 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.552+0000 7f18f8bc7700 1 Processor -- start 2024-03-26T03:35:03.709 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.552+0000 7f18f8bc7700 1 -- start start 2024-03-26T03:35:03.709 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.552+0000 7f18f8bc7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18f41063e0 0x7f18f4106800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:35:03.709 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.552+0000 7f18f8bc7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f18f4106dd0 con 0x7f18f41063e0 2024-03-26T03:35:03.709 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.552+0000 7f18f259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18f41063e0 0x7f18f4106800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:35:03.709 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.552+0000 7f18f259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18f41063e0 0x7f18f4106800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:44736/0 (socket says 172.21.15.175:44736) 2024-03-26T03:35:03.709 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.552+0000 7f18f259c700 1 -- 172.21.15.175:0/4071625976 learned_addr learned my addr 172.21.15.175:0/4071625976 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:35:03.709 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.552+0000 7f18f259c700 1 -- 172.21.15.175:0/4071625976 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18f41075e0 con 0x7f18f41063e0 2024-03-26T03:35:03.709 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.552+0000 7f18f259c700 1 --2- 172.21.15.175:0/4071625976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18f41063e0 0x7f18f4106800 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0098a0 tx=0x7f18ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268dce133571f556 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:35:03.709 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.552+0000 7f18f159a700 1 -- 172.21.15.175:0/4071625976 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18ec004160 con 0x7f18f41063e0 2024-03-26T03:35:03.709 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.552+0000 7f18f159a700 1 -- 172.21.15.175:0/4071625976 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f18ec004300 con 0x7f18f41063e0 2024-03-26T03:35:03.709 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.552+0000 7f18f159a700 1 -- 172.21.15.175:0/4071625976 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18ec0118f0 con 0x7f18f41063e0 2024-03-26T03:35:03.709 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.552+0000 7f18f8bc7700 1 -- 172.21.15.175:0/4071625976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18f41063e0 msgr2=0x7f18f4106800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:35:03.709 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.552+0000 7f18f8bc7700 1 --2- 172.21.15.175:0/4071625976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18f41063e0 0x7f18f4106800 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0098a0 tx=0x7f18ec009bb0 comp rx=0 tx=0).stop 2024-03-26T03:35:03.710 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.552+0000 7f18f8bc7700 1 -- 172.21.15.175:0/4071625976 shutdown_connections 2024-03-26T03:35:03.710 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.552+0000 7f18f8bc7700 1 --2- 172.21.15.175:0/4071625976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18f41063e0 0x7f18f4106800 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:35:03.710 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.552+0000 7f18f8bc7700 1 -- 172.21.15.175:0/4071625976 >> 172.21.15.175:0/4071625976 conn(0x7f18f4101940 msgr2=0x7f18f4103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:35:03.710 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.552+0000 7f18f8bc7700 1 -- 172.21.15.175:0/4071625976 shutdown_connections 2024-03-26T03:35:03.710 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.552+0000 7f18f8bc7700 1 -- 172.21.15.175:0/4071625976 wait complete. 2024-03-26T03:35:03.710 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.552+0000 7f18f8bc7700 1 Processor -- start 2024-03-26T03:35:03.710 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.552+0000 7f18f8bc7700 1 -- start start 2024-03-26T03:35:03.710 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.552+0000 7f18f8bc7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18f41063e0 0x7f18f419c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:35:03.710 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.552+0000 7f18f8bc7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f18f410a810 con 0x7f18f41063e0 2024-03-26T03:35:03.711 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.552+0000 7f18f259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18f41063e0 0x7f18f419c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:35:03.711 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.552+0000 7f18f259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18f41063e0 0x7f18f419c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:44752/0 (socket says 172.21.15.175:44752) 2024-03-26T03:35:03.711 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.552+0000 7f18f259c700 1 -- 172.21.15.175:0/3024309926 learned_addr learned my addr 172.21.15.175:0/3024309926 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:35:03.711 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.556+0000 7f18f259c700 1 -- 172.21.15.175:0/3024309926 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18f419cbe0 con 0x7f18f41063e0 2024-03-26T03:35:03.711 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.556+0000 7f18f259c700 1 --2- 172.21.15.175:0/3024309926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18f41063e0 0x7f18f419c6a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f18ec003f60 tx=0x7f18ec011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:35:03.711 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.556+0000 7f18eb7fe700 1 -- 172.21.15.175:0/3024309926 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18ec0038b0 con 0x7f18f41063e0 2024-03-26T03:35:03.711 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.556+0000 7f18eb7fe700 1 -- 172.21.15.175:0/3024309926 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f18ec003a50 con 0x7f18f41063e0 2024-03-26T03:35:03.711 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.556+0000 7f18f8bc7700 1 -- 172.21.15.175:0/3024309926 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18f419ce70 con 0x7f18f41063e0 2024-03-26T03:35:03.711 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.556+0000 7f18eb7fe700 1 -- 172.21.15.175:0/3024309926 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18ec024410 con 0x7f18f41063e0 2024-03-26T03:35:03.711 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.556+0000 7f18f8bc7700 1 -- 172.21.15.175:0/3024309926 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18f419fb60 con 0x7f18f41063e0 2024-03-26T03:35:03.711 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.556+0000 7f18eb7fe700 1 -- 172.21.15.175:0/3024309926 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 13) ==== 46523+0+0 (secure 0 0 0) 0x7f18ec022020 con 0x7f18f41063e0 2024-03-26T03:35:03.711 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.556+0000 7f18f8bc7700 1 -- 172.21.15.175:0/3024309926 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18f410a990 con 0x7f18f41063e0 2024-03-26T03:35:03.711 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.556+0000 7f18eb7fe700 1 --2- 172.21.15.175:0/3024309926 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f18e0039580 0x7f18e003ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:35:03.711 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.556+0000 7f18eb7fe700 1 -- 172.21.15.175:0/3024309926 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f18ec04d570 con 0x7f18f41063e0 2024-03-26T03:35:03.712 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.556+0000 7f18f1d9b700 1 --2- 172.21.15.175:0/3024309926 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f18e0039580 0x7f18e003ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:35:03.712 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.556+0000 7f18f1d9b700 1 --2- 172.21.15.175:0/3024309926 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f18e0039580 0x7f18e003ba40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f18dc006fd0 tx=0x7f18dc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:35:03.712 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:02.560+0000 7f18eb7fe700 1 -- 172.21.15.175:0/3024309926 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f18ec01a950 con 0x7f18f41063e0 2024-03-26T03:35:03.712 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:03.140+0000 7f18f8bc7700 1 -- 172.21.15.175:0/3024309926 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f18f419fd30 con 0x7f18f41063e0 2024-03-26T03:35:03.712 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:03.144+0000 7f18eb7fe700 1 -- 172.21.15.175:0/3024309926 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v26)=0 set mgr/dashboard/cluster/status v26) ==== 153+0+0 (secure 0 0 0) 0x7f18ec00f2e0 con 0x7f18f41063e0 2024-03-26T03:35:03.712 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:03.144+0000 7f18f8bc7700 1 -- 172.21.15.175:0/3024309926 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f18e0039580 msgr2=0x7f18e003ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:35:03.712 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:03.148+0000 7f18f8bc7700 1 --2- 172.21.15.175:0/3024309926 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f18e0039580 0x7f18e003ba40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f18dc006fd0 tx=0x7f18dc006e40 comp rx=0 tx=0).stop 2024-03-26T03:35:03.712 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:03.148+0000 7f18f8bc7700 1 -- 172.21.15.175:0/3024309926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18f41063e0 msgr2=0x7f18f419c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:35:03.712 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:03.148+0000 7f18f8bc7700 1 --2- 172.21.15.175:0/3024309926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18f41063e0 0x7f18f419c6a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f18ec003f60 tx=0x7f18ec011ef0 comp rx=0 tx=0).stop 2024-03-26T03:35:03.712 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:03.148+0000 7f18f8bc7700 1 -- 172.21.15.175:0/3024309926 shutdown_connections 2024-03-26T03:35:03.712 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:03.148+0000 7f18f8bc7700 1 --2- 172.21.15.175:0/3024309926 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f18e0039580 0x7f18e003ba40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:35:03.712 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:03.148+0000 7f18f8bc7700 1 --2- 172.21.15.175:0/3024309926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18f41063e0 0x7f18f419c6a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:35:03.712 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:03.148+0000 7f18f8bc7700 1 -- 172.21.15.175:0/3024309926 >> 172.21.15.175:0/3024309926 conn(0x7f18f4101940 msgr2=0x7f18f407ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:35:03.712 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:03.148+0000 7f18f8bc7700 1 -- 172.21.15.175:0/3024309926 shutdown_connections 2024-03-26T03:35:03.713 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr 2024-03-26T03:35:03.148+0000 7f18f8bc7700 1 -- 172.21.15.175:0/3024309926 wait complete. 2024-03-26T03:35:03.713 INFO:teuthology.orchestra.run.smithi175.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-26T03:35:03.713 INFO:teuthology.orchestra.run.smithi175.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-26T03:35:03.713 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-26T03:35:03.713 INFO:teuthology.orchestra.run.smithi175.stdout: sudo /usr/sbin/cephadm shell --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-26T03:35:03.713 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-26T03:35:03.713 INFO:teuthology.orchestra.run.smithi175.stdout:Or, if you are only running a single cluster on this host: 2024-03-26T03:35:03.713 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-26T03:35:03.713 INFO:teuthology.orchestra.run.smithi175.stdout: sudo /usr/sbin/cephadm shell 2024-03-26T03:35:03.713 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-26T03:35:03.713 INFO:teuthology.orchestra.run.smithi175.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-26T03:35:03.713 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-26T03:35:03.713 INFO:teuthology.orchestra.run.smithi175.stdout: ceph telemetry on 2024-03-26T03:35:03.713 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-26T03:35:03.714 INFO:teuthology.orchestra.run.smithi175.stdout:For more information see: 2024-03-26T03:35:03.714 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-26T03:35:03.714 INFO:teuthology.orchestra.run.smithi175.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-26T03:35:03.714 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-26T03:35:03.714 INFO:teuthology.orchestra.run.smithi175.stdout:Bootstrap complete. 2024-03-26T03:35:03.755 INFO:tasks.cephadm:Fetching config... 2024-03-26T03:35:03.755 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-26T03:35:03.755 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-26T03:35:03.804 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-26T03:35:03.804 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-26T03:35:03.804 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-26T03:35:03.852 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-26T03:35:03.852 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-26T03:35:03.852 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-26T03:35:03.907 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-26T03:35:03.907 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-26T03:35:03.907 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-26T03:35:03.956 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-26T03:35:03.956 DEBUG:teuthology.orchestra.run.smithi175:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIxbSJkqZCSs/lOfW0OK6oNIT2bcL4u25oaSQElk3cx0GxdAPDGMr23NqmfnCSvPKt3/GWsQRLo2qhZjMU3yCAErTt4mWgxPrgZIoIQY9fFlro7xHNoztqNYMeMDjrLLdVpL+Dc8Gw3heu2Pyh0YPsVgKVqlEwkcFRKhedWno8s1cEGDof8iXaaVsQM2FRV+KtnU//ZDapHDferKYmyb54R/x9PVwfrbOlR9h3EYvASpkmB+anwS6gfqkoI0NC4Y7ZJpwCfIZTiiZl4TYD8XbYXddepHTNt/lbdHwkBOBEdOU8H3iKWB1T0XiMFSXyW5QaWorLnSvzEA5qxtFdG/RzBxLB7SjTwIUDleCvEFsSjtIBQWPWec8m/7TKy6JB9vxUZqGGH9iyD++1nGL25wFU5ybdGNu1XnxP/yYywvcdXaBQydMB3Wf8Bf/hCA3nsxg0KgkN03r1BeqtTnidvoSjBnHtaqQdr0esYo2gBPlAw2Fe974XEJBxuAz5dRHWnoc= ceph-762e930c-eb21-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-26T03:35:04.020 INFO:teuthology.orchestra.run.smithi175.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIxbSJkqZCSs/lOfW0OK6oNIT2bcL4u25oaSQElk3cx0GxdAPDGMr23NqmfnCSvPKt3/GWsQRLo2qhZjMU3yCAErTt4mWgxPrgZIoIQY9fFlro7xHNoztqNYMeMDjrLLdVpL+Dc8Gw3heu2Pyh0YPsVgKVqlEwkcFRKhedWno8s1cEGDof8iXaaVsQM2FRV+KtnU//ZDapHDferKYmyb54R/x9PVwfrbOlR9h3EYvASpkmB+anwS6gfqkoI0NC4Y7ZJpwCfIZTiiZl4TYD8XbYXddepHTNt/lbdHwkBOBEdOU8H3iKWB1T0XiMFSXyW5QaWorLnSvzEA5qxtFdG/RzBxLB7SjTwIUDleCvEFsSjtIBQWPWec8m/7TKy6JB9vxUZqGGH9iyD++1nGL25wFU5ybdGNu1XnxP/yYywvcdXaBQydMB3Wf8Bf/hCA3nsxg0KgkN03r1BeqtTnidvoSjBnHtaqQdr0esYo2gBPlAw2Fe974XEJBxuAz5dRHWnoc= ceph-762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T03:35:04.029 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIxbSJkqZCSs/lOfW0OK6oNIT2bcL4u25oaSQElk3cx0GxdAPDGMr23NqmfnCSvPKt3/GWsQRLo2qhZjMU3yCAErTt4mWgxPrgZIoIQY9fFlro7xHNoztqNYMeMDjrLLdVpL+Dc8Gw3heu2Pyh0YPsVgKVqlEwkcFRKhedWno8s1cEGDof8iXaaVsQM2FRV+KtnU//ZDapHDferKYmyb54R/x9PVwfrbOlR9h3EYvASpkmB+anwS6gfqkoI0NC4Y7ZJpwCfIZTiiZl4TYD8XbYXddepHTNt/lbdHwkBOBEdOU8H3iKWB1T0XiMFSXyW5QaWorLnSvzEA5qxtFdG/RzBxLB7SjTwIUDleCvEFsSjtIBQWPWec8m/7TKy6JB9vxUZqGGH9iyD++1nGL25wFU5ybdGNu1XnxP/yYywvcdXaBQydMB3Wf8Bf/hCA3nsxg0KgkN03r1BeqtTnidvoSjBnHtaqQdr0esYo2gBPlAw2Fe974XEJBxuAz5dRHWnoc= ceph-762e930c-eb21-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-26T03:35:04.052 INFO:teuthology.orchestra.run.smithi190.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIxbSJkqZCSs/lOfW0OK6oNIT2bcL4u25oaSQElk3cx0GxdAPDGMr23NqmfnCSvPKt3/GWsQRLo2qhZjMU3yCAErTt4mWgxPrgZIoIQY9fFlro7xHNoztqNYMeMDjrLLdVpL+Dc8Gw3heu2Pyh0YPsVgKVqlEwkcFRKhedWno8s1cEGDof8iXaaVsQM2FRV+KtnU//ZDapHDferKYmyb54R/x9PVwfrbOlR9h3EYvASpkmB+anwS6gfqkoI0NC4Y7ZJpwCfIZTiiZl4TYD8XbYXddepHTNt/lbdHwkBOBEdOU8H3iKWB1T0XiMFSXyW5QaWorLnSvzEA5qxtFdG/RzBxLB7SjTwIUDleCvEFsSjtIBQWPWec8m/7TKy6JB9vxUZqGGH9iyD++1nGL25wFU5ybdGNu1XnxP/yYywvcdXaBQydMB3Wf8Bf/hCA3nsxg0KgkN03r1BeqtTnidvoSjBnHtaqQdr0esYo2gBPlAw2Fe974XEJBxuAz5dRHWnoc= ceph-762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T03:35:04.061 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-26T03:35:04.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:04 smithi175 bash[21527]: audit 2024-03-26T03:35:03.148665+0000 mon.a (mon.0) 96 : audit [INF] from='client.? 172.21.15.175:0/3024309926' entity='client.admin' 2024-03-26T03:35:08.695 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:35:11.624 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.620+0000 7f7211892700 1 -- 172.21.15.175:0/749427395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f720c100900 msgr2=0x7f720c100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:35:11.625 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.620+0000 7f7211892700 1 --2- 172.21.15.175:0/749427395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f720c100900 0x7f720c100d20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009910 tx=0x7f71fc009c20 comp rx=0 tx=0).stop 2024-03-26T03:35:11.625 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.620+0000 7f7211892700 1 -- 172.21.15.175:0/749427395 shutdown_connections 2024-03-26T03:35:11.625 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.620+0000 7f7211892700 1 --2- 172.21.15.175:0/749427395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f720c100900 0x7f720c100d20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:35:11.625 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.620+0000 7f7211892700 1 -- 172.21.15.175:0/749427395 >> 172.21.15.175:0/749427395 conn(0x7f720c0fc500 msgr2=0x7f720c0fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:35:11.625 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.620+0000 7f7211892700 1 -- 172.21.15.175:0/749427395 shutdown_connections 2024-03-26T03:35:11.625 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.620+0000 7f7211892700 1 -- 172.21.15.175:0/749427395 wait complete. 2024-03-26T03:35:11.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.620+0000 7f7211892700 1 Processor -- start 2024-03-26T03:35:11.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.620+0000 7f7211892700 1 -- start start 2024-03-26T03:35:11.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.620+0000 7f7211892700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f720c100900 0x7f720c198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:35:11.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.620+0000 7f7211892700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f720c109790 con 0x7f720c100900 2024-03-26T03:35:11.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.620+0000 7f720affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f720c100900 0x7f720c198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:35:11.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.620+0000 7f720affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f720c100900 0x7f720c198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:58826/0 (socket says 172.21.15.175:58826) 2024-03-26T03:35:11.628 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.620+0000 7f720affd700 1 -- 172.21.15.175:0/3135995142 learned_addr learned my addr 172.21.15.175:0/3135995142 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:35:11.629 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.620+0000 7f720affd700 1 -- 172.21.15.175:0/3135995142 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720c199410 con 0x7f720c100900 2024-03-26T03:35:11.629 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.620+0000 7f720affd700 1 --2- 172.21.15.175:0/3135995142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f720c100900 0x7f720c198ed0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f71fc00b5c0 tx=0x7f71fc0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:35:11.629 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.620+0000 7f7210890700 1 -- 172.21.15.175:0/3135995142 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71fc027070 con 0x7f720c100900 2024-03-26T03:35:11.629 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.620+0000 7f7211892700 1 -- 172.21.15.175:0/3135995142 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f720c1996a0 con 0x7f720c100900 2024-03-26T03:35:11.629 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.620+0000 7f7210890700 1 -- 172.21.15.175:0/3135995142 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f71fc00f460 con 0x7f720c100900 2024-03-26T03:35:11.630 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.620+0000 7f7210890700 1 -- 172.21.15.175:0/3135995142 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71fc017440 con 0x7f720c100900 2024-03-26T03:35:11.630 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.620+0000 7f7211892700 1 -- 172.21.15.175:0/3135995142 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f720c19c390 con 0x7f720c100900 2024-03-26T03:35:11.630 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.620+0000 7f7210890700 1 -- 172.21.15.175:0/3135995142 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 13) ==== 46523+0+0 (secure 0 0 0) 0x7f71fc02a070 con 0x7f720c100900 2024-03-26T03:35:11.630 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.620+0000 7f7211892700 1 -- 172.21.15.175:0/3135995142 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f720c109ef0 con 0x7f720c100900 2024-03-26T03:35:11.630 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.620+0000 7f7210890700 1 --2- 172.21.15.175:0/3135995142 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f71f8039470 0x7f71f803b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:35:11.630 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.620+0000 7f7210890700 1 -- 172.21.15.175:0/3135995142 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f71fc04d840 con 0x7f720c100900 2024-03-26T03:35:11.630 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.624+0000 7f7203fff700 1 --2- 172.21.15.175:0/3135995142 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f71f8039470 0x7f71f803b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:35:11.631 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.624+0000 7f7203fff700 1 --2- 172.21.15.175:0/3135995142 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f71f8039470 0x7f71f803b930 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f71f4006fd0 tx=0x7f71f4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:35:11.633 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.628+0000 7f7210890700 1 -- 172.21.15.175:0/3135995142 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f71fc013070 con 0x7f720c100900 2024-03-26T03:35:11.828 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.824+0000 7f7211892700 1 -- 172.21.15.175:0/3135995142 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f720c02dd30 con 0x7f720c100900 2024-03-26T03:35:11.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.828+0000 7f7210890700 1 -- 172.21.15.175:0/3135995142 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f71fc031020 con 0x7f720c100900 2024-03-26T03:35:11.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.832+0000 7f7211892700 1 -- 172.21.15.175:0/3135995142 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f71f8039470 msgr2=0x7f71f803b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:35:11.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.832+0000 7f7211892700 1 --2- 172.21.15.175:0/3135995142 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f71f8039470 0x7f71f803b930 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f71f4006fd0 tx=0x7f71f4006e40 comp rx=0 tx=0).stop 2024-03-26T03:35:11.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.832+0000 7f7211892700 1 -- 172.21.15.175:0/3135995142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f720c100900 msgr2=0x7f720c198ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:35:11.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.832+0000 7f7211892700 1 --2- 172.21.15.175:0/3135995142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f720c100900 0x7f720c198ed0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f71fc00b5c0 tx=0x7f71fc0057c0 comp rx=0 tx=0).stop 2024-03-26T03:35:11.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.832+0000 7f7211892700 1 -- 172.21.15.175:0/3135995142 shutdown_connections 2024-03-26T03:35:11.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.832+0000 7f7211892700 1 --2- 172.21.15.175:0/3135995142 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f71f8039470 0x7f71f803b930 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:35:11.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.832+0000 7f7211892700 1 --2- 172.21.15.175:0/3135995142 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f720c100900 0x7f720c198ed0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:35:11.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.832+0000 7f7211892700 1 -- 172.21.15.175:0/3135995142 >> 172.21.15.175:0/3135995142 conn(0x7f720c0fc500 msgr2=0x7f720c0fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:35:11.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.832+0000 7f7211892700 1 -- 172.21.15.175:0/3135995142 shutdown_connections 2024-03-26T03:35:11.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:11.832+0000 7f7211892700 1 -- 172.21.15.175:0/3135995142 wait complete. 2024-03-26T03:35:12.515 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-26T03:35:12.516 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-26T03:35:12.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:12 smithi175 bash[21527]: audit 2024-03-26T03:35:11.514594+0000 mon.a (mon.0) 97 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:35:12.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:12 smithi175 bash[21527]: audit 2024-03-26T03:35:11.517035+0000 mon.a (mon.0) 98 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:35:12.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:12 smithi175 bash[21527]: audit 2024-03-26T03:35:11.517737+0000 mon.a (mon.0) 99 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi175", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:35:12.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:12 smithi175 bash[21527]: audit 2024-03-26T03:35:11.520934+0000 mon.a (mon.0) 100 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:35:12.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:12 smithi175 bash[21527]: audit 2024-03-26T03:35:11.537887+0000 mon.a (mon.0) 101 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:35:12.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:12 smithi175 bash[21527]: audit 2024-03-26T03:35:11.541766+0000 mon.a (mon.0) 102 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:35:12.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:12 smithi175 bash[21527]: audit 2024-03-26T03:35:11.832569+0000 mon.a (mon.0) 103 : audit [INF] from='client.? 172.21.15.175:0/3135995142' entity='client.admin' 2024-03-26T03:35:12.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:12 smithi175 bash[21527]: audit 2024-03-26T03:35:11.872418+0000 mon.a (mon.0) 104 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:35:12.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:12 smithi175 bash[21527]: audit 2024-03-26T03:35:11.876392+0000 mon.a (mon.0) 105 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:35:14.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:13 smithi175 bash[21527]: cluster 2024-03-26T03:35:12.174907+0000 mgr.y (mgr.14152) 9 : cluster [DBG] pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:35:16.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:15 smithi175 bash[21527]: cluster 2024-03-26T03:35:14.175329+0000 mgr.y (mgr.14152) 10 : cluster [DBG] pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:35:17.226 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:35:18.144 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:17 smithi175 bash[21527]: cluster 2024-03-26T03:35:16.175772+0000 mgr.y (mgr.14152) 11 : cluster [DBG] pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:35:19.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.664+0000 7f61b0474700 1 -- 172.21.15.175:0/2408720222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61a8102540 msgr2=0x7f61a8102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:35:19.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.664+0000 7f61b0474700 1 --2- 172.21.15.175:0/2408720222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61a8102540 0x7f61a8102960 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f61a4009910 tx=0x7f61a4009c20 comp rx=0 tx=0).stop 2024-03-26T03:35:19.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.664+0000 7f61b0474700 1 -- 172.21.15.175:0/2408720222 shutdown_connections 2024-03-26T03:35:19.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.664+0000 7f61b0474700 1 --2- 172.21.15.175:0/2408720222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61a8102540 0x7f61a8102960 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:35:19.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.664+0000 7f61b0474700 1 -- 172.21.15.175:0/2408720222 >> 172.21.15.175:0/2408720222 conn(0x7f61a80fbc90 msgr2=0x7f61a80fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:35:19.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.664+0000 7f61b0474700 1 -- 172.21.15.175:0/2408720222 shutdown_connections 2024-03-26T03:35:19.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.664+0000 7f61b0474700 1 -- 172.21.15.175:0/2408720222 wait complete. 2024-03-26T03:35:19.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.668+0000 7f61b0474700 1 Processor -- start 2024-03-26T03:35:19.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.668+0000 7f61b0474700 1 -- start start 2024-03-26T03:35:19.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.668+0000 7f61b0474700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61a8102540 0x7f61a8196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:35:19.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.668+0000 7f61b0474700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f61a8106490 con 0x7f61a8102540 2024-03-26T03:35:19.672 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.668+0000 7f61ae210700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61a8102540 0x7f61a8196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:35:19.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.668+0000 7f61ae210700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61a8102540 0x7f61a8196cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:58856/0 (socket says 172.21.15.175:58856) 2024-03-26T03:35:19.673 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.668+0000 7f61ae210700 1 -- 172.21.15.175:0/285110039 learned_addr learned my addr 172.21.15.175:0/285110039 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:35:19.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.668+0000 7f61ae210700 1 -- 172.21.15.175:0/285110039 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61a8197230 con 0x7f61a8102540 2024-03-26T03:35:19.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.668+0000 7f61ae210700 1 --2- 172.21.15.175:0/285110039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61a8102540 0x7f61a8196cf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f61a4005710 tx=0x7f61a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:35:19.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.668+0000 7f619f7fe700 1 -- 172.21.15.175:0/285110039 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61a4027070 con 0x7f61a8102540 2024-03-26T03:35:19.674 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.668+0000 7f619f7fe700 1 -- 172.21.15.175:0/285110039 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f61a400f460 con 0x7f61a8102540 2024-03-26T03:35:19.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.668+0000 7f61b0474700 1 -- 172.21.15.175:0/285110039 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61a81974c0 con 0x7f61a8102540 2024-03-26T03:35:19.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.668+0000 7f619f7fe700 1 -- 172.21.15.175:0/285110039 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61a4017440 con 0x7f61a8102540 2024-03-26T03:35:19.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.668+0000 7f61b0474700 1 -- 172.21.15.175:0/285110039 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61a819a1a0 con 0x7f61a8102540 2024-03-26T03:35:19.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.668+0000 7f619f7fe700 1 -- 172.21.15.175:0/285110039 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 13) ==== 46523+0+0 (secure 0 0 0) 0x7f61a402a070 con 0x7f61a8102540 2024-03-26T03:35:19.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.668+0000 7f61b0474700 1 -- 172.21.15.175:0/285110039 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f618c005150 con 0x7f61a8102540 2024-03-26T03:35:19.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.668+0000 7f619f7fe700 1 --2- 172.21.15.175:0/285110039 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f6194039470 0x7f619403b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:35:19.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.668+0000 7f619f7fe700 1 -- 172.21.15.175:0/285110039 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f61a404d790 con 0x7f61a8102540 2024-03-26T03:35:19.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.668+0000 7f61ada0f700 1 --2- 172.21.15.175:0/285110039 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f6194039470 0x7f619403b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:35:19.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.668+0000 7f61ada0f700 1 --2- 172.21.15.175:0/285110039 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f6194039470 0x7f619403b930 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6198006fd0 tx=0x7f6198006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:35:19.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.672+0000 7f619f7fe700 1 -- 172.21.15.175:0/285110039 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f61a4013070 con 0x7f61a8102540 2024-03-26T03:35:19.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.892+0000 7f61b0474700 1 -- 172.21.15.175:0/285110039 --> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f618c000bf0 con 0x7f6194039470 2024-03-26T03:35:19.901 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.896+0000 7f619f7fe700 1 -- 172.21.15.175:0/285110039 <== mgr.14152 v2:172.21.15.175:6800/2826039805 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f618c000bf0 con 0x7f6194039470 2024-03-26T03:35:19.903 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.900+0000 7f61b0474700 1 -- 172.21.15.175:0/285110039 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f6194039470 msgr2=0x7f619403b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:35:19.903 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.900+0000 7f61b0474700 1 --2- 172.21.15.175:0/285110039 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f6194039470 0x7f619403b930 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6198006fd0 tx=0x7f6198006e40 comp rx=0 tx=0).stop 2024-03-26T03:35:19.904 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.900+0000 7f61b0474700 1 -- 172.21.15.175:0/285110039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61a8102540 msgr2=0x7f61a8196cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:35:19.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.900+0000 7f61b0474700 1 --2- 172.21.15.175:0/285110039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61a8102540 0x7f61a8196cf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f61a4005710 tx=0x7f61a40057f0 comp rx=0 tx=0).stop 2024-03-26T03:35:19.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.900+0000 7f61b0474700 1 -- 172.21.15.175:0/285110039 shutdown_connections 2024-03-26T03:35:19.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.900+0000 7f61b0474700 1 --2- 172.21.15.175:0/285110039 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f6194039470 0x7f619403b930 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:35:19.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.900+0000 7f61b0474700 1 --2- 172.21.15.175:0/285110039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61a8102540 0x7f61a8196cf0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:35:19.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.900+0000 7f61b0474700 1 -- 172.21.15.175:0/285110039 >> 172.21.15.175:0/285110039 conn(0x7f61a80fbc90 msgr2=0x7f61a8104260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:35:19.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.900+0000 7f61b0474700 1 -- 172.21.15.175:0/285110039 shutdown_connections 2024-03-26T03:35:19.906 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:19.900+0000 7f61b0474700 1 -- 172.21.15.175:0/285110039 wait complete. 2024-03-26T03:35:20.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:19 smithi175 bash[21527]: cluster 2024-03-26T03:35:18.176079+0000 mgr.y (mgr.14152) 12 : cluster [DBG] pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:35:20.571 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi190 2024-03-26T03:35:20.571 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-03-26T03:35:20.571 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/etc/ceph/ceph.conf 2024-03-26T03:35:20.577 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-03-26T03:35:20.577 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T03:35:20.626 INFO:tasks.cephadm:Adding host smithi190 to orchestrator... 2024-03-26T03:35:20.626 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph orch host add smithi190 2024-03-26T03:35:21.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:20 smithi175 bash[21527]: audit 2024-03-26T03:35:19.897487+0000 mgr.y (mgr.14152) 13 : audit [DBG] from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:35:21.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:20 smithi175 bash[21527]: audit 2024-03-26T03:35:19.900778+0000 mon.a (mon.0) 106 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:35:21.246 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:20 smithi175 bash[21527]: audit 2024-03-26T03:35:19.927066+0000 mon.a (mon.0) 107 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:35:21.246 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:20 smithi175 bash[21527]: audit 2024-03-26T03:35:19.928486+0000 mon.a (mon.0) 108 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:35:21.246 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:20 smithi175 bash[21527]: audit 2024-03-26T03:35:19.929184+0000 mon.a (mon.0) 109 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:35:21.246 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:20 smithi175 bash[21527]: cephadm 2024-03-26T03:35:19.930141+0000 mgr.y (mgr.14152) 14 : cephadm [INF] Updating smithi175:/etc/ceph/ceph.conf 2024-03-26T03:35:21.246 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:20 smithi175 bash[21527]: cephadm 2024-03-26T03:35:20.044269+0000 mgr.y (mgr.14152) 15 : cephadm [INF] Updating smithi175:/var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T03:35:21.246 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:20 smithi175 bash[21527]: audit 2024-03-26T03:35:20.544510+0000 mon.a (mon.0) 110 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:35:21.246 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:20 smithi175 bash[21527]: audit 2024-03-26T03:35:20.549222+0000 mon.a (mon.0) 111 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:35:21.246 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:20 smithi175 bash[21527]: audit 2024-03-26T03:35:20.553838+0000 mon.a (mon.0) 112 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:35:22.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:21 smithi175 bash[21527]: cluster 2024-03-26T03:35:20.176509+0000 mgr.y (mgr.14152) 16 : cluster [DBG] pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:35:22.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:21 smithi175 bash[21527]: cephadm 2024-03-26T03:35:20.208956+0000 mgr.y (mgr.14152) 17 : cephadm [INF] Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-03-26T03:35:22.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:21 smithi175 bash[21527]: cephadm 2024-03-26T03:35:20.374818+0000 mgr.y (mgr.14152) 18 : cephadm [INF] Updating smithi175:/var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T03:35:24.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:23 smithi175 bash[21527]: cluster 2024-03-26T03:35:22.176978+0000 mgr.y (mgr.14152) 19 : cluster [DBG] pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:35:25.335 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:35:26.236 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:25 smithi175 bash[21527]: cluster 2024-03-26T03:35:24.177479+0000 mgr.y (mgr.14152) 20 : cluster [DBG] pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:35:27.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:27.735+0000 7fae1526e700 1 -- 172.21.15.175:0/2197634020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae10102260 msgr2=0x7fae10102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:35:27.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:27.735+0000 7fae1526e700 1 --2- 172.21.15.175:0/2197634020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae10102260 0x7fae10102640 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fadf8009910 tx=0x7fadf8009c20 comp rx=0 tx=0).stop 2024-03-26T03:35:27.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:27.735+0000 7fae1526e700 1 -- 172.21.15.175:0/2197634020 shutdown_connections 2024-03-26T03:35:27.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:27.735+0000 7fae1526e700 1 --2- 172.21.15.175:0/2197634020 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae10102260 0x7fae10102640 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:35:27.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:27.735+0000 7fae1526e700 1 -- 172.21.15.175:0/2197634020 >> 172.21.15.175:0/2197634020 conn(0x7fae100fdb10 msgr2=0x7fae100fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:35:27.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:27.735+0000 7fae1526e700 1 -- 172.21.15.175:0/2197634020 shutdown_connections 2024-03-26T03:35:27.741 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:27.735+0000 7fae1526e700 1 -- 172.21.15.175:0/2197634020 wait complete. 2024-03-26T03:35:27.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:27.735+0000 7fae1526e700 1 Processor -- start 2024-03-26T03:35:27.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:27.735+0000 7fae1526e700 1 -- start start 2024-03-26T03:35:27.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:27.735+0000 7fae1526e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae10102260 0x7fae10197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:35:27.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:27.735+0000 7fae1526e700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fae1010baa0 con 0x7fae10102260 2024-03-26T03:35:27.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:27.735+0000 7fae0effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae10102260 0x7fae10197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:35:27.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:27.735+0000 7fae0effd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae10102260 0x7fae10197090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45858/0 (socket says 172.21.15.175:45858) 2024-03-26T03:35:27.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:27.735+0000 7fae0effd700 1 -- 172.21.15.175:0/1756556232 learned_addr learned my addr 172.21.15.175:0/1756556232 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:35:27.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:27.735+0000 7fae0effd700 1 -- 172.21.15.175:0/1756556232 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae101975d0 con 0x7fae10102260 2024-03-26T03:35:27.743 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:27.739+0000 7fae0effd700 1 --2- 172.21.15.175:0/1756556232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae10102260 0x7fae10197090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fadf8006010 tx=0x7fadf800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:35:27.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:27.739+0000 7fae07fff700 1 -- 172.21.15.175:0/1756556232 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadf8027070 con 0x7fae10102260 2024-03-26T03:35:27.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:27.739+0000 7fae07fff700 1 -- 172.21.15.175:0/1756556232 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fadf8022470 con 0x7fae10102260 2024-03-26T03:35:27.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:27.739+0000 7fae07fff700 1 -- 172.21.15.175:0/1756556232 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadf8017440 con 0x7fae10102260 2024-03-26T03:35:27.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:27.739+0000 7fae1526e700 1 -- 172.21.15.175:0/1756556232 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae101978c0 con 0x7fae10102260 2024-03-26T03:35:27.744 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:27.739+0000 7fae1526e700 1 -- 172.21.15.175:0/1756556232 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae1019b960 con 0x7fae10102260 2024-03-26T03:35:27.745 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:27.739+0000 7fae07fff700 1 -- 172.21.15.175:0/1756556232 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 13) ==== 46523+0+0 (secure 0 0 0) 0x7fadf800fb60 con 0x7fae10102260 2024-03-26T03:35:27.745 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:27.739+0000 7fae1526e700 1 -- 172.21.15.175:0/1756556232 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae10103740 con 0x7fae10102260 2024-03-26T03:35:27.745 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:27.739+0000 7fae07fff700 1 --2- 172.21.15.175:0/1756556232 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fadfc039470 0x7fadfc03b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:35:27.745 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:27.739+0000 7fae07fff700 1 -- 172.21.15.175:0/1756556232 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fadf801e030 con 0x7fae10102260 2024-03-26T03:35:27.745 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:27.739+0000 7fae0e7fc700 1 --2- 172.21.15.175:0/1756556232 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fadfc039470 0x7fadfc03b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:35:27.746 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:27.739+0000 7fae0e7fc700 1 --2- 172.21.15.175:0/1756556232 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fadfc039470 0x7fadfc03b930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fae00006fd0 tx=0x7fae00006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:35:27.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:27.743+0000 7fae07fff700 1 -- 172.21.15.175:0/1756556232 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7fadf8013070 con 0x7fae10102260 2024-03-26T03:35:27.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:27.959+0000 7fae1526e700 1 -- 172.21.15.175:0/1756556232 --> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi190", "target": ["mon-mgr", ""]}) -- 0x7fae101983f0 con 0x7fadfc039470 2024-03-26T03:35:28.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:27 smithi175 bash[21527]: cluster 2024-03-26T03:35:26.177963+0000 mgr.y (mgr.14152) 21 : cluster [DBG] pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:35:29.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:28 smithi175 bash[21527]: audit 2024-03-26T03:35:27.967131+0000 mgr.y (mgr.14152) 22 : audit [DBG] from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi190", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:35:30.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:29 smithi175 bash[21527]: cluster 2024-03-26T03:35:28.178276+0000 mgr.y (mgr.14152) 23 : cluster [DBG] pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:35:30.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:29 smithi175 bash[21527]: cephadm 2024-03-26T03:35:28.629364+0000 mgr.y (mgr.14152) 24 : cephadm [INF] Deploying cephadm binary to smithi190 2024-03-26T03:35:32.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:31 smithi175 bash[21527]: cluster 2024-03-26T03:35:30.178542+0000 mgr.y (mgr.14152) 25 : cluster [DBG] pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:35:33.822 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:33.815+0000 7fae07fff700 1 -- 172.21.15.175:0/1756556232 <== mgr.14152 v2:172.21.15.175:6800/2826039805 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fae101983f0 con 0x7fadfc039470 2024-03-26T03:35:33.822 INFO:teuthology.orchestra.run.smithi175.stdout:Added host 'smithi190' with addr '172.21.15.190' 2024-03-26T03:35:33.825 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:33.819+0000 7fae1526e700 1 -- 172.21.15.175:0/1756556232 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fadfc039470 msgr2=0x7fadfc03b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:35:33.825 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:33.819+0000 7fae1526e700 1 --2- 172.21.15.175:0/1756556232 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fadfc039470 0x7fadfc03b930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fae00006fd0 tx=0x7fae00006e40 comp rx=0 tx=0).stop 2024-03-26T03:35:33.825 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:33.819+0000 7fae1526e700 1 -- 172.21.15.175:0/1756556232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae10102260 msgr2=0x7fae10197090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:35:33.825 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:33.819+0000 7fae1526e700 1 --2- 172.21.15.175:0/1756556232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae10102260 0x7fae10197090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fadf8006010 tx=0x7fadf800f740 comp rx=0 tx=0).stop 2024-03-26T03:35:33.825 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:33.819+0000 7fae1526e700 1 -- 172.21.15.175:0/1756556232 shutdown_connections 2024-03-26T03:35:33.826 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:33.819+0000 7fae1526e700 1 --2- 172.21.15.175:0/1756556232 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fadfc039470 0x7fadfc03b930 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:35:33.826 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:33.819+0000 7fae1526e700 1 --2- 172.21.15.175:0/1756556232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae10102260 0x7fae10197090 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:35:33.826 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:33.819+0000 7fae1526e700 1 -- 172.21.15.175:0/1756556232 >> 172.21.15.175:0/1756556232 conn(0x7fae100fdb10 msgr2=0x7fae10109ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:35:33.826 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:33.819+0000 7fae1526e700 1 -- 172.21.15.175:0/1756556232 shutdown_connections 2024-03-26T03:35:33.826 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:33.819+0000 7fae1526e700 1 -- 172.21.15.175:0/1756556232 wait complete. 2024-03-26T03:35:34.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:33 smithi175 bash[21527]: cluster 2024-03-26T03:35:32.178954+0000 mgr.y (mgr.14152) 26 : cluster [DBG] pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:35:34.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:33 smithi175 bash[21527]: audit 2024-03-26T03:35:33.820712+0000 mon.a (mon.0) 113 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:35:34.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:33 smithi175 bash[21527]: audit 2024-03-26T03:35:33.866648+0000 mon.a (mon.0) 114 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:35:34.585 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph orch host ls --format=json 2024-03-26T03:35:35.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:34 smithi175 bash[21527]: cephadm 2024-03-26T03:35:33.821647+0000 mgr.y (mgr.14152) 27 : cephadm [INF] Added host smithi190 2024-03-26T03:35:35.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:34 smithi175 bash[21527]: audit 2024-03-26T03:35:34.344227+0000 mon.a (mon.0) 115 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:35:36.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:35 smithi175 bash[21527]: cluster 2024-03-26T03:35:34.179382+0000 mgr.y (mgr.14152) 28 : cluster [DBG] pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:35:36.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:35 smithi175 bash[21527]: audit 2024-03-26T03:35:35.819672+0000 mon.a (mon.0) 116 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:35:38.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:38 smithi175 bash[21527]: cluster 2024-03-26T03:35:36.179828+0000 mgr.y (mgr.14152) 29 : cluster [DBG] pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:35:38.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:38 smithi175 bash[21527]: audit 2024-03-26T03:35:37.023946+0000 mon.a (mon.0) 117 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:35:39.295 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:35:40.276 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:40 smithi175 bash[21527]: cluster 2024-03-26T03:35:38.180278+0000 mgr.y (mgr.14152) 30 : cluster [DBG] pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:35:41.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.659+0000 7f0af26c1700 1 -- 172.21.15.175:0/1253033206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0aec102260 msgr2=0x7f0aec102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:35:41.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.659+0000 7f0af26c1700 1 --2- 172.21.15.175:0/1253033206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0aec102260 0x7f0aec102640 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0ae400b0a0 tx=0x7f0ae400b3b0 comp rx=0 tx=0).stop 2024-03-26T03:35:41.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.659+0000 7f0af26c1700 1 -- 172.21.15.175:0/1253033206 shutdown_connections 2024-03-26T03:35:41.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.659+0000 7f0af26c1700 1 --2- 172.21.15.175:0/1253033206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0aec102260 0x7f0aec102640 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:35:41.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.663+0000 7f0af26c1700 1 -- 172.21.15.175:0/1253033206 >> 172.21.15.175:0/1253033206 conn(0x7f0aec0fdb10 msgr2=0x7f0aec0fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:35:41.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.663+0000 7f0af26c1700 1 -- 172.21.15.175:0/1253033206 shutdown_connections 2024-03-26T03:35:41.668 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.663+0000 7f0af26c1700 1 -- 172.21.15.175:0/1253033206 wait complete. 2024-03-26T03:35:41.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.663+0000 7f0af26c1700 1 Processor -- start 2024-03-26T03:35:41.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.663+0000 7f0af26c1700 1 -- start start 2024-03-26T03:35:41.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.663+0000 7f0af26c1700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0aec102260 0x7f0aec197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:35:41.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.663+0000 7f0af26c1700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0aec10baa0 con 0x7f0aec102260 2024-03-26T03:35:41.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.663+0000 7f0aebfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0aec102260 0x7f0aec197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:35:41.669 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.663+0000 7f0aebfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0aec102260 0x7f0aec197080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55938/0 (socket says 172.21.15.175:55938) 2024-03-26T03:35:41.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.663+0000 7f0aebfff700 1 -- 172.21.15.175:0/2258010751 learned_addr learned my addr 172.21.15.175:0/2258010751 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:35:41.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.663+0000 7f0aebfff700 1 -- 172.21.15.175:0/2258010751 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aec1975c0 con 0x7f0aec102260 2024-03-26T03:35:41.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.663+0000 7f0aebfff700 1 --2- 172.21.15.175:0/2258010751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0aec102260 0x7f0aec197080 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4009610 tx=0x7f0ae400bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:35:41.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.663+0000 7f0ae97fa700 1 -- 172.21.15.175:0/2258010751 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ae4017040 con 0x7f0aec102260 2024-03-26T03:35:41.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.663+0000 7f0ae97fa700 1 -- 172.21.15.175:0/2258010751 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f0ae4022070 con 0x7f0aec102260 2024-03-26T03:35:41.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.663+0000 7f0af26c1700 1 -- 172.21.15.175:0/2258010751 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aec197850 con 0x7f0aec102260 2024-03-26T03:35:41.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.663+0000 7f0ae97fa700 1 -- 172.21.15.175:0/2258010751 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ae4004790 con 0x7f0aec102260 2024-03-26T03:35:41.670 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.663+0000 7f0af26c1700 1 -- 172.21.15.175:0/2258010751 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aec19b950 con 0x7f0aec102260 2024-03-26T03:35:41.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.663+0000 7f0ae97fa700 1 -- 172.21.15.175:0/2258010751 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 13) ==== 46523+0+0 (secure 0 0 0) 0x7f0ae40040d0 con 0x7f0aec102260 2024-03-26T03:35:41.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.663+0000 7f0ae97fa700 1 --2- 172.21.15.175:0/2258010751 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f0ad40394d0 0x7f0ad403b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:35:41.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.663+0000 7f0ae97fa700 1 -- 172.21.15.175:0/2258010751 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0ae400f050 con 0x7f0aec102260 2024-03-26T03:35:41.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.663+0000 7f0af26c1700 1 -- 172.21.15.175:0/2258010751 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aec103740 con 0x7f0aec102260 2024-03-26T03:35:41.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.663+0000 7f0aeb7fe700 1 --2- 172.21.15.175:0/2258010751 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f0ad40394d0 0x7f0ad403b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:35:41.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.663+0000 7f0aeb7fe700 1 --2- 172.21.15.175:0/2258010751 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f0ad40394d0 0x7f0ad403b990 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0006fd0 tx=0x7f0ae0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:35:41.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.671+0000 7f0ae97fa700 1 -- 172.21.15.175:0/2258010751 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f0ae40093e0 con 0x7f0aec102260 2024-03-26T03:35:41.887 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.883+0000 7f0af26c1700 1 -- 172.21.15.175:0/2258010751 --> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0aec063c70 con 0x7f0ad40394d0 2024-03-26T03:35:41.889 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.883+0000 7f0ae97fa700 1 -- 172.21.15.175:0/2258010751 <== mgr.14152 v2:172.21.15.175:6800/2826039805 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7f0aec063c70 con 0x7f0ad40394d0 2024-03-26T03:35:41.890 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-26T03:35:41.890 INFO:teuthology.orchestra.run.smithi175.stdout:[{"addr": "172.21.15.175", "hostname": "smithi175", "labels": [], "status": ""}, {"addr": "172.21.15.190", "hostname": "smithi190", "labels": [], "status": ""}] 2024-03-26T03:35:41.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.887+0000 7f0af26c1700 1 -- 172.21.15.175:0/2258010751 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f0ad40394d0 msgr2=0x7f0ad403b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:35:41.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.887+0000 7f0af26c1700 1 --2- 172.21.15.175:0/2258010751 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f0ad40394d0 0x7f0ad403b990 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0006fd0 tx=0x7f0ae0006e40 comp rx=0 tx=0).stop 2024-03-26T03:35:41.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.887+0000 7f0af26c1700 1 -- 172.21.15.175:0/2258010751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0aec102260 msgr2=0x7f0aec197080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:35:41.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.887+0000 7f0af26c1700 1 --2- 172.21.15.175:0/2258010751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0aec102260 0x7f0aec197080 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4009610 tx=0x7f0ae400bd40 comp rx=0 tx=0).stop 2024-03-26T03:35:41.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.887+0000 7f0af26c1700 1 -- 172.21.15.175:0/2258010751 shutdown_connections 2024-03-26T03:35:41.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.887+0000 7f0af26c1700 1 --2- 172.21.15.175:0/2258010751 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f0ad40394d0 0x7f0ad403b990 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:35:41.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.887+0000 7f0af26c1700 1 --2- 172.21.15.175:0/2258010751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0aec102260 0x7f0aec197080 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:35:41.892 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.887+0000 7f0af26c1700 1 -- 172.21.15.175:0/2258010751 >> 172.21.15.175:0/2258010751 conn(0x7f0aec0fdb10 msgr2=0x7f0aec109f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:35:41.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.887+0000 7f0af26c1700 1 -- 172.21.15.175:0/2258010751 shutdown_connections 2024-03-26T03:35:41.893 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:41.887+0000 7f0af26c1700 1 -- 172.21.15.175:0/2258010751 wait complete. 2024-03-26T03:35:42.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:42 smithi175 bash[21527]: cluster 2024-03-26T03:35:40.180639+0000 mgr.y (mgr.14152) 31 : cluster [DBG] pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:35:42.563 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-26T03:35:42.564 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph osd crush tunables default 2024-03-26T03:35:43.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:43 smithi175 bash[21527]: audit 2024-03-26T03:35:41.888863+0000 mgr.y (mgr.14152) 32 : audit [DBG] from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T03:35:43.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:43 smithi175 bash[21527]: audit 2024-03-26T03:35:42.460422+0000 mon.a (mon.0) 118 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:35:43.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:43 smithi175 bash[21527]: audit 2024-03-26T03:35:42.465719+0000 mon.a (mon.0) 119 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:35:43.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:43 smithi175 bash[21527]: audit 2024-03-26T03:35:42.471991+0000 mon.a (mon.0) 120 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:35:43.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:43 smithi175 bash[21527]: audit 2024-03-26T03:35:42.477364+0000 mon.a (mon.0) 121 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:35:43.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:43 smithi175 bash[21527]: audit 2024-03-26T03:35:42.478992+0000 mon.a (mon.0) 122 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi190", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:35:43.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:43 smithi175 bash[21527]: audit 2024-03-26T03:35:42.481002+0000 mon.a (mon.0) 123 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:35:43.496 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:43 smithi175 bash[21527]: audit 2024-03-26T03:35:42.482756+0000 mon.a (mon.0) 124 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:35:44.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:44 smithi175 bash[21527]: cluster 2024-03-26T03:35:42.181021+0000 mgr.y (mgr.14152) 33 : cluster [DBG] pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:35:44.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:44 smithi175 bash[21527]: cephadm 2024-03-26T03:35:42.485371+0000 mgr.y (mgr.14152) 34 : cephadm [INF] Updating smithi190:/etc/ceph/ceph.conf 2024-03-26T03:35:44.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:44 smithi175 bash[21527]: cephadm 2024-03-26T03:35:42.650543+0000 mgr.y (mgr.14152) 35 : cephadm [INF] Updating smithi190:/var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T03:35:44.496 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:44 smithi175 bash[21527]: cephadm 2024-03-26T03:35:42.814852+0000 mgr.y (mgr.14152) 36 : cephadm [INF] Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-03-26T03:35:44.496 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:44 smithi175 bash[21527]: cephadm 2024-03-26T03:35:42.980175+0000 mgr.y (mgr.14152) 37 : cephadm [INF] Updating smithi190:/var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T03:35:44.496 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:44 smithi175 bash[21527]: audit 2024-03-26T03:35:43.152057+0000 mon.a (mon.0) 125 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:35:44.496 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:44 smithi175 bash[21527]: audit 2024-03-26T03:35:43.160519+0000 mon.a (mon.0) 126 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:35:44.496 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:44 smithi175 bash[21527]: audit 2024-03-26T03:35:43.169054+0000 mon.a (mon.0) 127 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:35:46.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:46 smithi175 bash[21527]: cluster 2024-03-26T03:35:44.181410+0000 mgr.y (mgr.14152) 38 : cluster [DBG] pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:35:47.275 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:35:48.295 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:48 smithi175 bash[21527]: cluster 2024-03-26T03:35:46.181768+0000 mgr.y (mgr.14152) 39 : cluster [DBG] pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:35:49.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:49.643+0000 7f531faf3700 1 -- 172.21.15.175:0/3309715257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5318102260 msgr2=0x7f5318102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:35:49.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:49.643+0000 7f531faf3700 1 --2- 172.21.15.175:0/3309715257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5318102260 0x7f5318102640 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5314009910 tx=0x7f5314009c20 comp rx=0 tx=0).stop 2024-03-26T03:35:49.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:49.643+0000 7f531faf3700 1 -- 172.21.15.175:0/3309715257 shutdown_connections 2024-03-26T03:35:49.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:49.643+0000 7f531faf3700 1 --2- 172.21.15.175:0/3309715257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5318102260 0x7f5318102640 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:35:49.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:49.643+0000 7f531faf3700 1 -- 172.21.15.175:0/3309715257 >> 172.21.15.175:0/3309715257 conn(0x7f53180fdb10 msgr2=0x7f53180fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:35:49.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:49.643+0000 7f531faf3700 1 -- 172.21.15.175:0/3309715257 shutdown_connections 2024-03-26T03:35:49.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:49.643+0000 7f531faf3700 1 -- 172.21.15.175:0/3309715257 wait complete. 2024-03-26T03:35:49.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:49.643+0000 7f531faf3700 1 Processor -- start 2024-03-26T03:35:49.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:49.643+0000 7f531faf3700 1 -- start start 2024-03-26T03:35:49.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:49.643+0000 7f531faf3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5318102260 0x7f5318197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:35:49.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:49.643+0000 7f531faf3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f531810baa0 con 0x7f5318102260 2024-03-26T03:35:49.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:49.643+0000 7f531d88f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5318102260 0x7f5318197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:35:49.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:49.643+0000 7f531d88f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5318102260 0x7f5318197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55962/0 (socket says 172.21.15.175:55962) 2024-03-26T03:35:49.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:49.643+0000 7f531d88f700 1 -- 172.21.15.175:0/1300667012 learned_addr learned my addr 172.21.15.175:0/1300667012 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:35:49.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:49.643+0000 7f531d88f700 1 -- 172.21.15.175:0/1300667012 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5318197640 con 0x7f5318102260 2024-03-26T03:35:49.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:49.643+0000 7f531d88f700 1 --2- 172.21.15.175:0/1300667012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5318102260 0x7f5318197100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f531400f690 tx=0x7f531400f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:35:49.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:49.647+0000 7f530effd700 1 -- 172.21.15.175:0/1300667012 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5314027070 con 0x7f5318102260 2024-03-26T03:35:49.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:49.647+0000 7f530effd700 1 -- 172.21.15.175:0/1300667012 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f531400fb90 con 0x7f5318102260 2024-03-26T03:35:49.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:49.647+0000 7f531faf3700 1 -- 172.21.15.175:0/1300667012 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5318197930 con 0x7f5318102260 2024-03-26T03:35:49.652 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:49.647+0000 7f530effd700 1 -- 172.21.15.175:0/1300667012 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5314017440 con 0x7f5318102260 2024-03-26T03:35:49.652 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:49.647+0000 7f531faf3700 1 -- 172.21.15.175:0/1300667012 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f531819b980 con 0x7f5318102260 2024-03-26T03:35:49.653 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:49.647+0000 7f530effd700 1 -- 172.21.15.175:0/1300667012 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 13) ==== 46523+0+0 (secure 0 0 0) 0x7f531402a070 con 0x7f5318102260 2024-03-26T03:35:49.653 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:49.647+0000 7f531faf3700 1 -- 172.21.15.175:0/1300667012 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5318103740 con 0x7f5318102260 2024-03-26T03:35:49.653 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:49.647+0000 7f530effd700 1 --2- 172.21.15.175:0/1300667012 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f5304039470 0x7f530403b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:35:49.653 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:49.647+0000 7f530effd700 1 -- 172.21.15.175:0/1300667012 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f531401e030 con 0x7f5318102260 2024-03-26T03:35:49.653 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:49.647+0000 7f531d08e700 1 --2- 172.21.15.175:0/1300667012 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f5304039470 0x7f530403b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:35:49.653 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:49.647+0000 7f531d08e700 1 --2- 172.21.15.175:0/1300667012 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f5304039470 0x7f530403b930 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5308006fd0 tx=0x7f5308006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:35:49.657 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:49.651+0000 7f530effd700 1 -- 172.21.15.175:0/1300667012 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f5314013070 con 0x7f5318102260 2024-03-26T03:35:49.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:49.867+0000 7f531faf3700 1 -- 172.21.15.175:0/1300667012 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f5318063c70 con 0x7f5318102260 2024-03-26T03:35:50.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:50.031+0000 7f530effd700 1 -- 172.21.15.175:0/1300667012 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f5314017a70 con 0x7f5318102260 2024-03-26T03:35:50.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:50.035+0000 7f531faf3700 1 -- 172.21.15.175:0/1300667012 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f5304039470 msgr2=0x7f530403b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:35:50.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:50.035+0000 7f531faf3700 1 --2- 172.21.15.175:0/1300667012 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f5304039470 0x7f530403b930 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5308006fd0 tx=0x7f5308006e40 comp rx=0 tx=0).stop 2024-03-26T03:35:50.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:50.035+0000 7f531faf3700 1 -- 172.21.15.175:0/1300667012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5318102260 msgr2=0x7f5318197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:35:50.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:50.035+0000 7f531faf3700 1 --2- 172.21.15.175:0/1300667012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5318102260 0x7f5318197100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f531400f690 tx=0x7f531400f770 comp rx=0 tx=0).stop 2024-03-26T03:35:50.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:50.035+0000 7f531faf3700 1 -- 172.21.15.175:0/1300667012 shutdown_connections 2024-03-26T03:35:50.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:50.035+0000 7f531faf3700 1 --2- 172.21.15.175:0/1300667012 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f5304039470 0x7f530403b930 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:35:50.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:50.035+0000 7f531faf3700 1 --2- 172.21.15.175:0/1300667012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5318102260 0x7f5318197100 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:35:50.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:50.035+0000 7f531faf3700 1 -- 172.21.15.175:0/1300667012 >> 172.21.15.175:0/1300667012 conn(0x7f53180fdb10 msgr2=0x7f5318109f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:35:50.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:50.035+0000 7f531faf3700 1 -- 172.21.15.175:0/1300667012 shutdown_connections 2024-03-26T03:35:50.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:35:50.035+0000 7f531faf3700 1 -- 172.21.15.175:0/1300667012 wait complete. 2024-03-26T03:35:50.043 INFO:teuthology.orchestra.run.smithi175.stderr:adjusted tunables profile to default 2024-03-26T03:35:50.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:50 smithi175 bash[21527]: cluster 2024-03-26T03:35:48.182087+0000 mgr.y (mgr.14152) 40 : cluster [DBG] pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:35:50.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:50 smithi175 bash[21527]: audit 2024-03-26T03:35:49.875498+0000 mon.a (mon.0) 128 : audit [INF] from='client.? 172.21.15.175:0/1300667012' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-26T03:35:50.739 INFO:tasks.cephadm:Adding mon.a on smithi175 2024-03-26T03:35:50.739 INFO:tasks.cephadm:Adding mon.c on smithi175 2024-03-26T03:35:50.739 INFO:tasks.cephadm:Adding mon.b on smithi190 2024-03-26T03:35:50.739 DEBUG:teuthology.orchestra.run.smithi190:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph orch apply mon '3;smithi175:172.21.15.175=a;smithi175:[v2:172.21.15.175:3301,v1:172.21.15.175:6790]=c;smithi190:172.21.15.190=b' 2024-03-26T03:35:51.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:51 smithi175 bash[21527]: audit 2024-03-26T03:35:50.037918+0000 mon.a (mon.0) 129 : audit [INF] from='client.? 172.21.15.175:0/1300667012' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-26T03:35:51.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:51 smithi175 bash[21527]: cluster 2024-03-26T03:35:50.042783+0000 mon.a (mon.0) 130 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-03-26T03:35:51.949 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T03:35:52.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:52 smithi175 bash[21527]: cluster 2024-03-26T03:35:50.182473+0000 mgr.y (mgr.14152) 41 : cluster [DBG] pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:35:54.224 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.221+0000 7f042e853700 1 -- 172.21.15.190:0/39267761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0428102280 msgr2=0x7f0428102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:35:54.224 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.221+0000 7f042e853700 1 --2- 172.21.15.190:0/39267761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0428102280 0x7f0428102660 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0418009910 tx=0x7f0418009c20 comp rx=0 tx=0).stop 2024-03-26T03:35:54.224 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.221+0000 7f042e853700 1 -- 172.21.15.190:0/39267761 shutdown_connections 2024-03-26T03:35:54.224 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.221+0000 7f042e853700 1 --2- 172.21.15.190:0/39267761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0428102280 0x7f0428102660 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:35:54.224 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.221+0000 7f042e853700 1 -- 172.21.15.190:0/39267761 >> 172.21.15.190:0/39267761 conn(0x7f04280fdb30 msgr2=0x7f04280fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:35:54.224 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.221+0000 7f042e853700 1 -- 172.21.15.190:0/39267761 shutdown_connections 2024-03-26T03:35:54.224 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.221+0000 7f042e853700 1 -- 172.21.15.190:0/39267761 wait complete. 2024-03-26T03:35:54.224 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.221+0000 7f042e853700 1 Processor -- start 2024-03-26T03:35:54.224 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.221+0000 7f042e853700 1 -- start start 2024-03-26T03:35:54.225 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.221+0000 7f042e853700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0428102280 0x7f0428197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:35:54.225 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.221+0000 7f042e853700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f042810bac0 con 0x7f0428102280 2024-03-26T03:35:54.226 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.221+0000 7f0427fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0428102280 0x7f0428197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:35:54.226 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.221+0000 7f0427fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0428102280 0x7f0428197130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.190:45824/0 (socket says 172.21.15.190:45824) 2024-03-26T03:35:54.226 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.221+0000 7f0427fff700 1 -- 172.21.15.190:0/2900666073 learned_addr learned my addr 172.21.15.190:0/2900666073 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-03-26T03:35:54.226 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.221+0000 7f0427fff700 1 -- 172.21.15.190:0/2900666073 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0428197670 con 0x7f0428102280 2024-03-26T03:35:54.226 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.221+0000 7f0427fff700 1 --2- 172.21.15.190:0/2900666073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0428102280 0x7f0428197130 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0418006010 tx=0x7f041800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:35:54.226 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.221+0000 7f04257fa700 1 -- 172.21.15.190:0/2900666073 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0418027070 con 0x7f0428102280 2024-03-26T03:35:54.226 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.221+0000 7f042e853700 1 -- 172.21.15.190:0/2900666073 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0428197960 con 0x7f0428102280 2024-03-26T03:35:54.226 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.221+0000 7f04257fa700 1 -- 172.21.15.190:0/2900666073 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f0418022470 con 0x7f0428102280 2024-03-26T03:35:54.226 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.221+0000 7f042e853700 1 -- 172.21.15.190:0/2900666073 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f042819ba00 con 0x7f0428102280 2024-03-26T03:35:54.227 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.221+0000 7f04257fa700 1 -- 172.21.15.190:0/2900666073 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0418017440 con 0x7f0428102280 2024-03-26T03:35:54.227 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.225+0000 7f042e853700 1 -- 172.21.15.190:0/2900666073 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0428103760 con 0x7f0428102280 2024-03-26T03:35:54.227 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.225+0000 7f04257fa700 1 -- 172.21.15.190:0/2900666073 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 13) ==== 46523+0+0 (secure 0 0 0) 0x7f041802a070 con 0x7f0428102280 2024-03-26T03:35:54.227 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.225+0000 7f04257fa700 1 --2- 172.21.15.190:0/2900666073 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f0410039470 0x7f041003b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:35:54.227 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.225+0000 7f04257fa700 1 -- 172.21.15.190:0/2900666073 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f041801e030 con 0x7f0428102280 2024-03-26T03:35:54.228 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.225+0000 7f04277fe700 1 --2- 172.21.15.190:0/2900666073 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f0410039470 0x7f041003b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:35:54.228 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.225+0000 7f04277fe700 1 --2- 172.21.15.190:0/2900666073 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f0410039470 0x7f041003b930 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f041c006fd0 tx=0x7f041c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:35:54.241 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.237+0000 7f04257fa700 1 -- 172.21.15.190:0/2900666073 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f0418013070 con 0x7f0428102280 2024-03-26T03:35:54.470 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.465+0000 7f042e853700 1 -- 172.21.15.190:0/2900666073 --> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi175:172.21.15.175=a;smithi175:[v2:172.21.15.175:3301,v1:172.21.15.175:6790]=c;smithi190:172.21.15.190=b", "target": ["mon-mgr", ""]}) -- 0x7f0428063c70 con 0x7f0410039470 2024-03-26T03:35:54.485 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.481+0000 7f04257fa700 1 -- 172.21.15.190:0/2900666073 <== mgr.14152 v2:172.21.15.175:6800/2826039805 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0428063c70 con 0x7f0410039470 2024-03-26T03:35:54.485 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled mon update... 2024-03-26T03:35:54.487 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.485+0000 7f042e853700 1 -- 172.21.15.190:0/2900666073 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f0410039470 msgr2=0x7f041003b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:35:54.487 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.485+0000 7f042e853700 1 --2- 172.21.15.190:0/2900666073 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f0410039470 0x7f041003b930 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f041c006fd0 tx=0x7f041c006e40 comp rx=0 tx=0).stop 2024-03-26T03:35:54.488 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.485+0000 7f042e853700 1 -- 172.21.15.190:0/2900666073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0428102280 msgr2=0x7f0428197130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:35:54.488 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.485+0000 7f042e853700 1 --2- 172.21.15.190:0/2900666073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0428102280 0x7f0428197130 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0418006010 tx=0x7f041800f740 comp rx=0 tx=0).stop 2024-03-26T03:35:54.488 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.485+0000 7f042e853700 1 -- 172.21.15.190:0/2900666073 shutdown_connections 2024-03-26T03:35:54.488 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.485+0000 7f042e853700 1 --2- 172.21.15.190:0/2900666073 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f0410039470 0x7f041003b930 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:35:54.488 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.485+0000 7f042e853700 1 --2- 172.21.15.190:0/2900666073 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0428102280 0x7f0428197130 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:35:54.488 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.485+0000 7f042e853700 1 -- 172.21.15.190:0/2900666073 >> 172.21.15.190:0/2900666073 conn(0x7f04280fdb30 msgr2=0x7f0428109ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:35:54.488 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.485+0000 7f042e853700 1 -- 172.21.15.190:0/2900666073 shutdown_connections 2024-03-26T03:35:54.488 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:54.485+0000 7f042e853700 1 -- 172.21.15.190:0/2900666073 wait complete. 2024-03-26T03:35:54.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:54 smithi175 bash[21527]: cluster 2024-03-26T03:35:52.182928+0000 mgr.y (mgr.14152) 42 : cluster [DBG] pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:35:55.255 DEBUG:teuthology.orchestra.run.smithi175:mon.c> sudo journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@mon.c.service 2024-03-26T03:35:55.258 DEBUG:teuthology.orchestra.run.smithi190:mon.b> sudo journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@mon.b.service 2024-03-26T03:35:55.260 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-26T03:35:55.261 DEBUG:teuthology.orchestra.run.smithi190:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-26T03:35:55.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:55 smithi175 bash[21527]: cluster 2024-03-26T03:35:54.183389+0000 mgr.y (mgr.14152) 43 : cluster [DBG] pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:35:55.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:55 smithi175 bash[21527]: audit 2024-03-26T03:35:54.472180+0000 mgr.y (mgr.14152) 44 : audit [DBG] from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi175:172.21.15.175=a;smithi175:[v2:172.21.15.175:3301,v1:172.21.15.175:6790]=c;smithi190:172.21.15.190=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:35:55.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:55 smithi175 bash[21527]: cephadm 2024-03-26T03:35:54.478506+0000 mgr.y (mgr.14152) 45 : cephadm [INF] Saving service mon spec with placement smithi175:172.21.15.175=a;smithi175:[v2:172.21.15.175:3301,v1:172.21.15.175:6790]=c;smithi190:172.21.15.190=b;count:3 2024-03-26T03:35:55.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:55 smithi175 bash[21527]: audit 2024-03-26T03:35:54.484432+0000 mon.a (mon.0) 131 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:35:55.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:55 smithi175 bash[21527]: audit 2024-03-26T03:35:54.528704+0000 mon.a (mon.0) 132 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:35:55.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:55 smithi175 bash[21527]: audit 2024-03-26T03:35:54.532230+0000 mon.a (mon.0) 133 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:35:55.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:55 smithi175 bash[21527]: audit 2024-03-26T03:35:54.534058+0000 mon.a (mon.0) 134 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:35:55.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:55 smithi175 bash[21527]: audit 2024-03-26T03:35:54.542521+0000 mon.a (mon.0) 135 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:35:55.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:55 smithi175 bash[21527]: audit 2024-03-26T03:35:54.546592+0000 mon.a (mon.0) 136 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T03:35:55.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:55 smithi175 bash[21527]: audit 2024-03-26T03:35:54.548283+0000 mon.a (mon.0) 137 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:35:55.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:55 smithi175 bash[21527]: cephadm 2024-03-26T03:35:54.550316+0000 mgr.y (mgr.14152) 46 : cephadm [INF] Deploying daemon mon.b on smithi190 2024-03-26T03:35:56.936 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T03:35:57.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:57 smithi175 bash[21527]: cluster 2024-03-26T03:35:56.183766+0000 mgr.y (mgr.14152) 47 : cluster [DBG] pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:35:59.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:35:59 smithi175 bash[21527]: cluster 2024-03-26T03:35:58.184160+0000 mgr.y (mgr.14152) 48 : cluster [DBG] pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:35:59.883 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:59.881+0000 7fc92b97b700 1 -- 172.21.15.190:0/3702115074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc924102280 msgr2=0x7fc924102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:35:59.883 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:59.881+0000 7fc92b97b700 1 --2- 172.21.15.190:0/3702115074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc924102280 0x7fc924102660 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc914009910 tx=0x7fc914009c20 comp rx=0 tx=0).stop 2024-03-26T03:35:59.883 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:59.881+0000 7fc92b97b700 1 -- 172.21.15.190:0/3702115074 shutdown_connections 2024-03-26T03:35:59.883 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:59.881+0000 7fc92b97b700 1 --2- 172.21.15.190:0/3702115074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc924102280 0x7fc924102660 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:35:59.883 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:59.881+0000 7fc92b97b700 1 -- 172.21.15.190:0/3702115074 >> 172.21.15.190:0/3702115074 conn(0x7fc9240fdb30 msgr2=0x7fc9240fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:35:59.883 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:59.881+0000 7fc92b97b700 1 -- 172.21.15.190:0/3702115074 shutdown_connections 2024-03-26T03:35:59.883 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:59.881+0000 7fc92b97b700 1 -- 172.21.15.190:0/3702115074 wait complete. 2024-03-26T03:35:59.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:59.881+0000 7fc92b97b700 1 Processor -- start 2024-03-26T03:35:59.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:59.881+0000 7fc92b97b700 1 -- start start 2024-03-26T03:35:59.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:59.881+0000 7fc92b97b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc924102280 0x7fc9241970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:35:59.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:59.881+0000 7fc92b97b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc92410bac0 con 0x7fc924102280 2024-03-26T03:35:59.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:59.881+0000 7fc929717700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc924102280 0x7fc9241970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:35:59.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:59.881+0000 7fc929717700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc924102280 0x7fc9241970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.190:53048/0 (socket says 172.21.15.190:53048) 2024-03-26T03:35:59.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:59.881+0000 7fc929717700 1 -- 172.21.15.190:0/4206446843 learned_addr learned my addr 172.21.15.190:0/4206446843 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-03-26T03:35:59.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:59.881+0000 7fc929717700 1 -- 172.21.15.190:0/4206446843 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc924197620 con 0x7fc924102280 2024-03-26T03:35:59.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:59.881+0000 7fc929717700 1 --2- 172.21.15.190:0/4206446843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc924102280 0x7fc9241970e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc91400f690 tx=0x7fc91400f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:35:59.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:59.881+0000 7fc9227fc700 1 -- 172.21.15.190:0/4206446843 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc914027070 con 0x7fc924102280 2024-03-26T03:35:59.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:59.881+0000 7fc9227fc700 1 -- 172.21.15.190:0/4206446843 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fc91400fb90 con 0x7fc924102280 2024-03-26T03:35:59.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:59.881+0000 7fc92b97b700 1 -- 172.21.15.190:0/4206446843 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc924197910 con 0x7fc924102280 2024-03-26T03:35:59.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:59.881+0000 7fc92b97b700 1 -- 172.21.15.190:0/4206446843 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc92419b9b0 con 0x7fc924102280 2024-03-26T03:35:59.886 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:59.881+0000 7fc9227fc700 1 -- 172.21.15.190:0/4206446843 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc914017440 con 0x7fc924102280 2024-03-26T03:35:59.886 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:59.881+0000 7fc92b97b700 1 -- 172.21.15.190:0/4206446843 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc924103760 con 0x7fc924102280 2024-03-26T03:35:59.886 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:59.885+0000 7fc9227fc700 1 -- 172.21.15.190:0/4206446843 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 13) ==== 46523+0+0 (secure 0 0 0) 0x7fc91402a070 con 0x7fc924102280 2024-03-26T03:35:59.887 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:59.885+0000 7fc9227fc700 1 --2- 172.21.15.190:0/4206446843 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fc90c039470 0x7fc90c03b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:35:59.887 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:59.885+0000 7fc9227fc700 1 -- 172.21.15.190:0/4206446843 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc91401e030 con 0x7fc924102280 2024-03-26T03:35:59.887 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:59.885+0000 7fc928f16700 1 --2- 172.21.15.190:0/4206446843 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fc90c039470 0x7fc90c03b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:35:59.887 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:59.885+0000 7fc928f16700 1 --2- 172.21.15.190:0/4206446843 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fc90c039470 0x7fc90c03b930 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc91800ad30 tx=0x7fc9180093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:35:59.893 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:35:59.889+0000 7fc9227fc700 1 -- 172.21.15.190:0/4206446843 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7fc914013070 con 0x7fc924102280 2024-03-26T03:36:00.177 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:00.173+0000 7fc92b97b700 1 -- 172.21.15.190:0/4206446843 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc924063c70 con 0x7fc924102280 2024-03-26T03:36:00.178 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:00.177+0000 7fc9227fc700 1 -- 172.21.15.190:0/4206446843 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7fc914031300 con 0x7fc924102280 2024-03-26T03:36:00.178 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-26T03:36:00.178 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":1,"fsid":"762e930c-eb21-11ee-95d0-87774f69a715","modified":"2024-03-26T03:33:41.181831Z","created":"2024-03-26T03:33:41.181831Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:3300","nonce":0},{"type":"v1","addr":"172.21.15.175:6789","nonce":0}]},"addr":"172.21.15.175:6789/0","public_addr":"172.21.15.175:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-26T03:36:00.180 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:00.177+0000 7fc92b97b700 1 -- 172.21.15.190:0/4206446843 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fc90c039470 msgr2=0x7fc90c03b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:36:00.180 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:00.177+0000 7fc92b97b700 1 --2- 172.21.15.190:0/4206446843 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fc90c039470 0x7fc90c03b930 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc91800ad30 tx=0x7fc9180093f0 comp rx=0 tx=0).stop 2024-03-26T03:36:00.180 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:00.177+0000 7fc92b97b700 1 -- 172.21.15.190:0/4206446843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc924102280 msgr2=0x7fc9241970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:36:00.181 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:00.177+0000 7fc92b97b700 1 --2- 172.21.15.190:0/4206446843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc924102280 0x7fc9241970e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc91400f690 tx=0x7fc91400f770 comp rx=0 tx=0).stop 2024-03-26T03:36:00.181 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:00.177+0000 7fc92b97b700 1 -- 172.21.15.190:0/4206446843 shutdown_connections 2024-03-26T03:36:00.181 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:00.177+0000 7fc92b97b700 1 --2- 172.21.15.190:0/4206446843 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fc90c039470 0x7fc90c03b930 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:36:00.181 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:00.177+0000 7fc92b97b700 1 --2- 172.21.15.190:0/4206446843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc924102280 0x7fc9241970e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:36:00.181 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:00.177+0000 7fc92b97b700 1 -- 172.21.15.190:0/4206446843 >> 172.21.15.190:0/4206446843 conn(0x7fc9240fdb30 msgr2=0x7fc924109f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:36:00.181 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:00.177+0000 7fc92b97b700 1 -- 172.21.15.190:0/4206446843 shutdown_connections 2024-03-26T03:36:00.181 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:00.177+0000 7fc92b97b700 1 -- 172.21.15.190:0/4206446843 wait complete. 2024-03-26T03:36:00.181 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 1 2024-03-26T03:36:01.155 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:00 smithi175 bash[21527]: audit 2024-03-26T03:35:59.786424+0000 mon.a (mon.0) 138 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:01.155 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:00 smithi175 bash[21527]: audit 2024-03-26T03:35:59.792772+0000 mon.a (mon.0) 139 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:01.155 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:00 smithi175 bash[21527]: audit 2024-03-26T03:35:59.833962+0000 mon.a (mon.0) 140 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:01.155 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:00 smithi175 bash[21527]: audit 2024-03-26T03:35:59.835541+0000 mon.a (mon.0) 141 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T03:36:01.155 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:00 smithi175 bash[21527]: audit 2024-03-26T03:35:59.837294+0000 mon.a (mon.0) 142 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:01.155 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:00 smithi175 bash[21527]: cephadm 2024-03-26T03:35:59.839307+0000 mgr.y (mgr.14152) 49 : cephadm [INF] Deploying daemon mon.c on smithi175 2024-03-26T03:36:01.155 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:00 smithi175 bash[21527]: audit 2024-03-26T03:36:00.178027+0000 mon.a (mon.0) 143 : audit [DBG] from='client.? 172.21.15.190:0/4206446843' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T03:36:01.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:01 smithi190 bash[23392]: debug 2024-03-26T03:36:01.297+0000 7f71bbe04700 1 mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-26T03:36:02.152 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-26T03:36:02.153 DEBUG:teuthology.orchestra.run.smithi190:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-26T03:36:03.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:03 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:36:03.745 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:36:03 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:36:04.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:03 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:36:04.245 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:36:03 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:36:04.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:04 smithi175 bash[29284]: debug 2024-03-26T03:36:04.795+0000 7fd72d4b7700 1 mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-26T03:36:05.851 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T03:36:06.690 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:06 smithi190 bash[23392]: audit 2024-03-26T03:36:01.305903+0000 mon.a (mon.0) 145 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T03:36:06.722 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:06 smithi190 bash[23392]: audit 2024-03-26T03:36:01.306067+0000 mon.a (mon.0) 146 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:36:06.722 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:06 smithi190 bash[23392]: cluster 2024-03-26T03:36:01.306185+0000 mon.a (mon.0) 147 : cluster [INF] mon.a calling monitor election 2024-03-26T03:36:06.722 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:06 smithi190 bash[23392]: cluster 2024-03-26T03:36:02.184948+0000 mgr.y (mgr.14152) 51 : cluster [DBG] pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:06.722 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:06 smithi190 bash[23392]: audit 2024-03-26T03:36:02.304033+0000 mon.a (mon.0) 148 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:36:06.723 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:06 smithi190 bash[23392]: audit 2024-03-26T03:36:03.304533+0000 mon.a (mon.0) 149 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:36:06.723 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:06 smithi190 bash[23392]: cluster 2024-03-26T03:36:03.306655+0000 mon.b (mon.1) 1 : cluster [INF] mon.b calling monitor election 2024-03-26T03:36:06.723 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:06 smithi190 bash[23392]: cluster 2024-03-26T03:36:04.185369+0000 mgr.y (mgr.14152) 52 : cluster [DBG] pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:06.723 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:06 smithi190 bash[23392]: audit 2024-03-26T03:36:04.304423+0000 mon.a (mon.0) 150 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:36:06.723 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:06 smithi190 bash[23392]: audit 2024-03-26T03:36:04.805166+0000 mon.a (mon.0) 151 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:36:06.723 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:06 smithi190 bash[23392]: audit 2024-03-26T03:36:05.304445+0000 mon.a (mon.0) 152 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:36:06.723 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:06 smithi190 bash[23392]: audit 2024-03-26T03:36:05.805945+0000 mon.a (mon.0) 153 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:36:06.723 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:06 smithi190 bash[23392]: audit 2024-03-26T03:36:06.304567+0000 mon.a (mon.0) 154 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:36:06.723 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:06 smithi190 bash[23392]: cluster 2024-03-26T03:36:06.344931+0000 mon.a (mon.0) 155 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-03-26T03:36:06.723 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:06 smithi190 bash[23392]: cluster 2024-03-26T03:36:06.355684+0000 mon.a (mon.0) 156 : cluster [DBG] monmap epoch 2 2024-03-26T03:36:06.723 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:06 smithi190 bash[23392]: cluster 2024-03-26T03:36:06.355723+0000 mon.a (mon.0) 157 : cluster [DBG] fsid 762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T03:36:06.723 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:06 smithi190 bash[23392]: cluster 2024-03-26T03:36:06.355747+0000 mon.a (mon.0) 158 : cluster [DBG] last_changed 2024-03-26T03:36:01.303163+0000 2024-03-26T03:36:06.723 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:06 smithi190 bash[23392]: cluster 2024-03-26T03:36:06.355775+0000 mon.a (mon.0) 159 : cluster [DBG] created 2024-03-26T03:33:41.181831+0000 2024-03-26T03:36:06.723 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:06 smithi190 bash[23392]: cluster 2024-03-26T03:36:06.355797+0000 mon.a (mon.0) 160 : cluster [DBG] min_mon_release 19 (squid) 2024-03-26T03:36:06.724 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:06 smithi190 bash[23392]: cluster 2024-03-26T03:36:06.355821+0000 mon.a (mon.0) 161 : cluster [DBG] election_strategy: 1 2024-03-26T03:36:06.724 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:06 smithi190 bash[23392]: cluster 2024-03-26T03:36:06.355851+0000 mon.a (mon.0) 162 : cluster [DBG] 0: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.a 2024-03-26T03:36:06.724 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:06 smithi190 bash[23392]: cluster 2024-03-26T03:36:06.355874+0000 mon.a (mon.0) 163 : cluster [DBG] 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.b 2024-03-26T03:36:06.724 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:06 smithi190 bash[23392]: cluster 2024-03-26T03:36:06.376348+0000 mon.a (mon.0) 164 : cluster [DBG] fsmap 2024-03-26T03:36:06.724 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:06 smithi190 bash[23392]: cluster 2024-03-26T03:36:06.376436+0000 mon.a (mon.0) 165 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-03-26T03:36:06.724 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:06 smithi190 bash[23392]: cluster 2024-03-26T03:36:06.376727+0000 mon.a (mon.0) 166 : cluster [DBG] mgrmap e13: y(active, since 74s) 2024-03-26T03:36:06.724 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:06 smithi190 bash[23392]: cluster 2024-03-26T03:36:06.376976+0000 mon.a (mon.0) 167 : cluster [INF] overall HEALTH_OK 2024-03-26T03:36:06.724 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:06 smithi190 bash[23392]: audit 2024-03-26T03:36:06.383809+0000 mon.a (mon.0) 168 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:06.724 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:06 smithi190 bash[23392]: audit 2024-03-26T03:36:06.392943+0000 mon.a (mon.0) 169 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:06.724 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:06 smithi190 bash[23392]: audit 2024-03-26T03:36:06.426304+0000 mon.a (mon.0) 170 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:06.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:06 smithi175 bash[21527]: audit 2024-03-26T03:36:01.305903+0000 mon.a (mon.0) 145 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T03:36:06.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:06 smithi175 bash[21527]: audit 2024-03-26T03:36:01.306067+0000 mon.a (mon.0) 146 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:36:06.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:06 smithi175 bash[21527]: cluster 2024-03-26T03:36:01.306185+0000 mon.a (mon.0) 147 : cluster [INF] mon.a calling monitor election 2024-03-26T03:36:06.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:06 smithi175 bash[21527]: cluster 2024-03-26T03:36:02.184948+0000 mgr.y (mgr.14152) 51 : cluster [DBG] pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:06.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:06 smithi175 bash[21527]: audit 2024-03-26T03:36:02.304033+0000 mon.a (mon.0) 148 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:36:06.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:06 smithi175 bash[21527]: audit 2024-03-26T03:36:03.304533+0000 mon.a (mon.0) 149 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:36:06.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:06 smithi175 bash[21527]: cluster 2024-03-26T03:36:03.306655+0000 mon.b (mon.1) 1 : cluster [INF] mon.b calling monitor election 2024-03-26T03:36:06.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:06 smithi175 bash[21527]: cluster 2024-03-26T03:36:04.185369+0000 mgr.y (mgr.14152) 52 : cluster [DBG] pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:06.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:06 smithi175 bash[21527]: audit 2024-03-26T03:36:04.304423+0000 mon.a (mon.0) 150 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:36:06.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:06 smithi175 bash[21527]: audit 2024-03-26T03:36:04.805166+0000 mon.a (mon.0) 151 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:36:06.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:06 smithi175 bash[21527]: audit 2024-03-26T03:36:05.304445+0000 mon.a (mon.0) 152 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:36:06.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:06 smithi175 bash[21527]: audit 2024-03-26T03:36:05.805945+0000 mon.a (mon.0) 153 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:36:06.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:06 smithi175 bash[21527]: audit 2024-03-26T03:36:06.304567+0000 mon.a (mon.0) 154 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:36:06.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:06 smithi175 bash[21527]: cluster 2024-03-26T03:36:06.344931+0000 mon.a (mon.0) 155 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-03-26T03:36:06.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:06 smithi175 bash[21527]: cluster 2024-03-26T03:36:06.355684+0000 mon.a (mon.0) 156 : cluster [DBG] monmap epoch 2 2024-03-26T03:36:06.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:06 smithi175 bash[21527]: cluster 2024-03-26T03:36:06.355723+0000 mon.a (mon.0) 157 : cluster [DBG] fsid 762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T03:36:06.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:06 smithi175 bash[21527]: cluster 2024-03-26T03:36:06.355747+0000 mon.a (mon.0) 158 : cluster [DBG] last_changed 2024-03-26T03:36:01.303163+0000 2024-03-26T03:36:06.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:06 smithi175 bash[21527]: cluster 2024-03-26T03:36:06.355775+0000 mon.a (mon.0) 159 : cluster [DBG] created 2024-03-26T03:33:41.181831+0000 2024-03-26T03:36:06.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:06 smithi175 bash[21527]: cluster 2024-03-26T03:36:06.355797+0000 mon.a (mon.0) 160 : cluster [DBG] min_mon_release 19 (squid) 2024-03-26T03:36:06.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:06 smithi175 bash[21527]: cluster 2024-03-26T03:36:06.355821+0000 mon.a (mon.0) 161 : cluster [DBG] election_strategy: 1 2024-03-26T03:36:06.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:06 smithi175 bash[21527]: cluster 2024-03-26T03:36:06.355851+0000 mon.a (mon.0) 162 : cluster [DBG] 0: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.a 2024-03-26T03:36:06.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:06 smithi175 bash[21527]: cluster 2024-03-26T03:36:06.355874+0000 mon.a (mon.0) 163 : cluster [DBG] 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.b 2024-03-26T03:36:06.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:06 smithi175 bash[21527]: cluster 2024-03-26T03:36:06.376348+0000 mon.a (mon.0) 164 : cluster [DBG] fsmap 2024-03-26T03:36:06.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:06 smithi175 bash[21527]: cluster 2024-03-26T03:36:06.376436+0000 mon.a (mon.0) 165 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-03-26T03:36:06.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:06 smithi175 bash[21527]: cluster 2024-03-26T03:36:06.376727+0000 mon.a (mon.0) 166 : cluster [DBG] mgrmap e13: y(active, since 74s) 2024-03-26T03:36:06.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:06 smithi175 bash[21527]: cluster 2024-03-26T03:36:06.376976+0000 mon.a (mon.0) 167 : cluster [INF] overall HEALTH_OK 2024-03-26T03:36:06.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:06 smithi175 bash[21527]: audit 2024-03-26T03:36:06.383809+0000 mon.a (mon.0) 168 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:06.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:06 smithi175 bash[21527]: audit 2024-03-26T03:36:06.392943+0000 mon.a (mon.0) 169 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:06.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:06 smithi175 bash[21527]: audit 2024-03-26T03:36:06.426304+0000 mon.a (mon.0) 170 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:07.744 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:36:07 smithi175 bash[21783]: debug 2024-03-26T03:36:07.299+0000 7f61b7560700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-26T03:36:11.989 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.984+0000 7f91a045d700 1 -- 172.21.15.190:0/1358597749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91840035c0 msgr2=0x7f9184005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:36:11.990 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.984+0000 7f91a045d700 1 --2- 172.21.15.190:0/1358597749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91840035c0 0x7f9184005a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9198104260 tx=0x7f9188014ec0 comp rx=0 tx=0).stop 2024-03-26T03:36:11.990 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.984+0000 7f91a045d700 1 -- 172.21.15.190:0/1358597749 shutdown_connections 2024-03-26T03:36:11.990 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.984+0000 7f91a045d700 1 --2- 172.21.15.190:0/1358597749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91840035c0 0x7f9184005a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:36:11.990 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.984+0000 7f91a045d700 1 -- 172.21.15.190:0/1358597749 >> 172.21.15.190:0/1358597749 conn(0x7f91980ffe40 msgr2=0x7f9198102260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:36:11.990 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.984+0000 7f91a045d700 1 -- 172.21.15.190:0/1358597749 shutdown_connections 2024-03-26T03:36:11.990 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.984+0000 7f91a045d700 1 -- 172.21.15.190:0/1358597749 wait complete. 2024-03-26T03:36:11.991 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.988+0000 7f91a045d700 1 Processor -- start 2024-03-26T03:36:11.991 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.988+0000 7f91a045d700 1 -- start start 2024-03-26T03:36:11.991 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.988+0000 7f91a045d700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91840035c0 0x7f91981995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:36:11.991 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.988+0000 7f91a045d700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9198199b20 0x7f919819de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:36:11.991 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.988+0000 7f91a045d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f919819e3a0 0x7f919819e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:36:11.992 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.988+0000 7f91a045d700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f919810b840 con 0x7f91840035c0 2024-03-26T03:36:11.992 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.988+0000 7f91a045d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f919810b6c0 con 0x7f919819e3a0 2024-03-26T03:36:11.992 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.988+0000 7f91a045d700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f919810b9c0 con 0x7f9198199b20 2024-03-26T03:36:11.993 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.988+0000 7f919e1f9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91840035c0 0x7f91981995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:36:11.993 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.988+0000 7f919e9fa700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f919819e3a0 0x7f919819e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:36:11.993 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.988+0000 7f919e9fa700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f919819e3a0 0x7f919819e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:34746/0 (socket says 172.21.15.190:34746) 2024-03-26T03:36:11.993 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.988+0000 7f919e1f9700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91840035c0 0x7f91981995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.190:59632/0 (socket says 172.21.15.190:59632) 2024-03-26T03:36:11.993 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.988+0000 7f919e9fa700 1 -- 172.21.15.190:0/3339193930 learned_addr learned my addr 172.21.15.190:0/3339193930 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-03-26T03:36:11.993 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.988+0000 7f919e9fa700 1 -- 172.21.15.190:0/3339193930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f919819e3a0 msgr2=0x7f919819e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-26T03:36:11.993 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.988+0000 7f919e9fa700 1 -- 172.21.15.190:0/3339193930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f919819e3a0 msgr2=0x7f919819e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-26T03:36:11.993 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.988+0000 7f919e9fa700 1 --2- 172.21.15.190:0/3339193930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f919819e3a0 0x7f919819e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-26T03:36:11.993 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.988+0000 7f919e9fa700 1 --2- 172.21.15.190:0/3339193930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f919819e3a0 0x7f919819e850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T03:36:11.993 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.988+0000 7f919d9f8700 1 --2- 172.21.15.190:0/3339193930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9198199b20 0x7f919819de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:36:11.993 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.988+0000 7f919d9f8700 1 -- 172.21.15.190:0/3339193930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9198199b20 msgr2=0x7f919819de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-03-26T03:36:11.993 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.988+0000 7f919d9f8700 1 -- 172.21.15.190:0/3339193930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9198199b20 msgr2=0x7f919819de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-26T03:36:11.993 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.988+0000 7f919d9f8700 1 --2- 172.21.15.190:0/3339193930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9198199b20 0x7f919819de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-26T03:36:11.994 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.988+0000 7f919d9f8700 1 --2- 172.21.15.190:0/3339193930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9198199b20 0x7f919819de60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T03:36:11.994 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.988+0000 7f919e1f9700 1 -- 172.21.15.190:0/3339193930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9198199b20 msgr2=0x7f919819de60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T03:36:11.994 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.988+0000 7f919e1f9700 1 --2- 172.21.15.190:0/3339193930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9198199b20 0x7f919819de60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:36:11.994 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.988+0000 7f919e1f9700 1 -- 172.21.15.190:0/3339193930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f919819e3a0 msgr2=0x7f919819e850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T03:36:11.994 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.988+0000 7f919e1f9700 1 --2- 172.21.15.190:0/3339193930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f919819e3a0 0x7f919819e850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:36:11.994 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.988+0000 7f919e1f9700 1 -- 172.21.15.190:0/3339193930 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91981a6110 con 0x7f91840035c0 2024-03-26T03:36:11.994 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.988+0000 7f919e1f9700 1 --2- 172.21.15.190:0/3339193930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91840035c0 0x7f91981995e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f918801d000 tx=0x7f918800ffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:36:11.995 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.988+0000 7f918f7fe700 1 -- 172.21.15.190:0/3339193930 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188004ff0 con 0x7f91840035c0 2024-03-26T03:36:11.995 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.992+0000 7f918f7fe700 1 -- 172.21.15.190:0/3339193930 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f9188005610 con 0x7f91840035c0 2024-03-26T03:36:11.995 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.992+0000 7f91a045d700 1 -- 172.21.15.190:0/3339193930 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91981a62e0 con 0x7f91840035c0 2024-03-26T03:36:11.996 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.992+0000 7f918f7fe700 1 -- 172.21.15.190:0/3339193930 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918801ebe0 con 0x7f91840035c0 2024-03-26T03:36:11.996 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.992+0000 7f91a045d700 1 -- 172.21.15.190:0/3339193930 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91981a6870 con 0x7f91840035c0 2024-03-26T03:36:11.996 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.992+0000 7f91a045d700 1 -- 172.21.15.190:0/3339193930 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9198063c70 con 0x7f91840035c0 2024-03-26T03:36:11.996 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.992+0000 7f918f7fe700 1 -- 172.21.15.190:0/3339193930 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 13) ==== 46523+0+0 (secure 0 0 0) 0x7f9188005190 con 0x7f91840035c0 2024-03-26T03:36:11.996 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.992+0000 7f918f7fe700 1 --2- 172.21.15.190:0/3339193930 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f917c039900 0x7f917c03bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:36:11.996 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.992+0000 7f918f7fe700 1 -- 172.21.15.190:0/3339193930 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f918804e5d0 con 0x7f91840035c0 2024-03-26T03:36:11.996 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.992+0000 7f919d9f8700 1 --2- 172.21.15.190:0/3339193930 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f917c039900 0x7f917c03bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:36:11.996 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:11.992+0000 7f919d9f8700 1 --2- 172.21.15.190:0/3339193930 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f917c039900 0x7f917c03bdc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9194009850 tx=0x7f9194008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:36:12.010 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:12.008+0000 7f918f7fe700 1 -- 172.21.15.190:0/3339193930 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f9188010630 con 0x7f91840035c0 2024-03-26T03:36:12.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: cluster 2024-03-26T03:36:06.185812+0000 mgr.y (mgr.14152) 53 : cluster [DBG] pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:12.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: cluster 2024-03-26T03:36:06.813816+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2024-03-26T03:36:12.246 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: audit 2024-03-26T03:36:06.814327+0000 mon.a (mon.0) 174 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T03:36:12.246 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: cluster 2024-03-26T03:36:06.814745+0000 mon.a (mon.0) 175 : cluster [INF] mon.a calling monitor election 2024-03-26T03:36:12.246 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: audit 2024-03-26T03:36:06.832777+0000 mon.a (mon.0) 176 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:36:12.246 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: audit 2024-03-26T03:36:06.833025+0000 mon.a (mon.0) 177 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:36:12.246 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: audit 2024-03-26T03:36:07.806552+0000 mon.a (mon.0) 178 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:36:12.246 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: cluster 2024-03-26T03:36:08.186221+0000 mgr.y (mgr.14152) 54 : cluster [DBG] pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:12.246 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: audit 2024-03-26T03:36:08.806616+0000 mon.a (mon.0) 179 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:36:12.246 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: audit 2024-03-26T03:36:09.806766+0000 mon.a (mon.0) 180 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:36:12.246 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: cluster 2024-03-26T03:36:10.186621+0000 mgr.y (mgr.14152) 55 : cluster [DBG] pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:12.246 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: audit 2024-03-26T03:36:10.806793+0000 mon.a (mon.0) 181 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:36:12.246 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: audit 2024-03-26T03:36:11.806547+0000 mon.a (mon.0) 182 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:36:12.246 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: cluster 2024-03-26T03:36:11.832719+0000 mon.a (mon.0) 183 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-03-26T03:36:12.246 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: cluster 2024-03-26T03:36:11.838590+0000 mon.a (mon.0) 184 : cluster [DBG] monmap epoch 3 2024-03-26T03:36:12.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: cluster 2024-03-26T03:36:11.838623+0000 mon.a (mon.0) 185 : cluster [DBG] fsid 762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T03:36:12.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: cluster 2024-03-26T03:36:11.838644+0000 mon.a (mon.0) 186 : cluster [DBG] last_changed 2024-03-26T03:36:06.806922+0000 2024-03-26T03:36:12.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: cluster 2024-03-26T03:36:11.838663+0000 mon.a (mon.0) 187 : cluster [DBG] created 2024-03-26T03:33:41.181831+0000 2024-03-26T03:36:12.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: cluster 2024-03-26T03:36:11.838684+0000 mon.a (mon.0) 188 : cluster [DBG] min_mon_release 19 (squid) 2024-03-26T03:36:12.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: cluster 2024-03-26T03:36:11.838712+0000 mon.a (mon.0) 189 : cluster [DBG] election_strategy: 1 2024-03-26T03:36:12.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: cluster 2024-03-26T03:36:11.838733+0000 mon.a (mon.0) 190 : cluster [DBG] 0: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.a 2024-03-26T03:36:12.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: cluster 2024-03-26T03:36:11.838756+0000 mon.a (mon.0) 191 : cluster [DBG] 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.b 2024-03-26T03:36:12.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: cluster 2024-03-26T03:36:11.838782+0000 mon.a (mon.0) 192 : cluster [DBG] 2: [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] mon.c 2024-03-26T03:36:12.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: cluster 2024-03-26T03:36:11.839814+0000 mon.a (mon.0) 193 : cluster [DBG] fsmap 2024-03-26T03:36:12.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: cluster 2024-03-26T03:36:11.839864+0000 mon.a (mon.0) 194 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-03-26T03:36:12.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: cluster 2024-03-26T03:36:11.840116+0000 mon.a (mon.0) 195 : cluster [DBG] mgrmap e13: y(active, since 79s) 2024-03-26T03:36:12.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: cluster 2024-03-26T03:36:11.840405+0000 mon.a (mon.0) 196 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-03-26T03:36:12.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: cluster 2024-03-26T03:36:11.860203+0000 mon.a (mon.0) 197 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-03-26T03:36:12.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: cluster 2024-03-26T03:36:11.860256+0000 mon.a (mon.0) 198 : cluster [WRN] [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-03-26T03:36:12.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: cluster 2024-03-26T03:36:11.860299+0000 mon.a (mon.0) 199 : cluster [WRN] mon.c (rank 2) addr [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] is down (out of quorum) 2024-03-26T03:36:12.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: audit 2024-03-26T03:36:11.865260+0000 mon.a (mon.0) 200 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:12.248 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: audit 2024-03-26T03:36:11.872298+0000 mon.a (mon.0) 201 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:12.248 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: audit 2024-03-26T03:36:11.877915+0000 mon.a (mon.0) 202 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:12.248 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: audit 2024-03-26T03:36:11.883943+0000 mon.a (mon.0) 203 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:12.248 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:11 smithi175 bash[21527]: audit 2024-03-26T03:36:11.890032+0000 mon.a (mon.0) 204 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:12.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: cluster 2024-03-26T03:36:06.185812+0000 mgr.y (mgr.14152) 53 : cluster [DBG] pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:12.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: cluster 2024-03-26T03:36:06.813816+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2024-03-26T03:36:12.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: audit 2024-03-26T03:36:06.814327+0000 mon.a (mon.0) 174 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T03:36:12.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: cluster 2024-03-26T03:36:06.814745+0000 mon.a (mon.0) 175 : cluster [INF] mon.a calling monitor election 2024-03-26T03:36:12.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: audit 2024-03-26T03:36:06.832777+0000 mon.a (mon.0) 176 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:36:12.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: audit 2024-03-26T03:36:06.833025+0000 mon.a (mon.0) 177 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:36:12.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: audit 2024-03-26T03:36:07.806552+0000 mon.a (mon.0) 178 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:36:12.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: cluster 2024-03-26T03:36:08.186221+0000 mgr.y (mgr.14152) 54 : cluster [DBG] pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:12.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: audit 2024-03-26T03:36:08.806616+0000 mon.a (mon.0) 179 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:36:12.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: audit 2024-03-26T03:36:09.806766+0000 mon.a (mon.0) 180 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:36:12.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: cluster 2024-03-26T03:36:10.186621+0000 mgr.y (mgr.14152) 55 : cluster [DBG] pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:12.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: audit 2024-03-26T03:36:10.806793+0000 mon.a (mon.0) 181 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:36:12.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: audit 2024-03-26T03:36:11.806547+0000 mon.a (mon.0) 182 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:36:12.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: cluster 2024-03-26T03:36:11.832719+0000 mon.a (mon.0) 183 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-03-26T03:36:12.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: cluster 2024-03-26T03:36:11.838590+0000 mon.a (mon.0) 184 : cluster [DBG] monmap epoch 3 2024-03-26T03:36:12.251 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: cluster 2024-03-26T03:36:11.838623+0000 mon.a (mon.0) 185 : cluster [DBG] fsid 762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T03:36:12.251 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: cluster 2024-03-26T03:36:11.838644+0000 mon.a (mon.0) 186 : cluster [DBG] last_changed 2024-03-26T03:36:06.806922+0000 2024-03-26T03:36:12.251 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: cluster 2024-03-26T03:36:11.838663+0000 mon.a (mon.0) 187 : cluster [DBG] created 2024-03-26T03:33:41.181831+0000 2024-03-26T03:36:12.251 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: cluster 2024-03-26T03:36:11.838684+0000 mon.a (mon.0) 188 : cluster [DBG] min_mon_release 19 (squid) 2024-03-26T03:36:12.251 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: cluster 2024-03-26T03:36:11.838712+0000 mon.a (mon.0) 189 : cluster [DBG] election_strategy: 1 2024-03-26T03:36:12.251 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: cluster 2024-03-26T03:36:11.838733+0000 mon.a (mon.0) 190 : cluster [DBG] 0: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.a 2024-03-26T03:36:12.251 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: cluster 2024-03-26T03:36:11.838756+0000 mon.a (mon.0) 191 : cluster [DBG] 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.b 2024-03-26T03:36:12.251 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: cluster 2024-03-26T03:36:11.838782+0000 mon.a (mon.0) 192 : cluster [DBG] 2: [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] mon.c 2024-03-26T03:36:12.251 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: cluster 2024-03-26T03:36:11.839814+0000 mon.a (mon.0) 193 : cluster [DBG] fsmap 2024-03-26T03:36:12.251 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: cluster 2024-03-26T03:36:11.839864+0000 mon.a (mon.0) 194 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-03-26T03:36:12.251 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: cluster 2024-03-26T03:36:11.840116+0000 mon.a (mon.0) 195 : cluster [DBG] mgrmap e13: y(active, since 79s) 2024-03-26T03:36:12.252 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: cluster 2024-03-26T03:36:11.840405+0000 mon.a (mon.0) 196 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-03-26T03:36:12.252 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: cluster 2024-03-26T03:36:11.860203+0000 mon.a (mon.0) 197 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-03-26T03:36:12.252 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: cluster 2024-03-26T03:36:11.860256+0000 mon.a (mon.0) 198 : cluster [WRN] [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-03-26T03:36:12.252 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: cluster 2024-03-26T03:36:11.860299+0000 mon.a (mon.0) 199 : cluster [WRN] mon.c (rank 2) addr [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] is down (out of quorum) 2024-03-26T03:36:12.252 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: audit 2024-03-26T03:36:11.865260+0000 mon.a (mon.0) 200 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:12.252 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: audit 2024-03-26T03:36:11.872298+0000 mon.a (mon.0) 201 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:12.252 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: audit 2024-03-26T03:36:11.877915+0000 mon.a (mon.0) 202 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:12.252 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: audit 2024-03-26T03:36:11.883943+0000 mon.a (mon.0) 203 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:12.252 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:11 smithi190 bash[23392]: audit 2024-03-26T03:36:11.890032+0000 mon.a (mon.0) 204 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:12.315 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:12.312+0000 7f91a045d700 1 -- 172.21.15.190:0/3339193930 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f91981a6ad0 con 0x7f91840035c0 2024-03-26T03:36:12.316 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:12.312+0000 7f918f7fe700 1 -- 172.21.15.190:0/3339193930 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7f91880103d0 con 0x7f91840035c0 2024-03-26T03:36:12.316 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-03-26T03:36:12.316 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":3,"fsid":"762e930c-eb21-11ee-95d0-87774f69a715","modified":"2024-03-26T03:36:06.806922Z","created":"2024-03-26T03:33:41.181831Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:3300","nonce":0},{"type":"v1","addr":"172.21.15.175:6789","nonce":0}]},"addr":"172.21.15.175:6789/0","public_addr":"172.21.15.175:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:3301","nonce":0},{"type":"v1","addr":"172.21.15.175:6790","nonce":0}]},"addr":"172.21.15.175:6790/0","public_addr":"172.21.15.175:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-26T03:36:12.318 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:12.316+0000 7f91a045d700 1 -- 172.21.15.190:0/3339193930 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f917c039900 msgr2=0x7f917c03bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:36:12.319 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:12.316+0000 7f91a045d700 1 --2- 172.21.15.190:0/3339193930 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f917c039900 0x7f917c03bdc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9194009850 tx=0x7f9194008040 comp rx=0 tx=0).stop 2024-03-26T03:36:12.319 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:12.316+0000 7f91a045d700 1 -- 172.21.15.190:0/3339193930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91840035c0 msgr2=0x7f91981995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:36:12.319 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:12.316+0000 7f91a045d700 1 --2- 172.21.15.190:0/3339193930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91840035c0 0x7f91981995e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f918801d000 tx=0x7f918800ffd0 comp rx=0 tx=0).stop 2024-03-26T03:36:12.319 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:12.316+0000 7f91a045d700 1 -- 172.21.15.190:0/3339193930 shutdown_connections 2024-03-26T03:36:12.319 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:12.316+0000 7f91a045d700 1 --2- 172.21.15.190:0/3339193930 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f917c039900 0x7f917c03bdc0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:36:12.319 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:12.316+0000 7f91a045d700 1 --2- 172.21.15.190:0/3339193930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f919819e3a0 0x7f919819e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:36:12.319 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:12.316+0000 7f91a045d700 1 --2- 172.21.15.190:0/3339193930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91840035c0 0x7f91981995e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:36:12.319 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:12.316+0000 7f91a045d700 1 --2- 172.21.15.190:0/3339193930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9198199b20 0x7f919819de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:36:12.319 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:12.316+0000 7f91a045d700 1 -- 172.21.15.190:0/3339193930 >> 172.21.15.190:0/3339193930 conn(0x7f91980ffe40 msgr2=0x7f9198107f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:36:12.319 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:12.316+0000 7f91a045d700 1 -- 172.21.15.190:0/3339193930 shutdown_connections 2024-03-26T03:36:12.319 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:12.316+0000 7f91a045d700 1 -- 172.21.15.190:0/3339193930 wait complete. 2024-03-26T03:36:12.320 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 3 2024-03-26T03:36:13.001 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-26T03:36:13.001 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph config generate-minimal-conf 2024-03-26T03:36:13.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:12 smithi175 bash[21527]: audit 2024-03-26T03:36:12.316018+0000 mon.a (mon.0) 205 : audit [DBG] from='client.? 172.21.15.190:0/3339193930' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T03:36:13.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:12 smithi175 bash[21527]: audit 2024-03-26T03:36:12.529552+0000 mon.a (mon.0) 206 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:13.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:12 smithi175 bash[21527]: audit 2024-03-26T03:36:12.530946+0000 mon.a (mon.0) 207 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:36:13.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:12 smithi175 bash[21527]: audit 2024-03-26T03:36:12.806934+0000 mon.a (mon.0) 208 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:36:13.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:12 smithi175 bash[21527]: audit 2024-03-26T03:36:12.869287+0000 mon.a (mon.0) 209 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:13.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:12 smithi175 bash[21527]: audit 2024-03-26T03:36:12.874321+0000 mon.a (mon.0) 210 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:13.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:12 smithi175 bash[21527]: audit 2024-03-26T03:36:12.878867+0000 mon.a (mon.0) 211 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:13.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:12 smithi175 bash[21527]: audit 2024-03-26T03:36:12.882959+0000 mon.a (mon.0) 212 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:13.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:12 smithi175 bash[21527]: audit 2024-03-26T03:36:12.888010+0000 mon.a (mon.0) 213 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:13.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:12 smithi190 bash[23392]: audit 2024-03-26T03:36:12.316018+0000 mon.a (mon.0) 205 : audit [DBG] from='client.? 172.21.15.190:0/3339193930' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T03:36:13.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:12 smithi190 bash[23392]: audit 2024-03-26T03:36:12.529552+0000 mon.a (mon.0) 206 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:13.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:12 smithi190 bash[23392]: audit 2024-03-26T03:36:12.530946+0000 mon.a (mon.0) 207 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:36:13.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:12 smithi190 bash[23392]: audit 2024-03-26T03:36:12.806934+0000 mon.a (mon.0) 208 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:36:13.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:12 smithi190 bash[23392]: audit 2024-03-26T03:36:12.869287+0000 mon.a (mon.0) 209 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:13.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:12 smithi190 bash[23392]: audit 2024-03-26T03:36:12.874321+0000 mon.a (mon.0) 210 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:13.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:12 smithi190 bash[23392]: audit 2024-03-26T03:36:12.878867+0000 mon.a (mon.0) 211 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:13.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:12 smithi190 bash[23392]: audit 2024-03-26T03:36:12.882959+0000 mon.a (mon.0) 212 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:13.251 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:12 smithi190 bash[23392]: audit 2024-03-26T03:36:12.888010+0000 mon.a (mon.0) 213 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:13.962 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:13 smithi190 bash[23392]: cluster 2024-03-26T03:36:08.808757+0000 mon.c (mon.2) 1 : cluster [INF] mon.c calling monitor election 2024-03-26T03:36:13.962 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:13 smithi190 bash[23392]: cluster 2024-03-26T03:36:13.842074+0000 mon.c (mon.2) 2 : cluster [INF] mon.c calling monitor election 2024-03-26T03:36:13.962 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:13 smithi190 bash[23392]: cluster 2024-03-26T03:36:13.842995+0000 mon.a (mon.0) 222 : cluster [INF] mon.a calling monitor election 2024-03-26T03:36:13.962 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:13 smithi190 bash[23392]: cluster 2024-03-26T03:36:13.843566+0000 mon.b (mon.1) 3 : cluster [INF] mon.b calling monitor election 2024-03-26T03:36:13.962 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:13 smithi190 bash[23392]: cluster 2024-03-26T03:36:13.865064+0000 mon.a (mon.0) 223 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-03-26T03:36:13.963 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:13 smithi190 bash[23392]: cluster 2024-03-26T03:36:13.872302+0000 mon.a (mon.0) 224 : cluster [DBG] monmap epoch 3 2024-03-26T03:36:13.963 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:13 smithi190 bash[23392]: cluster 2024-03-26T03:36:13.872344+0000 mon.a (mon.0) 225 : cluster [DBG] fsid 762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T03:36:13.963 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:13 smithi190 bash[23392]: cluster 2024-03-26T03:36:13.872367+0000 mon.a (mon.0) 226 : cluster [DBG] last_changed 2024-03-26T03:36:06.806922+0000 2024-03-26T03:36:13.963 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:13 smithi190 bash[23392]: cluster 2024-03-26T03:36:13.872383+0000 mon.a (mon.0) 227 : cluster [DBG] created 2024-03-26T03:33:41.181831+0000 2024-03-26T03:36:13.963 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:13 smithi190 bash[23392]: cluster 2024-03-26T03:36:13.872404+0000 mon.a (mon.0) 228 : cluster [DBG] min_mon_release 19 (squid) 2024-03-26T03:36:13.963 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:13 smithi190 bash[23392]: cluster 2024-03-26T03:36:13.872423+0000 mon.a (mon.0) 229 : cluster [DBG] election_strategy: 1 2024-03-26T03:36:13.963 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:13 smithi190 bash[23392]: cluster 2024-03-26T03:36:13.872448+0000 mon.a (mon.0) 230 : cluster [DBG] 0: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.a 2024-03-26T03:36:13.963 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:13 smithi190 bash[23392]: cluster 2024-03-26T03:36:13.872472+0000 mon.a (mon.0) 231 : cluster [DBG] 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.b 2024-03-26T03:36:13.963 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:13 smithi190 bash[23392]: cluster 2024-03-26T03:36:13.872492+0000 mon.a (mon.0) 232 : cluster [DBG] 2: [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] mon.c 2024-03-26T03:36:13.963 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:13 smithi190 bash[23392]: cluster 2024-03-26T03:36:13.873500+0000 mon.a (mon.0) 233 : cluster [DBG] fsmap 2024-03-26T03:36:13.963 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:13 smithi190 bash[23392]: cluster 2024-03-26T03:36:13.873561+0000 mon.a (mon.0) 234 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-03-26T03:36:13.963 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:13 smithi190 bash[23392]: cluster 2024-03-26T03:36:13.873846+0000 mon.a (mon.0) 235 : cluster [DBG] mgrmap e13: y(active, since 81s) 2024-03-26T03:36:13.963 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:13 smithi190 bash[23392]: cluster 2024-03-26T03:36:13.874126+0000 mon.a (mon.0) 236 : cluster [INF] Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-03-26T03:36:13.963 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:13 smithi190 bash[23392]: cluster 2024-03-26T03:36:13.874160+0000 mon.a (mon.0) 237 : cluster [INF] Cluster is now healthy 2024-03-26T03:36:14.237 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[21527]: cluster 2024-03-26T03:36:08.808757+0000 mon.c (mon.2) 1 : cluster [INF] mon.c calling monitor election 2024-03-26T03:36:14.237 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[21527]: cluster 2024-03-26T03:36:13.842074+0000 mon.c (mon.2) 2 : cluster [INF] mon.c calling monitor election 2024-03-26T03:36:14.237 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[21527]: cluster 2024-03-26T03:36:13.842995+0000 mon.a (mon.0) 222 : cluster [INF] mon.a calling monitor election 2024-03-26T03:36:14.237 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[21527]: cluster 2024-03-26T03:36:13.843566+0000 mon.b (mon.1) 3 : cluster [INF] mon.b calling monitor election 2024-03-26T03:36:14.237 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[21527]: cluster 2024-03-26T03:36:13.865064+0000 mon.a (mon.0) 223 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-03-26T03:36:14.238 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[21527]: cluster 2024-03-26T03:36:13.872302+0000 mon.a (mon.0) 224 : cluster [DBG] monmap epoch 3 2024-03-26T03:36:14.238 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[21527]: cluster 2024-03-26T03:36:13.872344+0000 mon.a (mon.0) 225 : cluster [DBG] fsid 762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T03:36:14.238 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[21527]: cluster 2024-03-26T03:36:13.872367+0000 mon.a (mon.0) 226 : cluster [DBG] last_changed 2024-03-26T03:36:06.806922+0000 2024-03-26T03:36:14.238 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[21527]: cluster 2024-03-26T03:36:13.872383+0000 mon.a (mon.0) 227 : cluster [DBG] created 2024-03-26T03:33:41.181831+0000 2024-03-26T03:36:14.238 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[21527]: cluster 2024-03-26T03:36:13.872404+0000 mon.a (mon.0) 228 : cluster [DBG] min_mon_release 19 (squid) 2024-03-26T03:36:14.238 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[21527]: cluster 2024-03-26T03:36:13.872423+0000 mon.a (mon.0) 229 : cluster [DBG] election_strategy: 1 2024-03-26T03:36:14.238 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[21527]: cluster 2024-03-26T03:36:13.872448+0000 mon.a (mon.0) 230 : cluster [DBG] 0: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.a 2024-03-26T03:36:14.238 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[21527]: cluster 2024-03-26T03:36:13.872472+0000 mon.a (mon.0) 231 : cluster [DBG] 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.b 2024-03-26T03:36:14.238 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[21527]: cluster 2024-03-26T03:36:13.872492+0000 mon.a (mon.0) 232 : cluster [DBG] 2: [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] mon.c 2024-03-26T03:36:14.238 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[21527]: cluster 2024-03-26T03:36:13.873500+0000 mon.a (mon.0) 233 : cluster [DBG] fsmap 2024-03-26T03:36:14.238 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[21527]: cluster 2024-03-26T03:36:13.873561+0000 mon.a (mon.0) 234 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-03-26T03:36:14.238 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[21527]: cluster 2024-03-26T03:36:13.873846+0000 mon.a (mon.0) 235 : cluster [DBG] mgrmap e13: y(active, since 81s) 2024-03-26T03:36:14.238 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[21527]: cluster 2024-03-26T03:36:13.874126+0000 mon.a (mon.0) 236 : cluster [INF] Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-03-26T03:36:14.239 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[21527]: cluster 2024-03-26T03:36:13.874160+0000 mon.a (mon.0) 237 : cluster [INF] Cluster is now healthy 2024-03-26T03:36:14.239 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[21527]: cluster 2024-03-26T03:36:13.880009+0000 mon.a (mon.0) 238 : cluster [INF] overall HEALTH_OK 2024-03-26T03:36:14.239 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:01.305903+0000 mon.a (mon.0) 145 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T03:36:14.239 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:01.306067+0000 mon.a (mon.0) 146 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:36:14.239 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:01.306185+0000 mon.a (mon.0) 147 : cluster [INF] mon.a calling monitor election 2024-03-26T03:36:14.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:02.184948+0000 mgr.y (mgr.14152) 51 : cluster [DBG] pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:14.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:02.304033+0000 mon.a (mon.0) 148 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:36:14.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:03.304533+0000 mon.a (mon.0) 149 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:36:14.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:03.306655+0000 mon.b (mon.1) 1 : cluster [INF] mon.b calling monitor election 2024-03-26T03:36:14.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:04.185369+0000 mgr.y (mgr.14152) 52 : cluster [DBG] pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:14.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:04.304423+0000 mon.a (mon.0) 150 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:36:14.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:04.805166+0000 mon.a (mon.0) 151 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:36:14.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:05.304445+0000 mon.a (mon.0) 152 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:36:14.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:05.805945+0000 mon.a (mon.0) 153 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:36:14.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:06.304567+0000 mon.a (mon.0) 154 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:36:14.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:06.344931+0000 mon.a (mon.0) 155 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-03-26T03:36:14.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:06.355684+0000 mon.a (mon.0) 156 : cluster [DBG] monmap epoch 2 2024-03-26T03:36:14.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:06.355723+0000 mon.a (mon.0) 157 : cluster [DBG] fsid 762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T03:36:14.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:06.355747+0000 mon.a (mon.0) 158 : cluster [DBG] last_changed 2024-03-26T03:36:01.303163+0000 2024-03-26T03:36:14.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:06.355775+0000 mon.a (mon.0) 159 : cluster [DBG] created 2024-03-26T03:33:41.181831+0000 2024-03-26T03:36:14.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:06.355797+0000 mon.a (mon.0) 160 : cluster [DBG] min_mon_release 19 (squid) 2024-03-26T03:36:14.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:06.355821+0000 mon.a (mon.0) 161 : cluster [DBG] election_strategy: 1 2024-03-26T03:36:14.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:06.355851+0000 mon.a (mon.0) 162 : cluster [DBG] 0: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.a 2024-03-26T03:36:14.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:06.355874+0000 mon.a (mon.0) 163 : cluster [DBG] 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.b 2024-03-26T03:36:14.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:06.376348+0000 mon.a (mon.0) 164 : cluster [DBG] fsmap 2024-03-26T03:36:14.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:06.376436+0000 mon.a (mon.0) 165 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-03-26T03:36:14.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:06.376727+0000 mon.a (mon.0) 166 : cluster [DBG] mgrmap e13: y(active, since 74s) 2024-03-26T03:36:14.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:06.376976+0000 mon.a (mon.0) 167 : cluster [INF] overall HEALTH_OK 2024-03-26T03:36:14.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:06.383809+0000 mon.a (mon.0) 168 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:14.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:06.392943+0000 mon.a (mon.0) 169 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:14.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:06.426304+0000 mon.a (mon.0) 170 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:14.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:06.185812+0000 mgr.y (mgr.14152) 53 : cluster [DBG] pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:14.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:06.813816+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2024-03-26T03:36:14.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:06.814327+0000 mon.a (mon.0) 174 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T03:36:14.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:06.814745+0000 mon.a (mon.0) 175 : cluster [INF] mon.a calling monitor election 2024-03-26T03:36:14.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:06.832777+0000 mon.a (mon.0) 176 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:36:14.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:06.833025+0000 mon.a (mon.0) 177 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:36:14.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:07.806552+0000 mon.a (mon.0) 178 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:36:14.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:08.186221+0000 mgr.y (mgr.14152) 54 : cluster [DBG] pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:14.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:08.806616+0000 mon.a (mon.0) 179 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:36:14.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:09.806766+0000 mon.a (mon.0) 180 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:36:14.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:10.186621+0000 mgr.y (mgr.14152) 55 : cluster [DBG] pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:14.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:10.806793+0000 mon.a (mon.0) 181 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:36:14.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:11.806547+0000 mon.a (mon.0) 182 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:36:14.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:11.832719+0000 mon.a (mon.0) 183 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-03-26T03:36:14.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:11.838590+0000 mon.a (mon.0) 184 : cluster [DBG] monmap epoch 3 2024-03-26T03:36:14.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:11.838623+0000 mon.a (mon.0) 185 : cluster [DBG] fsid 762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T03:36:14.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:11.838644+0000 mon.a (mon.0) 186 : cluster [DBG] last_changed 2024-03-26T03:36:06.806922+0000 2024-03-26T03:36:14.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:11.838663+0000 mon.a (mon.0) 187 : cluster [DBG] created 2024-03-26T03:33:41.181831+0000 2024-03-26T03:36:14.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:11.838684+0000 mon.a (mon.0) 188 : cluster [DBG] min_mon_release 19 (squid) 2024-03-26T03:36:14.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:11.838712+0000 mon.a (mon.0) 189 : cluster [DBG] election_strategy: 1 2024-03-26T03:36:14.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:11.838733+0000 mon.a (mon.0) 190 : cluster [DBG] 0: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.a 2024-03-26T03:36:14.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:11.838756+0000 mon.a (mon.0) 191 : cluster [DBG] 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.b 2024-03-26T03:36:14.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:11.838782+0000 mon.a (mon.0) 192 : cluster [DBG] 2: [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] mon.c 2024-03-26T03:36:14.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:11.839814+0000 mon.a (mon.0) 193 : cluster [DBG] fsmap 2024-03-26T03:36:14.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:11.839864+0000 mon.a (mon.0) 194 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-03-26T03:36:14.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:11.840116+0000 mon.a (mon.0) 195 : cluster [DBG] mgrmap e13: y(active, since 79s) 2024-03-26T03:36:14.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:11.840405+0000 mon.a (mon.0) 196 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-03-26T03:36:14.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:11.860203+0000 mon.a (mon.0) 197 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-03-26T03:36:14.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:11.860256+0000 mon.a (mon.0) 198 : cluster [WRN] [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-03-26T03:36:14.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:11.860299+0000 mon.a (mon.0) 199 : cluster [WRN] mon.c (rank 2) addr [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] is down (out of quorum) 2024-03-26T03:36:14.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:11.865260+0000 mon.a (mon.0) 200 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:14.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:11.872298+0000 mon.a (mon.0) 201 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:14.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:11.877915+0000 mon.a (mon.0) 202 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:14.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:11.883943+0000 mon.a (mon.0) 203 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:14.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:11.890032+0000 mon.a (mon.0) 204 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:14.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:12.316018+0000 mon.a (mon.0) 205 : audit [DBG] from='client.? 172.21.15.190:0/3339193930' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T03:36:14.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:12.529552+0000 mon.a (mon.0) 206 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:14.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:12.530946+0000 mon.a (mon.0) 207 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:36:14.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:12.806934+0000 mon.a (mon.0) 208 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:36:14.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:12.869287+0000 mon.a (mon.0) 209 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:14.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:12.874321+0000 mon.a (mon.0) 210 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:14.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:12.878867+0000 mon.a (mon.0) 211 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:14.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:12.882959+0000 mon.a (mon.0) 212 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:14.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: audit 2024-03-26T03:36:12.888010+0000 mon.a (mon.0) 213 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:14.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:08.808757+0000 mon.c (mon.2) 1 : cluster [INF] mon.c calling monitor election 2024-03-26T03:36:14.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:13.842074+0000 mon.c (mon.2) 2 : cluster [INF] mon.c calling monitor election 2024-03-26T03:36:14.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:13.842995+0000 mon.a (mon.0) 222 : cluster [INF] mon.a calling monitor election 2024-03-26T03:36:14.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:13.843566+0000 mon.b (mon.1) 3 : cluster [INF] mon.b calling monitor election 2024-03-26T03:36:14.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:13.865064+0000 mon.a (mon.0) 223 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-03-26T03:36:14.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:13.872302+0000 mon.a (mon.0) 224 : cluster [DBG] monmap epoch 3 2024-03-26T03:36:14.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:13.872344+0000 mon.a (mon.0) 225 : cluster [DBG] fsid 762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T03:36:14.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:13.872367+0000 mon.a (mon.0) 226 : cluster [DBG] last_changed 2024-03-26T03:36:06.806922+0000 2024-03-26T03:36:14.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:13.872383+0000 mon.a (mon.0) 227 : cluster [DBG] created 2024-03-26T03:33:41.181831+0000 2024-03-26T03:36:14.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:13.872404+0000 mon.a (mon.0) 228 : cluster [DBG] min_mon_release 19 (squid) 2024-03-26T03:36:14.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:13.872423+0000 mon.a (mon.0) 229 : cluster [DBG] election_strategy: 1 2024-03-26T03:36:14.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:13.872448+0000 mon.a (mon.0) 230 : cluster [DBG] 0: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.a 2024-03-26T03:36:14.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:13.872472+0000 mon.a (mon.0) 231 : cluster [DBG] 1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.b 2024-03-26T03:36:14.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:13.872492+0000 mon.a (mon.0) 232 : cluster [DBG] 2: [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] mon.c 2024-03-26T03:36:14.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:13.873500+0000 mon.a (mon.0) 233 : cluster [DBG] fsmap 2024-03-26T03:36:14.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:13.873561+0000 mon.a (mon.0) 234 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-03-26T03:36:14.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:13.873846+0000 mon.a (mon.0) 235 : cluster [DBG] mgrmap e13: y(active, since 81s) 2024-03-26T03:36:14.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:13.874126+0000 mon.a (mon.0) 236 : cluster [INF] Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-03-26T03:36:14.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:13.874160+0000 mon.a (mon.0) 237 : cluster [INF] Cluster is now healthy 2024-03-26T03:36:14.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:13 smithi175 bash[29284]: cluster 2024-03-26T03:36:13.880009+0000 mon.a (mon.0) 238 : cluster [INF] overall HEALTH_OK 2024-03-26T03:36:16.052 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:15 smithi175 bash[29284]: cluster 2024-03-26T03:36:14.187535+0000 mgr.y (mgr.14152) 63 : cluster [DBG] pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:16.052 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:15 smithi175 bash[29284]: audit 2024-03-26T03:36:14.807491+0000 mon.a (mon.0) 239 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:36:16.052 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:15 smithi175 bash[29284]: audit 2024-03-26T03:36:14.956346+0000 mon.a (mon.0) 240 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:16.052 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:15 smithi175 bash[29284]: audit 2024-03-26T03:36:14.971743+0000 mon.a (mon.0) 241 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:16.052 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:15 smithi175 bash[29284]: cephadm 2024-03-26T03:36:14.977800+0000 mgr.y (mgr.14152) 64 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2024-03-26T03:36:16.052 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:15 smithi175 bash[29284]: audit 2024-03-26T03:36:14.978405+0000 mon.a (mon.0) 242 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T03:36:16.052 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:15 smithi175 bash[29284]: audit 2024-03-26T03:36:14.979864+0000 mon.a (mon.0) 243 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T03:36:16.052 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:15 smithi175 bash[29284]: audit 2024-03-26T03:36:14.981184+0000 mon.a (mon.0) 244 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:16.053 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:15 smithi175 bash[29284]: cephadm 2024-03-26T03:36:14.982542+0000 mgr.y (mgr.14152) 65 : cephadm [INF] Reconfiguring daemon mon.c on smithi175 2024-03-26T03:36:16.053 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:15 smithi175 bash[21527]: cluster 2024-03-26T03:36:14.187535+0000 mgr.y (mgr.14152) 63 : cluster [DBG] pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:16.053 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:15 smithi175 bash[21527]: audit 2024-03-26T03:36:14.807491+0000 mon.a (mon.0) 239 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:36:16.053 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:15 smithi175 bash[21527]: audit 2024-03-26T03:36:14.956346+0000 mon.a (mon.0) 240 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:16.053 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:15 smithi175 bash[21527]: audit 2024-03-26T03:36:14.971743+0000 mon.a (mon.0) 241 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:16.053 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:15 smithi175 bash[21527]: cephadm 2024-03-26T03:36:14.977800+0000 mgr.y (mgr.14152) 64 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2024-03-26T03:36:16.053 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:15 smithi175 bash[21527]: audit 2024-03-26T03:36:14.978405+0000 mon.a (mon.0) 242 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T03:36:16.053 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:15 smithi175 bash[21527]: audit 2024-03-26T03:36:14.979864+0000 mon.a (mon.0) 243 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T03:36:16.053 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:15 smithi175 bash[21527]: audit 2024-03-26T03:36:14.981184+0000 mon.a (mon.0) 244 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:16.053 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:15 smithi175 bash[21527]: cephadm 2024-03-26T03:36:14.982542+0000 mgr.y (mgr.14152) 65 : cephadm [INF] Reconfiguring daemon mon.c on smithi175 2024-03-26T03:36:16.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:13 smithi190 bash[23392]: cluster 2024-03-26T03:36:13.880009+0000 mon.a (mon.0) 238 : cluster [INF] overall HEALTH_OK 2024-03-26T03:36:16.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:15 smithi190 bash[23392]: cluster 2024-03-26T03:36:14.187535+0000 mgr.y (mgr.14152) 63 : cluster [DBG] pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:16.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:15 smithi190 bash[23392]: audit 2024-03-26T03:36:14.807491+0000 mon.a (mon.0) 239 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:36:16.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:15 smithi190 bash[23392]: audit 2024-03-26T03:36:14.956346+0000 mon.a (mon.0) 240 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:16.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:15 smithi190 bash[23392]: audit 2024-03-26T03:36:14.971743+0000 mon.a (mon.0) 241 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:16.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:15 smithi190 bash[23392]: cephadm 2024-03-26T03:36:14.977800+0000 mgr.y (mgr.14152) 64 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2024-03-26T03:36:16.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:15 smithi190 bash[23392]: audit 2024-03-26T03:36:14.978405+0000 mon.a (mon.0) 242 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T03:36:16.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:15 smithi190 bash[23392]: audit 2024-03-26T03:36:14.979864+0000 mon.a (mon.0) 243 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T03:36:16.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:15 smithi190 bash[23392]: audit 2024-03-26T03:36:14.981184+0000 mon.a (mon.0) 244 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:16.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:15 smithi190 bash[23392]: cephadm 2024-03-26T03:36:14.982542+0000 mgr.y (mgr.14152) 65 : cephadm [INF] Reconfiguring daemon mon.c on smithi175 2024-03-26T03:36:17.712 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:36:17.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:17 smithi175 bash[21527]: cluster 2024-03-26T03:36:16.187832+0000 mgr.y (mgr.14152) 66 : cluster [DBG] pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:17.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:17 smithi175 bash[21527]: audit 2024-03-26T03:36:16.847977+0000 mon.a (mon.0) 245 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:17.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:17 smithi175 bash[21527]: audit 2024-03-26T03:36:16.853954+0000 mon.a (mon.0) 246 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:17.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:17 smithi175 bash[21527]: cephadm 2024-03-26T03:36:16.900520+0000 mgr.y (mgr.14152) 67 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2024-03-26T03:36:17.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:17 smithi175 bash[21527]: audit 2024-03-26T03:36:16.901152+0000 mon.a (mon.0) 247 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T03:36:17.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:17 smithi175 bash[21527]: audit 2024-03-26T03:36:16.902977+0000 mon.a (mon.0) 248 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T03:36:17.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:17 smithi175 bash[21527]: audit 2024-03-26T03:36:16.904715+0000 mon.a (mon.0) 249 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:17.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:17 smithi175 bash[21527]: cephadm 2024-03-26T03:36:16.906709+0000 mgr.y (mgr.14152) 68 : cephadm [INF] Reconfiguring daemon mon.b on smithi190 2024-03-26T03:36:17.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:17 smithi175 bash[29284]: cluster 2024-03-26T03:36:16.187832+0000 mgr.y (mgr.14152) 66 : cluster [DBG] pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:17.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:17 smithi175 bash[29284]: audit 2024-03-26T03:36:16.847977+0000 mon.a (mon.0) 245 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:17.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:17 smithi175 bash[29284]: audit 2024-03-26T03:36:16.853954+0000 mon.a (mon.0) 246 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:17.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:17 smithi175 bash[29284]: cephadm 2024-03-26T03:36:16.900520+0000 mgr.y (mgr.14152) 67 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2024-03-26T03:36:17.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:17 smithi175 bash[29284]: audit 2024-03-26T03:36:16.901152+0000 mon.a (mon.0) 247 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T03:36:17.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:17 smithi175 bash[29284]: audit 2024-03-26T03:36:16.902977+0000 mon.a (mon.0) 248 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T03:36:17.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:17 smithi175 bash[29284]: audit 2024-03-26T03:36:16.904715+0000 mon.a (mon.0) 249 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:17.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:17 smithi175 bash[29284]: cephadm 2024-03-26T03:36:16.906709+0000 mgr.y (mgr.14152) 68 : cephadm [INF] Reconfiguring daemon mon.b on smithi190 2024-03-26T03:36:18.122 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:17 smithi190 bash[23392]: cluster 2024-03-26T03:36:16.187832+0000 mgr.y (mgr.14152) 66 : cluster [DBG] pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:18.122 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:17 smithi190 bash[23392]: audit 2024-03-26T03:36:16.847977+0000 mon.a (mon.0) 245 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:18.123 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:17 smithi190 bash[23392]: audit 2024-03-26T03:36:16.853954+0000 mon.a (mon.0) 246 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:18.123 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:17 smithi190 bash[23392]: cephadm 2024-03-26T03:36:16.900520+0000 mgr.y (mgr.14152) 67 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2024-03-26T03:36:18.123 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:17 smithi190 bash[23392]: audit 2024-03-26T03:36:16.901152+0000 mon.a (mon.0) 247 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T03:36:18.123 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:17 smithi190 bash[23392]: audit 2024-03-26T03:36:16.902977+0000 mon.a (mon.0) 248 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T03:36:18.123 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:17 smithi190 bash[23392]: audit 2024-03-26T03:36:16.904715+0000 mon.a (mon.0) 249 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:18.123 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:17 smithi190 bash[23392]: cephadm 2024-03-26T03:36:16.906709+0000 mgr.y (mgr.14152) 68 : cephadm [INF] Reconfiguring daemon mon.b on smithi190 2024-03-26T03:36:20.008 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d4e7b7700 1 -- 172.21.15.175:0/41281052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d48106570 msgr2=0x7f9d48106950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:36:20.008 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d4e7b7700 1 --2- 172.21.15.175:0/41281052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d48106570 0x7f9d48106950 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9d3800d960 tx=0x7f9d3800dc70 comp rx=0 tx=0).stop 2024-03-26T03:36:20.009 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d4e7b7700 1 -- 172.21.15.175:0/41281052 shutdown_connections 2024-03-26T03:36:20.009 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d4e7b7700 1 --2- 172.21.15.175:0/41281052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d48101f20 0x7f9d4810e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:36:20.009 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d4e7b7700 1 --2- 172.21.15.175:0/41281052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d48106570 0x7f9d48106950 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:36:20.009 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d4e7b7700 1 --2- 172.21.15.175:0/41281052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d48101560 0x7f9d481019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:36:20.009 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d4e7b7700 1 -- 172.21.15.175:0/41281052 >> 172.21.15.175:0/41281052 conn(0x7f9d48078590 msgr2=0x7f9d480789a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:36:20.009 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d4e7b7700 1 -- 172.21.15.175:0/41281052 shutdown_connections 2024-03-26T03:36:20.009 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d4e7b7700 1 -- 172.21.15.175:0/41281052 wait complete. 2024-03-26T03:36:20.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d4e7b7700 1 Processor -- start 2024-03-26T03:36:20.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d4e7b7700 1 -- start start 2024-03-26T03:36:20.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d4e7b7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d48101560 0x7f9d48196140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:36:20.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d4e7b7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d48101f20 0x7f9d48196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:36:20.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d4e7b7700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d48106570 0x7f9d4819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:36:20.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d4e7b7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4810fcb0 con 0x7f9d48101f20 2024-03-26T03:36:20.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d4e7b7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4810fb30 con 0x7f9d48101560 2024-03-26T03:36:20.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d4e7b7700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9d4810fe30 con 0x7f9d48106570 2024-03-26T03:36:20.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d477fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d48101f20 0x7f9d48196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:36:20.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d4cd54700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d48106570 0x7f9d4819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:36:20.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d477fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d48101f20 0x7f9d48196680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:48298/0 (socket says 172.21.15.175:48298) 2024-03-26T03:36:20.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d477fe700 1 -- 172.21.15.175:0/789569440 learned_addr learned my addr 172.21.15.175:0/789569440 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:36:20.011 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d47fff700 1 --2- 172.21.15.175:0/789569440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d48101560 0x7f9d48196140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:36:20.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d477fe700 1 -- 172.21.15.175:0/789569440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d48106570 msgr2=0x7f9d4819aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:36:20.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d477fe700 1 --2- 172.21.15.175:0/789569440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d48106570 0x7f9d4819aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:36:20.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d477fe700 1 -- 172.21.15.175:0/789569440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d48101560 msgr2=0x7f9d48196140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:36:20.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d477fe700 1 --2- 172.21.15.175:0/789569440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d48101560 0x7f9d48196140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:36:20.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d477fe700 1 -- 172.21.15.175:0/789569440 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d4819b150 con 0x7f9d48101f20 2024-03-26T03:36:20.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d477fe700 1 --2- 172.21.15.175:0/789569440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d48101f20 0x7f9d48196680 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c00ec10 tx=0x7f9d3c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:36:20.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d457fa700 1 -- 172.21.15.175:0/789569440 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c00cd40 con 0x7f9d48101f20 2024-03-26T03:36:20.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d457fa700 1 -- 172.21.15.175:0/789569440 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f9d3c0044a0 con 0x7f9d48101f20 2024-03-26T03:36:20.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d457fa700 1 -- 172.21.15.175:0/789569440 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c010430 con 0x7f9d48101f20 2024-03-26T03:36:20.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d4e7b7700 1 -- 172.21.15.175:0/789569440 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d4819b440 con 0x7f9d48101f20 2024-03-26T03:36:20.012 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.003+0000 7f9d4e7b7700 1 -- 172.21.15.175:0/789569440 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d481a2cc0 con 0x7f9d48101f20 2024-03-26T03:36:20.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.007+0000 7f9d457fa700 1 -- 172.21.15.175:0/789569440 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 13) ==== 46523+0+0 (secure 0 0 0) 0x7f9d3c003f70 con 0x7f9d48101f20 2024-03-26T03:36:20.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.007+0000 7f9d4e7b7700 1 -- 172.21.15.175:0/789569440 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d48109b60 con 0x7f9d48101f20 2024-03-26T03:36:20.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.007+0000 7f9d457fa700 1 --2- 172.21.15.175:0/789569440 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f9d300396b0 0x7f9d3003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:36:20.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.007+0000 7f9d457fa700 1 -- 172.21.15.175:0/789569440 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9d3c04cc30 con 0x7f9d48101f20 2024-03-26T03:36:20.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.007+0000 7f9d47fff700 1 --2- 172.21.15.175:0/789569440 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f9d300396b0 0x7f9d3003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:36:20.013 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.007+0000 7f9d47fff700 1 --2- 172.21.15.175:0/789569440 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f9d300396b0 0x7f9d3003bb70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009230 tx=0x7f9d38005040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:36:20.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.011+0000 7f9d457fa700 1 -- 172.21.15.175:0/789569440 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f9d3c01fba0 con 0x7f9d48101f20 2024-03-26T03:36:20.221 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.215+0000 7f9d4e7b7700 1 -- 172.21.15.175:0/789569440 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f9d481974a0 con 0x7f9d48101f20 2024-03-26T03:36:20.222 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.215+0000 7f9d457fa700 1 -- 172.21.15.175:0/789569440 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f9d3c004260 con 0x7f9d48101f20 2024-03-26T03:36:20.222 INFO:teuthology.orchestra.run.smithi175.stdout:# minimal ceph.conf for 762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T03:36:20.222 INFO:teuthology.orchestra.run.smithi175.stdout:[global] 2024-03-26T03:36:20.222 INFO:teuthology.orchestra.run.smithi175.stdout: fsid = 762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T03:36:20.222 INFO:teuthology.orchestra.run.smithi175.stdout: mon_host = [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] 2024-03-26T03:36:20.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.219+0000 7f9d4e7b7700 1 -- 172.21.15.175:0/789569440 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f9d300396b0 msgr2=0x7f9d3003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:36:20.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.219+0000 7f9d4e7b7700 1 --2- 172.21.15.175:0/789569440 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f9d300396b0 0x7f9d3003bb70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009230 tx=0x7f9d38005040 comp rx=0 tx=0).stop 2024-03-26T03:36:20.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.219+0000 7f9d4e7b7700 1 -- 172.21.15.175:0/789569440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d48101f20 msgr2=0x7f9d48196680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:36:20.224 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.219+0000 7f9d4e7b7700 1 --2- 172.21.15.175:0/789569440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d48101f20 0x7f9d48196680 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c00ec10 tx=0x7f9d3c00ef20 comp rx=0 tx=0).stop 2024-03-26T03:36:20.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.219+0000 7f9d4e7b7700 1 -- 172.21.15.175:0/789569440 shutdown_connections 2024-03-26T03:36:20.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.219+0000 7f9d4e7b7700 1 --2- 172.21.15.175:0/789569440 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f9d300396b0 0x7f9d3003bb70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:36:20.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.219+0000 7f9d4e7b7700 1 --2- 172.21.15.175:0/789569440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d48106570 0x7f9d4819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:36:20.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.219+0000 7f9d4e7b7700 1 --2- 172.21.15.175:0/789569440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d48101560 0x7f9d48196140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:36:20.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.219+0000 7f9d4e7b7700 1 --2- 172.21.15.175:0/789569440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d48101f20 0x7f9d48196680 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:36:20.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.219+0000 7f9d4e7b7700 1 -- 172.21.15.175:0/789569440 >> 172.21.15.175:0/789569440 conn(0x7f9d48078590 msgr2=0x7f9d481041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:36:20.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.219+0000 7f9d4e7b7700 1 -- 172.21.15.175:0/789569440 shutdown_connections 2024-03-26T03:36:20.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:20.219+0000 7f9d4e7b7700 1 -- 172.21.15.175:0/789569440 wait complete. 2024-03-26T03:36:20.235 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:19 smithi175 bash[21527]: cluster 2024-03-26T03:36:18.188163+0000 mgr.y (mgr.14152) 69 : cluster [DBG] pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:20.235 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:19 smithi175 bash[21527]: audit 2024-03-26T03:36:18.920459+0000 mon.a (mon.0) 250 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:20.236 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:19 smithi175 bash[21527]: audit 2024-03-26T03:36:18.928176+0000 mon.a (mon.0) 251 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:20.236 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:19 smithi175 bash[21527]: audit 2024-03-26T03:36:18.976993+0000 mon.a (mon.0) 252 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:36:20.236 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:19 smithi175 bash[21527]: audit 2024-03-26T03:36:18.980160+0000 mon.a (mon.0) 253 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:20.236 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:19 smithi175 bash[21527]: audit 2024-03-26T03:36:18.981864+0000 mon.a (mon.0) 254 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:36:20.236 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:19 smithi175 bash[21527]: audit 2024-03-26T03:36:18.991131+0000 mon.a (mon.0) 255 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:20.236 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:19 smithi175 bash[29284]: cluster 2024-03-26T03:36:18.188163+0000 mgr.y (mgr.14152) 69 : cluster [DBG] pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:20.236 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:19 smithi175 bash[29284]: audit 2024-03-26T03:36:18.920459+0000 mon.a (mon.0) 250 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:20.236 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:19 smithi175 bash[29284]: audit 2024-03-26T03:36:18.928176+0000 mon.a (mon.0) 251 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:20.236 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:19 smithi175 bash[29284]: audit 2024-03-26T03:36:18.976993+0000 mon.a (mon.0) 252 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:36:20.236 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:19 smithi175 bash[29284]: audit 2024-03-26T03:36:18.980160+0000 mon.a (mon.0) 253 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:20.236 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:19 smithi175 bash[29284]: audit 2024-03-26T03:36:18.981864+0000 mon.a (mon.0) 254 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:36:20.236 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:19 smithi175 bash[29284]: audit 2024-03-26T03:36:18.991131+0000 mon.a (mon.0) 255 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:20.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:19 smithi190 bash[23392]: cluster 2024-03-26T03:36:18.188163+0000 mgr.y (mgr.14152) 69 : cluster [DBG] pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:20.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:19 smithi190 bash[23392]: audit 2024-03-26T03:36:18.920459+0000 mon.a (mon.0) 250 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:20.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:19 smithi190 bash[23392]: audit 2024-03-26T03:36:18.928176+0000 mon.a (mon.0) 251 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:20.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:19 smithi190 bash[23392]: audit 2024-03-26T03:36:18.976993+0000 mon.a (mon.0) 252 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:36:20.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:19 smithi190 bash[23392]: audit 2024-03-26T03:36:18.980160+0000 mon.a (mon.0) 253 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:20.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:19 smithi190 bash[23392]: audit 2024-03-26T03:36:18.981864+0000 mon.a (mon.0) 254 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:36:20.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:19 smithi190 bash[23392]: audit 2024-03-26T03:36:18.991131+0000 mon.a (mon.0) 255 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:20.863 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-26T03:36:20.864 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-26T03:36:20.864 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.conf 2024-03-26T03:36:20.878 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-26T03:36:20.878 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T03:36:20.936 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-03-26T03:36:20.936 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.conf 2024-03-26T03:36:20.951 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-03-26T03:36:20.951 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T03:36:21.005 INFO:tasks.cephadm:Adding mgr.y on smithi175 2024-03-26T03:36:21.005 INFO:tasks.cephadm:Adding mgr.x on smithi190 2024-03-26T03:36:21.006 DEBUG:teuthology.orchestra.run.smithi190:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph orch apply mgr '2;smithi175=y;smithi190=x' 2024-03-26T03:36:21.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:20 smithi175 bash[21527]: audit 2024-03-26T03:36:20.222156+0000 mon.a (mon.0) 256 : audit [DBG] from='client.? 172.21.15.175:0/789569440' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:21.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:20 smithi175 bash[29284]: audit 2024-03-26T03:36:20.222156+0000 mon.a (mon.0) 256 : audit [DBG] from='client.? 172.21.15.175:0/789569440' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:21.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:20 smithi190 bash[23392]: audit 2024-03-26T03:36:20.222156+0000 mon.a (mon.0) 256 : audit [DBG] from='client.? 172.21.15.175:0/789569440' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:22.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:21 smithi175 bash[21527]: cluster 2024-03-26T03:36:20.188625+0000 mgr.y (mgr.14152) 70 : cluster [DBG] pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:22.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:21 smithi175 bash[29284]: cluster 2024-03-26T03:36:20.188625+0000 mgr.y (mgr.14152) 70 : cluster [DBG] pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:22.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:21 smithi190 bash[23392]: cluster 2024-03-26T03:36:20.188625+0000 mgr.y (mgr.14152) 70 : cluster [DBG] pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:24.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:23 smithi175 bash[29284]: cluster 2024-03-26T03:36:22.189137+0000 mgr.y (mgr.14152) 71 : cluster [DBG] pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:24.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:23 smithi175 bash[21527]: cluster 2024-03-26T03:36:22.189137+0000 mgr.y (mgr.14152) 71 : cluster [DBG] pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:24.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:23 smithi190 bash[23392]: cluster 2024-03-26T03:36:22.189137+0000 mgr.y (mgr.14152) 71 : cluster [DBG] pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:24.749 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T03:36:26.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:25 smithi175 bash[21527]: cluster 2024-03-26T03:36:24.189515+0000 mgr.y (mgr.14152) 72 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:26.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:25 smithi175 bash[29284]: cluster 2024-03-26T03:36:24.189515+0000 mgr.y (mgr.14152) 72 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:26.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:25 smithi190 bash[23392]: cluster 2024-03-26T03:36:24.189515+0000 mgr.y (mgr.14152) 72 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:27.227 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d9842700 1 -- 172.21.15.190:0/751244488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8d4101860 msgr2=0x7fb8d4101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:36:27.227 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d9842700 1 --2- 172.21.15.190:0/751244488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8d4101860 0x7fb8d4101ce0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8009990 tx=0x7fb8c8009ca0 comp rx=0 tx=0).stop 2024-03-26T03:36:27.228 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d9842700 1 -- 172.21.15.190:0/751244488 shutdown_connections 2024-03-26T03:36:27.228 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d9842700 1 --2- 172.21.15.190:0/751244488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8d4102c00 0x7fb8d4103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:36:27.228 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d9842700 1 --2- 172.21.15.190:0/751244488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8d4102220 0x7fb8d4109ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:36:27.228 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d9842700 1 --2- 172.21.15.190:0/751244488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8d4101860 0x7fb8d4101ce0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:36:27.228 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d9842700 1 -- 172.21.15.190:0/751244488 >> 172.21.15.190:0/751244488 conn(0x7fb8d40fc310 msgr2=0x7fb8d40fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:36:27.228 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d9842700 1 -- 172.21.15.190:0/751244488 shutdown_connections 2024-03-26T03:36:27.228 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d9842700 1 -- 172.21.15.190:0/751244488 wait complete. 2024-03-26T03:36:27.229 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d9842700 1 Processor -- start 2024-03-26T03:36:27.230 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d9842700 1 -- start start 2024-03-26T03:36:27.230 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d9842700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8d4101860 0x7fb8d4101280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:36:27.230 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d9842700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8d4102220 0x7fb8d419cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:36:27.230 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d9842700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8d4102c00 0x7fb8d419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:36:27.230 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d9842700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d410c6b0 con 0x7fb8d4102220 2024-03-26T03:36:27.230 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d9842700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d410c530 con 0x7fb8d4102c00 2024-03-26T03:36:27.230 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d9842700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb8d410c830 con 0x7fb8d4101860 2024-03-26T03:36:27.230 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d37fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8d4102c00 0x7fb8d419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:36:27.230 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d37fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8d4102c00 0x7fb8d419f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:40460/0 (socket says 172.21.15.190:40460) 2024-03-26T03:36:27.230 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d37fe700 1 -- 172.21.15.190:0/1865842973 learned_addr learned my addr 172.21.15.190:0/1865842973 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-03-26T03:36:27.230 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d27fc700 1 --2- 172.21.15.190:0/1865842973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8d4102220 0x7fb8d419cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:36:27.230 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d2ffd700 1 --2- 172.21.15.190:0/1865842973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8d4101860 0x7fb8d4101280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:36:27.230 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d37fe700 1 -- 172.21.15.190:0/1865842973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8d4101860 msgr2=0x7fb8d4101280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:36:27.231 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d37fe700 1 --2- 172.21.15.190:0/1865842973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8d4101860 0x7fb8d4101280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:36:27.231 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d37fe700 1 -- 172.21.15.190:0/1865842973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8d4102220 msgr2=0x7fb8d419cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:36:27.231 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d37fe700 1 --2- 172.21.15.190:0/1865842973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8d4102220 0x7fb8d419cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:36:27.231 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d37fe700 1 -- 172.21.15.190:0/1865842973 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d419f9d0 con 0x7fb8d4102c00 2024-03-26T03:36:27.231 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d37fe700 1 --2- 172.21.15.190:0/1865842973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8d4102c00 0x7fb8d419f100 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc007ee0 tx=0x7fb8cc00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:36:27.231 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d8840700 1 -- 172.21.15.190:0/1865842973 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc00dbd0 con 0x7fb8d4102c00 2024-03-26T03:36:27.231 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d8840700 1 -- 172.21.15.190:0/1865842973 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fb8cc00dd70 con 0x7fb8d4102c00 2024-03-26T03:36:27.231 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d8840700 1 -- 172.21.15.190:0/1865842973 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc005890 con 0x7fb8d4102c00 2024-03-26T03:36:27.231 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d9842700 1 -- 172.21.15.190:0/1865842973 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d419fcc0 con 0x7fb8d4102c00 2024-03-26T03:36:27.231 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.224+0000 7fb8d9842700 1 -- 172.21.15.190:0/1865842973 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d41a01b0 con 0x7fb8d4102c00 2024-03-26T03:36:27.232 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.228+0000 7fb8d9842700 1 -- 172.21.15.190:0/1865842973 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8d4050300 con 0x7fb8d4102c00 2024-03-26T03:36:27.232 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.228+0000 7fb8d8840700 1 -- 172.21.15.190:0/1865842973 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 13) ==== 46523+0+0 (secure 0 0 0) 0x7fb8cc01a020 con 0x7fb8d4102c00 2024-03-26T03:36:27.232 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.228+0000 7fb8d8840700 1 --2- 172.21.15.190:0/1865842973 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fb8bc0396b0 0x7fb8bc03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:36:27.232 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.228+0000 7fb8d8840700 1 -- 172.21.15.190:0/1865842973 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb8cc04bf80 con 0x7fb8d4102c00 2024-03-26T03:36:27.232 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.228+0000 7fb8d2ffd700 1 --2- 172.21.15.190:0/1865842973 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fb8bc0396b0 0x7fb8bc03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:36:27.232 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.228+0000 7fb8d2ffd700 1 --2- 172.21.15.190:0/1865842973 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fb8bc0396b0 0x7fb8bc03bb70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4005fd0 tx=0x7fb8c4005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:36:27.236 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.232+0000 7fb8d8840700 1 -- 172.21.15.190:0/1865842973 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7fb8cc010070 con 0x7fb8d4102c00 2024-03-26T03:36:27.448 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.444+0000 7fb8d9842700 1 -- 172.21.15.190:0/1865842973 --> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi175=y;smithi190=x", "target": ["mon-mgr", ""]}) -- 0x7fb8d402e160 con 0x7fb8bc0396b0 2024-03-26T03:36:27.461 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.456+0000 7fb8d8840700 1 -- 172.21.15.190:0/1865842973 <== mgr.14152 v2:172.21.15.175:6800/2826039805 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fb8d402e160 con 0x7fb8bc0396b0 2024-03-26T03:36:27.461 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled mgr update... 2024-03-26T03:36:27.464 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.460+0000 7fb8d9842700 1 -- 172.21.15.190:0/1865842973 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fb8bc0396b0 msgr2=0x7fb8bc03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:36:27.464 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.460+0000 7fb8d9842700 1 --2- 172.21.15.190:0/1865842973 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fb8bc0396b0 0x7fb8bc03bb70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4005fd0 tx=0x7fb8c4005e50 comp rx=0 tx=0).stop 2024-03-26T03:36:27.464 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.460+0000 7fb8d9842700 1 -- 172.21.15.190:0/1865842973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8d4102c00 msgr2=0x7fb8d419f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:36:27.464 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.460+0000 7fb8d9842700 1 --2- 172.21.15.190:0/1865842973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8d4102c00 0x7fb8d419f100 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc007ee0 tx=0x7fb8cc00d3b0 comp rx=0 tx=0).stop 2024-03-26T03:36:27.464 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.460+0000 7fb8d9842700 1 -- 172.21.15.190:0/1865842973 shutdown_connections 2024-03-26T03:36:27.464 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.460+0000 7fb8d9842700 1 --2- 172.21.15.190:0/1865842973 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fb8bc0396b0 0x7fb8bc03bb70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:36:27.464 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.460+0000 7fb8d9842700 1 --2- 172.21.15.190:0/1865842973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8d4101860 0x7fb8d4101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:36:27.464 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.460+0000 7fb8d9842700 1 --2- 172.21.15.190:0/1865842973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8d4102c00 0x7fb8d419f100 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:36:27.464 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.460+0000 7fb8d9842700 1 --2- 172.21.15.190:0/1865842973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8d4102220 0x7fb8d419cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:36:27.464 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.460+0000 7fb8d9842700 1 -- 172.21.15.190:0/1865842973 >> 172.21.15.190:0/1865842973 conn(0x7fb8d40fc310 msgr2=0x7fb8d4105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:36:27.464 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.460+0000 7fb8d9842700 1 -- 172.21.15.190:0/1865842973 shutdown_connections 2024-03-26T03:36:27.465 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:36:27.460+0000 7fb8d9842700 1 -- 172.21.15.190:0/1865842973 wait complete. 2024-03-26T03:36:28.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:27 smithi175 bash[21527]: cluster 2024-03-26T03:36:26.189918+0000 mgr.y (mgr.14152) 73 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:28.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:27 smithi175 bash[21527]: audit 2024-03-26T03:36:27.460560+0000 mon.a (mon.0) 257 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:28.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:27 smithi175 bash[21527]: audit 2024-03-26T03:36:27.496759+0000 mon.a (mon.0) 258 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:36:28.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:27 smithi175 bash[21527]: audit 2024-03-26T03:36:27.500740+0000 mon.a (mon.0) 259 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:28.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:27 smithi175 bash[21527]: audit 2024-03-26T03:36:27.502511+0000 mon.a (mon.0) 260 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:36:28.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:27 smithi175 bash[21527]: audit 2024-03-26T03:36:27.511616+0000 mon.a (mon.0) 261 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:28.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:27 smithi175 bash[21527]: audit 2024-03-26T03:36:27.514752+0000 mon.a (mon.0) 262 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T03:36:28.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:27 smithi175 bash[21527]: audit 2024-03-26T03:36:27.517494+0000 mon.a (mon.0) 263 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T03:36:28.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:27 smithi175 bash[21527]: audit 2024-03-26T03:36:27.522656+0000 mon.a (mon.0) 264 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T03:36:28.246 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:27 smithi175 bash[21527]: audit 2024-03-26T03:36:27.524597+0000 mon.a (mon.0) 265 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:28.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:27 smithi175 bash[29284]: cluster 2024-03-26T03:36:26.189918+0000 mgr.y (mgr.14152) 73 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:28.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:27 smithi175 bash[29284]: audit 2024-03-26T03:36:27.460560+0000 mon.a (mon.0) 257 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:28.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:27 smithi175 bash[29284]: audit 2024-03-26T03:36:27.496759+0000 mon.a (mon.0) 258 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:36:28.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:27 smithi175 bash[29284]: audit 2024-03-26T03:36:27.500740+0000 mon.a (mon.0) 259 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:28.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:27 smithi175 bash[29284]: audit 2024-03-26T03:36:27.502511+0000 mon.a (mon.0) 260 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:36:28.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:27 smithi175 bash[29284]: audit 2024-03-26T03:36:27.511616+0000 mon.a (mon.0) 261 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:28.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:27 smithi175 bash[29284]: audit 2024-03-26T03:36:27.514752+0000 mon.a (mon.0) 262 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T03:36:28.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:27 smithi175 bash[29284]: audit 2024-03-26T03:36:27.517494+0000 mon.a (mon.0) 263 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T03:36:28.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:27 smithi175 bash[29284]: audit 2024-03-26T03:36:27.522656+0000 mon.a (mon.0) 264 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T03:36:28.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:27 smithi175 bash[29284]: audit 2024-03-26T03:36:27.524597+0000 mon.a (mon.0) 265 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:28.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:27 smithi190 bash[23392]: cluster 2024-03-26T03:36:26.189918+0000 mgr.y (mgr.14152) 73 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:28.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:27 smithi190 bash[23392]: audit 2024-03-26T03:36:27.460560+0000 mon.a (mon.0) 257 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:28.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:27 smithi190 bash[23392]: audit 2024-03-26T03:36:27.496759+0000 mon.a (mon.0) 258 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:36:28.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:27 smithi190 bash[23392]: audit 2024-03-26T03:36:27.500740+0000 mon.a (mon.0) 259 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:28.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:27 smithi190 bash[23392]: audit 2024-03-26T03:36:27.502511+0000 mon.a (mon.0) 260 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:36:28.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:27 smithi190 bash[23392]: audit 2024-03-26T03:36:27.511616+0000 mon.a (mon.0) 261 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:28.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:27 smithi190 bash[23392]: audit 2024-03-26T03:36:27.514752+0000 mon.a (mon.0) 262 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T03:36:28.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:27 smithi190 bash[23392]: audit 2024-03-26T03:36:27.517494+0000 mon.a (mon.0) 263 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T03:36:28.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:27 smithi190 bash[23392]: audit 2024-03-26T03:36:27.522656+0000 mon.a (mon.0) 264 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T03:36:28.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:27 smithi190 bash[23392]: audit 2024-03-26T03:36:27.524597+0000 mon.a (mon.0) 265 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:28.418 DEBUG:teuthology.orchestra.run.smithi190:mgr.x> sudo journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@mgr.x.service 2024-03-26T03:36:28.420 INFO:tasks.cephadm:Deploying OSDs... 2024-03-26T03:36:28.421 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-26T03:36:28.421 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/scratch_devs of=/dev/stdout 2024-03-26T03:36:28.427 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-26T03:36:28.427 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_1 2024-03-26T03:36:28.475 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-26T03:36:28.476 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T03:36:28.476 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 886 Links: 1 2024-03-26T03:36:28.476 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T03:36:28.476 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-03-26 03:27:06.523059591 +0000 2024-03-26T03:36:28.476 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-03-26 03:27:06.519059626 +0000 2024-03-26T03:36:28.476 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-03-26 03:27:06.519059626 +0000 2024-03-26T03:36:28.476 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: - 2024-03-26T03:36:28.476 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-26T03:36:28.530 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-03-26T03:36:28.530 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-03-26T03:36:28.530 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000454574 s, 1.1 MB/s 2024-03-26T03:36:28.531 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-26T03:36:28.580 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_2 2024-03-26T03:36:28.627 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-26T03:36:28.627 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T03:36:28.627 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 900 Links: 1 2024-03-26T03:36:28.627 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T03:36:28.627 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-03-26 03:27:07.235053231 +0000 2024-03-26T03:36:28.627 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-03-26 03:27:06.999055339 +0000 2024-03-26T03:36:28.627 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-03-26 03:27:06.999055339 +0000 2024-03-26T03:36:28.627 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: - 2024-03-26T03:36:28.628 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-26T03:36:28.681 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-03-26T03:36:28.681 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-03-26T03:36:28.681 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000399078 s, 1.3 MB/s 2024-03-26T03:36:28.682 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-26T03:36:28.732 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_3 2024-03-26T03:36:28.779 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-26T03:36:28.779 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T03:36:28.779 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 919 Links: 1 2024-03-26T03:36:28.779 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T03:36:28.779 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-03-26 03:27:07.727048836 +0000 2024-03-26T03:36:28.779 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-03-26 03:27:07.463051195 +0000 2024-03-26T03:36:28.779 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-03-26 03:27:07.463051195 +0000 2024-03-26T03:36:28.780 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: - 2024-03-26T03:36:28.780 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-26T03:36:28.834 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-03-26T03:36:28.834 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-03-26T03:36:28.834 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000420573 s, 1.2 MB/s 2024-03-26T03:36:28.836 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-26T03:36:28.884 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_4 2024-03-26T03:36:28.931 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-26T03:36:28.931 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T03:36:28.931 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 935 Links: 1 2024-03-26T03:36:28.931 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T03:36:28.931 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-03-26 03:27:08.219044442 +0000 2024-03-26T03:36:28.931 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-03-26 03:27:07.967046692 +0000 2024-03-26T03:36:28.931 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-03-26 03:27:07.967046692 +0000 2024-03-26T03:36:28.931 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: - 2024-03-26T03:36:28.932 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-26T03:36:28.986 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-03-26T03:36:28.986 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-03-26T03:36:28.986 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000392407 s, 1.3 MB/s 2024-03-26T03:36:28.987 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-26T03:36:29.036 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-03-26T03:36:29.037 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/scratch_devs of=/dev/stdout 2024-03-26T03:36:29.042 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-26T03:36:29.042 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_1 2024-03-26T03:36:29.090 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-26T03:36:29.090 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T03:36:29.090 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 880 Links: 1 2024-03-26T03:36:29.090 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T03:36:29.090 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-03-26 03:27:05.610938169 +0000 2024-03-26T03:36:29.090 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-03-26 03:27:05.342940559 +0000 2024-03-26T03:36:29.090 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-03-26 03:27:05.342940559 +0000 2024-03-26T03:36:29.090 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: - 2024-03-26T03:36:29.090 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-26T03:36:29.142 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-03-26T03:36:29.143 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-03-26T03:36:29.143 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000314242 s, 1.6 MB/s 2024-03-26T03:36:29.143 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-26T03:36:29.191 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_2 2024-03-26T03:36:29.223 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:28 smithi190 bash[23392]: audit 2024-03-26T03:36:27.450582+0000 mgr.y (mgr.14152) 74 : audit [DBG] from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi175=y;smithi190=x", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:36:29.223 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:28 smithi190 bash[23392]: cephadm 2024-03-26T03:36:27.454099+0000 mgr.y (mgr.14152) 75 : cephadm [INF] Saving service mgr spec with placement smithi175=y;smithi190=x;count:2 2024-03-26T03:36:29.223 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:28 smithi190 bash[23392]: cephadm 2024-03-26T03:36:27.526755+0000 mgr.y (mgr.14152) 76 : cephadm [INF] Deploying daemon mgr.x on smithi190 2024-03-26T03:36:29.227 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-26T03:36:29.227 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T03:36:29.227 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 900 Links: 1 2024-03-26T03:36:29.227 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T03:36:29.227 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-03-26 03:27:06.110933709 +0000 2024-03-26T03:36:29.227 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-03-26 03:27:05.850936029 +0000 2024-03-26T03:36:29.227 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-03-26 03:27:05.850936029 +0000 2024-03-26T03:36:29.227 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: - 2024-03-26T03:36:29.228 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-26T03:36:29.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:28 smithi175 bash[29284]: audit 2024-03-26T03:36:27.450582+0000 mgr.y (mgr.14152) 74 : audit [DBG] from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi175=y;smithi190=x", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:36:29.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:28 smithi175 bash[29284]: cephadm 2024-03-26T03:36:27.454099+0000 mgr.y (mgr.14152) 75 : cephadm [INF] Saving service mgr spec with placement smithi175=y;smithi190=x;count:2 2024-03-26T03:36:29.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:28 smithi175 bash[29284]: cephadm 2024-03-26T03:36:27.526755+0000 mgr.y (mgr.14152) 76 : cephadm [INF] Deploying daemon mgr.x on smithi190 2024-03-26T03:36:29.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:28 smithi175 bash[21527]: audit 2024-03-26T03:36:27.450582+0000 mgr.y (mgr.14152) 74 : audit [DBG] from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi175=y;smithi190=x", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:36:29.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:28 smithi175 bash[21527]: cephadm 2024-03-26T03:36:27.454099+0000 mgr.y (mgr.14152) 75 : cephadm [INF] Saving service mgr spec with placement smithi175=y;smithi190=x;count:2 2024-03-26T03:36:29.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:28 smithi175 bash[21527]: cephadm 2024-03-26T03:36:27.526755+0000 mgr.y (mgr.14152) 76 : cephadm [INF] Deploying daemon mgr.x on smithi190 2024-03-26T03:36:29.279 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-03-26T03:36:29.279 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-03-26T03:36:29.279 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000284156 s, 1.8 MB/s 2024-03-26T03:36:29.280 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-26T03:36:29.327 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_3 2024-03-26T03:36:29.374 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-26T03:36:29.374 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T03:36:29.375 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 916 Links: 1 2024-03-26T03:36:29.375 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T03:36:29.375 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-03-26 03:27:06.598929356 +0000 2024-03-26T03:36:29.375 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-03-26 03:27:06.350931568 +0000 2024-03-26T03:36:29.375 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-03-26 03:27:06.350931568 +0000 2024-03-26T03:36:29.375 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: - 2024-03-26T03:36:29.375 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-26T03:36:29.429 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-03-26T03:36:29.429 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-03-26T03:36:29.429 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000355367 s, 1.4 MB/s 2024-03-26T03:36:29.430 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-26T03:36:29.479 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_4 2024-03-26T03:36:29.526 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-26T03:36:29.526 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T03:36:29.526 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 930 Links: 1 2024-03-26T03:36:29.526 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T03:36:29.526 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-03-26 03:27:07.066925182 +0000 2024-03-26T03:36:29.526 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-03-26 03:27:06.830927285 +0000 2024-03-26T03:36:29.526 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-03-26 03:27:06.830927285 +0000 2024-03-26T03:36:29.526 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: - 2024-03-26T03:36:29.526 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-26T03:36:29.581 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-03-26T03:36:29.581 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-03-26T03:36:29.581 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000412753 s, 1.2 MB/s 2024-03-26T03:36:29.582 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-26T03:36:29.631 INFO:tasks.cephadm:Deploying osd.0 on smithi175 with /dev/vg_nvme/lv_4... 2024-03-26T03:36:29.631 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-26T03:36:30.117 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:29 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:36:30.117 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:29 smithi190 bash[23392]: cluster 2024-03-26T03:36:28.190321+0000 mgr.y (mgr.14152) 77 : cluster [DBG] pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:30.117 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:30 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:36:30.118 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:36:29 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:36:30.118 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:36:29 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:36:30.118 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:36:29 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:36:30.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:29 smithi175 bash[21527]: cluster 2024-03-26T03:36:28.190321+0000 mgr.y (mgr.14152) 77 : cluster [DBG] pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:30.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:29 smithi175 bash[29284]: cluster 2024-03-26T03:36:28.190321+0000 mgr.y (mgr.14152) 77 : cluster [DBG] pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:30.499 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:36:30 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:36:30.499 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:36:30 smithi190 systemd[1]: Started Ceph mgr.x for 762e930c-eb21-11ee-95d0-87774f69a715. 2024-03-26T03:36:31.399 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:36:31 smithi190 bash[24341]: debug 2024-03-26T03:36:31.252+0000 7fc1c1429700 1 -- 172.21.15.190:0/2793242 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c7f741a5a0 con 0x55c7f819a800 2024-03-26T03:36:31.400 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:31 smithi190 bash[23392]: cluster 2024-03-26T03:36:30.190678+0000 mgr.y (mgr.14152) 78 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:31.400 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:31 smithi190 bash[23392]: audit 2024-03-26T03:36:30.268681+0000 mon.a (mon.0) 266 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:31.400 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:31 smithi190 bash[23392]: audit 2024-03-26T03:36:30.276372+0000 mon.a (mon.0) 267 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:31.400 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:31 smithi190 bash[23392]: audit 2024-03-26T03:36:30.304266+0000 mon.a (mon.0) 268 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:31.400 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:31 smithi190 bash[23392]: audit 2024-03-26T03:36:30.313582+0000 mon.a (mon.0) 269 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:31.400 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:31 smithi190 bash[23392]: audit 2024-03-26T03:36:30.384998+0000 mon.a (mon.0) 270 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:36:31.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:31 smithi175 bash[21527]: cluster 2024-03-26T03:36:30.190678+0000 mgr.y (mgr.14152) 78 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:31.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:31 smithi175 bash[21527]: audit 2024-03-26T03:36:30.268681+0000 mon.a (mon.0) 266 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:31.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:31 smithi175 bash[21527]: audit 2024-03-26T03:36:30.276372+0000 mon.a (mon.0) 267 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:31.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:31 smithi175 bash[21527]: audit 2024-03-26T03:36:30.304266+0000 mon.a (mon.0) 268 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:31.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:31 smithi175 bash[21527]: audit 2024-03-26T03:36:30.313582+0000 mon.a (mon.0) 269 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:31.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:31 smithi175 bash[21527]: audit 2024-03-26T03:36:30.384998+0000 mon.a (mon.0) 270 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:36:31.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:31 smithi175 bash[29284]: cluster 2024-03-26T03:36:30.190678+0000 mgr.y (mgr.14152) 78 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:31.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:31 smithi175 bash[29284]: audit 2024-03-26T03:36:30.268681+0000 mon.a (mon.0) 266 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:31.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:31 smithi175 bash[29284]: audit 2024-03-26T03:36:30.276372+0000 mon.a (mon.0) 267 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:31.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:31 smithi175 bash[29284]: audit 2024-03-26T03:36:30.304266+0000 mon.a (mon.0) 268 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:31.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:31 smithi175 bash[29284]: audit 2024-03-26T03:36:30.313582+0000 mon.a (mon.0) 269 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:31.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:31 smithi175 bash[29284]: audit 2024-03-26T03:36:30.384998+0000 mon.a (mon.0) 270 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:36:33.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:32 smithi175 bash[21527]: audit 2024-03-26T03:36:31.874678+0000 mon.a (mon.0) 271 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:33.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:32 smithi175 bash[29284]: audit 2024-03-26T03:36:31.874678+0000 mon.a (mon.0) 271 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:33.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:32 smithi190 bash[23392]: audit 2024-03-26T03:36:31.874678+0000 mon.a (mon.0) 271 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:33.249 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:36:31 smithi190 bash[24341]: debug 2024-03-26T03:36:31.396+0000 7fc1cb638200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T03:36:33.249 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:36:32 smithi190 bash[24341]: debug 2024-03-26T03:36:32.956+0000 7fc1cb638200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T03:36:33.249 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:36:33 smithi190 bash[24341]: debug 2024-03-26T03:36:33.052+0000 7fc1cb638200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T03:36:33.749 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:36:33 smithi190 bash[24341]: debug 2024-03-26T03:36:33.300+0000 7fc1cb638200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T03:36:34.219 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:36:34 smithi190 bash[24341]: debug 2024-03-26T03:36:34.104+0000 7fc1cb638200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T03:36:34.219 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:33 smithi190 bash[23392]: cluster 2024-03-26T03:36:32.191168+0000 mgr.y (mgr.14152) 79 : cluster [DBG] pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:34.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:33 smithi175 bash[21527]: cluster 2024-03-26T03:36:32.191168+0000 mgr.y (mgr.14152) 79 : cluster [DBG] pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:34.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:33 smithi175 bash[29284]: cluster 2024-03-26T03:36:32.191168+0000 mgr.y (mgr.14152) 79 : cluster [DBG] pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:34.355 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:36:34.499 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:36:34 smithi190 bash[24341]: debug 2024-03-26T03:36:34.216+0000 7fc1cb638200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T03:36:34.499 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:36:34 smithi190 bash[24341]: debug 2024-03-26T03:36:34.316+0000 7fc1cb638200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T03:36:34.999 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:36:34 smithi190 bash[24341]: debug 2024-03-26T03:36:34.672+0000 7fc1cb638200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T03:36:34.999 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:36:34 smithi190 bash[24341]: debug 2024-03-26T03:36:34.784+0000 7fc1cb638200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T03:36:34.999 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:36:34 smithi190 bash[24341]: debug 2024-03-26T03:36:34.888+0000 7fc1cb638200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T03:36:35.499 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:36:35 smithi190 bash[24341]: debug 2024-03-26T03:36:35.240+0000 7fc1cb638200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T03:36:35.499 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:36:35 smithi190 bash[24341]: debug 2024-03-26T03:36:35.484+0000 7fc1cb638200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T03:36:36.153 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:36:35 smithi190 bash[24341]: debug 2024-03-26T03:36:35.768+0000 7fc1cb638200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T03:36:36.153 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:36:35 smithi190 bash[24341]: debug 2024-03-26T03:36:35.880+0000 7fc1cb638200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T03:36:36.154 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:35 smithi190 bash[23392]: cluster 2024-03-26T03:36:34.191628+0000 mgr.y (mgr.14152) 80 : cluster [DBG] pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:36.154 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:35 smithi190 bash[23392]: audit 2024-03-26T03:36:35.623787+0000 mon.a (mon.0) 272 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:36.154 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:35 smithi190 bash[23392]: audit 2024-03-26T03:36:35.629268+0000 mon.a (mon.0) 273 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:36.154 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:35 smithi190 bash[23392]: audit 2024-03-26T03:36:35.630600+0000 mon.a (mon.0) 274 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:36.154 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:35 smithi190 bash[23392]: audit 2024-03-26T03:36:35.631750+0000 mon.a (mon.0) 275 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:36:36.154 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:35 smithi190 bash[23392]: audit 2024-03-26T03:36:35.637808+0000 mon.a (mon.0) 276 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:36.154 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:35 smithi190 bash[23392]: audit 2024-03-26T03:36:35.705092+0000 mon.a (mon.0) 277 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T03:36:36.154 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:35 smithi190 bash[23392]: audit 2024-03-26T03:36:35.706956+0000 mon.a (mon.0) 278 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T03:36:36.154 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:35 smithi190 bash[23392]: audit 2024-03-26T03:36:35.708303+0000 mon.a (mon.0) 279 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:36.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:35 smithi175 bash[21527]: cluster 2024-03-26T03:36:34.191628+0000 mgr.y (mgr.14152) 80 : cluster [DBG] pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:36.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:35 smithi175 bash[21527]: audit 2024-03-26T03:36:35.623787+0000 mon.a (mon.0) 272 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:36.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:35 smithi175 bash[21527]: audit 2024-03-26T03:36:35.629268+0000 mon.a (mon.0) 273 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:36.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:35 smithi175 bash[21527]: audit 2024-03-26T03:36:35.630600+0000 mon.a (mon.0) 274 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:36.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:35 smithi175 bash[21527]: audit 2024-03-26T03:36:35.631750+0000 mon.a (mon.0) 275 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:36:36.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:35 smithi175 bash[21527]: audit 2024-03-26T03:36:35.637808+0000 mon.a (mon.0) 276 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:36.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:35 smithi175 bash[21527]: audit 2024-03-26T03:36:35.705092+0000 mon.a (mon.0) 277 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T03:36:36.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:35 smithi175 bash[21527]: audit 2024-03-26T03:36:35.706956+0000 mon.a (mon.0) 278 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T03:36:36.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:35 smithi175 bash[21527]: audit 2024-03-26T03:36:35.708303+0000 mon.a (mon.0) 279 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:36.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:35 smithi175 bash[29284]: cluster 2024-03-26T03:36:34.191628+0000 mgr.y (mgr.14152) 80 : cluster [DBG] pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:36.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:35 smithi175 bash[29284]: audit 2024-03-26T03:36:35.623787+0000 mon.a (mon.0) 272 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:36.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:35 smithi175 bash[29284]: audit 2024-03-26T03:36:35.629268+0000 mon.a (mon.0) 273 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:36.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:35 smithi175 bash[29284]: audit 2024-03-26T03:36:35.630600+0000 mon.a (mon.0) 274 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:36.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:35 smithi175 bash[29284]: audit 2024-03-26T03:36:35.631750+0000 mon.a (mon.0) 275 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:36:36.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:35 smithi175 bash[29284]: audit 2024-03-26T03:36:35.637808+0000 mon.a (mon.0) 276 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:36.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:35 smithi175 bash[29284]: audit 2024-03-26T03:36:35.705092+0000 mon.a (mon.0) 277 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T03:36:36.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:35 smithi175 bash[29284]: audit 2024-03-26T03:36:35.706956+0000 mon.a (mon.0) 278 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T03:36:36.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:35 smithi175 bash[29284]: audit 2024-03-26T03:36:35.708303+0000 mon.a (mon.0) 279 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:36.499 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:36:36 smithi190 bash[24341]: debug 2024-03-26T03:36:36.148+0000 7fc1cb638200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T03:36:36.892 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:36:36 smithi190 bash[24341]: debug 2024-03-26T03:36:36.564+0000 7fc1cb638200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T03:36:36.892 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:36:36 smithi190 bash[24341]: debug 2024-03-26T03:36:36.796+0000 7fc1cb638200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T03:36:37.161 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:36:36 smithi190 bash[24341]: debug 2024-03-26T03:36:36.912+0000 7fc1cb638200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T03:36:37.161 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:36 smithi190 bash[23392]: cephadm 2024-03-26T03:36:35.704434+0000 mgr.y (mgr.14152) 81 : cephadm [INF] Reconfiguring mgr.y (unknown last config time)... 2024-03-26T03:36:37.161 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:36 smithi190 bash[23392]: cephadm 2024-03-26T03:36:35.709766+0000 mgr.y (mgr.14152) 82 : cephadm [INF] Reconfiguring daemon mgr.y on smithi175 2024-03-26T03:36:37.163 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:36 smithi175 bash[21527]: cephadm 2024-03-26T03:36:35.704434+0000 mgr.y (mgr.14152) 81 : cephadm [INF] Reconfiguring mgr.y (unknown last config time)... 2024-03-26T03:36:37.163 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:36 smithi175 bash[21527]: cephadm 2024-03-26T03:36:35.709766+0000 mgr.y (mgr.14152) 82 : cephadm [INF] Reconfiguring daemon mgr.y on smithi175 2024-03-26T03:36:37.163 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:36 smithi175 bash[29284]: cephadm 2024-03-26T03:36:35.704434+0000 mgr.y (mgr.14152) 81 : cephadm [INF] Reconfiguring mgr.y (unknown last config time)... 2024-03-26T03:36:37.163 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:36 smithi175 bash[29284]: cephadm 2024-03-26T03:36:35.709766+0000 mgr.y (mgr.14152) 82 : cephadm [INF] Reconfiguring daemon mgr.y on smithi175 2024-03-26T03:36:37.499 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:36:37 smithi190 bash[24341]: debug 2024-03-26T03:36:37.156+0000 7fc1cb638200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T03:36:37.836 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-26T03:36:37.836 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-26T03:36:37.837 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2024-03-26T03:36:37.837 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2024-03-26T03:36:37.837 INFO:teuthology.orchestra.run.smithi175.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0129564 s, 809 MB/s 2024-03-26T03:36:37.837 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2024-03-26T03:36:38.170 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:37 smithi190 bash[23392]: cluster 2024-03-26T03:36:36.191982+0000 mgr.y (mgr.14152) 83 : cluster [DBG] pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:38.170 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:37 smithi190 bash[23392]: audit 2024-03-26T03:36:37.860821+0000 mon.a (mon.0) 280 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:38.170 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:37 smithi190 bash[23392]: audit 2024-03-26T03:36:37.868671+0000 mon.a (mon.0) 281 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:38.170 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:37 smithi190 bash[23392]: audit 2024-03-26T03:36:37.879164+0000 mon.a (mon.0) 282 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:36:38.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:37 smithi175 bash[21527]: cluster 2024-03-26T03:36:36.191982+0000 mgr.y (mgr.14152) 83 : cluster [DBG] pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:38.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:37 smithi175 bash[21527]: audit 2024-03-26T03:36:37.860821+0000 mon.a (mon.0) 280 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:38.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:37 smithi175 bash[21527]: audit 2024-03-26T03:36:37.868671+0000 mon.a (mon.0) 281 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:38.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:37 smithi175 bash[21527]: audit 2024-03-26T03:36:37.879164+0000 mon.a (mon.0) 282 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:36:38.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:37 smithi175 bash[29284]: cluster 2024-03-26T03:36:36.191982+0000 mgr.y (mgr.14152) 83 : cluster [DBG] pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:38.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:37 smithi175 bash[29284]: audit 2024-03-26T03:36:37.860821+0000 mon.a (mon.0) 280 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:38.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:37 smithi175 bash[29284]: audit 2024-03-26T03:36:37.868671+0000 mon.a (mon.0) 281 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:38.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:37 smithi175 bash[29284]: audit 2024-03-26T03:36:37.879164+0000 mon.a (mon.0) 282 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:36:38.459 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi175:vg_nvme/lv_4 2024-03-26T03:36:38.501 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:36:38 smithi190 bash[24341]: debug 2024-03-26T03:36:38.168+0000 7fc1cb638200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T03:36:38.501 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:36:38 smithi190 bash[24341]: debug 2024-03-26T03:36:38.284+0000 7fc1cb638200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T03:36:38.906 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:36:38 smithi190 bash[24341]: debug 2024-03-26T03:36:38.576+0000 7fc1cb638200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T03:36:38.906 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:36:38 smithi190 bash[24341]: debug 2024-03-26T03:36:38.688+0000 7fc1cb638200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T03:36:39.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:38 smithi175 bash[21527]: audit 2024-03-26T03:36:38.557814+0000 mon.a (mon.0) 283 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:39.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:38 smithi175 bash[21527]: audit 2024-03-26T03:36:38.559319+0000 mon.a (mon.0) 284 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:36:39.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:38 smithi175 bash[21527]: audit 2024-03-26T03:36:38.566504+0000 mon.a (mon.0) 285 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:39.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:38 smithi175 bash[29284]: audit 2024-03-26T03:36:38.557814+0000 mon.a (mon.0) 283 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:39.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:38 smithi175 bash[29284]: audit 2024-03-26T03:36:38.559319+0000 mon.a (mon.0) 284 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:36:39.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:38 smithi175 bash[29284]: audit 2024-03-26T03:36:38.566504+0000 mon.a (mon.0) 285 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:39.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:38 smithi190 bash[23392]: audit 2024-03-26T03:36:38.557814+0000 mon.a (mon.0) 283 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:39.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:38 smithi190 bash[23392]: audit 2024-03-26T03:36:38.559319+0000 mon.a (mon.0) 284 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:36:39.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:38 smithi190 bash[23392]: audit 2024-03-26T03:36:38.566504+0000 mon.a (mon.0) 285 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:36:39.249 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:36:39 smithi190 bash[24341]: debug 2024-03-26T03:36:39.132+0000 7fc1cb638200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T03:36:39.749 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:36:39 smithi190 bash[24341]: debug 2024-03-26T03:36:39.252+0000 7fc1cb638200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T03:36:40.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:39 smithi175 bash[21527]: cluster 2024-03-26T03:36:38.192335+0000 mgr.y (mgr.14152) 84 : cluster [DBG] pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:40.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:39 smithi175 bash[21527]: cluster 2024-03-26T03:36:39.277727+0000 mon.a (mon.0) 286 : cluster [DBG] Standby manager daemon x started 2024-03-26T03:36:40.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:39 smithi175 bash[21527]: audit 2024-03-26T03:36:39.281941+0000 mon.b (mon.1) 4 : audit [DBG] from='mgr.? 172.21.15.190:0/2073020682' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-26T03:36:40.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:39 smithi175 bash[21527]: audit 2024-03-26T03:36:39.282595+0000 mon.b (mon.1) 5 : audit [DBG] from='mgr.? 172.21.15.190:0/2073020682' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T03:36:40.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:39 smithi175 bash[21527]: audit 2024-03-26T03:36:39.284147+0000 mon.b (mon.1) 6 : audit [DBG] from='mgr.? 172.21.15.190:0/2073020682' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-26T03:36:40.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:39 smithi175 bash[21527]: audit 2024-03-26T03:36:39.284835+0000 mon.b (mon.1) 7 : audit [DBG] from='mgr.? 172.21.15.190:0/2073020682' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T03:36:40.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:39 smithi175 bash[29284]: cluster 2024-03-26T03:36:38.192335+0000 mgr.y (mgr.14152) 84 : cluster [DBG] pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:40.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:39 smithi175 bash[29284]: cluster 2024-03-26T03:36:39.277727+0000 mon.a (mon.0) 286 : cluster [DBG] Standby manager daemon x started 2024-03-26T03:36:40.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:39 smithi175 bash[29284]: audit 2024-03-26T03:36:39.281941+0000 mon.b (mon.1) 4 : audit [DBG] from='mgr.? 172.21.15.190:0/2073020682' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-26T03:36:40.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:39 smithi175 bash[29284]: audit 2024-03-26T03:36:39.282595+0000 mon.b (mon.1) 5 : audit [DBG] from='mgr.? 172.21.15.190:0/2073020682' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T03:36:40.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:39 smithi175 bash[29284]: audit 2024-03-26T03:36:39.284147+0000 mon.b (mon.1) 6 : audit [DBG] from='mgr.? 172.21.15.190:0/2073020682' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-26T03:36:40.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:39 smithi175 bash[29284]: audit 2024-03-26T03:36:39.284835+0000 mon.b (mon.1) 7 : audit [DBG] from='mgr.? 172.21.15.190:0/2073020682' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T03:36:40.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:39 smithi190 bash[23392]: cluster 2024-03-26T03:36:38.192335+0000 mgr.y (mgr.14152) 84 : cluster [DBG] pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:40.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:39 smithi190 bash[23392]: cluster 2024-03-26T03:36:39.277727+0000 mon.a (mon.0) 286 : cluster [DBG] Standby manager daemon x started 2024-03-26T03:36:40.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:39 smithi190 bash[23392]: audit 2024-03-26T03:36:39.281941+0000 mon.b (mon.1) 4 : audit [DBG] from='mgr.? 172.21.15.190:0/2073020682' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-26T03:36:40.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:39 smithi190 bash[23392]: audit 2024-03-26T03:36:39.282595+0000 mon.b (mon.1) 5 : audit [DBG] from='mgr.? 172.21.15.190:0/2073020682' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T03:36:40.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:39 smithi190 bash[23392]: audit 2024-03-26T03:36:39.284147+0000 mon.b (mon.1) 6 : audit [DBG] from='mgr.? 172.21.15.190:0/2073020682' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-26T03:36:40.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:39 smithi190 bash[23392]: audit 2024-03-26T03:36:39.284835+0000 mon.b (mon.1) 7 : audit [DBG] from='mgr.? 172.21.15.190:0/2073020682' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T03:36:41.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:40 smithi175 bash[29284]: cluster 2024-03-26T03:36:39.926434+0000 mon.a (mon.0) 287 : cluster [DBG] mgrmap e14: y(active, since 107s), standbys: x 2024-03-26T03:36:41.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:40 smithi175 bash[29284]: audit 2024-03-26T03:36:39.926705+0000 mon.a (mon.0) 288 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T03:36:41.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:40 smithi175 bash[21527]: cluster 2024-03-26T03:36:39.926434+0000 mon.a (mon.0) 287 : cluster [DBG] mgrmap e14: y(active, since 107s), standbys: x 2024-03-26T03:36:41.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:40 smithi175 bash[21527]: audit 2024-03-26T03:36:39.926705+0000 mon.a (mon.0) 288 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T03:36:41.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:40 smithi190 bash[23392]: cluster 2024-03-26T03:36:39.926434+0000 mon.a (mon.0) 287 : cluster [DBG] mgrmap e14: y(active, since 107s), standbys: x 2024-03-26T03:36:41.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:40 smithi190 bash[23392]: audit 2024-03-26T03:36:39.926705+0000 mon.a (mon.0) 288 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T03:36:42.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:41 smithi175 bash[29284]: cluster 2024-03-26T03:36:40.192703+0000 mgr.y (mgr.14152) 85 : cluster [DBG] pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:42.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:41 smithi175 bash[21527]: cluster 2024-03-26T03:36:40.192703+0000 mgr.y (mgr.14152) 85 : cluster [DBG] pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:42.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:41 smithi190 bash[23392]: cluster 2024-03-26T03:36:40.192703+0000 mgr.y (mgr.14152) 85 : cluster [DBG] pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:43.185 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:36:44.232 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:43 smithi175 bash[21527]: cluster 2024-03-26T03:36:42.193124+0000 mgr.y (mgr.14152) 86 : cluster [DBG] pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:44.232 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:43 smithi175 bash[29284]: cluster 2024-03-26T03:36:42.193124+0000 mgr.y (mgr.14152) 86 : cluster [DBG] pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:44.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:43 smithi190 bash[23392]: cluster 2024-03-26T03:36:42.193124+0000 mgr.y (mgr.14152) 86 : cluster [DBG] pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:45.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda4da8f700 1 -- 172.21.15.175:0/1242844124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda481009a0 msgr2=0x7fda48100dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:36:45.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda4da8f700 1 --2- 172.21.15.175:0/1242844124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda481009a0 0x7fda48100dc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fda38009990 tx=0x7fda38009ca0 comp rx=0 tx=0).stop 2024-03-26T03:36:45.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda4da8f700 1 -- 172.21.15.175:0/1242844124 shutdown_connections 2024-03-26T03:36:45.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda4da8f700 1 --2- 172.21.15.175:0/1242844124 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda481020d0 0x7fda48102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:36:45.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda4da8f700 1 --2- 172.21.15.175:0/1242844124 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda481009a0 0x7fda48100dc0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:36:45.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda4da8f700 1 --2- 172.21.15.175:0/1242844124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda48101710 0x7fda48101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:36:45.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda4da8f700 1 -- 172.21.15.175:0/1242844124 >> 172.21.15.175:0/1242844124 conn(0x7fda480fc300 msgr2=0x7fda480fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:36:45.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda4da8f700 1 -- 172.21.15.175:0/1242844124 shutdown_connections 2024-03-26T03:36:45.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda4da8f700 1 -- 172.21.15.175:0/1242844124 wait complete. 2024-03-26T03:36:45.509 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda4da8f700 1 Processor -- start 2024-03-26T03:36:45.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda4da8f700 1 -- start start 2024-03-26T03:36:45.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda4da8f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda481009a0 0x7fda4819a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:36:45.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda4da8f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda48101710 0x7fda4819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:36:45.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda4da8f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda481020d0 0x7fda481a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:36:45.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda4da8f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fda4810f8f0 con 0x7fda48101710 2024-03-26T03:36:45.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda4da8f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fda4810f770 con 0x7fda481009a0 2024-03-26T03:36:45.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda4da8f700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fda4810fa70 con 0x7fda481020d0 2024-03-26T03:36:45.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda46ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda48101710 0x7fda4819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:36:45.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda47fff700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda481020d0 0x7fda481a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:36:45.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda46ffd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda48101710 0x7fda4819a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:35316/0 (socket says 172.21.15.175:35316) 2024-03-26T03:36:45.510 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda46ffd700 1 -- 172.21.15.175:0/3143887228 learned_addr learned my addr 172.21.15.175:0/3143887228 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:36:45.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda477fe700 1 --2- 172.21.15.175:0/3143887228 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda481009a0 0x7fda4819a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:36:45.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda46ffd700 1 -- 172.21.15.175:0/3143887228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda481020d0 msgr2=0x7fda481a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:36:45.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda46ffd700 1 --2- 172.21.15.175:0/3143887228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda481020d0 0x7fda481a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:36:45.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda46ffd700 1 -- 172.21.15.175:0/3143887228 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda481009a0 msgr2=0x7fda4819a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:36:45.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda46ffd700 1 --2- 172.21.15.175:0/3143887228 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda481009a0 0x7fda4819a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:36:45.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda46ffd700 1 -- 172.21.15.175:0/3143887228 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda481a1ce0 con 0x7fda48101710 2024-03-26T03:36:45.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda46ffd700 1 --2- 172.21.15.175:0/3143887228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda48101710 0x7fda4819a5c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fda3000d980 tx=0x7fda3000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:36:45.512 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda44ff9700 1 -- 172.21.15.175:0/3143887228 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30016070 con 0x7fda48101710 2024-03-26T03:36:45.512 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda44ff9700 1 -- 172.21.15.175:0/3143887228 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fda3000de70 con 0x7fda48101710 2024-03-26T03:36:45.512 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda44ff9700 1 -- 172.21.15.175:0/3143887228 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30010e00 con 0x7fda48101710 2024-03-26T03:36:45.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda4da8f700 1 -- 172.21.15.175:0/3143887228 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda481a1fd0 con 0x7fda48101710 2024-03-26T03:36:45.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda4da8f700 1 -- 172.21.15.175:0/3143887228 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda481a2420 con 0x7fda48101710 2024-03-26T03:36:45.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.507+0000 7fda4da8f700 1 -- 172.21.15.175:0/3143887228 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda48050300 con 0x7fda48101710 2024-03-26T03:36:45.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.511+0000 7fda44ff9700 1 -- 172.21.15.175:0/3143887228 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 14) ==== 92419+0+0 (secure 0 0 0) 0x7fda30019070 con 0x7fda48101710 2024-03-26T03:36:45.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.511+0000 7fda44ff9700 1 --2- 172.21.15.175:0/3143887228 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fda3406e660 0x7fda34070b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:36:45.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.511+0000 7fda44ff9700 1 -- 172.21.15.175:0/3143887228 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fda3008ced0 con 0x7fda48101710 2024-03-26T03:36:45.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.511+0000 7fda477fe700 1 --2- 172.21.15.175:0/3143887228 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fda3406e660 0x7fda34070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:36:45.513 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.511+0000 7fda477fe700 1 --2- 172.21.15.175:0/3143887228 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fda3406e660 0x7fda34070b20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fda3800b2e0 tx=0x7fda3801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:36:45.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.515+0000 7fda44ff9700 1 -- 172.21.15.175:0/3143887228 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7fda3005b890 con 0x7fda48101710 2024-03-26T03:36:45.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:36:45.731+0000 7fda4da8f700 1 -- 172.21.15.175:0/3143887228 --> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fda48062a80 con 0x7fda3406e660 2024-03-26T03:36:46.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:45 smithi175 bash[29284]: cluster 2024-03-26T03:36:44.193521+0000 mgr.y (mgr.14152) 87 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:46.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:45 smithi175 bash[29284]: audit 2024-03-26T03:36:45.736783+0000 mon.a (mon.0) 289 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T03:36:46.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:45 smithi175 bash[29284]: audit 2024-03-26T03:36:45.740103+0000 mon.a (mon.0) 290 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T03:36:46.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:45 smithi175 bash[29284]: audit 2024-03-26T03:36:45.741078+0000 mon.a (mon.0) 291 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:46.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:45 smithi175 bash[21527]: cluster 2024-03-26T03:36:44.193521+0000 mgr.y (mgr.14152) 87 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:46.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:45 smithi175 bash[21527]: audit 2024-03-26T03:36:45.736783+0000 mon.a (mon.0) 289 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T03:36:46.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:45 smithi175 bash[21527]: audit 2024-03-26T03:36:45.740103+0000 mon.a (mon.0) 290 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T03:36:46.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:45 smithi175 bash[21527]: audit 2024-03-26T03:36:45.741078+0000 mon.a (mon.0) 291 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:46.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:45 smithi190 bash[23392]: cluster 2024-03-26T03:36:44.193521+0000 mgr.y (mgr.14152) 87 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:46.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:45 smithi190 bash[23392]: audit 2024-03-26T03:36:45.736783+0000 mon.a (mon.0) 289 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T03:36:46.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:45 smithi190 bash[23392]: audit 2024-03-26T03:36:45.740103+0000 mon.a (mon.0) 290 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T03:36:46.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:45 smithi190 bash[23392]: audit 2024-03-26T03:36:45.741078+0000 mon.a (mon.0) 291 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:36:47.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:46 smithi175 bash[21527]: audit 2024-03-26T03:36:45.732657+0000 mgr.y (mgr.14152) 88 : audit [DBG] from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:36:47.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:46 smithi175 bash[29284]: audit 2024-03-26T03:36:45.732657+0000 mgr.y (mgr.14152) 88 : audit [DBG] from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:36:47.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:46 smithi190 bash[23392]: audit 2024-03-26T03:36:45.732657+0000 mgr.y (mgr.14152) 88 : audit [DBG] from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:36:48.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:47 smithi175 bash[21527]: cluster 2024-03-26T03:36:46.193943+0000 mgr.y (mgr.14152) 89 : cluster [DBG] pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:48.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:47 smithi175 bash[29284]: cluster 2024-03-26T03:36:46.193943+0000 mgr.y (mgr.14152) 89 : cluster [DBG] pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:48.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:47 smithi190 bash[23392]: cluster 2024-03-26T03:36:46.193943+0000 mgr.y (mgr.14152) 89 : cluster [DBG] pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:50.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:49 smithi175 bash[21527]: cluster 2024-03-26T03:36:48.194355+0000 mgr.y (mgr.14152) 90 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:50.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:49 smithi175 bash[29284]: cluster 2024-03-26T03:36:48.194355+0000 mgr.y (mgr.14152) 90 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:50.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:49 smithi190 bash[23392]: cluster 2024-03-26T03:36:48.194355+0000 mgr.y (mgr.14152) 90 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:52.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:51 smithi175 bash[21527]: cluster 2024-03-26T03:36:50.194824+0000 mgr.y (mgr.14152) 91 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:52.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:51 smithi175 bash[29284]: cluster 2024-03-26T03:36:50.194824+0000 mgr.y (mgr.14152) 91 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:52.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:51 smithi190 bash[23392]: cluster 2024-03-26T03:36:50.194824+0000 mgr.y (mgr.14152) 91 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:54.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:53 smithi175 bash[21527]: cluster 2024-03-26T03:36:52.195183+0000 mgr.y (mgr.14152) 92 : cluster [DBG] pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:54.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:53 smithi175 bash[21527]: audit 2024-03-26T03:36:53.916472+0000 mon.c (mon.2) 3 : audit [INF] from='client.? 172.21.15.175:0/462415111' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41a2182b-5eeb-47fa-be41-1d7639df2f35"}]: dispatch 2024-03-26T03:36:54.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:53 smithi175 bash[21527]: audit 2024-03-26T03:36:53.917253+0000 mon.a (mon.0) 292 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41a2182b-5eeb-47fa-be41-1d7639df2f35"}]: dispatch 2024-03-26T03:36:54.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:53 smithi175 bash[21527]: audit 2024-03-26T03:36:53.919137+0000 mon.a (mon.0) 293 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41a2182b-5eeb-47fa-be41-1d7639df2f35"}]': finished 2024-03-26T03:36:54.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:53 smithi175 bash[21527]: cluster 2024-03-26T03:36:53.922003+0000 mon.a (mon.0) 294 : cluster [DBG] osdmap e5: 1 total, 0 up, 1 in 2024-03-26T03:36:54.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:53 smithi175 bash[21527]: audit 2024-03-26T03:36:53.922205+0000 mon.a (mon.0) 295 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T03:36:54.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:53 smithi175 bash[29284]: cluster 2024-03-26T03:36:52.195183+0000 mgr.y (mgr.14152) 92 : cluster [DBG] pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:54.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:53 smithi175 bash[29284]: audit 2024-03-26T03:36:53.916472+0000 mon.c (mon.2) 3 : audit [INF] from='client.? 172.21.15.175:0/462415111' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41a2182b-5eeb-47fa-be41-1d7639df2f35"}]: dispatch 2024-03-26T03:36:54.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:53 smithi175 bash[29284]: audit 2024-03-26T03:36:53.917253+0000 mon.a (mon.0) 292 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41a2182b-5eeb-47fa-be41-1d7639df2f35"}]: dispatch 2024-03-26T03:36:54.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:53 smithi175 bash[29284]: audit 2024-03-26T03:36:53.919137+0000 mon.a (mon.0) 293 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41a2182b-5eeb-47fa-be41-1d7639df2f35"}]': finished 2024-03-26T03:36:54.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:53 smithi175 bash[29284]: cluster 2024-03-26T03:36:53.922003+0000 mon.a (mon.0) 294 : cluster [DBG] osdmap e5: 1 total, 0 up, 1 in 2024-03-26T03:36:54.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:53 smithi175 bash[29284]: audit 2024-03-26T03:36:53.922205+0000 mon.a (mon.0) 295 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T03:36:54.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:53 smithi190 bash[23392]: cluster 2024-03-26T03:36:52.195183+0000 mgr.y (mgr.14152) 92 : cluster [DBG] pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:54.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:53 smithi190 bash[23392]: audit 2024-03-26T03:36:53.916472+0000 mon.c (mon.2) 3 : audit [INF] from='client.? 172.21.15.175:0/462415111' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41a2182b-5eeb-47fa-be41-1d7639df2f35"}]: dispatch 2024-03-26T03:36:54.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:53 smithi190 bash[23392]: audit 2024-03-26T03:36:53.917253+0000 mon.a (mon.0) 292 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41a2182b-5eeb-47fa-be41-1d7639df2f35"}]: dispatch 2024-03-26T03:36:54.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:53 smithi190 bash[23392]: audit 2024-03-26T03:36:53.919137+0000 mon.a (mon.0) 293 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41a2182b-5eeb-47fa-be41-1d7639df2f35"}]': finished 2024-03-26T03:36:54.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:53 smithi190 bash[23392]: cluster 2024-03-26T03:36:53.922003+0000 mon.a (mon.0) 294 : cluster [DBG] osdmap e5: 1 total, 0 up, 1 in 2024-03-26T03:36:54.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:53 smithi190 bash[23392]: audit 2024-03-26T03:36:53.922205+0000 mon.a (mon.0) 295 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T03:36:55.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:54 smithi175 bash[21527]: audit 2024-03-26T03:36:54.608438+0000 mon.c (mon.2) 4 : audit [DBG] from='client.? 172.21.15.175:0/576709619' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T03:36:55.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:54 smithi175 bash[29284]: audit 2024-03-26T03:36:54.608438+0000 mon.c (mon.2) 4 : audit [DBG] from='client.? 172.21.15.175:0/576709619' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T03:36:55.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:54 smithi190 bash[23392]: audit 2024-03-26T03:36:54.608438+0000 mon.c (mon.2) 4 : audit [DBG] from='client.? 172.21.15.175:0/576709619' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T03:36:56.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:55 smithi175 bash[21527]: cluster 2024-03-26T03:36:54.195605+0000 mgr.y (mgr.14152) 93 : cluster [DBG] pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:56.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:55 smithi175 bash[29284]: cluster 2024-03-26T03:36:54.195605+0000 mgr.y (mgr.14152) 93 : cluster [DBG] pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:56.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:55 smithi190 bash[23392]: cluster 2024-03-26T03:36:54.195605+0000 mgr.y (mgr.14152) 93 : cluster [DBG] pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:58.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:57 smithi175 bash[21527]: cluster 2024-03-26T03:36:56.195981+0000 mgr.y (mgr.14152) 94 : cluster [DBG] pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:58.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:57 smithi175 bash[29284]: cluster 2024-03-26T03:36:56.195981+0000 mgr.y (mgr.14152) 94 : cluster [DBG] pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:36:58.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:57 smithi190 bash[23392]: cluster 2024-03-26T03:36:56.195981+0000 mgr.y (mgr.14152) 94 : cluster [DBG] pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:00.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:36:59 smithi175 bash[21527]: cluster 2024-03-26T03:36:58.196452+0000 mgr.y (mgr.14152) 95 : cluster [DBG] pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:00.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:36:59 smithi175 bash[29284]: cluster 2024-03-26T03:36:58.196452+0000 mgr.y (mgr.14152) 95 : cluster [DBG] pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:00.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:36:59 smithi190 bash[23392]: cluster 2024-03-26T03:36:58.196452+0000 mgr.y (mgr.14152) 95 : cluster [DBG] pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:02.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:01 smithi175 bash[21527]: cluster 2024-03-26T03:37:00.196989+0000 mgr.y (mgr.14152) 96 : cluster [DBG] pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:02.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:01 smithi175 bash[29284]: cluster 2024-03-26T03:37:00.196989+0000 mgr.y (mgr.14152) 96 : cluster [DBG] pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:02.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:01 smithi190 bash[23392]: cluster 2024-03-26T03:37:00.196989+0000 mgr.y (mgr.14152) 96 : cluster [DBG] pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:04.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:03 smithi175 bash[21527]: cluster 2024-03-26T03:37:02.197538+0000 mgr.y (mgr.14152) 97 : cluster [DBG] pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:04.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:03 smithi175 bash[29284]: cluster 2024-03-26T03:37:02.197538+0000 mgr.y (mgr.14152) 97 : cluster [DBG] pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:04.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:03 smithi190 bash[23392]: cluster 2024-03-26T03:37:02.197538+0000 mgr.y (mgr.14152) 97 : cluster [DBG] pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:06.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:05 smithi175 bash[29284]: cluster 2024-03-26T03:37:04.197915+0000 mgr.y (mgr.14152) 98 : cluster [DBG] pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:06.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:05 smithi175 bash[21527]: cluster 2024-03-26T03:37:04.197915+0000 mgr.y (mgr.14152) 98 : cluster [DBG] pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:06.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:05 smithi190 bash[23392]: cluster 2024-03-26T03:37:04.197915+0000 mgr.y (mgr.14152) 98 : cluster [DBG] pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:07.292 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:07 smithi175 bash[29284]: audit 2024-03-26T03:37:06.079428+0000 mon.a (mon.0) 296 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-26T03:37:07.292 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:07 smithi175 bash[29284]: audit 2024-03-26T03:37:06.080633+0000 mon.a (mon.0) 297 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:37:07.292 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:07 smithi175 bash[29284]: cephadm 2024-03-26T03:37:06.081854+0000 mgr.y (mgr.14152) 99 : cephadm [INF] Deploying daemon osd.0 on smithi175 2024-03-26T03:37:07.292 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:07 smithi175 bash[21527]: audit 2024-03-26T03:37:06.079428+0000 mon.a (mon.0) 296 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-26T03:37:07.292 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:07 smithi175 bash[21527]: audit 2024-03-26T03:37:06.080633+0000 mon.a (mon.0) 297 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:37:07.292 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:07 smithi175 bash[21527]: cephadm 2024-03-26T03:37:06.081854+0000 mgr.y (mgr.14152) 99 : cephadm [INF] Deploying daemon osd.0 on smithi175 2024-03-26T03:37:07.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:07 smithi190 bash[23392]: audit 2024-03-26T03:37:06.079428+0000 mon.a (mon.0) 296 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-26T03:37:07.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:07 smithi190 bash[23392]: audit 2024-03-26T03:37:06.080633+0000 mon.a (mon.0) 297 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:37:07.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:07 smithi190 bash[23392]: cephadm 2024-03-26T03:37:06.081854+0000 mgr.y (mgr.14152) 99 : cephadm [INF] Deploying daemon osd.0 on smithi175 2024-03-26T03:37:08.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:08 smithi175 bash[29284]: cluster 2024-03-26T03:37:06.198390+0000 mgr.y (mgr.14152) 100 : cluster [DBG] pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:08.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:08 smithi175 bash[21527]: cluster 2024-03-26T03:37:06.198390+0000 mgr.y (mgr.14152) 100 : cluster [DBG] pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:08.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:08 smithi190 bash[23392]: cluster 2024-03-26T03:37:06.198390+0000 mgr.y (mgr.14152) 100 : cluster [DBG] pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:10.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:10 smithi175 bash[21527]: cluster 2024-03-26T03:37:08.198887+0000 mgr.y (mgr.14152) 101 : cluster [DBG] pgmap v63: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:10.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:10 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:37:10.245 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:37:10 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:37:10.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:10 smithi175 bash[29284]: cluster 2024-03-26T03:37:08.198887+0000 mgr.y (mgr.14152) 101 : cluster [DBG] pgmap v63: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:10.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:10 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:37:10.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:10 smithi190 bash[23392]: cluster 2024-03-26T03:37:08.198887+0000 mgr.y (mgr.14152) 101 : cluster [DBG] pgmap v63: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:10.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:10 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:37:10.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:10 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:37:10.745 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:37:10 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:37:11.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:11 smithi175 bash[21527]: cluster 2024-03-26T03:37:10.199192+0000 mgr.y (mgr.14152) 102 : cluster [DBG] pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:11.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:11 smithi175 bash[21527]: audit 2024-03-26T03:37:10.543541+0000 mon.a (mon.0) 298 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:11.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:11 smithi175 bash[21527]: audit 2024-03-26T03:37:10.548828+0000 mon.a (mon.0) 299 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:11.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:11 smithi175 bash[21527]: audit 2024-03-26T03:37:10.577250+0000 mon.a (mon.0) 300 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:37:11.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:11 smithi175 bash[29284]: cluster 2024-03-26T03:37:10.199192+0000 mgr.y (mgr.14152) 102 : cluster [DBG] pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:11.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:11 smithi175 bash[29284]: audit 2024-03-26T03:37:10.543541+0000 mon.a (mon.0) 298 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:11.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:11 smithi175 bash[29284]: audit 2024-03-26T03:37:10.548828+0000 mon.a (mon.0) 299 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:11.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:11 smithi175 bash[29284]: audit 2024-03-26T03:37:10.577250+0000 mon.a (mon.0) 300 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:37:11.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:11 smithi190 bash[23392]: cluster 2024-03-26T03:37:10.199192+0000 mgr.y (mgr.14152) 102 : cluster [DBG] pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:11.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:11 smithi190 bash[23392]: audit 2024-03-26T03:37:10.543541+0000 mon.a (mon.0) 298 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:11.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:11 smithi190 bash[23392]: audit 2024-03-26T03:37:10.548828+0000 mon.a (mon.0) 299 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:11.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:11 smithi190 bash[23392]: audit 2024-03-26T03:37:10.577250+0000 mon.a (mon.0) 300 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:37:13.857 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:13 smithi175 bash[21527]: cluster 2024-03-26T03:37:12.199598+0000 mgr.y (mgr.14152) 103 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:13.857 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:13 smithi175 bash[29284]: cluster 2024-03-26T03:37:12.199598+0000 mgr.y (mgr.14152) 103 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:13.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:13 smithi190 bash[23392]: cluster 2024-03-26T03:37:12.199598+0000 mgr.y (mgr.14152) 103 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:15.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:15 smithi175 bash[21527]: cluster 2024-03-26T03:37:14.199965+0000 mgr.y (mgr.14152) 104 : cluster [DBG] pgmap v66: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:15.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:15 smithi175 bash[29284]: cluster 2024-03-26T03:37:14.199965+0000 mgr.y (mgr.14152) 104 : cluster [DBG] pgmap v66: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:15.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:15 smithi190 bash[23392]: cluster 2024-03-26T03:37:14.199965+0000 mgr.y (mgr.14152) 104 : cluster [DBG] pgmap v66: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:17.972 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:17 smithi175 bash[21527]: cluster 2024-03-26T03:37:16.200402+0000 mgr.y (mgr.14152) 105 : cluster [DBG] pgmap v67: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:17.972 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:17 smithi175 bash[21527]: audit 2024-03-26T03:37:16.821201+0000 mon.c (mon.2) 5 : audit [INF] from='osd.0 [v2:172.21.15.175:6802/241042466,v1:172.21.15.175:6803/241042466]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-26T03:37:17.972 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:17 smithi175 bash[21527]: audit 2024-03-26T03:37:16.821570+0000 mon.a (mon.0) 301 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-26T03:37:17.972 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:17 smithi175 bash[21527]: audit 2024-03-26T03:37:17.413450+0000 mon.a (mon.0) 302 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:17.972 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:17 smithi175 bash[21527]: audit 2024-03-26T03:37:17.418063+0000 mon.a (mon.0) 303 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:17.972 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:17 smithi175 bash[29284]: cluster 2024-03-26T03:37:16.200402+0000 mgr.y (mgr.14152) 105 : cluster [DBG] pgmap v67: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:17.972 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:17 smithi175 bash[29284]: audit 2024-03-26T03:37:16.821201+0000 mon.c (mon.2) 5 : audit [INF] from='osd.0 [v2:172.21.15.175:6802/241042466,v1:172.21.15.175:6803/241042466]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-26T03:37:17.973 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:17 smithi175 bash[29284]: audit 2024-03-26T03:37:16.821570+0000 mon.a (mon.0) 301 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-26T03:37:17.973 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:17 smithi175 bash[29284]: audit 2024-03-26T03:37:17.413450+0000 mon.a (mon.0) 302 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:17.973 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:17 smithi175 bash[29284]: audit 2024-03-26T03:37:17.418063+0000 mon.a (mon.0) 303 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:17.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:17 smithi190 bash[23392]: cluster 2024-03-26T03:37:16.200402+0000 mgr.y (mgr.14152) 105 : cluster [DBG] pgmap v67: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:17.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:17 smithi190 bash[23392]: audit 2024-03-26T03:37:16.821201+0000 mon.c (mon.2) 5 : audit [INF] from='osd.0 [v2:172.21.15.175:6802/241042466,v1:172.21.15.175:6803/241042466]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-26T03:37:17.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:17 smithi190 bash[23392]: audit 2024-03-26T03:37:16.821570+0000 mon.a (mon.0) 301 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-26T03:37:17.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:17 smithi190 bash[23392]: audit 2024-03-26T03:37:17.413450+0000 mon.a (mon.0) 302 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:17.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:17 smithi190 bash[23392]: audit 2024-03-26T03:37:17.418063+0000 mon.a (mon.0) 303 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:18.843 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:18 smithi175 bash[29284]: audit 2024-03-26T03:37:17.564322+0000 mon.a (mon.0) 304 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-26T03:37:18.843 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:18 smithi175 bash[29284]: cluster 2024-03-26T03:37:17.573895+0000 mon.a (mon.0) 305 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2024-03-26T03:37:18.843 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:18 smithi175 bash[29284]: audit 2024-03-26T03:37:17.574558+0000 mon.a (mon.0) 306 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T03:37:18.844 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:18 smithi175 bash[29284]: audit 2024-03-26T03:37:17.574593+0000 mon.c (mon.2) 6 : audit [INF] from='osd.0 [v2:172.21.15.175:6802/241042466,v1:172.21.15.175:6803/241042466]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-26T03:37:18.844 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:18 smithi175 bash[29284]: audit 2024-03-26T03:37:17.575312+0000 mon.a (mon.0) 307 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-26T03:37:18.844 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:18 smithi175 bash[29284]: audit 2024-03-26T03:37:17.987642+0000 mon.a (mon.0) 308 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:37:18.844 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:18 smithi175 bash[29284]: audit 2024-03-26T03:37:17.988470+0000 mon.a (mon.0) 309 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:37:18.844 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:18 smithi175 bash[29284]: audit 2024-03-26T03:37:17.995144+0000 mon.a (mon.0) 310 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:18.844 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:18 smithi175 bash[21527]: audit 2024-03-26T03:37:17.564322+0000 mon.a (mon.0) 304 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-26T03:37:18.844 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:18 smithi175 bash[21527]: cluster 2024-03-26T03:37:17.573895+0000 mon.a (mon.0) 305 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2024-03-26T03:37:18.844 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:18 smithi175 bash[21527]: audit 2024-03-26T03:37:17.574558+0000 mon.a (mon.0) 306 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T03:37:18.844 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:18 smithi175 bash[21527]: audit 2024-03-26T03:37:17.574593+0000 mon.c (mon.2) 6 : audit [INF] from='osd.0 [v2:172.21.15.175:6802/241042466,v1:172.21.15.175:6803/241042466]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-26T03:37:18.844 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:18 smithi175 bash[21527]: audit 2024-03-26T03:37:17.575312+0000 mon.a (mon.0) 307 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-26T03:37:18.844 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:18 smithi175 bash[21527]: audit 2024-03-26T03:37:17.987642+0000 mon.a (mon.0) 308 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:37:18.844 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:18 smithi175 bash[21527]: audit 2024-03-26T03:37:17.988470+0000 mon.a (mon.0) 309 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:37:18.844 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:18 smithi175 bash[21527]: audit 2024-03-26T03:37:17.995144+0000 mon.a (mon.0) 310 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:18.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:18 smithi190 bash[23392]: audit 2024-03-26T03:37:17.564322+0000 mon.a (mon.0) 304 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-26T03:37:18.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:18 smithi190 bash[23392]: cluster 2024-03-26T03:37:17.573895+0000 mon.a (mon.0) 305 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2024-03-26T03:37:18.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:18 smithi190 bash[23392]: audit 2024-03-26T03:37:17.574558+0000 mon.a (mon.0) 306 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T03:37:18.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:18 smithi190 bash[23392]: audit 2024-03-26T03:37:17.574593+0000 mon.c (mon.2) 6 : audit [INF] from='osd.0 [v2:172.21.15.175:6802/241042466,v1:172.21.15.175:6803/241042466]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-26T03:37:18.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:18 smithi190 bash[23392]: audit 2024-03-26T03:37:17.575312+0000 mon.a (mon.0) 307 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-26T03:37:18.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:18 smithi190 bash[23392]: audit 2024-03-26T03:37:17.987642+0000 mon.a (mon.0) 308 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:37:19.000 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:18 smithi190 bash[23392]: audit 2024-03-26T03:37:17.988470+0000 mon.a (mon.0) 309 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:37:19.000 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:18 smithi190 bash[23392]: audit 2024-03-26T03:37:17.995144+0000 mon.a (mon.0) 310 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:19.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:19 smithi175 bash[21527]: cluster 2024-03-26T03:37:18.200806+0000 mgr.y (mgr.14152) 106 : cluster [DBG] pgmap v69: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:19.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:19 smithi175 bash[21527]: audit 2024-03-26T03:37:18.566608+0000 mon.a (mon.0) 311 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-03-26T03:37:19.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:19 smithi175 bash[21527]: cluster 2024-03-26T03:37:18.574369+0000 mon.a (mon.0) 312 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2024-03-26T03:37:19.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:19 smithi175 bash[21527]: audit 2024-03-26T03:37:18.575389+0000 mon.a (mon.0) 313 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T03:37:19.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:19 smithi175 bash[21527]: audit 2024-03-26T03:37:18.587185+0000 mon.a (mon.0) 314 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T03:37:19.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:19 smithi175 bash[29284]: cluster 2024-03-26T03:37:18.200806+0000 mgr.y (mgr.14152) 106 : cluster [DBG] pgmap v69: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:19.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:19 smithi175 bash[29284]: audit 2024-03-26T03:37:18.566608+0000 mon.a (mon.0) 311 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-03-26T03:37:19.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:19 smithi175 bash[29284]: cluster 2024-03-26T03:37:18.574369+0000 mon.a (mon.0) 312 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2024-03-26T03:37:19.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:19 smithi175 bash[29284]: audit 2024-03-26T03:37:18.575389+0000 mon.a (mon.0) 313 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T03:37:19.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:19 smithi175 bash[29284]: audit 2024-03-26T03:37:18.587185+0000 mon.a (mon.0) 314 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T03:37:19.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:19 smithi190 bash[23392]: cluster 2024-03-26T03:37:18.200806+0000 mgr.y (mgr.14152) 106 : cluster [DBG] pgmap v69: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:19.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:19 smithi190 bash[23392]: audit 2024-03-26T03:37:18.566608+0000 mon.a (mon.0) 311 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-03-26T03:37:19.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:19 smithi190 bash[23392]: cluster 2024-03-26T03:37:18.574369+0000 mon.a (mon.0) 312 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2024-03-26T03:37:19.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:19 smithi190 bash[23392]: audit 2024-03-26T03:37:18.575389+0000 mon.a (mon.0) 313 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T03:37:19.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:19 smithi190 bash[23392]: audit 2024-03-26T03:37:18.587185+0000 mon.a (mon.0) 314 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T03:37:20.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:20 smithi175 bash[21527]: cluster 2024-03-26T03:37:17.820354+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2024-03-26T03:37:20.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:20 smithi175 bash[21527]: cluster 2024-03-26T03:37:17.820420+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2024-03-26T03:37:20.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:20 smithi175 bash[21527]: cluster 2024-03-26T03:37:19.586074+0000 mon.a (mon.0) 315 : cluster [INF] osd.0 [v2:172.21.15.175:6802/241042466,v1:172.21.15.175:6803/241042466] boot 2024-03-26T03:37:20.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:20 smithi175 bash[21527]: cluster 2024-03-26T03:37:19.586169+0000 mon.a (mon.0) 316 : cluster [DBG] osdmap e8: 1 total, 1 up, 1 in 2024-03-26T03:37:20.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:20 smithi175 bash[21527]: audit 2024-03-26T03:37:19.586911+0000 mon.a (mon.0) 317 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T03:37:20.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:20 smithi175 bash[29284]: cluster 2024-03-26T03:37:17.820354+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2024-03-26T03:37:20.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:20 smithi175 bash[29284]: cluster 2024-03-26T03:37:17.820420+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2024-03-26T03:37:20.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:20 smithi175 bash[29284]: cluster 2024-03-26T03:37:19.586074+0000 mon.a (mon.0) 315 : cluster [INF] osd.0 [v2:172.21.15.175:6802/241042466,v1:172.21.15.175:6803/241042466] boot 2024-03-26T03:37:20.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:20 smithi175 bash[29284]: cluster 2024-03-26T03:37:19.586169+0000 mon.a (mon.0) 316 : cluster [DBG] osdmap e8: 1 total, 1 up, 1 in 2024-03-26T03:37:20.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:20 smithi175 bash[29284]: audit 2024-03-26T03:37:19.586911+0000 mon.a (mon.0) 317 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T03:37:20.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:20 smithi190 bash[23392]: cluster 2024-03-26T03:37:17.820354+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2024-03-26T03:37:20.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:20 smithi190 bash[23392]: cluster 2024-03-26T03:37:17.820420+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2024-03-26T03:37:20.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:20 smithi190 bash[23392]: cluster 2024-03-26T03:37:19.586074+0000 mon.a (mon.0) 315 : cluster [INF] osd.0 [v2:172.21.15.175:6802/241042466,v1:172.21.15.175:6803/241042466] boot 2024-03-26T03:37:20.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:20 smithi190 bash[23392]: cluster 2024-03-26T03:37:19.586169+0000 mon.a (mon.0) 316 : cluster [DBG] osdmap e8: 1 total, 1 up, 1 in 2024-03-26T03:37:20.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:20 smithi190 bash[23392]: audit 2024-03-26T03:37:19.586911+0000 mon.a (mon.0) 317 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T03:37:21.094 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:21.090+0000 7fda44ff9700 1 -- 172.21.15.175:0/3143887228 <== mgr.14152 v2:172.21.15.175:6800/2826039805 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fda48062a80 con 0x7fda3406e660 2024-03-26T03:37:21.095 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 0 on host 'smithi175' 2024-03-26T03:37:21.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:21.098+0000 7fda4da8f700 1 -- 172.21.15.175:0/3143887228 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fda3406e660 msgr2=0x7fda34070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:37:21.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:21.098+0000 7fda4da8f700 1 --2- 172.21.15.175:0/3143887228 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fda3406e660 0x7fda34070b20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fda3800b2e0 tx=0x7fda3801a040 comp rx=0 tx=0).stop 2024-03-26T03:37:21.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:21.098+0000 7fda4da8f700 1 -- 172.21.15.175:0/3143887228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda48101710 msgr2=0x7fda4819a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:37:21.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:21.098+0000 7fda4da8f700 1 --2- 172.21.15.175:0/3143887228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda48101710 0x7fda4819a5c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fda3000d980 tx=0x7fda3000dc90 comp rx=0 tx=0).stop 2024-03-26T03:37:21.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:21.098+0000 7fda4da8f700 1 -- 172.21.15.175:0/3143887228 shutdown_connections 2024-03-26T03:37:21.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:21.098+0000 7fda4da8f700 1 --2- 172.21.15.175:0/3143887228 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fda3406e660 0x7fda34070b20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:37:21.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:21.098+0000 7fda4da8f700 1 --2- 172.21.15.175:0/3143887228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda481020d0 0x7fda481a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:37:21.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:21.098+0000 7fda4da8f700 1 --2- 172.21.15.175:0/3143887228 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda481009a0 0x7fda4819a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:37:21.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:21.098+0000 7fda4da8f700 1 --2- 172.21.15.175:0/3143887228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda48101710 0x7fda4819a5c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:37:21.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:21.098+0000 7fda4da8f700 1 -- 172.21.15.175:0/3143887228 >> 172.21.15.175:0/3143887228 conn(0x7fda480fc300 msgr2=0x7fda480fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:37:21.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:21.098+0000 7fda4da8f700 1 -- 172.21.15.175:0/3143887228 shutdown_connections 2024-03-26T03:37:21.101 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:21.098+0000 7fda4da8f700 1 -- 172.21.15.175:0/3143887228 wait complete. 2024-03-26T03:37:21.800 DEBUG:teuthology.orchestra.run.smithi175:osd.0> sudo journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@osd.0.service 2024-03-26T03:37:21.803 INFO:tasks.cephadm:Deploying osd.1 on smithi175 with /dev/vg_nvme/lv_3... 2024-03-26T03:37:21.803 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-26T03:37:21.859 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:21 smithi175 bash[21527]: cluster 2024-03-26T03:37:20.201251+0000 mgr.y (mgr.14152) 107 : cluster [DBG] pgmap v72: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:21.859 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:21 smithi175 bash[21527]: cluster 2024-03-26T03:37:20.606853+0000 mon.a (mon.0) 318 : cluster [DBG] osdmap e9: 1 total, 1 up, 1 in 2024-03-26T03:37:21.859 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:21 smithi175 bash[21527]: audit 2024-03-26T03:37:21.040542+0000 mon.a (mon.0) 319 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:21.859 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:21 smithi175 bash[21527]: audit 2024-03-26T03:37:21.048816+0000 mon.a (mon.0) 320 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:21.860 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:21 smithi175 bash[21527]: audit 2024-03-26T03:37:21.067180+0000 mon.a (mon.0) 321 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:37:21.860 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:21 smithi175 bash[29284]: cluster 2024-03-26T03:37:20.201251+0000 mgr.y (mgr.14152) 107 : cluster [DBG] pgmap v72: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:21.860 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:21 smithi175 bash[29284]: cluster 2024-03-26T03:37:20.606853+0000 mon.a (mon.0) 318 : cluster [DBG] osdmap e9: 1 total, 1 up, 1 in 2024-03-26T03:37:21.860 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:21 smithi175 bash[29284]: audit 2024-03-26T03:37:21.040542+0000 mon.a (mon.0) 319 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:21.860 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:21 smithi175 bash[29284]: audit 2024-03-26T03:37:21.048816+0000 mon.a (mon.0) 320 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:21.860 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:21 smithi175 bash[29284]: audit 2024-03-26T03:37:21.067180+0000 mon.a (mon.0) 321 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:37:21.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:21 smithi190 bash[23392]: cluster 2024-03-26T03:37:20.201251+0000 mgr.y (mgr.14152) 107 : cluster [DBG] pgmap v72: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T03:37:21.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:21 smithi190 bash[23392]: cluster 2024-03-26T03:37:20.606853+0000 mon.a (mon.0) 318 : cluster [DBG] osdmap e9: 1 total, 1 up, 1 in 2024-03-26T03:37:21.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:21 smithi190 bash[23392]: audit 2024-03-26T03:37:21.040542+0000 mon.a (mon.0) 319 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:21.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:21 smithi190 bash[23392]: audit 2024-03-26T03:37:21.048816+0000 mon.a (mon.0) 320 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:21.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:21 smithi190 bash[23392]: audit 2024-03-26T03:37:21.067180+0000 mon.a (mon.0) 321 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:37:23.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:23 smithi175 bash[21527]: cluster 2024-03-26T03:37:22.201751+0000 mgr.y (mgr.14152) 108 : cluster [DBG] pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:23.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:23 smithi175 bash[29284]: cluster 2024-03-26T03:37:22.201751+0000 mgr.y (mgr.14152) 108 : cluster [DBG] pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:23.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:23 smithi190 bash[23392]: cluster 2024-03-26T03:37:22.201751+0000 mgr.y (mgr.14152) 108 : cluster [DBG] pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:25.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:25 smithi175 bash[21527]: cluster 2024-03-26T03:37:24.202325+0000 mgr.y (mgr.14152) 109 : cluster [DBG] pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:25.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:25 smithi175 bash[29284]: cluster 2024-03-26T03:37:24.202325+0000 mgr.y (mgr.14152) 109 : cluster [DBG] pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:25.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:25 smithi190 bash[23392]: cluster 2024-03-26T03:37:24.202325+0000 mgr.y (mgr.14152) 109 : cluster [DBG] pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:26.045 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:37:27.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:27 smithi175 bash[21527]: cluster 2024-03-26T03:37:26.202931+0000 mgr.y (mgr.14152) 110 : cluster [DBG] pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:27.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:27 smithi175 bash[29284]: cluster 2024-03-26T03:37:26.202931+0000 mgr.y (mgr.14152) 110 : cluster [DBG] pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:27.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:27 smithi190 bash[23392]: cluster 2024-03-26T03:37:26.202931+0000 mgr.y (mgr.14152) 110 : cluster [DBG] pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:29.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:29 smithi175 bash[21527]: cluster 2024-03-26T03:37:28.203523+0000 mgr.y (mgr.14152) 111 : cluster [DBG] pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:29.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:29 smithi175 bash[29284]: cluster 2024-03-26T03:37:28.203523+0000 mgr.y (mgr.14152) 111 : cluster [DBG] pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:29.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:29 smithi190 bash[23392]: cluster 2024-03-26T03:37:28.203523+0000 mgr.y (mgr.14152) 111 : cluster [DBG] pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:30.082 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-26T03:37:30.082 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-26T03:37:30.083 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2024-03-26T03:37:30.083 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2024-03-26T03:37:30.083 INFO:teuthology.orchestra.run.smithi175.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.013298 s, 789 MB/s 2024-03-26T03:37:30.083 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2024-03-26T03:37:30.922 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi175:vg_nvme/lv_3 2024-03-26T03:37:31.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:31 smithi175 bash[29284]: cluster 2024-03-26T03:37:30.203851+0000 mgr.y (mgr.14152) 112 : cluster [DBG] pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:31.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:31 smithi175 bash[29284]: audit 2024-03-26T03:37:31.317687+0000 mon.a (mon.0) 322 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:31.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:31 smithi175 bash[29284]: audit 2024-03-26T03:37:31.326031+0000 mon.a (mon.0) 323 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:31.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:31 smithi175 bash[29284]: audit 2024-03-26T03:37:31.328138+0000 mon.a (mon.0) 324 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:37:31.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:31 smithi175 bash[29284]: audit 2024-03-26T03:37:31.335326+0000 mon.a (mon.0) 325 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:31.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:31 smithi175 bash[29284]: audit 2024-03-26T03:37:31.339324+0000 mon.a (mon.0) 326 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:37:31.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:31 smithi175 bash[29284]: audit 2024-03-26T03:37:31.340995+0000 mon.a (mon.0) 327 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:37:31.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:31 smithi175 bash[29284]: audit 2024-03-26T03:37:31.350022+0000 mon.a (mon.0) 328 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:31.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:31 smithi175 bash[21527]: cluster 2024-03-26T03:37:30.203851+0000 mgr.y (mgr.14152) 112 : cluster [DBG] pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:31.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:31 smithi175 bash[21527]: audit 2024-03-26T03:37:31.317687+0000 mon.a (mon.0) 322 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:31.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:31 smithi175 bash[21527]: audit 2024-03-26T03:37:31.326031+0000 mon.a (mon.0) 323 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:31.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:31 smithi175 bash[21527]: audit 2024-03-26T03:37:31.328138+0000 mon.a (mon.0) 324 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:37:31.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:31 smithi175 bash[21527]: audit 2024-03-26T03:37:31.335326+0000 mon.a (mon.0) 325 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:31.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:31 smithi175 bash[21527]: audit 2024-03-26T03:37:31.339324+0000 mon.a (mon.0) 326 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:37:31.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:31 smithi175 bash[21527]: audit 2024-03-26T03:37:31.340995+0000 mon.a (mon.0) 327 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:37:31.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:31 smithi175 bash[21527]: audit 2024-03-26T03:37:31.350022+0000 mon.a (mon.0) 328 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:31.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:31 smithi190 bash[23392]: cluster 2024-03-26T03:37:30.203851+0000 mgr.y (mgr.14152) 112 : cluster [DBG] pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:31.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:31 smithi190 bash[23392]: audit 2024-03-26T03:37:31.317687+0000 mon.a (mon.0) 322 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:31.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:31 smithi190 bash[23392]: audit 2024-03-26T03:37:31.326031+0000 mon.a (mon.0) 323 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:31.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:31 smithi190 bash[23392]: audit 2024-03-26T03:37:31.328138+0000 mon.a (mon.0) 324 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:37:31.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:31 smithi190 bash[23392]: audit 2024-03-26T03:37:31.335326+0000 mon.a (mon.0) 325 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:31.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:31 smithi190 bash[23392]: audit 2024-03-26T03:37:31.339324+0000 mon.a (mon.0) 326 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:37:31.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:31 smithi190 bash[23392]: audit 2024-03-26T03:37:31.340995+0000 mon.a (mon.0) 327 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:37:31.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:31 smithi190 bash[23392]: audit 2024-03-26T03:37:31.350022+0000 mon.a (mon.0) 328 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:37:32.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:32 smithi175 bash[21527]: cephadm 2024-03-26T03:37:31.309475+0000 mgr.y (mgr.14152) 113 : cephadm [INF] Detected new or changed devices on smithi175 2024-03-26T03:37:32.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:32 smithi175 bash[21527]: cephadm 2024-03-26T03:37:31.329483+0000 mgr.y (mgr.14152) 114 : cephadm [INF] Adjusting osd_memory_target on smithi175 to 16241M 2024-03-26T03:37:32.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:32 smithi175 bash[29284]: cephadm 2024-03-26T03:37:31.309475+0000 mgr.y (mgr.14152) 113 : cephadm [INF] Detected new or changed devices on smithi175 2024-03-26T03:37:32.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:32 smithi175 bash[29284]: cephadm 2024-03-26T03:37:31.329483+0000 mgr.y (mgr.14152) 114 : cephadm [INF] Adjusting osd_memory_target on smithi175 to 16241M 2024-03-26T03:37:32.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:32 smithi190 bash[23392]: cephadm 2024-03-26T03:37:31.309475+0000 mgr.y (mgr.14152) 113 : cephadm [INF] Detected new or changed devices on smithi175 2024-03-26T03:37:32.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:32 smithi190 bash[23392]: cephadm 2024-03-26T03:37:31.329483+0000 mgr.y (mgr.14152) 114 : cephadm [INF] Adjusting osd_memory_target on smithi175 to 16241M 2024-03-26T03:37:33.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:33 smithi175 bash[21527]: cluster 2024-03-26T03:37:32.204395+0000 mgr.y (mgr.14152) 115 : cluster [DBG] pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:33.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:33 smithi175 bash[29284]: cluster 2024-03-26T03:37:32.204395+0000 mgr.y (mgr.14152) 115 : cluster [DBG] pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:33.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:33 smithi190 bash[23392]: cluster 2024-03-26T03:37:32.204395+0000 mgr.y (mgr.14152) 115 : cluster [DBG] pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:35.700 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:37:35.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:35 smithi175 bash[21527]: cluster 2024-03-26T03:37:34.204874+0000 mgr.y (mgr.14152) 116 : cluster [DBG] pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:35.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:35 smithi175 bash[29284]: cluster 2024-03-26T03:37:34.204874+0000 mgr.y (mgr.14152) 116 : cluster [DBG] pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:35.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:35 smithi190 bash[23392]: cluster 2024-03-26T03:37:34.204874+0000 mgr.y (mgr.14152) 116 : cluster [DBG] pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:37.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:37 smithi175 bash[21527]: cluster 2024-03-26T03:37:36.205367+0000 mgr.y (mgr.14152) 117 : cluster [DBG] pgmap v81: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:37.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:37 smithi175 bash[29284]: cluster 2024-03-26T03:37:36.205367+0000 mgr.y (mgr.14152) 117 : cluster [DBG] pgmap v81: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:37.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:37 smithi190 bash[23392]: cluster 2024-03-26T03:37:36.205367+0000 mgr.y (mgr.14152) 117 : cluster [DBG] pgmap v81: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:38.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f43097dd700 1 -- 172.21.15.175:0/1626137043 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4304100f50 msgr2=0x7f4304101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:37:38.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f43097dd700 1 --2- 172.21.15.175:0/1626137043 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4304100f50 0x7f4304101370 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f42fc00b0a0 tx=0x7f42fc00b3b0 comp rx=0 tx=0).stop 2024-03-26T03:37:38.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f43097dd700 1 -- 172.21.15.175:0/1626137043 shutdown_connections 2024-03-26T03:37:38.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f43097dd700 1 --2- 172.21.15.175:0/1626137043 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4304100f50 0x7f4304101370 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:37:38.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f43097dd700 1 --2- 172.21.15.175:0/1626137043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4304102270 0x7f43041026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:37:38.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f43097dd700 1 --2- 172.21.15.175:0/1626137043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43041018b0 0x7f4304101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:37:38.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f43097dd700 1 -- 172.21.15.175:0/1626137043 >> 172.21.15.175:0/1626137043 conn(0x7f43040fcbb0 msgr2=0x7f43040fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:37:38.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f43097dd700 1 -- 172.21.15.175:0/1626137043 shutdown_connections 2024-03-26T03:37:38.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f43097dd700 1 -- 172.21.15.175:0/1626137043 wait complete. 2024-03-26T03:37:38.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f43097dd700 1 Processor -- start 2024-03-26T03:37:38.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f43097dd700 1 -- start start 2024-03-26T03:37:38.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f43097dd700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4304100f50 0x7f4304199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:37:38.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f43097dd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43041018b0 0x7f430419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:37:38.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f43097dd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4304102270 0x7f43041a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:37:38.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f43097dd700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f430410f8e0 con 0x7f4304102270 2024-03-26T03:37:38.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f43097dd700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f430410f760 con 0x7f43041018b0 2024-03-26T03:37:38.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f43097dd700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f430410fa60 con 0x7f4304100f50 2024-03-26T03:37:38.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f4302ffd700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4304100f50 0x7f4304199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:37:38.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f43037fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4304102270 0x7f43041a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:37:38.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f43037fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4304102270 0x7f43041a15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:35512/0 (socket says 172.21.15.175:35512) 2024-03-26T03:37:38.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f43037fe700 1 -- 172.21.15.175:0/4241739282 learned_addr learned my addr 172.21.15.175:0/4241739282 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:37:38.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f4302ffd700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4304100f50 0x7f4304199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:52052/0 (socket says 172.21.15.175:52052) 2024-03-26T03:37:38.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f43027fc700 1 --2- 172.21.15.175:0/4241739282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43041018b0 0x7f430419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:37:38.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f43037fe700 1 -- 172.21.15.175:0/4241739282 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4304100f50 msgr2=0x7f4304199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:37:38.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f43037fe700 1 --2- 172.21.15.175:0/4241739282 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4304100f50 0x7f4304199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:37:38.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f43037fe700 1 -- 172.21.15.175:0/4241739282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43041018b0 msgr2=0x7f430419a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:37:38.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f43037fe700 1 --2- 172.21.15.175:0/4241739282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43041018b0 0x7f430419a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:37:38.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f43037fe700 1 -- 172.21.15.175:0/4241739282 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43041a1c40 con 0x7f4304102270 2024-03-26T03:37:38.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f43027fc700 1 --2- 172.21.15.175:0/4241739282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43041018b0 0x7f430419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T03:37:38.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f43037fe700 1 --2- 172.21.15.175:0/4241739282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4304102270 0x7f43041a15a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f42f400e970 tx=0x7f42f400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:37:38.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f42fbfff700 1 -- 172.21.15.175:0/4241739282 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f400cbd0 con 0x7f4304102270 2024-03-26T03:37:38.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f42fbfff700 1 -- 172.21.15.175:0/4241739282 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f42f400cd70 con 0x7f4304102270 2024-03-26T03:37:38.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f43097dd700 1 -- 172.21.15.175:0/4241739282 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43041a1f30 con 0x7f4304102270 2024-03-26T03:37:38.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f42fbfff700 1 -- 172.21.15.175:0/4241739282 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f40054e0 con 0x7f4304102270 2024-03-26T03:37:38.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.346+0000 7f43097dd700 1 -- 172.21.15.175:0/4241739282 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43041a2390 con 0x7f4304102270 2024-03-26T03:37:38.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.350+0000 7f43097dd700 1 -- 172.21.15.175:0/4241739282 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4304050300 con 0x7f4304102270 2024-03-26T03:37:38.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.350+0000 7f42fbfff700 1 -- 172.21.15.175:0/4241739282 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 14) ==== 92419+0+0 (secure 0 0 0) 0x7f42f4004750 con 0x7f4304102270 2024-03-26T03:37:38.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.350+0000 7f42fbfff700 1 --2- 172.21.15.175:0/4241739282 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f42f006e660 0x7f42f0070b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:37:38.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.350+0000 7f42fbfff700 1 -- 172.21.15.175:0/4241739282 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f42f408d850 con 0x7f4304102270 2024-03-26T03:37:38.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.350+0000 7f4302ffd700 1 --2- 172.21.15.175:0/4241739282 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f42f006e660 0x7f42f0070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:37:38.353 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.350+0000 7f4302ffd700 1 --2- 172.21.15.175:0/4241739282 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f42f006e660 0x7f42f0070b20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f42fc009e60 tx=0x7f42fc00b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:37:38.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.354+0000 7f42fbfff700 1 -- 172.21.15.175:0/4241739282 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f42f405bfb0 con 0x7f4304102270 2024-03-26T03:37:38.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:37:38.566+0000 7f43097dd700 1 -- 172.21.15.175:0/4241739282 --> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4304062a80 con 0x7f42f006e660 2024-03-26T03:37:38.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:38 smithi175 bash[29284]: audit 2024-03-26T03:37:38.578146+0000 mon.a (mon.0) 329 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T03:37:38.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:38 smithi175 bash[29284]: audit 2024-03-26T03:37:38.583728+0000 mon.a (mon.0) 330 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T03:37:38.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:38 smithi175 bash[29284]: audit 2024-03-26T03:37:38.585120+0000 mon.a (mon.0) 331 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:37:38.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:38 smithi175 bash[21527]: audit 2024-03-26T03:37:38.578146+0000 mon.a (mon.0) 329 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T03:37:38.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:38 smithi175 bash[21527]: audit 2024-03-26T03:37:38.583728+0000 mon.a (mon.0) 330 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T03:37:38.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:38 smithi175 bash[21527]: audit 2024-03-26T03:37:38.585120+0000 mon.a (mon.0) 331 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:37:38.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:38 smithi190 bash[23392]: audit 2024-03-26T03:37:38.578146+0000 mon.a (mon.0) 329 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T03:37:38.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:38 smithi190 bash[23392]: audit 2024-03-26T03:37:38.583728+0000 mon.a (mon.0) 330 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T03:37:38.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:38 smithi190 bash[23392]: audit 2024-03-26T03:37:38.585120+0000 mon.a (mon.0) 331 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:37:39.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:39 smithi175 bash[21527]: cluster 2024-03-26T03:37:38.205829+0000 mgr.y (mgr.14152) 118 : cluster [DBG] pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:39.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:39 smithi175 bash[21527]: audit 2024-03-26T03:37:38.571386+0000 mgr.y (mgr.14152) 119 : audit [DBG] from='client.14238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:37:39.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:39 smithi175 bash[29284]: cluster 2024-03-26T03:37:38.205829+0000 mgr.y (mgr.14152) 118 : cluster [DBG] pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:39.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:39 smithi175 bash[29284]: audit 2024-03-26T03:37:38.571386+0000 mgr.y (mgr.14152) 119 : audit [DBG] from='client.14238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:37:39.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:39 smithi190 bash[23392]: cluster 2024-03-26T03:37:38.205829+0000 mgr.y (mgr.14152) 118 : cluster [DBG] pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:39.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:39 smithi190 bash[23392]: audit 2024-03-26T03:37:38.571386+0000 mgr.y (mgr.14152) 119 : audit [DBG] from='client.14238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:37:41.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:41 smithi175 bash[29284]: cluster 2024-03-26T03:37:40.206264+0000 mgr.y (mgr.14152) 120 : cluster [DBG] pgmap v83: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:41.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:41 smithi175 bash[21527]: cluster 2024-03-26T03:37:40.206264+0000 mgr.y (mgr.14152) 120 : cluster [DBG] pgmap v83: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:41.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:41 smithi190 bash[23392]: cluster 2024-03-26T03:37:40.206264+0000 mgr.y (mgr.14152) 120 : cluster [DBG] pgmap v83: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:43.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:43 smithi175 bash[21527]: cluster 2024-03-26T03:37:42.206759+0000 mgr.y (mgr.14152) 121 : cluster [DBG] pgmap v84: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:43.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:43 smithi175 bash[29284]: cluster 2024-03-26T03:37:42.206759+0000 mgr.y (mgr.14152) 121 : cluster [DBG] pgmap v84: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:43.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:43 smithi190 bash[23392]: cluster 2024-03-26T03:37:42.206759+0000 mgr.y (mgr.14152) 121 : cluster [DBG] pgmap v84: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:45.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:45 smithi175 bash[29284]: cluster 2024-03-26T03:37:44.207265+0000 mgr.y (mgr.14152) 122 : cluster [DBG] pgmap v85: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:45.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:45 smithi175 bash[21527]: cluster 2024-03-26T03:37:44.207265+0000 mgr.y (mgr.14152) 122 : cluster [DBG] pgmap v85: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:45.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:45 smithi190 bash[23392]: cluster 2024-03-26T03:37:44.207265+0000 mgr.y (mgr.14152) 122 : cluster [DBG] pgmap v85: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:47.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:47 smithi175 bash[29284]: cluster 2024-03-26T03:37:46.207588+0000 mgr.y (mgr.14152) 123 : cluster [DBG] pgmap v86: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:47.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:47 smithi175 bash[29284]: audit 2024-03-26T03:37:46.901965+0000 mon.a (mon.0) 332 : audit [INF] from='client.? 172.21.15.175:0/1707505012' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fbe50bf-5723-4174-9c85-6010a32bd55b"}]: dispatch 2024-03-26T03:37:47.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:47 smithi175 bash[29284]: audit 2024-03-26T03:37:46.906677+0000 mon.a (mon.0) 333 : audit [INF] from='client.? 172.21.15.175:0/1707505012' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fbe50bf-5723-4174-9c85-6010a32bd55b"}]': finished 2024-03-26T03:37:47.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:47 smithi175 bash[29284]: cluster 2024-03-26T03:37:46.912674+0000 mon.a (mon.0) 334 : cluster [DBG] osdmap e10: 2 total, 1 up, 2 in 2024-03-26T03:37:47.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:47 smithi175 bash[29284]: audit 2024-03-26T03:37:46.912942+0000 mon.a (mon.0) 335 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T03:37:47.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:47 smithi175 bash[29284]: audit 2024-03-26T03:37:47.655424+0000 mon.a (mon.0) 336 : audit [DBG] from='client.? 172.21.15.175:0/210817120' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T03:37:47.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:47 smithi175 bash[21527]: cluster 2024-03-26T03:37:46.207588+0000 mgr.y (mgr.14152) 123 : cluster [DBG] pgmap v86: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:47.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:47 smithi175 bash[21527]: audit 2024-03-26T03:37:46.901965+0000 mon.a (mon.0) 332 : audit [INF] from='client.? 172.21.15.175:0/1707505012' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fbe50bf-5723-4174-9c85-6010a32bd55b"}]: dispatch 2024-03-26T03:37:47.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:47 smithi175 bash[21527]: audit 2024-03-26T03:37:46.906677+0000 mon.a (mon.0) 333 : audit [INF] from='client.? 172.21.15.175:0/1707505012' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fbe50bf-5723-4174-9c85-6010a32bd55b"}]': finished 2024-03-26T03:37:47.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:47 smithi175 bash[21527]: cluster 2024-03-26T03:37:46.912674+0000 mon.a (mon.0) 334 : cluster [DBG] osdmap e10: 2 total, 1 up, 2 in 2024-03-26T03:37:47.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:47 smithi175 bash[21527]: audit 2024-03-26T03:37:46.912942+0000 mon.a (mon.0) 335 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T03:37:47.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:47 smithi175 bash[21527]: audit 2024-03-26T03:37:47.655424+0000 mon.a (mon.0) 336 : audit [DBG] from='client.? 172.21.15.175:0/210817120' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T03:37:47.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:47 smithi190 bash[23392]: cluster 2024-03-26T03:37:46.207588+0000 mgr.y (mgr.14152) 123 : cluster [DBG] pgmap v86: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:47.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:47 smithi190 bash[23392]: audit 2024-03-26T03:37:46.901965+0000 mon.a (mon.0) 332 : audit [INF] from='client.? 172.21.15.175:0/1707505012' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fbe50bf-5723-4174-9c85-6010a32bd55b"}]: dispatch 2024-03-26T03:37:47.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:47 smithi190 bash[23392]: audit 2024-03-26T03:37:46.906677+0000 mon.a (mon.0) 333 : audit [INF] from='client.? 172.21.15.175:0/1707505012' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fbe50bf-5723-4174-9c85-6010a32bd55b"}]': finished 2024-03-26T03:37:47.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:47 smithi190 bash[23392]: cluster 2024-03-26T03:37:46.912674+0000 mon.a (mon.0) 334 : cluster [DBG] osdmap e10: 2 total, 1 up, 2 in 2024-03-26T03:37:47.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:47 smithi190 bash[23392]: audit 2024-03-26T03:37:46.912942+0000 mon.a (mon.0) 335 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T03:37:47.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:47 smithi190 bash[23392]: audit 2024-03-26T03:37:47.655424+0000 mon.a (mon.0) 336 : audit [DBG] from='client.? 172.21.15.175:0/210817120' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T03:37:49.923 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:49 smithi175 bash[29284]: cluster 2024-03-26T03:37:48.208040+0000 mgr.y (mgr.14152) 124 : cluster [DBG] pgmap v88: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:49.924 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:49 smithi175 bash[21527]: cluster 2024-03-26T03:37:48.208040+0000 mgr.y (mgr.14152) 124 : cluster [DBG] pgmap v88: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:49.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:49 smithi190 bash[23392]: cluster 2024-03-26T03:37:48.208040+0000 mgr.y (mgr.14152) 124 : cluster [DBG] pgmap v88: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:51.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:51 smithi175 bash[21527]: cluster 2024-03-26T03:37:50.208550+0000 mgr.y (mgr.14152) 125 : cluster [DBG] pgmap v89: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:51.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:51 smithi175 bash[29284]: cluster 2024-03-26T03:37:50.208550+0000 mgr.y (mgr.14152) 125 : cluster [DBG] pgmap v89: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:51.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:51 smithi190 bash[23392]: cluster 2024-03-26T03:37:50.208550+0000 mgr.y (mgr.14152) 125 : cluster [DBG] pgmap v89: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:53.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:53 smithi175 bash[21527]: cluster 2024-03-26T03:37:52.208994+0000 mgr.y (mgr.14152) 126 : cluster [DBG] pgmap v90: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:53.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:53 smithi175 bash[29284]: cluster 2024-03-26T03:37:52.208994+0000 mgr.y (mgr.14152) 126 : cluster [DBG] pgmap v90: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:53.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:53 smithi190 bash[23392]: cluster 2024-03-26T03:37:52.208994+0000 mgr.y (mgr.14152) 126 : cluster [DBG] pgmap v90: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:55.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:55 smithi175 bash[21527]: cluster 2024-03-26T03:37:54.209562+0000 mgr.y (mgr.14152) 127 : cluster [DBG] pgmap v91: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:55.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:55 smithi175 bash[29284]: cluster 2024-03-26T03:37:54.209562+0000 mgr.y (mgr.14152) 127 : cluster [DBG] pgmap v91: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:55.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:55 smithi190 bash[23392]: cluster 2024-03-26T03:37:54.209562+0000 mgr.y (mgr.14152) 127 : cluster [DBG] pgmap v91: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:57.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:57 smithi175 bash[21527]: cluster 2024-03-26T03:37:56.210056+0000 mgr.y (mgr.14152) 128 : cluster [DBG] pgmap v92: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:57.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:57 smithi175 bash[29284]: cluster 2024-03-26T03:37:56.210056+0000 mgr.y (mgr.14152) 128 : cluster [DBG] pgmap v92: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:57.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:57 smithi190 bash[23392]: cluster 2024-03-26T03:37:56.210056+0000 mgr.y (mgr.14152) 128 : cluster [DBG] pgmap v92: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:59.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:59 smithi175 bash[29284]: cluster 2024-03-26T03:37:58.210483+0000 mgr.y (mgr.14152) 129 : cluster [DBG] pgmap v93: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:59.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:59 smithi175 bash[29284]: audit 2024-03-26T03:37:59.474847+0000 mon.a (mon.0) 337 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-26T03:37:59.957 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:37:59 smithi175 bash[29284]: audit 2024-03-26T03:37:59.476504+0000 mon.a (mon.0) 338 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:37:59.958 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:59 smithi175 bash[21527]: cluster 2024-03-26T03:37:58.210483+0000 mgr.y (mgr.14152) 129 : cluster [DBG] pgmap v93: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:59.958 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:59 smithi175 bash[21527]: audit 2024-03-26T03:37:59.474847+0000 mon.a (mon.0) 337 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-26T03:37:59.958 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:37:59 smithi175 bash[21527]: audit 2024-03-26T03:37:59.476504+0000 mon.a (mon.0) 338 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:37:59.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:59 smithi190 bash[23392]: cluster 2024-03-26T03:37:58.210483+0000 mgr.y (mgr.14152) 129 : cluster [DBG] pgmap v93: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:37:59.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:59 smithi190 bash[23392]: audit 2024-03-26T03:37:59.474847+0000 mon.a (mon.0) 337 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-26T03:37:59.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:37:59 smithi190 bash[23392]: audit 2024-03-26T03:37:59.476504+0000 mon.a (mon.0) 338 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:38:00.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:00 smithi175 bash[21527]: cephadm 2024-03-26T03:37:59.478049+0000 mgr.y (mgr.14152) 130 : cephadm [INF] Deploying daemon osd.1 on smithi175 2024-03-26T03:38:00.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:00 smithi175 bash[29284]: cephadm 2024-03-26T03:37:59.478049+0000 mgr.y (mgr.14152) 130 : cephadm [INF] Deploying daemon osd.1 on smithi175 2024-03-26T03:38:00.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:00 smithi190 bash[23392]: cephadm 2024-03-26T03:37:59.478049+0000 mgr.y (mgr.14152) 130 : cephadm [INF] Deploying daemon osd.1 on smithi175 2024-03-26T03:38:01.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:01 smithi175 bash[21527]: cluster 2024-03-26T03:38:00.210922+0000 mgr.y (mgr.14152) 131 : cluster [DBG] pgmap v94: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:38:01.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:01 smithi175 bash[29284]: cluster 2024-03-26T03:38:00.210922+0000 mgr.y (mgr.14152) 131 : cluster [DBG] pgmap v94: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:38:01.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:01 smithi190 bash[23392]: cluster 2024-03-26T03:38:00.210922+0000 mgr.y (mgr.14152) 131 : cluster [DBG] pgmap v94: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:38:03.697 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:03 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:38:03.698 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:38:03 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:38:03.698 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:03 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:38:03.698 INFO:journalctl@ceph.osd.0.smithi175.stdout:Mar 26 03:38:03 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:38:03.994 INFO:journalctl@ceph.osd.0.smithi175.stdout:Mar 26 03:38:03 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:38:03.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:03 smithi175 bash[21527]: cluster 2024-03-26T03:38:02.211235+0000 mgr.y (mgr.14152) 132 : cluster [DBG] pgmap v95: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:38:03.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:03 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:38:03.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:03 smithi175 bash[29284]: cluster 2024-03-26T03:38:02.211235+0000 mgr.y (mgr.14152) 132 : cluster [DBG] pgmap v95: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:38:03.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:03 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:38:03.995 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:38:03 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:38:03.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:03 smithi190 bash[23392]: cluster 2024-03-26T03:38:02.211235+0000 mgr.y (mgr.14152) 132 : cluster [DBG] pgmap v95: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:38:04.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:04 smithi175 bash[29284]: audit 2024-03-26T03:38:03.891256+0000 mon.a (mon.0) 339 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:04.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:04 smithi175 bash[29284]: audit 2024-03-26T03:38:03.895726+0000 mon.a (mon.0) 340 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:04.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:04 smithi175 bash[29284]: audit 2024-03-26T03:38:03.917738+0000 mon.a (mon.0) 341 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:38:04.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:04 smithi175 bash[21527]: audit 2024-03-26T03:38:03.891256+0000 mon.a (mon.0) 339 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:04.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:04 smithi175 bash[21527]: audit 2024-03-26T03:38:03.895726+0000 mon.a (mon.0) 340 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:04.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:04 smithi175 bash[21527]: audit 2024-03-26T03:38:03.917738+0000 mon.a (mon.0) 341 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:38:05.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:04 smithi190 bash[23392]: audit 2024-03-26T03:38:03.891256+0000 mon.a (mon.0) 339 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:05.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:04 smithi190 bash[23392]: audit 2024-03-26T03:38:03.895726+0000 mon.a (mon.0) 340 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:05.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:04 smithi190 bash[23392]: audit 2024-03-26T03:38:03.917738+0000 mon.a (mon.0) 341 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:38:06.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:05 smithi175 bash[21527]: cluster 2024-03-26T03:38:04.211650+0000 mgr.y (mgr.14152) 133 : cluster [DBG] pgmap v96: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:38:06.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:05 smithi175 bash[29284]: cluster 2024-03-26T03:38:04.211650+0000 mgr.y (mgr.14152) 133 : cluster [DBG] pgmap v96: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:38:06.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:05 smithi190 bash[23392]: cluster 2024-03-26T03:38:04.211650+0000 mgr.y (mgr.14152) 133 : cluster [DBG] pgmap v96: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:38:08.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:07 smithi175 bash[21527]: cluster 2024-03-26T03:38:06.212075+0000 mgr.y (mgr.14152) 134 : cluster [DBG] pgmap v97: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:38:08.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:07 smithi175 bash[29284]: cluster 2024-03-26T03:38:06.212075+0000 mgr.y (mgr.14152) 134 : cluster [DBG] pgmap v97: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:38:08.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:07 smithi190 bash[23392]: cluster 2024-03-26T03:38:06.212075+0000 mgr.y (mgr.14152) 134 : cluster [DBG] pgmap v97: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:38:10.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:09 smithi175 bash[21527]: cluster 2024-03-26T03:38:08.212558+0000 mgr.y (mgr.14152) 135 : cluster [DBG] pgmap v98: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:38:10.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:09 smithi175 bash[29284]: cluster 2024-03-26T03:38:08.212558+0000 mgr.y (mgr.14152) 135 : cluster [DBG] pgmap v98: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:38:10.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:09 smithi190 bash[23392]: cluster 2024-03-26T03:38:08.212558+0000 mgr.y (mgr.14152) 135 : cluster [DBG] pgmap v98: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:38:11.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:10 smithi175 bash[21527]: audit 2024-03-26T03:38:10.291531+0000 mon.a (mon.0) 342 : audit [INF] from='osd.1 [v2:172.21.15.175:6810/931168929,v1:172.21.15.175:6811/931168929]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T03:38:11.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:10 smithi175 bash[21527]: audit 2024-03-26T03:38:10.817020+0000 mon.a (mon.0) 343 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:11.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:10 smithi175 bash[21527]: audit 2024-03-26T03:38:10.822561+0000 mon.a (mon.0) 344 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:11.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:10 smithi175 bash[21527]: audit 2024-03-26T03:38:10.823759+0000 mon.a (mon.0) 345 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:38:11.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:10 smithi175 bash[21527]: audit 2024-03-26T03:38:10.824798+0000 mon.a (mon.0) 346 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:38:11.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:10 smithi175 bash[21527]: audit 2024-03-26T03:38:10.832111+0000 mon.a (mon.0) 347 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:11.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:10 smithi175 bash[29284]: audit 2024-03-26T03:38:10.291531+0000 mon.a (mon.0) 342 : audit [INF] from='osd.1 [v2:172.21.15.175:6810/931168929,v1:172.21.15.175:6811/931168929]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T03:38:11.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:10 smithi175 bash[29284]: audit 2024-03-26T03:38:10.817020+0000 mon.a (mon.0) 343 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:11.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:10 smithi175 bash[29284]: audit 2024-03-26T03:38:10.822561+0000 mon.a (mon.0) 344 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:11.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:10 smithi175 bash[29284]: audit 2024-03-26T03:38:10.823759+0000 mon.a (mon.0) 345 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:38:11.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:10 smithi175 bash[29284]: audit 2024-03-26T03:38:10.824798+0000 mon.a (mon.0) 346 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:38:11.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:10 smithi175 bash[29284]: audit 2024-03-26T03:38:10.832111+0000 mon.a (mon.0) 347 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:11.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:10 smithi190 bash[23392]: audit 2024-03-26T03:38:10.291531+0000 mon.a (mon.0) 342 : audit [INF] from='osd.1 [v2:172.21.15.175:6810/931168929,v1:172.21.15.175:6811/931168929]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T03:38:11.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:10 smithi190 bash[23392]: audit 2024-03-26T03:38:10.817020+0000 mon.a (mon.0) 343 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:11.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:10 smithi190 bash[23392]: audit 2024-03-26T03:38:10.822561+0000 mon.a (mon.0) 344 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:11.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:10 smithi190 bash[23392]: audit 2024-03-26T03:38:10.823759+0000 mon.a (mon.0) 345 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:38:11.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:10 smithi190 bash[23392]: audit 2024-03-26T03:38:10.824798+0000 mon.a (mon.0) 346 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:38:11.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:10 smithi190 bash[23392]: audit 2024-03-26T03:38:10.832111+0000 mon.a (mon.0) 347 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:11.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:11 smithi175 bash[29284]: cluster 2024-03-26T03:38:10.213035+0000 mgr.y (mgr.14152) 136 : cluster [DBG] pgmap v99: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:38:11.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:11 smithi175 bash[29284]: audit 2024-03-26T03:38:10.919305+0000 mon.a (mon.0) 348 : audit [INF] from='osd.1 [v2:172.21.15.175:6810/931168929,v1:172.21.15.175:6811/931168929]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-26T03:38:11.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:11 smithi175 bash[29284]: cluster 2024-03-26T03:38:10.924159+0000 mon.a (mon.0) 349 : cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2024-03-26T03:38:11.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:11 smithi175 bash[29284]: audit 2024-03-26T03:38:10.924728+0000 mon.a (mon.0) 350 : audit [INF] from='osd.1 [v2:172.21.15.175:6810/931168929,v1:172.21.15.175:6811/931168929]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-26T03:38:11.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:11 smithi175 bash[29284]: audit 2024-03-26T03:38:10.925186+0000 mon.a (mon.0) 351 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T03:38:11.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:11 smithi175 bash[21527]: cluster 2024-03-26T03:38:10.213035+0000 mgr.y (mgr.14152) 136 : cluster [DBG] pgmap v99: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:38:11.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:11 smithi175 bash[21527]: audit 2024-03-26T03:38:10.919305+0000 mon.a (mon.0) 348 : audit [INF] from='osd.1 [v2:172.21.15.175:6810/931168929,v1:172.21.15.175:6811/931168929]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-26T03:38:11.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:11 smithi175 bash[21527]: cluster 2024-03-26T03:38:10.924159+0000 mon.a (mon.0) 349 : cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2024-03-26T03:38:11.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:11 smithi175 bash[21527]: audit 2024-03-26T03:38:10.924728+0000 mon.a (mon.0) 350 : audit [INF] from='osd.1 [v2:172.21.15.175:6810/931168929,v1:172.21.15.175:6811/931168929]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-26T03:38:11.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:11 smithi175 bash[21527]: audit 2024-03-26T03:38:10.925186+0000 mon.a (mon.0) 351 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T03:38:12.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:11 smithi190 bash[23392]: cluster 2024-03-26T03:38:10.213035+0000 mgr.y (mgr.14152) 136 : cluster [DBG] pgmap v99: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:38:12.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:11 smithi190 bash[23392]: audit 2024-03-26T03:38:10.919305+0000 mon.a (mon.0) 348 : audit [INF] from='osd.1 [v2:172.21.15.175:6810/931168929,v1:172.21.15.175:6811/931168929]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-26T03:38:12.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:11 smithi190 bash[23392]: cluster 2024-03-26T03:38:10.924159+0000 mon.a (mon.0) 349 : cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2024-03-26T03:38:12.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:11 smithi190 bash[23392]: audit 2024-03-26T03:38:10.924728+0000 mon.a (mon.0) 350 : audit [INF] from='osd.1 [v2:172.21.15.175:6810/931168929,v1:172.21.15.175:6811/931168929]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-26T03:38:12.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:11 smithi190 bash[23392]: audit 2024-03-26T03:38:10.925186+0000 mon.a (mon.0) 351 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T03:38:13.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:12 smithi175 bash[29284]: audit 2024-03-26T03:38:11.921391+0000 mon.a (mon.0) 352 : audit [INF] from='osd.1 [v2:172.21.15.175:6810/931168929,v1:172.21.15.175:6811/931168929]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-03-26T03:38:13.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:12 smithi175 bash[29284]: cluster 2024-03-26T03:38:11.927027+0000 mon.a (mon.0) 353 : cluster [DBG] osdmap e12: 2 total, 1 up, 2 in 2024-03-26T03:38:13.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:12 smithi175 bash[29284]: audit 2024-03-26T03:38:11.928365+0000 mon.a (mon.0) 354 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T03:38:13.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:12 smithi175 bash[29284]: audit 2024-03-26T03:38:11.937842+0000 mon.a (mon.0) 355 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T03:38:13.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:12 smithi175 bash[29284]: audit 2024-03-26T03:38:12.406695+0000 mon.a (mon.0) 356 : audit [INF] from='osd.1 [v2:172.21.15.175:6810/931168929,v1:172.21.15.175:6811/931168929]' entity='osd.1' 2024-03-26T03:38:13.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:12 smithi175 bash[21527]: audit 2024-03-26T03:38:11.921391+0000 mon.a (mon.0) 352 : audit [INF] from='osd.1 [v2:172.21.15.175:6810/931168929,v1:172.21.15.175:6811/931168929]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-03-26T03:38:13.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:12 smithi175 bash[21527]: cluster 2024-03-26T03:38:11.927027+0000 mon.a (mon.0) 353 : cluster [DBG] osdmap e12: 2 total, 1 up, 2 in 2024-03-26T03:38:13.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:12 smithi175 bash[21527]: audit 2024-03-26T03:38:11.928365+0000 mon.a (mon.0) 354 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T03:38:13.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:12 smithi175 bash[21527]: audit 2024-03-26T03:38:11.937842+0000 mon.a (mon.0) 355 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T03:38:13.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:12 smithi175 bash[21527]: audit 2024-03-26T03:38:12.406695+0000 mon.a (mon.0) 356 : audit [INF] from='osd.1 [v2:172.21.15.175:6810/931168929,v1:172.21.15.175:6811/931168929]' entity='osd.1' 2024-03-26T03:38:13.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:12 smithi190 bash[23392]: audit 2024-03-26T03:38:11.921391+0000 mon.a (mon.0) 352 : audit [INF] from='osd.1 [v2:172.21.15.175:6810/931168929,v1:172.21.15.175:6811/931168929]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-03-26T03:38:13.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:12 smithi190 bash[23392]: cluster 2024-03-26T03:38:11.927027+0000 mon.a (mon.0) 353 : cluster [DBG] osdmap e12: 2 total, 1 up, 2 in 2024-03-26T03:38:13.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:12 smithi190 bash[23392]: audit 2024-03-26T03:38:11.928365+0000 mon.a (mon.0) 354 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T03:38:13.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:12 smithi190 bash[23392]: audit 2024-03-26T03:38:11.937842+0000 mon.a (mon.0) 355 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T03:38:13.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:12 smithi190 bash[23392]: audit 2024-03-26T03:38:12.406695+0000 mon.a (mon.0) 356 : audit [INF] from='osd.1 [v2:172.21.15.175:6810/931168929,v1:172.21.15.175:6811/931168929]' entity='osd.1' 2024-03-26T03:38:13.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:13 smithi175 bash[21527]: cluster 2024-03-26T03:38:11.242169+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2024-03-26T03:38:13.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:13 smithi175 bash[21527]: cluster 2024-03-26T03:38:11.242272+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2024-03-26T03:38:13.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:13 smithi175 bash[21527]: cluster 2024-03-26T03:38:12.213508+0000 mgr.y (mgr.14152) 137 : cluster [DBG] pgmap v102: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:38:13.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:13 smithi175 bash[21527]: audit 2024-03-26T03:38:12.937410+0000 mon.a (mon.0) 357 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T03:38:13.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:13 smithi175 bash[21527]: cluster 2024-03-26T03:38:13.409350+0000 mon.a (mon.0) 358 : cluster [INF] osd.1 [v2:172.21.15.175:6810/931168929,v1:172.21.15.175:6811/931168929] boot 2024-03-26T03:38:13.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:13 smithi175 bash[21527]: cluster 2024-03-26T03:38:13.409432+0000 mon.a (mon.0) 359 : cluster [DBG] osdmap e13: 2 total, 2 up, 2 in 2024-03-26T03:38:13.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:13 smithi175 bash[21527]: audit 2024-03-26T03:38:13.410183+0000 mon.a (mon.0) 360 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T03:38:13.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:13 smithi175 bash[29284]: cluster 2024-03-26T03:38:11.242169+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2024-03-26T03:38:13.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:13 smithi175 bash[29284]: cluster 2024-03-26T03:38:11.242272+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2024-03-26T03:38:13.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:13 smithi175 bash[29284]: cluster 2024-03-26T03:38:12.213508+0000 mgr.y (mgr.14152) 137 : cluster [DBG] pgmap v102: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:38:13.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:13 smithi175 bash[29284]: audit 2024-03-26T03:38:12.937410+0000 mon.a (mon.0) 357 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T03:38:13.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:13 smithi175 bash[29284]: cluster 2024-03-26T03:38:13.409350+0000 mon.a (mon.0) 358 : cluster [INF] osd.1 [v2:172.21.15.175:6810/931168929,v1:172.21.15.175:6811/931168929] boot 2024-03-26T03:38:13.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:13 smithi175 bash[29284]: cluster 2024-03-26T03:38:13.409432+0000 mon.a (mon.0) 359 : cluster [DBG] osdmap e13: 2 total, 2 up, 2 in 2024-03-26T03:38:13.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:13 smithi175 bash[29284]: audit 2024-03-26T03:38:13.410183+0000 mon.a (mon.0) 360 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T03:38:14.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:13 smithi190 bash[23392]: cluster 2024-03-26T03:38:11.242169+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2024-03-26T03:38:14.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:13 smithi190 bash[23392]: cluster 2024-03-26T03:38:11.242272+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2024-03-26T03:38:14.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:13 smithi190 bash[23392]: cluster 2024-03-26T03:38:12.213508+0000 mgr.y (mgr.14152) 137 : cluster [DBG] pgmap v102: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T03:38:14.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:13 smithi190 bash[23392]: audit 2024-03-26T03:38:12.937410+0000 mon.a (mon.0) 357 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T03:38:14.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:13 smithi190 bash[23392]: cluster 2024-03-26T03:38:13.409350+0000 mon.a (mon.0) 358 : cluster [INF] osd.1 [v2:172.21.15.175:6810/931168929,v1:172.21.15.175:6811/931168929] boot 2024-03-26T03:38:14.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:13 smithi190 bash[23392]: cluster 2024-03-26T03:38:13.409432+0000 mon.a (mon.0) 359 : cluster [DBG] osdmap e13: 2 total, 2 up, 2 in 2024-03-26T03:38:14.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:13 smithi190 bash[23392]: audit 2024-03-26T03:38:13.410183+0000 mon.a (mon.0) 360 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T03:38:14.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:14.446+0000 7f42fbfff700 1 -- 172.21.15.175:0/4241739282 <== mgr.14152 v2:172.21.15.175:6800/2826039805 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4304062a80 con 0x7f42f006e660 2024-03-26T03:38:14.447 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 1 on host 'smithi175' 2024-03-26T03:38:14.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:14.446+0000 7f43097dd700 1 -- 172.21.15.175:0/4241739282 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f42f006e660 msgr2=0x7f42f0070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:38:14.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:14.446+0000 7f43097dd700 1 --2- 172.21.15.175:0/4241739282 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f42f006e660 0x7f42f0070b20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f42fc009e60 tx=0x7f42fc00b700 comp rx=0 tx=0).stop 2024-03-26T03:38:14.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:14.446+0000 7f43097dd700 1 -- 172.21.15.175:0/4241739282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4304102270 msgr2=0x7f43041a15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:38:14.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:14.446+0000 7f43097dd700 1 --2- 172.21.15.175:0/4241739282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4304102270 0x7f43041a15a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f42f400e970 tx=0x7f42f400ec80 comp rx=0 tx=0).stop 2024-03-26T03:38:14.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:14.446+0000 7f43097dd700 1 -- 172.21.15.175:0/4241739282 shutdown_connections 2024-03-26T03:38:14.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:14.446+0000 7f43097dd700 1 --2- 172.21.15.175:0/4241739282 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f42f006e660 0x7f42f0070b20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:38:14.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:14.446+0000 7f43097dd700 1 --2- 172.21.15.175:0/4241739282 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4304100f50 0x7f4304199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:38:14.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:14.446+0000 7f43097dd700 1 --2- 172.21.15.175:0/4241739282 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4304102270 0x7f43041a15a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:38:14.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:14.446+0000 7f43097dd700 1 --2- 172.21.15.175:0/4241739282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43041018b0 0x7f430419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:38:14.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:14.446+0000 7f43097dd700 1 -- 172.21.15.175:0/4241739282 >> 172.21.15.175:0/4241739282 conn(0x7f43040fcbb0 msgr2=0x7f43040fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:38:14.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:14.446+0000 7f43097dd700 1 -- 172.21.15.175:0/4241739282 shutdown_connections 2024-03-26T03:38:14.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:14.450+0000 7f43097dd700 1 -- 172.21.15.175:0/4241739282 wait complete. 2024-03-26T03:38:15.102 DEBUG:teuthology.orchestra.run.smithi175:osd.1> sudo journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@osd.1.service 2024-03-26T03:38:15.105 INFO:tasks.cephadm:Deploying osd.2 on smithi175 with /dev/vg_nvme/lv_2... 2024-03-26T03:38:15.105 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-26T03:38:15.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:15 smithi175 bash[21527]: cluster 2024-03-26T03:38:14.214044+0000 mgr.y (mgr.14152) 138 : cluster [DBG] pgmap v104: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:15.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:15 smithi175 bash[21527]: audit 2024-03-26T03:38:14.411339+0000 mon.a (mon.0) 361 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:15.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:15 smithi175 bash[21527]: audit 2024-03-26T03:38:14.420841+0000 mon.a (mon.0) 362 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:15.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:15 smithi175 bash[21527]: audit 2024-03-26T03:38:14.425995+0000 mon.a (mon.0) 363 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:38:15.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:15 smithi175 bash[21527]: cluster 2024-03-26T03:38:14.953888+0000 mon.a (mon.0) 364 : cluster [DBG] osdmap e14: 2 total, 2 up, 2 in 2024-03-26T03:38:15.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:15 smithi175 bash[29284]: cluster 2024-03-26T03:38:14.214044+0000 mgr.y (mgr.14152) 138 : cluster [DBG] pgmap v104: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:15.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:15 smithi175 bash[29284]: audit 2024-03-26T03:38:14.411339+0000 mon.a (mon.0) 361 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:15.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:15 smithi175 bash[29284]: audit 2024-03-26T03:38:14.420841+0000 mon.a (mon.0) 362 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:15.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:15 smithi175 bash[29284]: audit 2024-03-26T03:38:14.425995+0000 mon.a (mon.0) 363 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:38:15.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:15 smithi175 bash[29284]: cluster 2024-03-26T03:38:14.953888+0000 mon.a (mon.0) 364 : cluster [DBG] osdmap e14: 2 total, 2 up, 2 in 2024-03-26T03:38:15.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:15 smithi190 bash[23392]: cluster 2024-03-26T03:38:14.214044+0000 mgr.y (mgr.14152) 138 : cluster [DBG] pgmap v104: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:15.749 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:15 smithi190 bash[23392]: audit 2024-03-26T03:38:14.411339+0000 mon.a (mon.0) 361 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:15.749 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:15 smithi190 bash[23392]: audit 2024-03-26T03:38:14.420841+0000 mon.a (mon.0) 362 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:15.749 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:15 smithi190 bash[23392]: audit 2024-03-26T03:38:14.425995+0000 mon.a (mon.0) 363 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:38:15.749 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:15 smithi190 bash[23392]: cluster 2024-03-26T03:38:14.953888+0000 mon.a (mon.0) 364 : cluster [DBG] osdmap e14: 2 total, 2 up, 2 in 2024-03-26T03:38:17.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:17 smithi175 bash[21527]: cluster 2024-03-26T03:38:16.214621+0000 mgr.y (mgr.14152) 139 : cluster [DBG] pgmap v106: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:17.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:17 smithi175 bash[29284]: cluster 2024-03-26T03:38:16.214621+0000 mgr.y (mgr.14152) 139 : cluster [DBG] pgmap v106: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:17.749 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:17 smithi190 bash[23392]: cluster 2024-03-26T03:38:16.214621+0000 mgr.y (mgr.14152) 139 : cluster [DBG] pgmap v106: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:19.401 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:38:19.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:19 smithi175 bash[29284]: cluster 2024-03-26T03:38:18.215043+0000 mgr.y (mgr.14152) 140 : cluster [DBG] pgmap v107: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:19.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:19 smithi175 bash[21527]: cluster 2024-03-26T03:38:18.215043+0000 mgr.y (mgr.14152) 140 : cluster [DBG] pgmap v107: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:19.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:19 smithi190 bash[23392]: cluster 2024-03-26T03:38:18.215043+0000 mgr.y (mgr.14152) 140 : cluster [DBG] pgmap v107: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:21.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:21 smithi175 bash[21527]: cluster 2024-03-26T03:38:20.215536+0000 mgr.y (mgr.14152) 141 : cluster [DBG] pgmap v108: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:21.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:21 smithi175 bash[29284]: cluster 2024-03-26T03:38:20.215536+0000 mgr.y (mgr.14152) 141 : cluster [DBG] pgmap v108: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:21.749 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:21 smithi190 bash[23392]: cluster 2024-03-26T03:38:20.215536+0000 mgr.y (mgr.14152) 141 : cluster [DBG] pgmap v108: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:23.624 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-26T03:38:23.624 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-26T03:38:23.624 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2024-03-26T03:38:23.624 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2024-03-26T03:38:23.625 INFO:teuthology.orchestra.run.smithi175.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0153492 s, 683 MB/s 2024-03-26T03:38:23.625 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2024-03-26T03:38:23.686 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:23 smithi175 bash[21527]: cluster 2024-03-26T03:38:22.216042+0000 mgr.y (mgr.14152) 142 : cluster [DBG] pgmap v109: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:23.686 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:23 smithi175 bash[29284]: cluster 2024-03-26T03:38:22.216042+0000 mgr.y (mgr.14152) 142 : cluster [DBG] pgmap v109: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:23.749 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:23 smithi190 bash[23392]: cluster 2024-03-26T03:38:22.216042+0000 mgr.y (mgr.14152) 142 : cluster [DBG] pgmap v109: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:24.829 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi175:vg_nvme/lv_2 2024-03-26T03:38:26.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:26 smithi175 bash[21527]: cluster 2024-03-26T03:38:24.216490+0000 mgr.y (mgr.14152) 143 : cluster [DBG] pgmap v110: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:26.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:26 smithi175 bash[21527]: cephadm 2024-03-26T03:38:25.069544+0000 mgr.y (mgr.14152) 144 : cephadm [INF] Detected new or changed devices on smithi175 2024-03-26T03:38:26.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:26 smithi175 bash[21527]: audit 2024-03-26T03:38:25.074004+0000 mon.a (mon.0) 365 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:26.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:26 smithi175 bash[21527]: audit 2024-03-26T03:38:25.078938+0000 mon.a (mon.0) 366 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:26.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:26 smithi175 bash[21527]: audit 2024-03-26T03:38:25.079919+0000 mon.a (mon.0) 367 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:38:26.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:26 smithi175 bash[21527]: audit 2024-03-26T03:38:25.080785+0000 mon.a (mon.0) 368 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:38:26.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:26 smithi175 bash[21527]: cephadm 2024-03-26T03:38:25.081424+0000 mgr.y (mgr.14152) 145 : cephadm [INF] Adjusting osd_memory_target on smithi175 to 8120M 2024-03-26T03:38:26.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:26 smithi175 bash[21527]: audit 2024-03-26T03:38:25.086490+0000 mon.a (mon.0) 369 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:26.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:26 smithi175 bash[21527]: audit 2024-03-26T03:38:25.090047+0000 mon.a (mon.0) 370 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:38:26.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:26 smithi175 bash[21527]: audit 2024-03-26T03:38:25.090926+0000 mon.a (mon.0) 371 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:38:26.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:26 smithi175 bash[21527]: audit 2024-03-26T03:38:25.098227+0000 mon.a (mon.0) 372 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:26.495 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:26 smithi175 bash[29284]: cluster 2024-03-26T03:38:24.216490+0000 mgr.y (mgr.14152) 143 : cluster [DBG] pgmap v110: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:26.495 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:26 smithi175 bash[29284]: cephadm 2024-03-26T03:38:25.069544+0000 mgr.y (mgr.14152) 144 : cephadm [INF] Detected new or changed devices on smithi175 2024-03-26T03:38:26.495 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:26 smithi175 bash[29284]: audit 2024-03-26T03:38:25.074004+0000 mon.a (mon.0) 365 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:26.495 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:26 smithi175 bash[29284]: audit 2024-03-26T03:38:25.078938+0000 mon.a (mon.0) 366 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:26.495 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:26 smithi175 bash[29284]: audit 2024-03-26T03:38:25.079919+0000 mon.a (mon.0) 367 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:38:26.496 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:26 smithi175 bash[29284]: audit 2024-03-26T03:38:25.080785+0000 mon.a (mon.0) 368 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:38:26.496 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:26 smithi175 bash[29284]: cephadm 2024-03-26T03:38:25.081424+0000 mgr.y (mgr.14152) 145 : cephadm [INF] Adjusting osd_memory_target on smithi175 to 8120M 2024-03-26T03:38:26.496 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:26 smithi175 bash[29284]: audit 2024-03-26T03:38:25.086490+0000 mon.a (mon.0) 369 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:26.496 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:26 smithi175 bash[29284]: audit 2024-03-26T03:38:25.090047+0000 mon.a (mon.0) 370 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:38:26.496 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:26 smithi175 bash[29284]: audit 2024-03-26T03:38:25.090926+0000 mon.a (mon.0) 371 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:38:26.496 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:26 smithi175 bash[29284]: audit 2024-03-26T03:38:25.098227+0000 mon.a (mon.0) 372 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:26.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:26 smithi190 bash[23392]: cluster 2024-03-26T03:38:24.216490+0000 mgr.y (mgr.14152) 143 : cluster [DBG] pgmap v110: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:26.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:26 smithi190 bash[23392]: cephadm 2024-03-26T03:38:25.069544+0000 mgr.y (mgr.14152) 144 : cephadm [INF] Detected new or changed devices on smithi175 2024-03-26T03:38:26.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:26 smithi190 bash[23392]: audit 2024-03-26T03:38:25.074004+0000 mon.a (mon.0) 365 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:26.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:26 smithi190 bash[23392]: audit 2024-03-26T03:38:25.078938+0000 mon.a (mon.0) 366 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:26.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:26 smithi190 bash[23392]: audit 2024-03-26T03:38:25.079919+0000 mon.a (mon.0) 367 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:38:26.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:26 smithi190 bash[23392]: audit 2024-03-26T03:38:25.080785+0000 mon.a (mon.0) 368 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:38:26.500 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:26 smithi190 bash[23392]: cephadm 2024-03-26T03:38:25.081424+0000 mgr.y (mgr.14152) 145 : cephadm [INF] Adjusting osd_memory_target on smithi175 to 8120M 2024-03-26T03:38:26.500 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:26 smithi190 bash[23392]: audit 2024-03-26T03:38:25.086490+0000 mon.a (mon.0) 369 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:26.500 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:26 smithi190 bash[23392]: audit 2024-03-26T03:38:25.090047+0000 mon.a (mon.0) 370 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:38:26.500 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:26 smithi190 bash[23392]: audit 2024-03-26T03:38:25.090926+0000 mon.a (mon.0) 371 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:38:26.500 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:26 smithi190 bash[23392]: audit 2024-03-26T03:38:25.098227+0000 mon.a (mon.0) 372 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:28.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:28 smithi175 bash[21527]: cluster 2024-03-26T03:38:26.216939+0000 mgr.y (mgr.14152) 146 : cluster [DBG] pgmap v111: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:28.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:28 smithi175 bash[29284]: cluster 2024-03-26T03:38:26.216939+0000 mgr.y (mgr.14152) 146 : cluster [DBG] pgmap v111: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:28.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:28 smithi190 bash[23392]: cluster 2024-03-26T03:38:26.216939+0000 mgr.y (mgr.14152) 146 : cluster [DBG] pgmap v111: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:29.582 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:38:30.476 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:30 smithi175 bash[21527]: cluster 2024-03-26T03:38:28.217478+0000 mgr.y (mgr.14152) 147 : cluster [DBG] pgmap v112: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:30.476 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:30 smithi175 bash[29284]: cluster 2024-03-26T03:38:28.217478+0000 mgr.y (mgr.14152) 147 : cluster [DBG] pgmap v112: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:30.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:30 smithi190 bash[23392]: cluster 2024-03-26T03:38:28.217478+0000 mgr.y (mgr.14152) 147 : cluster [DBG] pgmap v112: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:32.285 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e906a2700 1 -- 172.21.15.175:0/1217674509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e88102270 msgr2=0x7f8e881026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:38:32.285 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e906a2700 1 --2- 172.21.15.175:0/1217674509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e88102270 0x7f8e881026f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8e8000b0a0 tx=0x7f8e8000b3b0 comp rx=0 tx=0).stop 2024-03-26T03:38:32.285 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e906a2700 1 -- 172.21.15.175:0/1217674509 shutdown_connections 2024-03-26T03:38:32.285 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e906a2700 1 --2- 172.21.15.175:0/1217674509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e88102270 0x7f8e881026f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:38:32.285 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e906a2700 1 --2- 172.21.15.175:0/1217674509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e88100f50 0x7f8e88101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:38:32.285 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e906a2700 1 --2- 172.21.15.175:0/1217674509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e881018b0 0x7f8e88101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:38:32.286 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e906a2700 1 -- 172.21.15.175:0/1217674509 >> 172.21.15.175:0/1217674509 conn(0x7f8e880fcb70 msgr2=0x7f8e880fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:38:32.286 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e906a2700 1 -- 172.21.15.175:0/1217674509 shutdown_connections 2024-03-26T03:38:32.286 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e906a2700 1 -- 172.21.15.175:0/1217674509 wait complete. 2024-03-26T03:38:32.286 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e906a2700 1 Processor -- start 2024-03-26T03:38:32.286 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e906a2700 1 -- start start 2024-03-26T03:38:32.286 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e906a2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e88100f50 0x7f8e88199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:38:32.286 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e906a2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e881018b0 0x7f8e8819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:38:32.286 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e906a2700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e88102270 0x7f8e881a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:38:32.286 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e906a2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8810f930 con 0x7f8e88100f50 2024-03-26T03:38:32.287 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e906a2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8810f7b0 con 0x7f8e881018b0 2024-03-26T03:38:32.287 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e906a2700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8e8810fab0 con 0x7f8e88102270 2024-03-26T03:38:32.287 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e8e43e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e88100f50 0x7f8e88199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:38:32.287 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e8ec3f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e88102270 0x7f8e881a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:38:32.287 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e8e43e700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e88100f50 0x7f8e88199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:57212/0 (socket says 172.21.15.175:57212) 2024-03-26T03:38:32.287 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e8e43e700 1 -- 172.21.15.175:0/3033077209 learned_addr learned my addr 172.21.15.175:0/3033077209 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:38:32.287 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e8dc3d700 1 --2- 172.21.15.175:0/3033077209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e881018b0 0x7f8e8819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:38:32.287 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e8e43e700 1 -- 172.21.15.175:0/3033077209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e88102270 msgr2=0x7f8e881a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:38:32.287 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e8e43e700 1 --2- 172.21.15.175:0/3033077209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e88102270 0x7f8e881a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:38:32.287 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e8e43e700 1 -- 172.21.15.175:0/3033077209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e881018b0 msgr2=0x7f8e8819a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:38:32.287 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e8e43e700 1 --2- 172.21.15.175:0/3033077209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e881018b0 0x7f8e8819a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:38:32.287 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e8e43e700 1 -- 172.21.15.175:0/3033077209 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e881a1bc0 con 0x7f8e88100f50 2024-03-26T03:38:32.287 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e8ec3f700 1 --2- 172.21.15.175:0/3033077209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e88102270 0x7f8e881a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T03:38:32.287 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e8e43e700 1 --2- 172.21.15.175:0/3033077209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e88100f50 0x7f8e88199fb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400eb60 tx=0x7f8e8400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:38:32.288 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e7b7fe700 1 -- 172.21.15.175:0/3033077209 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8400cc90 con 0x7f8e88100f50 2024-03-26T03:38:32.288 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e7b7fe700 1 -- 172.21.15.175:0/3033077209 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f8e8400ce30 con 0x7f8e88100f50 2024-03-26T03:38:32.288 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e906a2700 1 -- 172.21.15.175:0/3033077209 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e881a1eb0 con 0x7f8e88100f50 2024-03-26T03:38:32.288 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e7b7fe700 1 -- 172.21.15.175:0/3033077209 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e84003e60 con 0x7f8e88100f50 2024-03-26T03:38:32.288 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.282+0000 7f8e906a2700 1 -- 172.21.15.175:0/3033077209 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e881a2300 con 0x7f8e88100f50 2024-03-26T03:38:32.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.286+0000 7f8e906a2700 1 -- 172.21.15.175:0/3033077209 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e88050300 con 0x7f8e88100f50 2024-03-26T03:38:32.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.286+0000 7f8e7b7fe700 1 -- 172.21.15.175:0/3033077209 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 14) ==== 92419+0+0 (secure 0 0 0) 0x7f8e84004000 con 0x7f8e88100f50 2024-03-26T03:38:32.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.286+0000 7f8e7b7fe700 1 --2- 172.21.15.175:0/3033077209 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f8e7406e660 0x7f8e74070b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:38:32.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.286+0000 7f8e7b7fe700 1 -- 172.21.15.175:0/3033077209 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2199+0+0 (secure 0 0 0) 0x7f8e8408d9e0 con 0x7f8e88100f50 2024-03-26T03:38:32.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.286+0000 7f8e8dc3d700 1 --2- 172.21.15.175:0/3033077209 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f8e7406e660 0x7f8e74070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:38:32.289 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.286+0000 7f8e8dc3d700 1 --2- 172.21.15.175:0/3033077209 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f8e7406e660 0x7f8e74070b20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00ba60 tx=0x7f8e7c005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:38:32.292 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.290+0000 7f8e7b7fe700 1 -- 172.21.15.175:0/3033077209 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f8e8405bf90 con 0x7f8e88100f50 2024-03-26T03:38:32.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:32 smithi175 bash[21527]: cluster 2024-03-26T03:38:30.217979+0000 mgr.y (mgr.14152) 148 : cluster [DBG] pgmap v113: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:32.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:32 smithi175 bash[29284]: cluster 2024-03-26T03:38:30.217979+0000 mgr.y (mgr.14152) 148 : cluster [DBG] pgmap v113: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:32.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:32 smithi190 bash[23392]: cluster 2024-03-26T03:38:30.217979+0000 mgr.y (mgr.14152) 148 : cluster [DBG] pgmap v113: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:32.511 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:38:32.510+0000 7f8e906a2700 1 -- 172.21.15.175:0/3033077209 --> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8e88062a80 con 0x7f8e7406e660 2024-03-26T03:38:33.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:33 smithi175 bash[21527]: audit 2024-03-26T03:38:32.517721+0000 mon.a (mon.0) 373 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T03:38:33.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:33 smithi175 bash[21527]: audit 2024-03-26T03:38:32.521465+0000 mon.a (mon.0) 374 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T03:38:33.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:33 smithi175 bash[21527]: audit 2024-03-26T03:38:32.522604+0000 mon.a (mon.0) 375 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:38:33.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:33 smithi175 bash[29284]: audit 2024-03-26T03:38:32.517721+0000 mon.a (mon.0) 373 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T03:38:33.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:33 smithi175 bash[29284]: audit 2024-03-26T03:38:32.521465+0000 mon.a (mon.0) 374 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T03:38:33.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:33 smithi175 bash[29284]: audit 2024-03-26T03:38:32.522604+0000 mon.a (mon.0) 375 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:38:33.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:33 smithi190 bash[23392]: audit 2024-03-26T03:38:32.517721+0000 mon.a (mon.0) 373 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T03:38:33.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:33 smithi190 bash[23392]: audit 2024-03-26T03:38:32.521465+0000 mon.a (mon.0) 374 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T03:38:33.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:33 smithi190 bash[23392]: audit 2024-03-26T03:38:32.522604+0000 mon.a (mon.0) 375 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:38:34.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:34 smithi175 bash[21527]: cluster 2024-03-26T03:38:32.218406+0000 mgr.y (mgr.14152) 149 : cluster [DBG] pgmap v114: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:34.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:34 smithi175 bash[21527]: audit 2024-03-26T03:38:32.514043+0000 mgr.y (mgr.14152) 150 : audit [DBG] from='client.14265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:38:34.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:34 smithi175 bash[29284]: cluster 2024-03-26T03:38:32.218406+0000 mgr.y (mgr.14152) 149 : cluster [DBG] pgmap v114: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:34.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:34 smithi175 bash[29284]: audit 2024-03-26T03:38:32.514043+0000 mgr.y (mgr.14152) 150 : audit [DBG] from='client.14265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:38:34.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:34 smithi190 bash[23392]: cluster 2024-03-26T03:38:32.218406+0000 mgr.y (mgr.14152) 149 : cluster [DBG] pgmap v114: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:34.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:34 smithi190 bash[23392]: audit 2024-03-26T03:38:32.514043+0000 mgr.y (mgr.14152) 150 : audit [DBG] from='client.14265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:38:36.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:36 smithi175 bash[21527]: cluster 2024-03-26T03:38:34.218893+0000 mgr.y (mgr.14152) 151 : cluster [DBG] pgmap v115: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:36.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:36 smithi175 bash[29284]: cluster 2024-03-26T03:38:34.218893+0000 mgr.y (mgr.14152) 151 : cluster [DBG] pgmap v115: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:36.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:36 smithi190 bash[23392]: cluster 2024-03-26T03:38:34.218893+0000 mgr.y (mgr.14152) 151 : cluster [DBG] pgmap v115: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:38.383 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:38 smithi175 bash[21527]: cluster 2024-03-26T03:38:36.219360+0000 mgr.y (mgr.14152) 152 : cluster [DBG] pgmap v116: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:38.383 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:38 smithi175 bash[29284]: cluster 2024-03-26T03:38:36.219360+0000 mgr.y (mgr.14152) 152 : cluster [DBG] pgmap v116: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:38.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:38 smithi190 bash[23392]: cluster 2024-03-26T03:38:36.219360+0000 mgr.y (mgr.14152) 152 : cluster [DBG] pgmap v116: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:40.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:40 smithi175 bash[21527]: cluster 2024-03-26T03:38:38.219823+0000 mgr.y (mgr.14152) 153 : cluster [DBG] pgmap v117: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:40.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:40 smithi175 bash[29284]: cluster 2024-03-26T03:38:38.219823+0000 mgr.y (mgr.14152) 153 : cluster [DBG] pgmap v117: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:40.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:40 smithi190 bash[23392]: cluster 2024-03-26T03:38:38.219823+0000 mgr.y (mgr.14152) 153 : cluster [DBG] pgmap v117: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:41.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:41 smithi175 bash[21527]: audit 2024-03-26T03:38:40.910843+0000 mon.c (mon.2) 7 : audit [INF] from='client.? 172.21.15.175:0/3408453580' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02333e36-f498-4ac2-ab6e-5ae1e1148e97"}]: dispatch 2024-03-26T03:38:41.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:41 smithi175 bash[21527]: audit 2024-03-26T03:38:40.911640+0000 mon.a (mon.0) 376 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02333e36-f498-4ac2-ab6e-5ae1e1148e97"}]: dispatch 2024-03-26T03:38:41.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:41 smithi175 bash[21527]: audit 2024-03-26T03:38:40.914945+0000 mon.a (mon.0) 377 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "02333e36-f498-4ac2-ab6e-5ae1e1148e97"}]': finished 2024-03-26T03:38:41.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:41 smithi175 bash[21527]: cluster 2024-03-26T03:38:40.920355+0000 mon.a (mon.0) 378 : cluster [DBG] osdmap e15: 3 total, 2 up, 3 in 2024-03-26T03:38:41.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:41 smithi175 bash[21527]: audit 2024-03-26T03:38:40.920906+0000 mon.a (mon.0) 379 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T03:38:41.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:41 smithi175 bash[29284]: audit 2024-03-26T03:38:40.910843+0000 mon.c (mon.2) 7 : audit [INF] from='client.? 172.21.15.175:0/3408453580' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02333e36-f498-4ac2-ab6e-5ae1e1148e97"}]: dispatch 2024-03-26T03:38:41.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:41 smithi175 bash[29284]: audit 2024-03-26T03:38:40.911640+0000 mon.a (mon.0) 376 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02333e36-f498-4ac2-ab6e-5ae1e1148e97"}]: dispatch 2024-03-26T03:38:41.495 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:41 smithi175 bash[29284]: audit 2024-03-26T03:38:40.914945+0000 mon.a (mon.0) 377 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "02333e36-f498-4ac2-ab6e-5ae1e1148e97"}]': finished 2024-03-26T03:38:41.495 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:41 smithi175 bash[29284]: cluster 2024-03-26T03:38:40.920355+0000 mon.a (mon.0) 378 : cluster [DBG] osdmap e15: 3 total, 2 up, 3 in 2024-03-26T03:38:41.495 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:41 smithi175 bash[29284]: audit 2024-03-26T03:38:40.920906+0000 mon.a (mon.0) 379 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T03:38:41.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:41 smithi190 bash[23392]: audit 2024-03-26T03:38:40.910843+0000 mon.c (mon.2) 7 : audit [INF] from='client.? 172.21.15.175:0/3408453580' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02333e36-f498-4ac2-ab6e-5ae1e1148e97"}]: dispatch 2024-03-26T03:38:41.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:41 smithi190 bash[23392]: audit 2024-03-26T03:38:40.911640+0000 mon.a (mon.0) 376 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "02333e36-f498-4ac2-ab6e-5ae1e1148e97"}]: dispatch 2024-03-26T03:38:41.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:41 smithi190 bash[23392]: audit 2024-03-26T03:38:40.914945+0000 mon.a (mon.0) 377 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "02333e36-f498-4ac2-ab6e-5ae1e1148e97"}]': finished 2024-03-26T03:38:41.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:41 smithi190 bash[23392]: cluster 2024-03-26T03:38:40.920355+0000 mon.a (mon.0) 378 : cluster [DBG] osdmap e15: 3 total, 2 up, 3 in 2024-03-26T03:38:41.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:41 smithi190 bash[23392]: audit 2024-03-26T03:38:40.920906+0000 mon.a (mon.0) 379 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T03:38:42.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:42 smithi175 bash[21527]: cluster 2024-03-26T03:38:40.220224+0000 mgr.y (mgr.14152) 154 : cluster [DBG] pgmap v118: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:42.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:42 smithi175 bash[21527]: audit 2024-03-26T03:38:41.689175+0000 mon.c (mon.2) 8 : audit [DBG] from='client.? 172.21.15.175:0/430660496' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T03:38:42.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:42 smithi175 bash[29284]: cluster 2024-03-26T03:38:40.220224+0000 mgr.y (mgr.14152) 154 : cluster [DBG] pgmap v118: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:42.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:42 smithi175 bash[29284]: audit 2024-03-26T03:38:41.689175+0000 mon.c (mon.2) 8 : audit [DBG] from='client.? 172.21.15.175:0/430660496' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T03:38:42.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:42 smithi190 bash[23392]: cluster 2024-03-26T03:38:40.220224+0000 mgr.y (mgr.14152) 154 : cluster [DBG] pgmap v118: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:42.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:42 smithi190 bash[23392]: audit 2024-03-26T03:38:41.689175+0000 mon.c (mon.2) 8 : audit [DBG] from='client.? 172.21.15.175:0/430660496' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T03:38:44.376 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:44 smithi175 bash[21527]: cluster 2024-03-26T03:38:42.220675+0000 mgr.y (mgr.14152) 155 : cluster [DBG] pgmap v120: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:44.376 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:44 smithi175 bash[29284]: cluster 2024-03-26T03:38:42.220675+0000 mgr.y (mgr.14152) 155 : cluster [DBG] pgmap v120: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:44.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:44 smithi190 bash[23392]: cluster 2024-03-26T03:38:42.220675+0000 mgr.y (mgr.14152) 155 : cluster [DBG] pgmap v120: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:46.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:46 smithi175 bash[21527]: cluster 2024-03-26T03:38:44.221027+0000 mgr.y (mgr.14152) 156 : cluster [DBG] pgmap v121: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:46.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:46 smithi175 bash[29284]: cluster 2024-03-26T03:38:44.221027+0000 mgr.y (mgr.14152) 156 : cluster [DBG] pgmap v121: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:46.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:46 smithi190 bash[23392]: cluster 2024-03-26T03:38:44.221027+0000 mgr.y (mgr.14152) 156 : cluster [DBG] pgmap v121: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:48.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:48 smithi175 bash[21527]: cluster 2024-03-26T03:38:46.221497+0000 mgr.y (mgr.14152) 157 : cluster [DBG] pgmap v122: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:48.495 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:48 smithi175 bash[29284]: cluster 2024-03-26T03:38:46.221497+0000 mgr.y (mgr.14152) 157 : cluster [DBG] pgmap v122: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:48.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:48 smithi190 bash[23392]: cluster 2024-03-26T03:38:46.221497+0000 mgr.y (mgr.14152) 157 : cluster [DBG] pgmap v122: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:50.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:50 smithi175 bash[21527]: cluster 2024-03-26T03:38:48.221956+0000 mgr.y (mgr.14152) 158 : cluster [DBG] pgmap v123: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:50.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:50 smithi175 bash[29284]: cluster 2024-03-26T03:38:48.221956+0000 mgr.y (mgr.14152) 158 : cluster [DBG] pgmap v123: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:50.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:50 smithi190 bash[23392]: cluster 2024-03-26T03:38:48.221956+0000 mgr.y (mgr.14152) 158 : cluster [DBG] pgmap v123: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:52.431 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:52 smithi175 bash[21527]: cluster 2024-03-26T03:38:50.222414+0000 mgr.y (mgr.14152) 159 : cluster [DBG] pgmap v124: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:52.432 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:52 smithi175 bash[29284]: cluster 2024-03-26T03:38:50.222414+0000 mgr.y (mgr.14152) 159 : cluster [DBG] pgmap v124: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:52.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:52 smithi190 bash[23392]: cluster 2024-03-26T03:38:50.222414+0000 mgr.y (mgr.14152) 159 : cluster [DBG] pgmap v124: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:54.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:54 smithi175 bash[29284]: cluster 2024-03-26T03:38:52.222983+0000 mgr.y (mgr.14152) 160 : cluster [DBG] pgmap v125: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:54.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:54 smithi175 bash[29284]: audit 2024-03-26T03:38:53.710506+0000 mon.a (mon.0) 380 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-26T03:38:54.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:54 smithi175 bash[29284]: audit 2024-03-26T03:38:53.712291+0000 mon.a (mon.0) 381 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:38:54.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:54 smithi175 bash[21527]: cluster 2024-03-26T03:38:52.222983+0000 mgr.y (mgr.14152) 160 : cluster [DBG] pgmap v125: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:54.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:54 smithi175 bash[21527]: audit 2024-03-26T03:38:53.710506+0000 mon.a (mon.0) 380 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-26T03:38:54.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:54 smithi175 bash[21527]: audit 2024-03-26T03:38:53.712291+0000 mon.a (mon.0) 381 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:38:54.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:54 smithi190 bash[23392]: cluster 2024-03-26T03:38:52.222983+0000 mgr.y (mgr.14152) 160 : cluster [DBG] pgmap v125: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:54.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:54 smithi190 bash[23392]: audit 2024-03-26T03:38:53.710506+0000 mon.a (mon.0) 380 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-26T03:38:54.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:54 smithi190 bash[23392]: audit 2024-03-26T03:38:53.712291+0000 mon.a (mon.0) 381 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:38:55.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:55 smithi175 bash[21527]: cephadm 2024-03-26T03:38:53.713888+0000 mgr.y (mgr.14152) 161 : cephadm [INF] Deploying daemon osd.2 on smithi175 2024-03-26T03:38:55.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:55 smithi175 bash[29284]: cephadm 2024-03-26T03:38:53.713888+0000 mgr.y (mgr.14152) 161 : cephadm [INF] Deploying daemon osd.2 on smithi175 2024-03-26T03:38:55.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:55 smithi190 bash[23392]: cephadm 2024-03-26T03:38:53.713888+0000 mgr.y (mgr.14152) 161 : cephadm [INF] Deploying daemon osd.2 on smithi175 2024-03-26T03:38:56.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:56 smithi175 bash[29284]: cluster 2024-03-26T03:38:54.223423+0000 mgr.y (mgr.14152) 162 : cluster [DBG] pgmap v126: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:56.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:56 smithi175 bash[21527]: cluster 2024-03-26T03:38:54.223423+0000 mgr.y (mgr.14152) 162 : cluster [DBG] pgmap v126: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:56.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:56 smithi190 bash[23392]: cluster 2024-03-26T03:38:54.223423+0000 mgr.y (mgr.14152) 162 : cluster [DBG] pgmap v126: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:57.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:57 smithi175 bash[29284]: cluster 2024-03-26T03:38:56.223870+0000 mgr.y (mgr.14152) 163 : cluster [DBG] pgmap v127: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:57.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:57 smithi175 bash[21527]: cluster 2024-03-26T03:38:56.223870+0000 mgr.y (mgr.14152) 163 : cluster [DBG] pgmap v127: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:57.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:57 smithi190 bash[23392]: cluster 2024-03-26T03:38:56.223870+0000 mgr.y (mgr.14152) 163 : cluster [DBG] pgmap v127: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:58.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:58 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:38:58.244 INFO:journalctl@ceph.osd.0.smithi175.stdout:Mar 26 03:38:58 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:38:58.244 INFO:journalctl@ceph.osd.1.smithi175.stdout:Mar 26 03:38:58 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:38:58.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:58 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:38:58.245 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:38:58 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:38:58.522 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:38:58 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:38:58.523 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:58 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:38:58.523 INFO:journalctl@ceph.osd.1.smithi175.stdout:Mar 26 03:38:58 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:38:58.523 INFO:journalctl@ceph.osd.0.smithi175.stdout:Mar 26 03:38:58 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:38:58.524 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:58 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:38:59.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:59 smithi175 bash[21527]: cluster 2024-03-26T03:38:58.224270+0000 mgr.y (mgr.14152) 164 : cluster [DBG] pgmap v128: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:59.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:59 smithi175 bash[21527]: audit 2024-03-26T03:38:58.523473+0000 mon.a (mon.0) 382 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:59.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:59 smithi175 bash[21527]: audit 2024-03-26T03:38:58.528226+0000 mon.a (mon.0) 383 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:59.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:38:59 smithi175 bash[21527]: audit 2024-03-26T03:38:58.560181+0000 mon.a (mon.0) 384 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:38:59.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:59 smithi175 bash[29284]: cluster 2024-03-26T03:38:58.224270+0000 mgr.y (mgr.14152) 164 : cluster [DBG] pgmap v128: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:59.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:59 smithi175 bash[29284]: audit 2024-03-26T03:38:58.523473+0000 mon.a (mon.0) 382 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:59.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:59 smithi175 bash[29284]: audit 2024-03-26T03:38:58.528226+0000 mon.a (mon.0) 383 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:59.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:38:59 smithi175 bash[29284]: audit 2024-03-26T03:38:58.560181+0000 mon.a (mon.0) 384 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:38:59.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:59 smithi190 bash[23392]: cluster 2024-03-26T03:38:58.224270+0000 mgr.y (mgr.14152) 164 : cluster [DBG] pgmap v128: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:38:59.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:59 smithi190 bash[23392]: audit 2024-03-26T03:38:58.523473+0000 mon.a (mon.0) 382 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:59.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:59 smithi190 bash[23392]: audit 2024-03-26T03:38:58.528226+0000 mon.a (mon.0) 383 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:38:59.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:38:59 smithi190 bash[23392]: audit 2024-03-26T03:38:58.560181+0000 mon.a (mon.0) 384 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:39:01.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:01 smithi175 bash[21527]: cluster 2024-03-26T03:39:00.224783+0000 mgr.y (mgr.14152) 165 : cluster [DBG] pgmap v129: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:39:01.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:01 smithi175 bash[29284]: cluster 2024-03-26T03:39:00.224783+0000 mgr.y (mgr.14152) 165 : cluster [DBG] pgmap v129: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:39:01.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:01 smithi190 bash[23392]: cluster 2024-03-26T03:39:00.224783+0000 mgr.y (mgr.14152) 165 : cluster [DBG] pgmap v129: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:39:03.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:03 smithi175 bash[21527]: cluster 2024-03-26T03:39:02.225291+0000 mgr.y (mgr.14152) 166 : cluster [DBG] pgmap v130: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:39:03.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:03 smithi175 bash[29284]: cluster 2024-03-26T03:39:02.225291+0000 mgr.y (mgr.14152) 166 : cluster [DBG] pgmap v130: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:39:03.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:03 smithi190 bash[23392]: cluster 2024-03-26T03:39:02.225291+0000 mgr.y (mgr.14152) 166 : cluster [DBG] pgmap v130: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:39:05.152 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:04 smithi175 bash[29284]: audit 2024-03-26T03:39:03.844091+0000 mon.a (mon.0) 385 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:05.152 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:04 smithi175 bash[29284]: audit 2024-03-26T03:39:03.853249+0000 mon.a (mon.0) 386 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:05.152 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:04 smithi175 bash[29284]: audit 2024-03-26T03:39:04.470774+0000 mon.a (mon.0) 387 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:39:05.152 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:04 smithi175 bash[29284]: audit 2024-03-26T03:39:04.472566+0000 mon.a (mon.0) 388 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:39:05.153 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:04 smithi175 bash[29284]: audit 2024-03-26T03:39:04.482115+0000 mon.a (mon.0) 389 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:05.153 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:04 smithi175 bash[21527]: audit 2024-03-26T03:39:03.844091+0000 mon.a (mon.0) 385 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:05.153 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:04 smithi175 bash[21527]: audit 2024-03-26T03:39:03.853249+0000 mon.a (mon.0) 386 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:05.153 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:04 smithi175 bash[21527]: audit 2024-03-26T03:39:04.470774+0000 mon.a (mon.0) 387 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:39:05.153 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:04 smithi175 bash[21527]: audit 2024-03-26T03:39:04.472566+0000 mon.a (mon.0) 388 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:39:05.153 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:04 smithi175 bash[21527]: audit 2024-03-26T03:39:04.482115+0000 mon.a (mon.0) 389 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:05.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:04 smithi190 bash[23392]: audit 2024-03-26T03:39:03.844091+0000 mon.a (mon.0) 385 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:05.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:04 smithi190 bash[23392]: audit 2024-03-26T03:39:03.853249+0000 mon.a (mon.0) 386 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:05.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:04 smithi190 bash[23392]: audit 2024-03-26T03:39:04.470774+0000 mon.a (mon.0) 387 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:39:05.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:04 smithi190 bash[23392]: audit 2024-03-26T03:39:04.472566+0000 mon.a (mon.0) 388 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:39:05.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:04 smithi190 bash[23392]: audit 2024-03-26T03:39:04.482115+0000 mon.a (mon.0) 389 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:06.156 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:05 smithi175 bash[29284]: cluster 2024-03-26T03:39:04.225591+0000 mgr.y (mgr.14152) 167 : cluster [DBG] pgmap v131: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:39:06.156 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:05 smithi175 bash[29284]: audit 2024-03-26T03:39:05.157373+0000 mon.a (mon.0) 390 : audit [INF] from='osd.2 [v2:172.21.15.175:6818/3775031674,v1:172.21.15.175:6819/3775031674]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T03:39:06.156 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:05 smithi175 bash[21527]: cluster 2024-03-26T03:39:04.225591+0000 mgr.y (mgr.14152) 167 : cluster [DBG] pgmap v131: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:39:06.156 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:05 smithi175 bash[21527]: audit 2024-03-26T03:39:05.157373+0000 mon.a (mon.0) 390 : audit [INF] from='osd.2 [v2:172.21.15.175:6818/3775031674,v1:172.21.15.175:6819/3775031674]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T03:39:06.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:05 smithi190 bash[23392]: cluster 2024-03-26T03:39:04.225591+0000 mgr.y (mgr.14152) 167 : cluster [DBG] pgmap v131: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:39:06.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:05 smithi190 bash[23392]: audit 2024-03-26T03:39:05.157373+0000 mon.a (mon.0) 390 : audit [INF] from='osd.2 [v2:172.21.15.175:6818/3775031674,v1:172.21.15.175:6819/3775031674]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T03:39:07.338 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:07 smithi175 bash[21527]: audit 2024-03-26T03:39:05.855227+0000 mon.a (mon.0) 391 : audit [INF] from='osd.2 [v2:172.21.15.175:6818/3775031674,v1:172.21.15.175:6819/3775031674]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-26T03:39:07.338 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:07 smithi175 bash[21527]: cluster 2024-03-26T03:39:05.861585+0000 mon.a (mon.0) 392 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2024-03-26T03:39:07.338 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:07 smithi175 bash[21527]: audit 2024-03-26T03:39:05.862568+0000 mon.a (mon.0) 393 : audit [INF] from='osd.2 [v2:172.21.15.175:6818/3775031674,v1:172.21.15.175:6819/3775031674]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-26T03:39:07.338 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:07 smithi175 bash[21527]: audit 2024-03-26T03:39:05.862878+0000 mon.a (mon.0) 394 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T03:39:07.338 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:07 smithi175 bash[21527]: audit 2024-03-26T03:39:06.857444+0000 mon.a (mon.0) 395 : audit [INF] from='osd.2 [v2:172.21.15.175:6818/3775031674,v1:172.21.15.175:6819/3775031674]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-03-26T03:39:07.338 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:07 smithi175 bash[29284]: audit 2024-03-26T03:39:05.855227+0000 mon.a (mon.0) 391 : audit [INF] from='osd.2 [v2:172.21.15.175:6818/3775031674,v1:172.21.15.175:6819/3775031674]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-26T03:39:07.338 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:07 smithi175 bash[29284]: cluster 2024-03-26T03:39:05.861585+0000 mon.a (mon.0) 392 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2024-03-26T03:39:07.338 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:07 smithi175 bash[29284]: audit 2024-03-26T03:39:05.862568+0000 mon.a (mon.0) 393 : audit [INF] from='osd.2 [v2:172.21.15.175:6818/3775031674,v1:172.21.15.175:6819/3775031674]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-26T03:39:07.339 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:07 smithi175 bash[29284]: audit 2024-03-26T03:39:05.862878+0000 mon.a (mon.0) 394 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T03:39:07.339 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:07 smithi175 bash[29284]: audit 2024-03-26T03:39:06.857444+0000 mon.a (mon.0) 395 : audit [INF] from='osd.2 [v2:172.21.15.175:6818/3775031674,v1:172.21.15.175:6819/3775031674]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-03-26T03:39:07.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:07 smithi190 bash[23392]: audit 2024-03-26T03:39:05.855227+0000 mon.a (mon.0) 391 : audit [INF] from='osd.2 [v2:172.21.15.175:6818/3775031674,v1:172.21.15.175:6819/3775031674]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-26T03:39:07.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:07 smithi190 bash[23392]: cluster 2024-03-26T03:39:05.861585+0000 mon.a (mon.0) 392 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2024-03-26T03:39:07.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:07 smithi190 bash[23392]: audit 2024-03-26T03:39:05.862568+0000 mon.a (mon.0) 393 : audit [INF] from='osd.2 [v2:172.21.15.175:6818/3775031674,v1:172.21.15.175:6819/3775031674]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-26T03:39:07.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:07 smithi190 bash[23392]: audit 2024-03-26T03:39:05.862878+0000 mon.a (mon.0) 394 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T03:39:07.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:07 smithi190 bash[23392]: audit 2024-03-26T03:39:06.857444+0000 mon.a (mon.0) 395 : audit [INF] from='osd.2 [v2:172.21.15.175:6818/3775031674,v1:172.21.15.175:6819/3775031674]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-03-26T03:39:07.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:07.525+0000 7f8e7b7fe700 1 -- 172.21.15.175:0/3033077209 <== mgr.14152 v2:172.21.15.175:6800/2826039805 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8e88062a80 con 0x7f8e7406e660 2024-03-26T03:39:07.526 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 2 on host 'smithi175' 2024-03-26T03:39:07.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:07.529+0000 7f8e906a2700 1 -- 172.21.15.175:0/3033077209 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f8e7406e660 msgr2=0x7f8e74070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:39:07.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:07.529+0000 7f8e906a2700 1 --2- 172.21.15.175:0/3033077209 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f8e7406e660 0x7f8e74070b20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00ba60 tx=0x7f8e7c005d50 comp rx=0 tx=0).stop 2024-03-26T03:39:07.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:07.529+0000 7f8e906a2700 1 -- 172.21.15.175:0/3033077209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e88100f50 msgr2=0x7f8e88199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:39:07.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:07.529+0000 7f8e906a2700 1 --2- 172.21.15.175:0/3033077209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e88100f50 0x7f8e88199fb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400eb60 tx=0x7f8e8400ee70 comp rx=0 tx=0).stop 2024-03-26T03:39:07.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:07.529+0000 7f8e906a2700 1 -- 172.21.15.175:0/3033077209 shutdown_connections 2024-03-26T03:39:07.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:07.529+0000 7f8e906a2700 1 --2- 172.21.15.175:0/3033077209 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f8e7406e660 0x7f8e74070b20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:39:07.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:07.529+0000 7f8e906a2700 1 --2- 172.21.15.175:0/3033077209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e88102270 0x7f8e881a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:39:07.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:07.529+0000 7f8e906a2700 1 --2- 172.21.15.175:0/3033077209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e88100f50 0x7f8e88199fb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:39:07.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:07.529+0000 7f8e906a2700 1 --2- 172.21.15.175:0/3033077209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e881018b0 0x7f8e8819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:39:07.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:07.529+0000 7f8e906a2700 1 -- 172.21.15.175:0/3033077209 >> 172.21.15.175:0/3033077209 conn(0x7f8e880fcb70 msgr2=0x7f8e880fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:39:07.534 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:07.533+0000 7f8e906a2700 1 -- 172.21.15.175:0/3033077209 shutdown_connections 2024-03-26T03:39:07.535 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:07.533+0000 7f8e906a2700 1 -- 172.21.15.175:0/3033077209 wait complete. 2024-03-26T03:39:08.263 DEBUG:teuthology.orchestra.run.smithi175:osd.2> sudo journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@osd.2.service 2024-03-26T03:39:08.265 INFO:tasks.cephadm:Deploying osd.3 on smithi175 with /dev/vg_nvme/lv_1... 2024-03-26T03:39:08.266 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-26T03:39:08.273 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:08 smithi175 bash[21527]: cluster 2024-03-26T03:39:06.225861+0000 mgr.y (mgr.14152) 168 : cluster [DBG] pgmap v133: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:39:08.273 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:08 smithi175 bash[21527]: cluster 2024-03-26T03:39:06.865724+0000 mon.a (mon.0) 396 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2024-03-26T03:39:08.273 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:08 smithi175 bash[21527]: audit 2024-03-26T03:39:06.889176+0000 mon.a (mon.0) 397 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T03:39:08.274 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:08 smithi175 bash[21527]: audit 2024-03-26T03:39:06.896926+0000 mon.a (mon.0) 398 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T03:39:08.274 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:08 smithi175 bash[21527]: audit 2024-03-26T03:39:07.342769+0000 mon.a (mon.0) 399 : audit [INF] from='osd.2 [v2:172.21.15.175:6818/3775031674,v1:172.21.15.175:6819/3775031674]' entity='osd.2' 2024-03-26T03:39:08.274 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:08 smithi175 bash[21527]: audit 2024-03-26T03:39:07.492262+0000 mon.a (mon.0) 400 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:08.274 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:08 smithi175 bash[21527]: audit 2024-03-26T03:39:07.504338+0000 mon.a (mon.0) 401 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:08.274 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:08 smithi175 bash[21527]: audit 2024-03-26T03:39:07.521960+0000 mon.a (mon.0) 402 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:39:08.274 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:08 smithi175 bash[21527]: audit 2024-03-26T03:39:07.895252+0000 mon.a (mon.0) 403 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T03:39:08.274 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:08 smithi175 bash[29284]: cluster 2024-03-26T03:39:06.225861+0000 mgr.y (mgr.14152) 168 : cluster [DBG] pgmap v133: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:39:08.274 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:08 smithi175 bash[29284]: cluster 2024-03-26T03:39:06.865724+0000 mon.a (mon.0) 396 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2024-03-26T03:39:08.274 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:08 smithi175 bash[29284]: audit 2024-03-26T03:39:06.889176+0000 mon.a (mon.0) 397 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T03:39:08.274 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:08 smithi175 bash[29284]: audit 2024-03-26T03:39:06.896926+0000 mon.a (mon.0) 398 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T03:39:08.274 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:08 smithi175 bash[29284]: audit 2024-03-26T03:39:07.342769+0000 mon.a (mon.0) 399 : audit [INF] from='osd.2 [v2:172.21.15.175:6818/3775031674,v1:172.21.15.175:6819/3775031674]' entity='osd.2' 2024-03-26T03:39:08.275 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:08 smithi175 bash[29284]: audit 2024-03-26T03:39:07.492262+0000 mon.a (mon.0) 400 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:08.275 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:08 smithi175 bash[29284]: audit 2024-03-26T03:39:07.504338+0000 mon.a (mon.0) 401 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:08.275 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:08 smithi175 bash[29284]: audit 2024-03-26T03:39:07.521960+0000 mon.a (mon.0) 402 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:39:08.275 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:08 smithi175 bash[29284]: audit 2024-03-26T03:39:07.895252+0000 mon.a (mon.0) 403 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T03:39:08.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:08 smithi190 bash[23392]: cluster 2024-03-26T03:39:06.225861+0000 mgr.y (mgr.14152) 168 : cluster [DBG] pgmap v133: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:39:08.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:08 smithi190 bash[23392]: cluster 2024-03-26T03:39:06.865724+0000 mon.a (mon.0) 396 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2024-03-26T03:39:08.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:08 smithi190 bash[23392]: audit 2024-03-26T03:39:06.889176+0000 mon.a (mon.0) 397 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T03:39:08.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:08 smithi190 bash[23392]: audit 2024-03-26T03:39:06.896926+0000 mon.a (mon.0) 398 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T03:39:08.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:08 smithi190 bash[23392]: audit 2024-03-26T03:39:07.342769+0000 mon.a (mon.0) 399 : audit [INF] from='osd.2 [v2:172.21.15.175:6818/3775031674,v1:172.21.15.175:6819/3775031674]' entity='osd.2' 2024-03-26T03:39:08.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:08 smithi190 bash[23392]: audit 2024-03-26T03:39:07.492262+0000 mon.a (mon.0) 400 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:08.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:08 smithi190 bash[23392]: audit 2024-03-26T03:39:07.504338+0000 mon.a (mon.0) 401 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:08.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:08 smithi190 bash[23392]: audit 2024-03-26T03:39:07.521960+0000 mon.a (mon.0) 402 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:39:08.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:08 smithi190 bash[23392]: audit 2024-03-26T03:39:07.895252+0000 mon.a (mon.0) 403 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T03:39:09.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:09 smithi175 bash[29284]: cluster 2024-03-26T03:39:06.127345+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2024-03-26T03:39:09.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:09 smithi175 bash[29284]: cluster 2024-03-26T03:39:06.127485+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2024-03-26T03:39:09.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:09 smithi175 bash[29284]: cluster 2024-03-26T03:39:08.351041+0000 mon.a (mon.0) 404 : cluster [INF] osd.2 [v2:172.21.15.175:6818/3775031674,v1:172.21.15.175:6819/3775031674] boot 2024-03-26T03:39:09.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:09 smithi175 bash[29284]: cluster 2024-03-26T03:39:08.351109+0000 mon.a (mon.0) 405 : cluster [DBG] osdmap e18: 3 total, 3 up, 3 in 2024-03-26T03:39:09.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:09 smithi175 bash[29284]: audit 2024-03-26T03:39:08.351518+0000 mon.a (mon.0) 406 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T03:39:09.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:09 smithi175 bash[29284]: audit 2024-03-26T03:39:08.362309+0000 mon.a (mon.0) 407 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:39:09.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:09 smithi175 bash[21527]: cluster 2024-03-26T03:39:06.127345+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2024-03-26T03:39:09.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:09 smithi175 bash[21527]: cluster 2024-03-26T03:39:06.127485+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2024-03-26T03:39:09.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:09 smithi175 bash[21527]: cluster 2024-03-26T03:39:08.351041+0000 mon.a (mon.0) 404 : cluster [INF] osd.2 [v2:172.21.15.175:6818/3775031674,v1:172.21.15.175:6819/3775031674] boot 2024-03-26T03:39:09.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:09 smithi175 bash[21527]: cluster 2024-03-26T03:39:08.351109+0000 mon.a (mon.0) 405 : cluster [DBG] osdmap e18: 3 total, 3 up, 3 in 2024-03-26T03:39:09.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:09 smithi175 bash[21527]: audit 2024-03-26T03:39:08.351518+0000 mon.a (mon.0) 406 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T03:39:09.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:09 smithi175 bash[21527]: audit 2024-03-26T03:39:08.362309+0000 mon.a (mon.0) 407 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:39:09.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:09 smithi190 bash[23392]: cluster 2024-03-26T03:39:06.127345+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2024-03-26T03:39:09.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:09 smithi190 bash[23392]: cluster 2024-03-26T03:39:06.127485+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2024-03-26T03:39:09.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:09 smithi190 bash[23392]: cluster 2024-03-26T03:39:08.351041+0000 mon.a (mon.0) 404 : cluster [INF] osd.2 [v2:172.21.15.175:6818/3775031674,v1:172.21.15.175:6819/3775031674] boot 2024-03-26T03:39:09.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:09 smithi190 bash[23392]: cluster 2024-03-26T03:39:08.351109+0000 mon.a (mon.0) 405 : cluster [DBG] osdmap e18: 3 total, 3 up, 3 in 2024-03-26T03:39:09.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:09 smithi190 bash[23392]: audit 2024-03-26T03:39:08.351518+0000 mon.a (mon.0) 406 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T03:39:09.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:09 smithi190 bash[23392]: audit 2024-03-26T03:39:08.362309+0000 mon.a (mon.0) 407 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:39:10.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:10 smithi175 bash[21527]: cluster 2024-03-26T03:39:08.226393+0000 mgr.y (mgr.14152) 169 : cluster [DBG] pgmap v135: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:39:10.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:10 smithi175 bash[21527]: audit 2024-03-26T03:39:09.347621+0000 mon.a (mon.0) 408 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-26T03:39:10.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:10 smithi175 bash[21527]: cluster 2024-03-26T03:39:09.352978+0000 mon.a (mon.0) 409 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2024-03-26T03:39:10.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:10 smithi175 bash[21527]: audit 2024-03-26T03:39:09.354849+0000 mon.a (mon.0) 410 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:39:10.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:10 smithi175 bash[29284]: cluster 2024-03-26T03:39:08.226393+0000 mgr.y (mgr.14152) 169 : cluster [DBG] pgmap v135: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:39:10.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:10 smithi175 bash[29284]: audit 2024-03-26T03:39:09.347621+0000 mon.a (mon.0) 408 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-26T03:39:10.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:10 smithi175 bash[29284]: cluster 2024-03-26T03:39:09.352978+0000 mon.a (mon.0) 409 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2024-03-26T03:39:10.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:10 smithi175 bash[29284]: audit 2024-03-26T03:39:09.354849+0000 mon.a (mon.0) 410 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:39:10.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:10 smithi190 bash[23392]: cluster 2024-03-26T03:39:08.226393+0000 mgr.y (mgr.14152) 169 : cluster [DBG] pgmap v135: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-26T03:39:10.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:10 smithi190 bash[23392]: audit 2024-03-26T03:39:09.347621+0000 mon.a (mon.0) 408 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-26T03:39:10.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:10 smithi190 bash[23392]: cluster 2024-03-26T03:39:09.352978+0000 mon.a (mon.0) 409 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2024-03-26T03:39:10.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:10 smithi190 bash[23392]: audit 2024-03-26T03:39:09.354849+0000 mon.a (mon.0) 410 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:39:11.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:11 smithi175 bash[21527]: cluster 2024-03-26T03:39:10.227163+0000 mgr.y (mgr.14152) 170 : cluster [DBG] pgmap v138: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:11.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:11 smithi175 bash[21527]: audit 2024-03-26T03:39:10.350028+0000 mon.a (mon.0) 411 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-26T03:39:11.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:11 smithi175 bash[21527]: cluster 2024-03-26T03:39:10.357366+0000 mon.a (mon.0) 412 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2024-03-26T03:39:11.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:11 smithi175 bash[21527]: cluster 2024-03-26T03:39:11.033063+0000 mon.a (mon.0) 413 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:39:11.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:11 smithi175 bash[29284]: cluster 2024-03-26T03:39:10.227163+0000 mgr.y (mgr.14152) 170 : cluster [DBG] pgmap v138: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:11.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:11 smithi175 bash[29284]: audit 2024-03-26T03:39:10.350028+0000 mon.a (mon.0) 411 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-26T03:39:11.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:11 smithi175 bash[29284]: cluster 2024-03-26T03:39:10.357366+0000 mon.a (mon.0) 412 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2024-03-26T03:39:11.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:11 smithi175 bash[29284]: cluster 2024-03-26T03:39:11.033063+0000 mon.a (mon.0) 413 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:39:11.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:11 smithi190 bash[23392]: cluster 2024-03-26T03:39:10.227163+0000 mgr.y (mgr.14152) 170 : cluster [DBG] pgmap v138: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:11.749 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:11 smithi190 bash[23392]: audit 2024-03-26T03:39:10.350028+0000 mon.a (mon.0) 411 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-26T03:39:11.749 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:11 smithi190 bash[23392]: cluster 2024-03-26T03:39:10.357366+0000 mon.a (mon.0) 412 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2024-03-26T03:39:11.749 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:11 smithi190 bash[23392]: cluster 2024-03-26T03:39:11.033063+0000 mon.a (mon.0) 413 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:39:12.510 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:39:12.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[29284]: audit 2024-03-26T03:39:11.391360+0000 mon.a (mon.0) 414 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:12.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[29284]: audit 2024-03-26T03:39:11.476727+0000 mon.a (mon.0) 415 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T03:39:12.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[29284]: audit 2024-03-26T03:39:11.513526+0000 mon.a (mon.0) 416 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T03:39:12.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[29284]: audit 2024-03-26T03:39:11.513793+0000 mon.a (mon.0) 417 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T03:39:12.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[29284]: audit 2024-03-26T03:39:11.513994+0000 mon.a (mon.0) 418 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:39:12.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[29284]: audit 2024-03-26T03:39:11.514217+0000 mon.a (mon.0) 419 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:39:12.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[29284]: audit 2024-03-26T03:39:11.516507+0000 mon.b (mon.1) 8 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T03:39:12.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[29284]: audit 2024-03-26T03:39:11.516853+0000 mon.a (mon.0) 420 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T03:39:12.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[29284]: audit 2024-03-26T03:39:11.517058+0000 mon.a (mon.0) 421 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:39:12.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[29284]: audit 2024-03-26T03:39:11.517251+0000 mon.a (mon.0) 422 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:39:12.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[29284]: audit 2024-03-26T03:39:11.571299+0000 mon.b (mon.1) 9 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T03:39:12.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[29284]: audit 2024-03-26T03:39:11.605495+0000 mon.a (mon.0) 423 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:12.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[29284]: audit 2024-03-26T03:39:11.607104+0000 mon.c (mon.2) 9 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T03:39:12.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[29284]: audit 2024-03-26T03:39:11.607457+0000 mon.a (mon.0) 424 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T03:39:12.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[29284]: audit 2024-03-26T03:39:11.607727+0000 mon.a (mon.0) 425 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:39:12.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[29284]: audit 2024-03-26T03:39:11.607932+0000 mon.a (mon.0) 426 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:39:12.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[29284]: audit 2024-03-26T03:39:11.653498+0000 mon.c (mon.2) 10 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T03:39:12.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[29284]: cluster 2024-03-26T03:39:12.045020+0000 mon.a (mon.0) 427 : cluster [DBG] osdmap e21: 3 total, 3 up, 3 in 2024-03-26T03:39:12.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[21527]: audit 2024-03-26T03:39:11.391360+0000 mon.a (mon.0) 414 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:12.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[21527]: audit 2024-03-26T03:39:11.476727+0000 mon.a (mon.0) 415 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T03:39:12.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[21527]: audit 2024-03-26T03:39:11.513526+0000 mon.a (mon.0) 416 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T03:39:12.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[21527]: audit 2024-03-26T03:39:11.513793+0000 mon.a (mon.0) 417 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T03:39:12.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[21527]: audit 2024-03-26T03:39:11.513994+0000 mon.a (mon.0) 418 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:39:12.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[21527]: audit 2024-03-26T03:39:11.514217+0000 mon.a (mon.0) 419 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:39:12.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[21527]: audit 2024-03-26T03:39:11.516507+0000 mon.b (mon.1) 8 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T03:39:12.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[21527]: audit 2024-03-26T03:39:11.516853+0000 mon.a (mon.0) 420 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T03:39:12.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[21527]: audit 2024-03-26T03:39:11.517058+0000 mon.a (mon.0) 421 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:39:12.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[21527]: audit 2024-03-26T03:39:11.517251+0000 mon.a (mon.0) 422 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:39:12.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[21527]: audit 2024-03-26T03:39:11.571299+0000 mon.b (mon.1) 9 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T03:39:12.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[21527]: audit 2024-03-26T03:39:11.605495+0000 mon.a (mon.0) 423 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:12.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[21527]: audit 2024-03-26T03:39:11.607104+0000 mon.c (mon.2) 9 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T03:39:12.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[21527]: audit 2024-03-26T03:39:11.607457+0000 mon.a (mon.0) 424 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T03:39:12.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[21527]: audit 2024-03-26T03:39:11.607727+0000 mon.a (mon.0) 425 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:39:12.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[21527]: audit 2024-03-26T03:39:11.607932+0000 mon.a (mon.0) 426 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:39:12.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[21527]: audit 2024-03-26T03:39:11.653498+0000 mon.c (mon.2) 10 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T03:39:12.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:12 smithi175 bash[21527]: cluster 2024-03-26T03:39:12.045020+0000 mon.a (mon.0) 427 : cluster [DBG] osdmap e21: 3 total, 3 up, 3 in 2024-03-26T03:39:12.749 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:12 smithi190 bash[23392]: audit 2024-03-26T03:39:11.391360+0000 mon.a (mon.0) 414 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:12.749 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:12 smithi190 bash[23392]: audit 2024-03-26T03:39:11.476727+0000 mon.a (mon.0) 415 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T03:39:12.749 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:12 smithi190 bash[23392]: audit 2024-03-26T03:39:11.513526+0000 mon.a (mon.0) 416 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T03:39:12.749 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:12 smithi190 bash[23392]: audit 2024-03-26T03:39:11.513793+0000 mon.a (mon.0) 417 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T03:39:12.749 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:12 smithi190 bash[23392]: audit 2024-03-26T03:39:11.513994+0000 mon.a (mon.0) 418 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:39:12.749 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:12 smithi190 bash[23392]: audit 2024-03-26T03:39:11.514217+0000 mon.a (mon.0) 419 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:39:12.749 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:12 smithi190 bash[23392]: audit 2024-03-26T03:39:11.516507+0000 mon.b (mon.1) 8 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T03:39:12.749 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:12 smithi190 bash[23392]: audit 2024-03-26T03:39:11.516853+0000 mon.a (mon.0) 420 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T03:39:12.749 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:12 smithi190 bash[23392]: audit 2024-03-26T03:39:11.517058+0000 mon.a (mon.0) 421 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:39:12.749 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:12 smithi190 bash[23392]: audit 2024-03-26T03:39:11.517251+0000 mon.a (mon.0) 422 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:39:12.749 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:12 smithi190 bash[23392]: audit 2024-03-26T03:39:11.571299+0000 mon.b (mon.1) 9 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T03:39:12.750 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:12 smithi190 bash[23392]: audit 2024-03-26T03:39:11.605495+0000 mon.a (mon.0) 423 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:12.750 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:12 smithi190 bash[23392]: audit 2024-03-26T03:39:11.607104+0000 mon.c (mon.2) 9 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T03:39:12.750 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:12 smithi190 bash[23392]: audit 2024-03-26T03:39:11.607457+0000 mon.a (mon.0) 424 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T03:39:12.750 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:12 smithi190 bash[23392]: audit 2024-03-26T03:39:11.607727+0000 mon.a (mon.0) 425 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:39:12.750 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:12 smithi190 bash[23392]: audit 2024-03-26T03:39:11.607932+0000 mon.a (mon.0) 426 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:39:12.750 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:12 smithi190 bash[23392]: audit 2024-03-26T03:39:11.653498+0000 mon.c (mon.2) 10 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T03:39:12.750 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:12 smithi190 bash[23392]: cluster 2024-03-26T03:39:12.045020+0000 mon.a (mon.0) 427 : cluster [DBG] osdmap e21: 3 total, 3 up, 3 in 2024-03-26T03:39:13.684 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:13 smithi175 bash[21527]: cluster 2024-03-26T03:39:12.227835+0000 mgr.y (mgr.14152) 171 : cluster [DBG] pgmap v141: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:13.684 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:13 smithi175 bash[21527]: cluster 2024-03-26T03:39:13.037207+0000 mon.a (mon.0) 428 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-26T03:39:13.684 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:13 smithi175 bash[21527]: cluster 2024-03-26T03:39:13.037287+0000 mon.a (mon.0) 429 : cluster [INF] Cluster is now healthy 2024-03-26T03:39:13.684 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:13 smithi175 bash[29284]: cluster 2024-03-26T03:39:12.227835+0000 mgr.y (mgr.14152) 171 : cluster [DBG] pgmap v141: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:13.684 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:13 smithi175 bash[29284]: cluster 2024-03-26T03:39:13.037207+0000 mon.a (mon.0) 428 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-26T03:39:13.684 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:13 smithi175 bash[29284]: cluster 2024-03-26T03:39:13.037287+0000 mon.a (mon.0) 429 : cluster [INF] Cluster is now healthy 2024-03-26T03:39:13.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:13 smithi190 bash[23392]: cluster 2024-03-26T03:39:12.227835+0000 mgr.y (mgr.14152) 171 : cluster [DBG] pgmap v141: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:13.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:13 smithi190 bash[23392]: cluster 2024-03-26T03:39:13.037207+0000 mon.a (mon.0) 428 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-26T03:39:13.749 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:13 smithi190 bash[23392]: cluster 2024-03-26T03:39:13.037287+0000 mon.a (mon.0) 429 : cluster [INF] Cluster is now healthy 2024-03-26T03:39:14.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:14 smithi175 bash[21527]: cluster 2024-03-26T03:39:13.411916+0000 mon.a (mon.0) 430 : cluster [DBG] mgrmap e15: y(active, since 4m), standbys: x 2024-03-26T03:39:14.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:14 smithi175 bash[29284]: cluster 2024-03-26T03:39:13.411916+0000 mon.a (mon.0) 430 : cluster [DBG] mgrmap e15: y(active, since 4m), standbys: x 2024-03-26T03:39:14.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:14 smithi190 bash[23392]: cluster 2024-03-26T03:39:13.411916+0000 mon.a (mon.0) 430 : cluster [DBG] mgrmap e15: y(active, since 4m), standbys: x 2024-03-26T03:39:15.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:15 smithi175 bash[21527]: cluster 2024-03-26T03:39:14.228477+0000 mgr.y (mgr.14152) 172 : cluster [DBG] pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:15.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:15 smithi175 bash[29284]: cluster 2024-03-26T03:39:14.228477+0000 mgr.y (mgr.14152) 172 : cluster [DBG] pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:15.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:15 smithi190 bash[23392]: cluster 2024-03-26T03:39:14.228477+0000 mgr.y (mgr.14152) 172 : cluster [DBG] pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:17.431 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-26T03:39:17.431 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-26T03:39:17.431 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2024-03-26T03:39:17.431 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2024-03-26T03:39:17.431 INFO:teuthology.orchestra.run.smithi175.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0159234 s, 659 MB/s 2024-03-26T03:39:17.431 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2024-03-26T03:39:17.698 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:17 smithi175 bash[29284]: cluster 2024-03-26T03:39:16.228970+0000 mgr.y (mgr.14152) 173 : cluster [DBG] pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:17.698 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:17 smithi175 bash[21527]: cluster 2024-03-26T03:39:16.228970+0000 mgr.y (mgr.14152) 173 : cluster [DBG] pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:17.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:17 smithi190 bash[23392]: cluster 2024-03-26T03:39:16.228970+0000 mgr.y (mgr.14152) 173 : cluster [DBG] pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:18.297 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi175:vg_nvme/lv_1 2024-03-26T03:39:19.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:19 smithi175 bash[21527]: cluster 2024-03-26T03:39:18.229493+0000 mgr.y (mgr.14152) 174 : cluster [DBG] pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:19.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:19 smithi175 bash[21527]: cephadm 2024-03-26T03:39:18.732692+0000 mgr.y (mgr.14152) 175 : cephadm [INF] Detected new or changed devices on smithi175 2024-03-26T03:39:19.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:19 smithi175 bash[21527]: audit 2024-03-26T03:39:18.742098+0000 mon.a (mon.0) 431 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:19.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:19 smithi175 bash[21527]: audit 2024-03-26T03:39:18.753040+0000 mon.a (mon.0) 432 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:19.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:19 smithi175 bash[21527]: audit 2024-03-26T03:39:18.756085+0000 mon.a (mon.0) 433 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:39:19.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:19 smithi175 bash[21527]: audit 2024-03-26T03:39:18.757820+0000 mon.a (mon.0) 434 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:39:19.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:19 smithi175 bash[21527]: audit 2024-03-26T03:39:18.759459+0000 mon.a (mon.0) 435 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:39:19.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:19 smithi175 bash[21527]: cephadm 2024-03-26T03:39:18.760813+0000 mgr.y (mgr.14152) 176 : cephadm [INF] Adjusting osd_memory_target on smithi175 to 5413M 2024-03-26T03:39:19.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:19 smithi175 bash[21527]: audit 2024-03-26T03:39:18.766961+0000 mon.a (mon.0) 436 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:19.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:19 smithi175 bash[21527]: audit 2024-03-26T03:39:18.771973+0000 mon.a (mon.0) 437 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:39:19.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:19 smithi175 bash[21527]: audit 2024-03-26T03:39:18.773878+0000 mon.a (mon.0) 438 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:39:19.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:19 smithi175 bash[21527]: audit 2024-03-26T03:39:18.784252+0000 mon.a (mon.0) 439 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:19.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:19 smithi175 bash[29284]: cluster 2024-03-26T03:39:18.229493+0000 mgr.y (mgr.14152) 174 : cluster [DBG] pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:19.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:19 smithi175 bash[29284]: cephadm 2024-03-26T03:39:18.732692+0000 mgr.y (mgr.14152) 175 : cephadm [INF] Detected new or changed devices on smithi175 2024-03-26T03:39:19.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:19 smithi175 bash[29284]: audit 2024-03-26T03:39:18.742098+0000 mon.a (mon.0) 431 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:19.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:19 smithi175 bash[29284]: audit 2024-03-26T03:39:18.753040+0000 mon.a (mon.0) 432 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:19.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:19 smithi175 bash[29284]: audit 2024-03-26T03:39:18.756085+0000 mon.a (mon.0) 433 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:39:19.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:19 smithi175 bash[29284]: audit 2024-03-26T03:39:18.757820+0000 mon.a (mon.0) 434 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:39:19.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:19 smithi175 bash[29284]: audit 2024-03-26T03:39:18.759459+0000 mon.a (mon.0) 435 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:39:19.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:19 smithi175 bash[29284]: cephadm 2024-03-26T03:39:18.760813+0000 mgr.y (mgr.14152) 176 : cephadm [INF] Adjusting osd_memory_target on smithi175 to 5413M 2024-03-26T03:39:19.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:19 smithi175 bash[29284]: audit 2024-03-26T03:39:18.766961+0000 mon.a (mon.0) 436 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:19.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:19 smithi175 bash[29284]: audit 2024-03-26T03:39:18.771973+0000 mon.a (mon.0) 437 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:39:19.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:19 smithi175 bash[29284]: audit 2024-03-26T03:39:18.773878+0000 mon.a (mon.0) 438 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:39:19.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:19 smithi175 bash[29284]: audit 2024-03-26T03:39:18.784252+0000 mon.a (mon.0) 439 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:19.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:19 smithi190 bash[23392]: cluster 2024-03-26T03:39:18.229493+0000 mgr.y (mgr.14152) 174 : cluster [DBG] pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:19.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:19 smithi190 bash[23392]: cephadm 2024-03-26T03:39:18.732692+0000 mgr.y (mgr.14152) 175 : cephadm [INF] Detected new or changed devices on smithi175 2024-03-26T03:39:19.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:19 smithi190 bash[23392]: audit 2024-03-26T03:39:18.742098+0000 mon.a (mon.0) 431 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:19.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:19 smithi190 bash[23392]: audit 2024-03-26T03:39:18.753040+0000 mon.a (mon.0) 432 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:19.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:19 smithi190 bash[23392]: audit 2024-03-26T03:39:18.756085+0000 mon.a (mon.0) 433 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:39:19.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:19 smithi190 bash[23392]: audit 2024-03-26T03:39:18.757820+0000 mon.a (mon.0) 434 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:39:19.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:19 smithi190 bash[23392]: audit 2024-03-26T03:39:18.759459+0000 mon.a (mon.0) 435 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:39:19.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:19 smithi190 bash[23392]: cephadm 2024-03-26T03:39:18.760813+0000 mgr.y (mgr.14152) 176 : cephadm [INF] Adjusting osd_memory_target on smithi175 to 5413M 2024-03-26T03:39:19.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:19 smithi190 bash[23392]: audit 2024-03-26T03:39:18.766961+0000 mon.a (mon.0) 436 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:19.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:19 smithi190 bash[23392]: audit 2024-03-26T03:39:18.771973+0000 mon.a (mon.0) 437 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:39:19.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:19 smithi190 bash[23392]: audit 2024-03-26T03:39:18.773878+0000 mon.a (mon.0) 438 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:39:19.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:19 smithi190 bash[23392]: audit 2024-03-26T03:39:18.784252+0000 mon.a (mon.0) 439 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:21.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:21 smithi175 bash[21527]: cluster 2024-03-26T03:39:20.230178+0000 mgr.y (mgr.14152) 177 : cluster [DBG] pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:21.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:21 smithi175 bash[29284]: cluster 2024-03-26T03:39:20.230178+0000 mgr.y (mgr.14152) 177 : cluster [DBG] pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:21.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:21 smithi190 bash[23392]: cluster 2024-03-26T03:39:20.230178+0000 mgr.y (mgr.14152) 177 : cluster [DBG] pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:23.047 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:39:23.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:23 smithi190 bash[23392]: cluster 2024-03-26T03:39:22.230849+0000 mgr.y (mgr.14152) 178 : cluster [DBG] pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:24.004 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:23 smithi175 bash[21527]: cluster 2024-03-26T03:39:22.230849+0000 mgr.y (mgr.14152) 178 : cluster [DBG] pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:24.004 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:23 smithi175 bash[29284]: cluster 2024-03-26T03:39:22.230849+0000 mgr.y (mgr.14152) 178 : cluster [DBG] pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:25.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f1855623700 1 -- 172.21.15.175:0/311553234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18501083c0 msgr2=0x7f185010a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:39:25.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f1855623700 1 --2- 172.21.15.175:0/311553234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18501083c0 0x7f185010a830 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1848009990 tx=0x7f1848009ca0 comp rx=0 tx=0).stop 2024-03-26T03:39:25.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f1855623700 1 -- 172.21.15.175:0/311553234 shutdown_connections 2024-03-26T03:39:25.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f1855623700 1 --2- 172.21.15.175:0/311553234 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1850103160 0x7f1850105550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:39:25.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f1855623700 1 --2- 172.21.15.175:0/311553234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18501083c0 0x7f185010a830 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:39:25.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f1855623700 1 --2- 172.21.15.175:0/311553234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1850105a90 0x7f1850107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:39:25.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f1855623700 1 -- 172.21.15.175:0/311553234 >> 172.21.15.175:0/311553234 conn(0x7f18500fcb70 msgr2=0x7f18500fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:39:25.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f1855623700 1 -- 172.21.15.175:0/311553234 shutdown_connections 2024-03-26T03:39:25.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f1855623700 1 -- 172.21.15.175:0/311553234 wait complete. 2024-03-26T03:39:25.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f1855623700 1 Processor -- start 2024-03-26T03:39:25.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f1855623700 1 -- start start 2024-03-26T03:39:25.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f1855623700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1850103160 0x7f1850102ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:39:25.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f1855623700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1850105a90 0x7f1850101110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:39:25.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f1855623700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18501083c0 0x7f1850101650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:39:25.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f1855623700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f185010d770 con 0x7f18501083c0 2024-03-26T03:39:25.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f1855623700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f185010d5f0 con 0x7f1850105a90 2024-03-26T03:39:25.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f1855623700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f185010d8f0 con 0x7f1850103160 2024-03-26T03:39:25.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f184effd700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1850103160 0x7f1850102ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:39:25.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f184f7fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18501083c0 0x7f1850101650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:39:25.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f184effd700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1850103160 0x7f1850102ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:37778/0 (socket says 172.21.15.175:37778) 2024-03-26T03:39:25.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f184effd700 1 -- 172.21.15.175:0/528847777 learned_addr learned my addr 172.21.15.175:0/528847777 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:39:25.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f184e7fc700 1 --2- 172.21.15.175:0/528847777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1850105a90 0x7f1850101110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:39:25.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f184effd700 1 -- 172.21.15.175:0/528847777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1850105a90 msgr2=0x7f1850101110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:39:25.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f184effd700 1 --2- 172.21.15.175:0/528847777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1850105a90 0x7f1850101110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:39:25.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f184effd700 1 -- 172.21.15.175:0/528847777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18501083c0 msgr2=0x7f1850101650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:39:25.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f184effd700 1 --2- 172.21.15.175:0/528847777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18501083c0 0x7f1850101650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:39:25.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f184effd700 1 -- 172.21.15.175:0/528847777 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1850101f00 con 0x7f1850103160 2024-03-26T03:39:25.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f184f7fe700 1 --2- 172.21.15.175:0/528847777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18501083c0 0x7f1850101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T03:39:25.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f184e7fc700 1 --2- 172.21.15.175:0/528847777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1850105a90 0x7f1850101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T03:39:25.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f184effd700 1 --2- 172.21.15.175:0/528847777 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1850103160 0x7f1850102ac0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f184000d980 tx=0x7f184000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:39:25.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f1847fff700 1 -- 172.21.15.175:0/528847777 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840017070 con 0x7f1850103160 2024-03-26T03:39:25.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f1847fff700 1 -- 172.21.15.175:0/528847777 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f1840005020 con 0x7f1850103160 2024-03-26T03:39:25.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f1847fff700 1 -- 172.21.15.175:0/528847777 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840009a30 con 0x7f1850103160 2024-03-26T03:39:25.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f1855623700 1 -- 172.21.15.175:0/528847777 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18501a1f40 con 0x7f1850103160 2024-03-26T03:39:25.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.725+0000 7f1855623700 1 -- 172.21.15.175:0/528847777 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f18501a22e0 con 0x7f1850103160 2024-03-26T03:39:25.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.729+0000 7f1855623700 1 -- 172.21.15.175:0/528847777 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1850050300 con 0x7f1850103160 2024-03-26T03:39:25.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.729+0000 7f1847fff700 1 -- 172.21.15.175:0/528847777 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 15) ==== 92475+0+0 (secure 0 0 0) 0x7f1840009bd0 con 0x7f1850103160 2024-03-26T03:39:25.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.729+0000 7f1847fff700 1 --2- 172.21.15.175:0/528847777 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f183c06e6f0 0x7f183c070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:39:25.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.729+0000 7f1847fff700 1 -- 172.21.15.175:0/528847777 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 3017+0+0 (secure 0 0 0) 0x7f184008dbb0 con 0x7f1850103160 2024-03-26T03:39:25.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.729+0000 7f184e7fc700 1 --2- 172.21.15.175:0/528847777 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f183c06e6f0 0x7f183c070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:39:25.732 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.729+0000 7f184e7fc700 1 --2- 172.21.15.175:0/528847777 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f183c06e6f0 0x7f183c070bb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f18380060b0 tx=0x7f18380076a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:39:25.736 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.733+0000 7f1847fff700 1 -- 172.21.15.175:0/528847777 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f1840020020 con 0x7f1850103160 2024-03-26T03:39:25.954 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:39:25.953+0000 7f1855623700 1 -- 172.21.15.175:0/528847777 --> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1850062a80 con 0x7f183c06e6f0 2024-03-26T03:39:25.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:25 smithi175 bash[21527]: cluster 2024-03-26T03:39:24.231587+0000 mgr.y (mgr.14152) 179 : cluster [DBG] pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:25.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:25 smithi175 bash[29284]: cluster 2024-03-26T03:39:24.231587+0000 mgr.y (mgr.14152) 179 : cluster [DBG] pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:26.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:25 smithi190 bash[23392]: cluster 2024-03-26T03:39:24.231587+0000 mgr.y (mgr.14152) 179 : cluster [DBG] pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:27.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:26 smithi175 bash[21527]: audit 2024-03-26T03:39:25.957787+0000 mgr.y (mgr.14152) 180 : audit [DBG] from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:39:27.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:26 smithi175 bash[21527]: audit 2024-03-26T03:39:25.965118+0000 mon.a (mon.0) 440 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T03:39:27.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:26 smithi175 bash[21527]: audit 2024-03-26T03:39:25.970796+0000 mon.a (mon.0) 441 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T03:39:27.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:26 smithi175 bash[21527]: audit 2024-03-26T03:39:25.972289+0000 mon.a (mon.0) 442 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:39:27.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:26 smithi175 bash[29284]: audit 2024-03-26T03:39:25.957787+0000 mgr.y (mgr.14152) 180 : audit [DBG] from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:39:27.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:26 smithi175 bash[29284]: audit 2024-03-26T03:39:25.965118+0000 mon.a (mon.0) 440 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T03:39:27.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:26 smithi175 bash[29284]: audit 2024-03-26T03:39:25.970796+0000 mon.a (mon.0) 441 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T03:39:27.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:26 smithi175 bash[29284]: audit 2024-03-26T03:39:25.972289+0000 mon.a (mon.0) 442 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:39:27.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:26 smithi190 bash[23392]: audit 2024-03-26T03:39:25.957787+0000 mgr.y (mgr.14152) 180 : audit [DBG] from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:39:27.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:26 smithi190 bash[23392]: audit 2024-03-26T03:39:25.965118+0000 mon.a (mon.0) 440 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T03:39:27.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:26 smithi190 bash[23392]: audit 2024-03-26T03:39:25.970796+0000 mon.a (mon.0) 441 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T03:39:27.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:26 smithi190 bash[23392]: audit 2024-03-26T03:39:25.972289+0000 mon.a (mon.0) 442 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:39:28.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:27 smithi175 bash[21527]: cluster 2024-03-26T03:39:26.232085+0000 mgr.y (mgr.14152) 181 : cluster [DBG] pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:28.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:27 smithi175 bash[29284]: cluster 2024-03-26T03:39:26.232085+0000 mgr.y (mgr.14152) 181 : cluster [DBG] pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:28.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:27 smithi190 bash[23392]: cluster 2024-03-26T03:39:26.232085+0000 mgr.y (mgr.14152) 181 : cluster [DBG] pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:30.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:29 smithi175 bash[21527]: cluster 2024-03-26T03:39:28.232682+0000 mgr.y (mgr.14152) 182 : cluster [DBG] pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:30.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:29 smithi175 bash[29284]: cluster 2024-03-26T03:39:28.232682+0000 mgr.y (mgr.14152) 182 : cluster [DBG] pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:30.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:29 smithi190 bash[23392]: cluster 2024-03-26T03:39:28.232682+0000 mgr.y (mgr.14152) 182 : cluster [DBG] pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:32.176 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:31 smithi175 bash[21527]: cluster 2024-03-26T03:39:30.233257+0000 mgr.y (mgr.14152) 183 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:32.176 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:31 smithi175 bash[29284]: cluster 2024-03-26T03:39:30.233257+0000 mgr.y (mgr.14152) 183 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:32.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:31 smithi190 bash[23392]: cluster 2024-03-26T03:39:30.233257+0000 mgr.y (mgr.14152) 183 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:34.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:33 smithi175 bash[21527]: cluster 2024-03-26T03:39:32.233829+0000 mgr.y (mgr.14152) 184 : cluster [DBG] pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:34.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:33 smithi175 bash[29284]: cluster 2024-03-26T03:39:32.233829+0000 mgr.y (mgr.14152) 184 : cluster [DBG] pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:34.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:33 smithi190 bash[23392]: cluster 2024-03-26T03:39:32.233829+0000 mgr.y (mgr.14152) 184 : cluster [DBG] pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:35.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:34 smithi175 bash[21527]: audit 2024-03-26T03:39:34.369597+0000 mon.c (mon.2) 11 : audit [INF] from='client.? 172.21.15.175:0/4191448328' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8add17f6-b408-4dc0-af05-b3b64845d8ed"}]: dispatch 2024-03-26T03:39:35.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:34 smithi175 bash[21527]: audit 2024-03-26T03:39:34.370149+0000 mon.a (mon.0) 443 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8add17f6-b408-4dc0-af05-b3b64845d8ed"}]: dispatch 2024-03-26T03:39:35.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:34 smithi175 bash[21527]: audit 2024-03-26T03:39:34.375233+0000 mon.a (mon.0) 444 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8add17f6-b408-4dc0-af05-b3b64845d8ed"}]': finished 2024-03-26T03:39:35.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:34 smithi175 bash[21527]: cluster 2024-03-26T03:39:34.382647+0000 mon.a (mon.0) 445 : cluster [DBG] osdmap e22: 4 total, 3 up, 4 in 2024-03-26T03:39:35.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:34 smithi175 bash[21527]: audit 2024-03-26T03:39:34.383441+0000 mon.a (mon.0) 446 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T03:39:35.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:34 smithi175 bash[29284]: audit 2024-03-26T03:39:34.369597+0000 mon.c (mon.2) 11 : audit [INF] from='client.? 172.21.15.175:0/4191448328' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8add17f6-b408-4dc0-af05-b3b64845d8ed"}]: dispatch 2024-03-26T03:39:35.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:34 smithi175 bash[29284]: audit 2024-03-26T03:39:34.370149+0000 mon.a (mon.0) 443 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8add17f6-b408-4dc0-af05-b3b64845d8ed"}]: dispatch 2024-03-26T03:39:35.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:34 smithi175 bash[29284]: audit 2024-03-26T03:39:34.375233+0000 mon.a (mon.0) 444 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8add17f6-b408-4dc0-af05-b3b64845d8ed"}]': finished 2024-03-26T03:39:35.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:34 smithi175 bash[29284]: cluster 2024-03-26T03:39:34.382647+0000 mon.a (mon.0) 445 : cluster [DBG] osdmap e22: 4 total, 3 up, 4 in 2024-03-26T03:39:35.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:34 smithi175 bash[29284]: audit 2024-03-26T03:39:34.383441+0000 mon.a (mon.0) 446 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T03:39:35.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:34 smithi190 bash[23392]: audit 2024-03-26T03:39:34.369597+0000 mon.c (mon.2) 11 : audit [INF] from='client.? 172.21.15.175:0/4191448328' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8add17f6-b408-4dc0-af05-b3b64845d8ed"}]: dispatch 2024-03-26T03:39:35.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:34 smithi190 bash[23392]: audit 2024-03-26T03:39:34.370149+0000 mon.a (mon.0) 443 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8add17f6-b408-4dc0-af05-b3b64845d8ed"}]: dispatch 2024-03-26T03:39:35.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:34 smithi190 bash[23392]: audit 2024-03-26T03:39:34.375233+0000 mon.a (mon.0) 444 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8add17f6-b408-4dc0-af05-b3b64845d8ed"}]': finished 2024-03-26T03:39:35.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:34 smithi190 bash[23392]: cluster 2024-03-26T03:39:34.382647+0000 mon.a (mon.0) 445 : cluster [DBG] osdmap e22: 4 total, 3 up, 4 in 2024-03-26T03:39:35.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:34 smithi190 bash[23392]: audit 2024-03-26T03:39:34.383441+0000 mon.a (mon.0) 446 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T03:39:36.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:35 smithi175 bash[21527]: cluster 2024-03-26T03:39:34.234352+0000 mgr.y (mgr.14152) 185 : cluster [DBG] pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:36.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:35 smithi175 bash[21527]: audit 2024-03-26T03:39:35.115086+0000 mon.c (mon.2) 12 : audit [DBG] from='client.? 172.21.15.175:0/2272395898' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T03:39:36.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:35 smithi175 bash[29284]: cluster 2024-03-26T03:39:34.234352+0000 mgr.y (mgr.14152) 185 : cluster [DBG] pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:36.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:35 smithi175 bash[29284]: audit 2024-03-26T03:39:35.115086+0000 mon.c (mon.2) 12 : audit [DBG] from='client.? 172.21.15.175:0/2272395898' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T03:39:36.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:35 smithi190 bash[23392]: cluster 2024-03-26T03:39:34.234352+0000 mgr.y (mgr.14152) 185 : cluster [DBG] pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:36.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:35 smithi190 bash[23392]: audit 2024-03-26T03:39:35.115086+0000 mon.c (mon.2) 12 : audit [DBG] from='client.? 172.21.15.175:0/2272395898' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T03:39:37.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:37 smithi175 bash[29284]: cluster 2024-03-26T03:39:36.234880+0000 mgr.y (mgr.14152) 186 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:37.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:37 smithi175 bash[21527]: cluster 2024-03-26T03:39:36.234880+0000 mgr.y (mgr.14152) 186 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:38.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:37 smithi190 bash[23392]: cluster 2024-03-26T03:39:36.234880+0000 mgr.y (mgr.14152) 186 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:40.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:39 smithi175 bash[21527]: cluster 2024-03-26T03:39:38.235545+0000 mgr.y (mgr.14152) 187 : cluster [DBG] pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:40.592 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:39 smithi175 bash[29284]: cluster 2024-03-26T03:39:38.235545+0000 mgr.y (mgr.14152) 187 : cluster [DBG] pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:40.593 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:39 smithi190 bash[23392]: cluster 2024-03-26T03:39:38.235545+0000 mgr.y (mgr.14152) 187 : cluster [DBG] pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:42.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:41 smithi175 bash[29284]: cluster 2024-03-26T03:39:40.236184+0000 mgr.y (mgr.14152) 188 : cluster [DBG] pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:42.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:41 smithi175 bash[21527]: cluster 2024-03-26T03:39:40.236184+0000 mgr.y (mgr.14152) 188 : cluster [DBG] pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:42.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:41 smithi190 bash[23392]: cluster 2024-03-26T03:39:40.236184+0000 mgr.y (mgr.14152) 188 : cluster [DBG] pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:44.179 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:43 smithi175 bash[21527]: cluster 2024-03-26T03:39:42.236794+0000 mgr.y (mgr.14152) 189 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:44.179 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:43 smithi175 bash[29284]: cluster 2024-03-26T03:39:42.236794+0000 mgr.y (mgr.14152) 189 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:44.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:43 smithi190 bash[23392]: cluster 2024-03-26T03:39:42.236794+0000 mgr.y (mgr.14152) 189 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:46.156 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:45 smithi175 bash[21527]: cluster 2024-03-26T03:39:44.237241+0000 mgr.y (mgr.14152) 190 : cluster [DBG] pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:46.157 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:45 smithi175 bash[29284]: cluster 2024-03-26T03:39:44.237241+0000 mgr.y (mgr.14152) 190 : cluster [DBG] pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:46.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:45 smithi190 bash[23392]: cluster 2024-03-26T03:39:44.237241+0000 mgr.y (mgr.14152) 190 : cluster [DBG] pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:48.120 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:47 smithi175 bash[21527]: cluster 2024-03-26T03:39:46.237597+0000 mgr.y (mgr.14152) 191 : cluster [DBG] pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:48.120 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:47 smithi175 bash[21527]: audit 2024-03-26T03:39:46.881220+0000 mon.a (mon.0) 447 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-26T03:39:48.120 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:47 smithi175 bash[21527]: audit 2024-03-26T03:39:46.882899+0000 mon.a (mon.0) 448 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:39:48.120 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:47 smithi175 bash[21527]: cephadm 2024-03-26T03:39:46.884424+0000 mgr.y (mgr.14152) 192 : cephadm [INF] Deploying daemon osd.3 on smithi175 2024-03-26T03:39:48.120 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:47 smithi175 bash[29284]: cluster 2024-03-26T03:39:46.237597+0000 mgr.y (mgr.14152) 191 : cluster [DBG] pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:48.120 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:47 smithi175 bash[29284]: audit 2024-03-26T03:39:46.881220+0000 mon.a (mon.0) 447 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-26T03:39:48.120 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:47 smithi175 bash[29284]: audit 2024-03-26T03:39:46.882899+0000 mon.a (mon.0) 448 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:39:48.120 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:47 smithi175 bash[29284]: cephadm 2024-03-26T03:39:46.884424+0000 mgr.y (mgr.14152) 192 : cephadm [INF] Deploying daemon osd.3 on smithi175 2024-03-26T03:39:48.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:47 smithi190 bash[23392]: cluster 2024-03-26T03:39:46.237597+0000 mgr.y (mgr.14152) 191 : cluster [DBG] pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:48.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:47 smithi190 bash[23392]: audit 2024-03-26T03:39:46.881220+0000 mon.a (mon.0) 447 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-26T03:39:48.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:47 smithi190 bash[23392]: audit 2024-03-26T03:39:46.882899+0000 mon.a (mon.0) 448 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:39:48.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:47 smithi190 bash[23392]: cephadm 2024-03-26T03:39:46.884424+0000 mgr.y (mgr.14152) 192 : cephadm [INF] Deploying daemon osd.3 on smithi175 2024-03-26T03:39:50.086 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:49 smithi175 bash[21527]: cluster 2024-03-26T03:39:48.238091+0000 mgr.y (mgr.14152) 193 : cluster [DBG] pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:50.086 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:49 smithi175 bash[29284]: cluster 2024-03-26T03:39:48.238091+0000 mgr.y (mgr.14152) 193 : cluster [DBG] pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:50.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:49 smithi190 bash[23392]: cluster 2024-03-26T03:39:48.238091+0000 mgr.y (mgr.14152) 193 : cluster [DBG] pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:51.300 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:51 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:39:51.300 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:51 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:39:51.300 INFO:journalctl@ceph.osd.1.smithi175.stdout:Mar 26 03:39:51 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:39:51.301 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:39:51 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:39:51.301 INFO:journalctl@ceph.osd.0.smithi175.stdout:Mar 26 03:39:51 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:39:51.301 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 26 03:39:51 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:39:51.564 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:39:51 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:39:51.564 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:51 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:39:51.564 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 26 03:39:51 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:39:51.565 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:51 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:39:51.565 INFO:journalctl@ceph.osd.1.smithi175.stdout:Mar 26 03:39:51 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:39:51.565 INFO:journalctl@ceph.osd.0.smithi175.stdout:Mar 26 03:39:51 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:39:52.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:51 smithi175 bash[21527]: cluster 2024-03-26T03:39:50.238710+0000 mgr.y (mgr.14152) 194 : cluster [DBG] pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:52.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:51 smithi175 bash[21527]: audit 2024-03-26T03:39:51.609421+0000 mon.a (mon.0) 449 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:52.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:51 smithi175 bash[21527]: audit 2024-03-26T03:39:51.616093+0000 mon.a (mon.0) 450 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:52.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:51 smithi175 bash[21527]: audit 2024-03-26T03:39:51.651540+0000 mon.a (mon.0) 451 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:39:52.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:51 smithi175 bash[29284]: cluster 2024-03-26T03:39:50.238710+0000 mgr.y (mgr.14152) 194 : cluster [DBG] pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:52.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:51 smithi175 bash[29284]: audit 2024-03-26T03:39:51.609421+0000 mon.a (mon.0) 449 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:52.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:51 smithi175 bash[29284]: audit 2024-03-26T03:39:51.616093+0000 mon.a (mon.0) 450 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:52.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:51 smithi175 bash[29284]: audit 2024-03-26T03:39:51.651540+0000 mon.a (mon.0) 451 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:39:52.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:51 smithi190 bash[23392]: cluster 2024-03-26T03:39:50.238710+0000 mgr.y (mgr.14152) 194 : cluster [DBG] pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:52.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:51 smithi190 bash[23392]: audit 2024-03-26T03:39:51.609421+0000 mon.a (mon.0) 449 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:52.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:51 smithi190 bash[23392]: audit 2024-03-26T03:39:51.616093+0000 mon.a (mon.0) 450 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:52.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:51 smithi190 bash[23392]: audit 2024-03-26T03:39:51.651540+0000 mon.a (mon.0) 451 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:39:54.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:53 smithi175 bash[29284]: cluster 2024-03-26T03:39:52.239223+0000 mgr.y (mgr.14152) 195 : cluster [DBG] pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:54.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:53 smithi175 bash[21527]: cluster 2024-03-26T03:39:52.239223+0000 mgr.y (mgr.14152) 195 : cluster [DBG] pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:54.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:53 smithi190 bash[23392]: cluster 2024-03-26T03:39:52.239223+0000 mgr.y (mgr.14152) 195 : cluster [DBG] pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:56.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:55 smithi175 bash[29284]: cluster 2024-03-26T03:39:54.239813+0000 mgr.y (mgr.14152) 196 : cluster [DBG] pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:56.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:55 smithi175 bash[21527]: cluster 2024-03-26T03:39:54.239813+0000 mgr.y (mgr.14152) 196 : cluster [DBG] pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:56.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:55 smithi190 bash[23392]: cluster 2024-03-26T03:39:54.239813+0000 mgr.y (mgr.14152) 196 : cluster [DBG] pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:58.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:57 smithi175 bash[21527]: cluster 2024-03-26T03:39:56.240316+0000 mgr.y (mgr.14152) 197 : cluster [DBG] pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:58.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:57 smithi175 bash[21527]: audit 2024-03-26T03:39:56.987194+0000 mon.a (mon.0) 452 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:58.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:57 smithi175 bash[21527]: audit 2024-03-26T03:39:56.995752+0000 mon.a (mon.0) 453 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:58.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:57 smithi175 bash[21527]: audit 2024-03-26T03:39:56.997388+0000 mon.a (mon.0) 454 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:39:58.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:57 smithi175 bash[21527]: audit 2024-03-26T03:39:56.998861+0000 mon.a (mon.0) 455 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:39:58.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:57 smithi175 bash[21527]: audit 2024-03-26T03:39:57.007683+0000 mon.a (mon.0) 456 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:58.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:57 smithi175 bash[29284]: cluster 2024-03-26T03:39:56.240316+0000 mgr.y (mgr.14152) 197 : cluster [DBG] pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:58.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:57 smithi175 bash[29284]: audit 2024-03-26T03:39:56.987194+0000 mon.a (mon.0) 452 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:58.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:57 smithi175 bash[29284]: audit 2024-03-26T03:39:56.995752+0000 mon.a (mon.0) 453 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:58.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:57 smithi175 bash[29284]: audit 2024-03-26T03:39:56.997388+0000 mon.a (mon.0) 454 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:39:58.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:57 smithi175 bash[29284]: audit 2024-03-26T03:39:56.998861+0000 mon.a (mon.0) 455 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:39:58.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:57 smithi175 bash[29284]: audit 2024-03-26T03:39:57.007683+0000 mon.a (mon.0) 456 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:58.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:57 smithi190 bash[23392]: cluster 2024-03-26T03:39:56.240316+0000 mgr.y (mgr.14152) 197 : cluster [DBG] pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:39:58.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:57 smithi190 bash[23392]: audit 2024-03-26T03:39:56.987194+0000 mon.a (mon.0) 452 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:58.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:57 smithi190 bash[23392]: audit 2024-03-26T03:39:56.995752+0000 mon.a (mon.0) 453 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:58.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:57 smithi190 bash[23392]: audit 2024-03-26T03:39:56.997388+0000 mon.a (mon.0) 454 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:39:58.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:57 smithi190 bash[23392]: audit 2024-03-26T03:39:56.998861+0000 mon.a (mon.0) 455 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:39:58.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:57 smithi190 bash[23392]: audit 2024-03-26T03:39:57.007683+0000 mon.a (mon.0) 456 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:39:59.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:58 smithi175 bash[21527]: audit 2024-03-26T03:39:58.327491+0000 mon.c (mon.2) 13 : audit [INF] from='osd.3 [v2:172.21.15.175:6826/672326154,v1:172.21.15.175:6827/672326154]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-26T03:39:59.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:39:58 smithi175 bash[21527]: audit 2024-03-26T03:39:58.327841+0000 mon.a (mon.0) 457 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-26T03:39:59.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:58 smithi175 bash[29284]: audit 2024-03-26T03:39:58.327491+0000 mon.c (mon.2) 13 : audit [INF] from='osd.3 [v2:172.21.15.175:6826/672326154,v1:172.21.15.175:6827/672326154]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-26T03:39:59.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:39:58 smithi175 bash[29284]: audit 2024-03-26T03:39:58.327841+0000 mon.a (mon.0) 457 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-26T03:39:59.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:58 smithi190 bash[23392]: audit 2024-03-26T03:39:58.327491+0000 mon.c (mon.2) 13 : audit [INF] from='osd.3 [v2:172.21.15.175:6826/672326154,v1:172.21.15.175:6827/672326154]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-26T03:39:59.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:39:58 smithi190 bash[23392]: audit 2024-03-26T03:39:58.327841+0000 mon.a (mon.0) 457 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-26T03:40:00.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:00 smithi175 bash[21527]: cluster 2024-03-26T03:39:58.240814+0000 mgr.y (mgr.14152) 198 : cluster [DBG] pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:40:00.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:00 smithi175 bash[21527]: audit 2024-03-26T03:39:58.998010+0000 mon.a (mon.0) 458 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-26T03:40:00.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:00 smithi175 bash[21527]: cluster 2024-03-26T03:39:59.003889+0000 mon.a (mon.0) 459 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2024-03-26T03:40:00.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:00 smithi175 bash[21527]: audit 2024-03-26T03:39:59.004511+0000 mon.a (mon.0) 460 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T03:40:00.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:00 smithi175 bash[21527]: audit 2024-03-26T03:39:59.005081+0000 mon.c (mon.2) 14 : audit [INF] from='osd.3 [v2:172.21.15.175:6826/672326154,v1:172.21.15.175:6827/672326154]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-26T03:40:00.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:00 smithi175 bash[21527]: audit 2024-03-26T03:39:59.005619+0000 mon.a (mon.0) 461 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-26T03:40:00.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:00 smithi175 bash[29284]: cluster 2024-03-26T03:39:58.240814+0000 mgr.y (mgr.14152) 198 : cluster [DBG] pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:40:00.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:00 smithi175 bash[29284]: audit 2024-03-26T03:39:58.998010+0000 mon.a (mon.0) 458 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-26T03:40:00.495 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:00 smithi175 bash[29284]: cluster 2024-03-26T03:39:59.003889+0000 mon.a (mon.0) 459 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2024-03-26T03:40:00.495 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:00 smithi175 bash[29284]: audit 2024-03-26T03:39:59.004511+0000 mon.a (mon.0) 460 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T03:40:00.495 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:00 smithi175 bash[29284]: audit 2024-03-26T03:39:59.005081+0000 mon.c (mon.2) 14 : audit [INF] from='osd.3 [v2:172.21.15.175:6826/672326154,v1:172.21.15.175:6827/672326154]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-26T03:40:00.495 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:00 smithi175 bash[29284]: audit 2024-03-26T03:39:59.005619+0000 mon.a (mon.0) 461 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-26T03:40:00.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:00 smithi190 bash[23392]: cluster 2024-03-26T03:39:58.240814+0000 mgr.y (mgr.14152) 198 : cluster [DBG] pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:40:00.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:00 smithi190 bash[23392]: audit 2024-03-26T03:39:58.998010+0000 mon.a (mon.0) 458 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-26T03:40:00.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:00 smithi190 bash[23392]: cluster 2024-03-26T03:39:59.003889+0000 mon.a (mon.0) 459 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2024-03-26T03:40:00.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:00 smithi190 bash[23392]: audit 2024-03-26T03:39:59.004511+0000 mon.a (mon.0) 460 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T03:40:00.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:00 smithi190 bash[23392]: audit 2024-03-26T03:39:59.005081+0000 mon.c (mon.2) 14 : audit [INF] from='osd.3 [v2:172.21.15.175:6826/672326154,v1:172.21.15.175:6827/672326154]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-26T03:40:00.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:00 smithi190 bash[23392]: audit 2024-03-26T03:39:59.005619+0000 mon.a (mon.0) 461 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-03-26T03:40:00.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:40:00.613+0000 7f1847fff700 1 -- 172.21.15.175:0/528847777 <== mgr.14152 v2:172.21.15.175:6800/2826039805 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1850062a80 con 0x7f183c06e6f0 2024-03-26T03:40:00.615 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 3 on host 'smithi175' 2024-03-26T03:40:00.618 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:40:00.617+0000 7f1855623700 1 -- 172.21.15.175:0/528847777 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f183c06e6f0 msgr2=0x7f183c070bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:40:00.618 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:40:00.617+0000 7f1855623700 1 --2- 172.21.15.175:0/528847777 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f183c06e6f0 0x7f183c070bb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f18380060b0 tx=0x7f18380076a0 comp rx=0 tx=0).stop 2024-03-26T03:40:00.618 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:40:00.617+0000 7f1855623700 1 -- 172.21.15.175:0/528847777 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1850103160 msgr2=0x7f1850102ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:40:00.618 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:40:00.617+0000 7f1855623700 1 --2- 172.21.15.175:0/528847777 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1850103160 0x7f1850102ac0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f184000d980 tx=0x7f184000dc90 comp rx=0 tx=0).stop 2024-03-26T03:40:00.619 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:40:00.617+0000 7f1855623700 1 -- 172.21.15.175:0/528847777 shutdown_connections 2024-03-26T03:40:00.619 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:40:00.617+0000 7f1855623700 1 --2- 172.21.15.175:0/528847777 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f183c06e6f0 0x7f183c070bb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:40:00.619 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:40:00.617+0000 7f1855623700 1 --2- 172.21.15.175:0/528847777 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1850103160 0x7f1850102ac0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:40:00.619 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:40:00.617+0000 7f1855623700 1 --2- 172.21.15.175:0/528847777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18501083c0 0x7f1850101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:40:00.619 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:40:00.617+0000 7f1855623700 1 --2- 172.21.15.175:0/528847777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1850105a90 0x7f1850101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:40:00.619 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:40:00.617+0000 7f1855623700 1 -- 172.21.15.175:0/528847777 >> 172.21.15.175:0/528847777 conn(0x7f18500fcb70 msgr2=0x7f1850103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:40:00.619 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:40:00.617+0000 7f1855623700 1 -- 172.21.15.175:0/528847777 shutdown_connections 2024-03-26T03:40:00.619 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:40:00.617+0000 7f1855623700 1 -- 172.21.15.175:0/528847777 wait complete. 2024-03-26T03:40:01.350 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:01 smithi175 bash[21527]: cluster 2024-03-26T03:40:00.000131+0000 mon.a (mon.0) 462 : cluster [INF] overall HEALTH_OK 2024-03-26T03:40:01.350 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:01 smithi175 bash[21527]: audit 2024-03-26T03:40:00.000650+0000 mon.a (mon.0) 463 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-03-26T03:40:01.350 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:01 smithi175 bash[21527]: cluster 2024-03-26T03:40:00.008780+0000 mon.a (mon.0) 464 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2024-03-26T03:40:01.350 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:01 smithi175 bash[21527]: audit 2024-03-26T03:40:00.010153+0000 mon.a (mon.0) 465 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T03:40:01.350 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:01 smithi175 bash[21527]: audit 2024-03-26T03:40:00.019745+0000 mon.a (mon.0) 466 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T03:40:01.350 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:01 smithi175 bash[21527]: audit 2024-03-26T03:40:00.594630+0000 mon.a (mon.0) 467 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:01.350 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:01 smithi175 bash[21527]: audit 2024-03-26T03:40:00.603318+0000 mon.a (mon.0) 468 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:01.351 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:01 smithi175 bash[21527]: audit 2024-03-26T03:40:00.604010+0000 mon.a (mon.0) 469 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:40:01.351 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:01 smithi175 bash[29284]: cluster 2024-03-26T03:40:00.000131+0000 mon.a (mon.0) 462 : cluster [INF] overall HEALTH_OK 2024-03-26T03:40:01.351 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:01 smithi175 bash[29284]: audit 2024-03-26T03:40:00.000650+0000 mon.a (mon.0) 463 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-03-26T03:40:01.351 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:01 smithi175 bash[29284]: cluster 2024-03-26T03:40:00.008780+0000 mon.a (mon.0) 464 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2024-03-26T03:40:01.351 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:01 smithi175 bash[29284]: audit 2024-03-26T03:40:00.010153+0000 mon.a (mon.0) 465 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T03:40:01.351 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:01 smithi175 bash[29284]: audit 2024-03-26T03:40:00.019745+0000 mon.a (mon.0) 466 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T03:40:01.351 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:01 smithi175 bash[29284]: audit 2024-03-26T03:40:00.594630+0000 mon.a (mon.0) 467 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:01.352 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:01 smithi175 bash[29284]: audit 2024-03-26T03:40:00.603318+0000 mon.a (mon.0) 468 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:01.352 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:01 smithi175 bash[29284]: audit 2024-03-26T03:40:00.604010+0000 mon.a (mon.0) 469 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:40:01.353 DEBUG:teuthology.orchestra.run.smithi175:osd.3> sudo journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@osd.3.service 2024-03-26T03:40:01.355 INFO:tasks.cephadm:Deploying osd.4 on smithi190 with /dev/vg_nvme/lv_4... 2024-03-26T03:40:01.356 DEBUG:teuthology.orchestra.run.smithi190:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-26T03:40:01.367 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:01 smithi190 bash[23392]: cluster 2024-03-26T03:40:00.000131+0000 mon.a (mon.0) 462 : cluster [INF] overall HEALTH_OK 2024-03-26T03:40:01.367 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:01 smithi190 bash[23392]: audit 2024-03-26T03:40:00.000650+0000 mon.a (mon.0) 463 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-03-26T03:40:01.368 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:01 smithi190 bash[23392]: cluster 2024-03-26T03:40:00.008780+0000 mon.a (mon.0) 464 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2024-03-26T03:40:01.368 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:01 smithi190 bash[23392]: audit 2024-03-26T03:40:00.010153+0000 mon.a (mon.0) 465 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T03:40:01.368 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:01 smithi190 bash[23392]: audit 2024-03-26T03:40:00.019745+0000 mon.a (mon.0) 466 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T03:40:01.368 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:01 smithi190 bash[23392]: audit 2024-03-26T03:40:00.594630+0000 mon.a (mon.0) 467 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:01.368 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:01 smithi190 bash[23392]: audit 2024-03-26T03:40:00.603318+0000 mon.a (mon.0) 468 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:01.368 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:01 smithi190 bash[23392]: audit 2024-03-26T03:40:00.604010+0000 mon.a (mon.0) 469 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:40:02.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:02 smithi175 bash[29284]: cluster 2024-03-26T03:39:59.306194+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2024-03-26T03:40:02.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:02 smithi175 bash[29284]: cluster 2024-03-26T03:39:59.306273+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2024-03-26T03:40:02.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:02 smithi175 bash[29284]: cluster 2024-03-26T03:40:00.241407+0000 mgr.y (mgr.14152) 199 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:40:02.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:02 smithi175 bash[29284]: audit 2024-03-26T03:40:01.015649+0000 mon.a (mon.0) 470 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T03:40:02.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:02 smithi175 bash[29284]: cluster 2024-03-26T03:40:01.022661+0000 mon.a (mon.0) 471 : cluster [INF] osd.3 [v2:172.21.15.175:6826/672326154,v1:172.21.15.175:6827/672326154] boot 2024-03-26T03:40:02.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:02 smithi175 bash[29284]: cluster 2024-03-26T03:40:01.022852+0000 mon.a (mon.0) 472 : cluster [DBG] osdmap e25: 4 total, 4 up, 4 in 2024-03-26T03:40:02.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:02 smithi175 bash[29284]: audit 2024-03-26T03:40:01.023547+0000 mon.a (mon.0) 473 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T03:40:02.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:02 smithi175 bash[21527]: cluster 2024-03-26T03:39:59.306194+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2024-03-26T03:40:02.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:02 smithi175 bash[21527]: cluster 2024-03-26T03:39:59.306273+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2024-03-26T03:40:02.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:02 smithi175 bash[21527]: cluster 2024-03-26T03:40:00.241407+0000 mgr.y (mgr.14152) 199 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:40:02.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:02 smithi175 bash[21527]: audit 2024-03-26T03:40:01.015649+0000 mon.a (mon.0) 470 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T03:40:02.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:02 smithi175 bash[21527]: cluster 2024-03-26T03:40:01.022661+0000 mon.a (mon.0) 471 : cluster [INF] osd.3 [v2:172.21.15.175:6826/672326154,v1:172.21.15.175:6827/672326154] boot 2024-03-26T03:40:02.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:02 smithi175 bash[21527]: cluster 2024-03-26T03:40:01.022852+0000 mon.a (mon.0) 472 : cluster [DBG] osdmap e25: 4 total, 4 up, 4 in 2024-03-26T03:40:02.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:02 smithi175 bash[21527]: audit 2024-03-26T03:40:01.023547+0000 mon.a (mon.0) 473 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T03:40:02.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:02 smithi190 bash[23392]: cluster 2024-03-26T03:39:59.306194+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2024-03-26T03:40:02.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:02 smithi190 bash[23392]: cluster 2024-03-26T03:39:59.306273+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2024-03-26T03:40:02.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:02 smithi190 bash[23392]: cluster 2024-03-26T03:40:00.241407+0000 mgr.y (mgr.14152) 199 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-26T03:40:02.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:02 smithi190 bash[23392]: audit 2024-03-26T03:40:01.015649+0000 mon.a (mon.0) 470 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T03:40:02.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:02 smithi190 bash[23392]: cluster 2024-03-26T03:40:01.022661+0000 mon.a (mon.0) 471 : cluster [INF] osd.3 [v2:172.21.15.175:6826/672326154,v1:172.21.15.175:6827/672326154] boot 2024-03-26T03:40:02.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:02 smithi190 bash[23392]: cluster 2024-03-26T03:40:01.022852+0000 mon.a (mon.0) 472 : cluster [DBG] osdmap e25: 4 total, 4 up, 4 in 2024-03-26T03:40:02.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:02 smithi190 bash[23392]: audit 2024-03-26T03:40:01.023547+0000 mon.a (mon.0) 473 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T03:40:03.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:03 smithi175 bash[21527]: cluster 2024-03-26T03:40:02.041239+0000 mon.a (mon.0) 474 : cluster [DBG] osdmap e26: 4 total, 4 up, 4 in 2024-03-26T03:40:03.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:03 smithi175 bash[29284]: cluster 2024-03-26T03:40:02.041239+0000 mon.a (mon.0) 474 : cluster [DBG] osdmap e26: 4 total, 4 up, 4 in 2024-03-26T03:40:03.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:03 smithi190 bash[23392]: cluster 2024-03-26T03:40:02.041239+0000 mon.a (mon.0) 474 : cluster [DBG] osdmap e26: 4 total, 4 up, 4 in 2024-03-26T03:40:04.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:04 smithi175 bash[21527]: cluster 2024-03-26T03:40:02.242077+0000 mgr.y (mgr.14152) 200 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:04.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:04 smithi175 bash[29284]: cluster 2024-03-26T03:40:02.242077+0000 mgr.y (mgr.14152) 200 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:04.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:04 smithi190 bash[23392]: cluster 2024-03-26T03:40:02.242077+0000 mgr.y (mgr.14152) 200 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:06.065 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T03:40:06.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:06 smithi175 bash[29284]: cluster 2024-03-26T03:40:04.242676+0000 mgr.y (mgr.14152) 201 : cluster [DBG] pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:06.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:06 smithi175 bash[21527]: cluster 2024-03-26T03:40:04.242676+0000 mgr.y (mgr.14152) 201 : cluster [DBG] pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:06.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:06 smithi190 bash[23392]: cluster 2024-03-26T03:40:04.242676+0000 mgr.y (mgr.14152) 201 : cluster [DBG] pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:08.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:08 smithi190 bash[23392]: cluster 2024-03-26T03:40:06.243307+0000 mgr.y (mgr.14152) 202 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:08.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:08 smithi175 bash[21527]: cluster 2024-03-26T03:40:06.243307+0000 mgr.y (mgr.14152) 202 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:08.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:08 smithi175 bash[29284]: cluster 2024-03-26T03:40:06.243307+0000 mgr.y (mgr.14152) 202 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:09.209 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-26T03:40:09.209 INFO:teuthology.orchestra.run.smithi190.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-26T03:40:09.209 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10+0 records in 2024-03-26T03:40:09.209 INFO:teuthology.orchestra.run.smithi190.stderr:10+0 records out 2024-03-26T03:40:09.209 INFO:teuthology.orchestra.run.smithi190.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0137067 s, 765 MB/s 2024-03-26T03:40:09.209 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping successful for: 2024-03-26T03:40:09.918 DEBUG:teuthology.orchestra.run.smithi190:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi190:vg_nvme/lv_4 2024-03-26T03:40:10.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:10 smithi190 bash[23392]: cluster 2024-03-26T03:40:08.243800+0000 mgr.y (mgr.14152) 203 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:10.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:10 smithi175 bash[21527]: cluster 2024-03-26T03:40:08.243800+0000 mgr.y (mgr.14152) 203 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:10.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:10 smithi175 bash[29284]: cluster 2024-03-26T03:40:08.243800+0000 mgr.y (mgr.14152) 203 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:11.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:11 smithi175 bash[21527]: cluster 2024-03-26T03:40:10.244438+0000 mgr.y (mgr.14152) 204 : cluster [DBG] pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:11.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:11 smithi175 bash[21527]: cephadm 2024-03-26T03:40:10.248093+0000 mgr.y (mgr.14152) 205 : cephadm [INF] Detected new or changed devices on smithi175 2024-03-26T03:40:11.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:11 smithi175 bash[21527]: audit 2024-03-26T03:40:10.257807+0000 mon.a (mon.0) 475 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:11.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:11 smithi175 bash[21527]: audit 2024-03-26T03:40:10.267892+0000 mon.a (mon.0) 476 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:11.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:11 smithi175 bash[21527]: audit 2024-03-26T03:40:10.270363+0000 mon.a (mon.0) 477 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:40:11.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:11 smithi175 bash[21527]: audit 2024-03-26T03:40:10.272076+0000 mon.a (mon.0) 478 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:40:11.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:11 smithi175 bash[21527]: audit 2024-03-26T03:40:10.273914+0000 mon.a (mon.0) 479 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:40:11.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:11 smithi175 bash[21527]: audit 2024-03-26T03:40:10.275545+0000 mon.a (mon.0) 480 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:40:11.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:11 smithi175 bash[21527]: cephadm 2024-03-26T03:40:10.276726+0000 mgr.y (mgr.14152) 206 : cephadm [INF] Adjusting osd_memory_target on smithi175 to 4060M 2024-03-26T03:40:11.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:11 smithi175 bash[21527]: audit 2024-03-26T03:40:10.282920+0000 mon.a (mon.0) 481 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:11.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:11 smithi175 bash[21527]: audit 2024-03-26T03:40:10.287731+0000 mon.a (mon.0) 482 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:40:11.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:11 smithi175 bash[21527]: audit 2024-03-26T03:40:10.289315+0000 mon.a (mon.0) 483 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:40:11.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:11 smithi175 bash[21527]: audit 2024-03-26T03:40:10.297814+0000 mon.a (mon.0) 484 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:11.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:11 smithi175 bash[29284]: cluster 2024-03-26T03:40:10.244438+0000 mgr.y (mgr.14152) 204 : cluster [DBG] pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:11.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:11 smithi175 bash[29284]: cephadm 2024-03-26T03:40:10.248093+0000 mgr.y (mgr.14152) 205 : cephadm [INF] Detected new or changed devices on smithi175 2024-03-26T03:40:11.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:11 smithi175 bash[29284]: audit 2024-03-26T03:40:10.257807+0000 mon.a (mon.0) 475 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:11.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:11 smithi175 bash[29284]: audit 2024-03-26T03:40:10.267892+0000 mon.a (mon.0) 476 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:11.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:11 smithi175 bash[29284]: audit 2024-03-26T03:40:10.270363+0000 mon.a (mon.0) 477 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:40:11.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:11 smithi175 bash[29284]: audit 2024-03-26T03:40:10.272076+0000 mon.a (mon.0) 478 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:40:11.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:11 smithi175 bash[29284]: audit 2024-03-26T03:40:10.273914+0000 mon.a (mon.0) 479 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:40:11.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:11 smithi175 bash[29284]: audit 2024-03-26T03:40:10.275545+0000 mon.a (mon.0) 480 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:40:11.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:11 smithi175 bash[29284]: cephadm 2024-03-26T03:40:10.276726+0000 mgr.y (mgr.14152) 206 : cephadm [INF] Adjusting osd_memory_target on smithi175 to 4060M 2024-03-26T03:40:11.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:11 smithi175 bash[29284]: audit 2024-03-26T03:40:10.282920+0000 mon.a (mon.0) 481 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:11.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:11 smithi175 bash[29284]: audit 2024-03-26T03:40:10.287731+0000 mon.a (mon.0) 482 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:40:11.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:11 smithi175 bash[29284]: audit 2024-03-26T03:40:10.289315+0000 mon.a (mon.0) 483 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:40:11.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:11 smithi175 bash[29284]: audit 2024-03-26T03:40:10.297814+0000 mon.a (mon.0) 484 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:11.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:11 smithi190 bash[23392]: cluster 2024-03-26T03:40:10.244438+0000 mgr.y (mgr.14152) 204 : cluster [DBG] pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:11.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:11 smithi190 bash[23392]: cephadm 2024-03-26T03:40:10.248093+0000 mgr.y (mgr.14152) 205 : cephadm [INF] Detected new or changed devices on smithi175 2024-03-26T03:40:11.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:11 smithi190 bash[23392]: audit 2024-03-26T03:40:10.257807+0000 mon.a (mon.0) 475 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:11.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:11 smithi190 bash[23392]: audit 2024-03-26T03:40:10.267892+0000 mon.a (mon.0) 476 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:11.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:11 smithi190 bash[23392]: audit 2024-03-26T03:40:10.270363+0000 mon.a (mon.0) 477 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:40:11.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:11 smithi190 bash[23392]: audit 2024-03-26T03:40:10.272076+0000 mon.a (mon.0) 478 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:40:11.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:11 smithi190 bash[23392]: audit 2024-03-26T03:40:10.273914+0000 mon.a (mon.0) 479 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:40:11.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:11 smithi190 bash[23392]: audit 2024-03-26T03:40:10.275545+0000 mon.a (mon.0) 480 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:40:11.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:11 smithi190 bash[23392]: cephadm 2024-03-26T03:40:10.276726+0000 mgr.y (mgr.14152) 206 : cephadm [INF] Adjusting osd_memory_target on smithi175 to 4060M 2024-03-26T03:40:11.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:11 smithi190 bash[23392]: audit 2024-03-26T03:40:10.282920+0000 mon.a (mon.0) 481 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:11.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:11 smithi190 bash[23392]: audit 2024-03-26T03:40:10.287731+0000 mon.a (mon.0) 482 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:40:11.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:11 smithi190 bash[23392]: audit 2024-03-26T03:40:10.289315+0000 mon.a (mon.0) 483 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:40:11.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:11 smithi190 bash[23392]: audit 2024-03-26T03:40:10.297814+0000 mon.a (mon.0) 484 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:13.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:13 smithi175 bash[21527]: cluster 2024-03-26T03:40:12.245006+0000 mgr.y (mgr.14152) 207 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:13.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:13 smithi175 bash[29284]: cluster 2024-03-26T03:40:12.245006+0000 mgr.y (mgr.14152) 207 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:13.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:13 smithi190 bash[23392]: cluster 2024-03-26T03:40:12.245006+0000 mgr.y (mgr.14152) 207 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:14.667 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T03:40:15.545 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:15 smithi190 bash[23392]: cluster 2024-03-26T03:40:14.245549+0000 mgr.y (mgr.14152) 208 : cluster [DBG] pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:15.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:15 smithi175 bash[21527]: cluster 2024-03-26T03:40:14.245549+0000 mgr.y (mgr.14152) 208 : cluster [DBG] pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:15.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:15 smithi175 bash[29284]: cluster 2024-03-26T03:40:14.245549+0000 mgr.y (mgr.14152) 208 : cluster [DBG] pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:17.160 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe34cdb2700 1 -- 172.21.15.190:0/3476037210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe34810a880 msgr2=0x7fe348111130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:40:17.160 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe34cdb2700 1 --2- 172.21.15.190:0/3476037210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe34810a880 0x7fe348111130 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe3380099c0 tx=0x7fe338009cd0 comp rx=0 tx=0).stop 2024-03-26T03:40:17.160 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe34cdb2700 1 -- 172.21.15.190:0/3476037210 shutdown_connections 2024-03-26T03:40:17.161 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe34cdb2700 1 --2- 172.21.15.190:0/3476037210 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe348108ce0 0x7fe348109100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:40:17.161 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe34cdb2700 1 --2- 172.21.15.190:0/3476037210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe34810a880 0x7fe348111130 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:40:17.161 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe34cdb2700 1 --2- 172.21.15.190:0/3476037210 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe348109ee0 0x7fe34810a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:40:17.161 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe34cdb2700 1 -- 172.21.15.190:0/3476037210 >> 172.21.15.190:0/3476037210 conn(0x7fe348077cd0 msgr2=0x7fe34807a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:40:17.161 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe34cdb2700 1 -- 172.21.15.190:0/3476037210 shutdown_connections 2024-03-26T03:40:17.161 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe34cdb2700 1 -- 172.21.15.190:0/3476037210 wait complete. 2024-03-26T03:40:17.163 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe34cdb2700 1 Processor -- start 2024-03-26T03:40:17.163 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe34cdb2700 1 -- start start 2024-03-26T03:40:17.163 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe34cdb2700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe348108ce0 0x7fe34819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:40:17.163 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe34cdb2700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe348109ee0 0x7fe34819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:40:17.163 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe34cdb2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe34810a880 0x7fe3481a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:40:17.163 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe34cdb2700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe348113dc0 con 0x7fe348108ce0 2024-03-26T03:40:17.163 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe34cdb2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe348113c40 con 0x7fe34810a880 2024-03-26T03:40:17.163 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe34cdb2700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe348113f40 con 0x7fe348109ee0 2024-03-26T03:40:17.163 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe346d9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe34810a880 0x7fe3481a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:40:17.163 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe346d9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe34810a880 0x7fe3481a59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:55308/0 (socket says 172.21.15.190:55308) 2024-03-26T03:40:17.163 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe346d9d700 1 -- 172.21.15.190:0/1625985311 learned_addr learned my addr 172.21.15.190:0/1625985311 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-03-26T03:40:17.163 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe34659c700 1 --2- 172.21.15.190:0/1625985311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe348108ce0 0x7fe34819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:40:17.164 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe345d9b700 1 --2- 172.21.15.190:0/1625985311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe348109ee0 0x7fe34819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:40:17.164 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe346d9d700 1 -- 172.21.15.190:0/1625985311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe348109ee0 msgr2=0x7fe34819e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:40:17.164 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe346d9d700 1 --2- 172.21.15.190:0/1625985311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe348109ee0 0x7fe34819e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:40:17.164 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe346d9d700 1 -- 172.21.15.190:0/1625985311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe348108ce0 msgr2=0x7fe34819e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:40:17.164 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe346d9d700 1 --2- 172.21.15.190:0/1625985311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe348108ce0 0x7fe34819e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:40:17.164 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe346d9d700 1 -- 172.21.15.190:0/1625985311 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3481a6050 con 0x7fe34810a880 2024-03-26T03:40:17.164 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe346d9d700 1 --2- 172.21.15.190:0/1625985311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe34810a880 0x7fe3481a59b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe3380047b0 tx=0x7fe338009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:40:17.164 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe33f7fe700 1 -- 172.21.15.190:0/1625985311 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe338004830 con 0x7fe34810a880 2024-03-26T03:40:17.164 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe33f7fe700 1 -- 172.21.15.190:0/1625985311 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fe338004e50 con 0x7fe34810a880 2024-03-26T03:40:17.164 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe33f7fe700 1 -- 172.21.15.190:0/1625985311 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe338005280 con 0x7fe34810a880 2024-03-26T03:40:17.164 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe34cdb2700 1 -- 172.21.15.190:0/1625985311 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3481a62e0 con 0x7fe34810a880 2024-03-26T03:40:17.164 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe34cdb2700 1 -- 172.21.15.190:0/1625985311 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3481a6730 con 0x7fe34810a880 2024-03-26T03:40:17.164 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe34cdb2700 1 -- 172.21.15.190:0/1625985311 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe348050300 con 0x7fe34810a880 2024-03-26T03:40:17.165 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.159+0000 7fe33f7fe700 1 -- 172.21.15.190:0/1625985311 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 15) ==== 92475+0+0 (secure 0 0 0) 0x7fe338005420 con 0x7fe34810a880 2024-03-26T03:40:17.165 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.163+0000 7fe33f7fe700 1 --2- 172.21.15.190:0/1625985311 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fe33406e6f0 0x7fe334070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:40:17.165 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.163+0000 7fe33f7fe700 1 -- 172.21.15.190:0/1625985311 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 3449+0+0 (secure 0 0 0) 0x7fe33808ea00 con 0x7fe34810a880 2024-03-26T03:40:17.165 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.163+0000 7fe34659c700 1 --2- 172.21.15.190:0/1625985311 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fe33406e6f0 0x7fe334070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:40:17.165 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.163+0000 7fe34659c700 1 --2- 172.21.15.190:0/1625985311 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fe33406e6f0 0x7fe334070bb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe348109d40 tx=0x7fe340009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:40:17.169 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.167+0000 7fe33f7fe700 1 -- 172.21.15.190:0/1625985311 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7fe338013070 con 0x7fe34810a880 2024-03-26T03:40:17.388 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:17.383+0000 7fe34cdb2700 1 -- 172.21.15.190:0/1625985311 --> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe348062a80 con 0x7fe33406e6f0 2024-03-26T03:40:17.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:17 smithi175 bash[21527]: cluster 2024-03-26T03:40:16.246075+0000 mgr.y (mgr.14152) 209 : cluster [DBG] pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:17.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:17 smithi175 bash[29284]: cluster 2024-03-26T03:40:16.246075+0000 mgr.y (mgr.14152) 209 : cluster [DBG] pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:17.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:17 smithi190 bash[23392]: cluster 2024-03-26T03:40:16.246075+0000 mgr.y (mgr.14152) 209 : cluster [DBG] pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:18.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:18 smithi175 bash[21527]: audit 2024-03-26T03:40:17.391878+0000 mgr.y (mgr.14152) 210 : audit [DBG] from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:40:18.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:18 smithi175 bash[21527]: audit 2024-03-26T03:40:17.399893+0000 mon.a (mon.0) 485 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T03:40:18.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:18 smithi175 bash[21527]: audit 2024-03-26T03:40:17.406367+0000 mon.a (mon.0) 486 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T03:40:18.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:18 smithi175 bash[21527]: audit 2024-03-26T03:40:17.407959+0000 mon.a (mon.0) 487 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:40:18.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:18 smithi175 bash[29284]: audit 2024-03-26T03:40:17.391878+0000 mgr.y (mgr.14152) 210 : audit [DBG] from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:40:18.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:18 smithi175 bash[29284]: audit 2024-03-26T03:40:17.399893+0000 mon.a (mon.0) 485 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T03:40:18.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:18 smithi175 bash[29284]: audit 2024-03-26T03:40:17.406367+0000 mon.a (mon.0) 486 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T03:40:18.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:18 smithi175 bash[29284]: audit 2024-03-26T03:40:17.407959+0000 mon.a (mon.0) 487 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:40:18.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:18 smithi190 bash[23392]: audit 2024-03-26T03:40:17.391878+0000 mgr.y (mgr.14152) 210 : audit [DBG] from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:40:18.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:18 smithi190 bash[23392]: audit 2024-03-26T03:40:17.399893+0000 mon.a (mon.0) 485 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T03:40:18.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:18 smithi190 bash[23392]: audit 2024-03-26T03:40:17.406367+0000 mon.a (mon.0) 486 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T03:40:18.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:18 smithi190 bash[23392]: audit 2024-03-26T03:40:17.407959+0000 mon.a (mon.0) 487 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:40:19.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:19 smithi175 bash[21527]: cluster 2024-03-26T03:40:18.246599+0000 mgr.y (mgr.14152) 211 : cluster [DBG] pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:19.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:19 smithi175 bash[29284]: cluster 2024-03-26T03:40:18.246599+0000 mgr.y (mgr.14152) 211 : cluster [DBG] pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:19.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:19 smithi190 bash[23392]: cluster 2024-03-26T03:40:18.246599+0000 mgr.y (mgr.14152) 211 : cluster [DBG] pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:21.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:21 smithi175 bash[21527]: cluster 2024-03-26T03:40:20.247204+0000 mgr.y (mgr.14152) 212 : cluster [DBG] pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:21.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:21 smithi175 bash[29284]: cluster 2024-03-26T03:40:20.247204+0000 mgr.y (mgr.14152) 212 : cluster [DBG] pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:21.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:21 smithi190 bash[23392]: cluster 2024-03-26T03:40:20.247204+0000 mgr.y (mgr.14152) 212 : cluster [DBG] pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:23.487 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:23 smithi190 bash[23392]: cluster 2024-03-26T03:40:22.247760+0000 mgr.y (mgr.14152) 213 : cluster [DBG] pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:23.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:23 smithi175 bash[29284]: cluster 2024-03-26T03:40:22.247760+0000 mgr.y (mgr.14152) 213 : cluster [DBG] pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:23.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:23 smithi175 bash[21527]: cluster 2024-03-26T03:40:22.247760+0000 mgr.y (mgr.14152) 213 : cluster [DBG] pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:25.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:25 smithi175 bash[21527]: cluster 2024-03-26T03:40:24.248312+0000 mgr.y (mgr.14152) 214 : cluster [DBG] pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:25.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:25 smithi175 bash[29284]: cluster 2024-03-26T03:40:24.248312+0000 mgr.y (mgr.14152) 214 : cluster [DBG] pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:25.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:25 smithi190 bash[23392]: cluster 2024-03-26T03:40:24.248312+0000 mgr.y (mgr.14152) 214 : cluster [DBG] pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:26.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:26 smithi175 bash[21527]: audit 2024-03-26T03:40:25.748160+0000 mon.a (mon.0) 488 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54aac0c2-d77e-4bf4-b1e5-856b4e997426"}]: dispatch 2024-03-26T03:40:26.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:26 smithi175 bash[21527]: audit 2024-03-26T03:40:25.748575+0000 mon.b (mon.1) 10 : audit [INF] from='client.? 172.21.15.190:0/3116548657' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54aac0c2-d77e-4bf4-b1e5-856b4e997426"}]: dispatch 2024-03-26T03:40:26.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:26 smithi175 bash[21527]: audit 2024-03-26T03:40:25.751859+0000 mon.a (mon.0) 489 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54aac0c2-d77e-4bf4-b1e5-856b4e997426"}]': finished 2024-03-26T03:40:26.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:26 smithi175 bash[21527]: cluster 2024-03-26T03:40:25.757780+0000 mon.a (mon.0) 490 : cluster [DBG] osdmap e27: 5 total, 4 up, 5 in 2024-03-26T03:40:26.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:26 smithi175 bash[21527]: audit 2024-03-26T03:40:25.758421+0000 mon.a (mon.0) 491 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T03:40:26.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:26 smithi175 bash[29284]: audit 2024-03-26T03:40:25.748160+0000 mon.a (mon.0) 488 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54aac0c2-d77e-4bf4-b1e5-856b4e997426"}]: dispatch 2024-03-26T03:40:26.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:26 smithi175 bash[29284]: audit 2024-03-26T03:40:25.748575+0000 mon.b (mon.1) 10 : audit [INF] from='client.? 172.21.15.190:0/3116548657' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54aac0c2-d77e-4bf4-b1e5-856b4e997426"}]: dispatch 2024-03-26T03:40:26.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:26 smithi175 bash[29284]: audit 2024-03-26T03:40:25.751859+0000 mon.a (mon.0) 489 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54aac0c2-d77e-4bf4-b1e5-856b4e997426"}]': finished 2024-03-26T03:40:26.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:26 smithi175 bash[29284]: cluster 2024-03-26T03:40:25.757780+0000 mon.a (mon.0) 490 : cluster [DBG] osdmap e27: 5 total, 4 up, 5 in 2024-03-26T03:40:26.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:26 smithi175 bash[29284]: audit 2024-03-26T03:40:25.758421+0000 mon.a (mon.0) 491 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T03:40:26.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:26 smithi190 bash[23392]: audit 2024-03-26T03:40:25.748160+0000 mon.a (mon.0) 488 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54aac0c2-d77e-4bf4-b1e5-856b4e997426"}]: dispatch 2024-03-26T03:40:26.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:26 smithi190 bash[23392]: audit 2024-03-26T03:40:25.748575+0000 mon.b (mon.1) 10 : audit [INF] from='client.? 172.21.15.190:0/3116548657' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54aac0c2-d77e-4bf4-b1e5-856b4e997426"}]: dispatch 2024-03-26T03:40:26.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:26 smithi190 bash[23392]: audit 2024-03-26T03:40:25.751859+0000 mon.a (mon.0) 489 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54aac0c2-d77e-4bf4-b1e5-856b4e997426"}]': finished 2024-03-26T03:40:26.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:26 smithi190 bash[23392]: cluster 2024-03-26T03:40:25.757780+0000 mon.a (mon.0) 490 : cluster [DBG] osdmap e27: 5 total, 4 up, 5 in 2024-03-26T03:40:26.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:26 smithi190 bash[23392]: audit 2024-03-26T03:40:25.758421+0000 mon.a (mon.0) 491 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T03:40:27.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:27 smithi175 bash[21527]: cluster 2024-03-26T03:40:26.248812+0000 mgr.y (mgr.14152) 215 : cluster [DBG] pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:27.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:27 smithi175 bash[21527]: audit 2024-03-26T03:40:26.480694+0000 mon.b (mon.1) 11 : audit [DBG] from='client.? 172.21.15.190:0/3707622353' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T03:40:27.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:27 smithi175 bash[29284]: cluster 2024-03-26T03:40:26.248812+0000 mgr.y (mgr.14152) 215 : cluster [DBG] pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:27.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:27 smithi175 bash[29284]: audit 2024-03-26T03:40:26.480694+0000 mon.b (mon.1) 11 : audit [DBG] from='client.? 172.21.15.190:0/3707622353' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T03:40:27.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:27 smithi190 bash[23392]: cluster 2024-03-26T03:40:26.248812+0000 mgr.y (mgr.14152) 215 : cluster [DBG] pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:27.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:27 smithi190 bash[23392]: audit 2024-03-26T03:40:26.480694+0000 mon.b (mon.1) 11 : audit [DBG] from='client.? 172.21.15.190:0/3707622353' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T03:40:29.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:29 smithi190 bash[23392]: cluster 2024-03-26T03:40:28.249381+0000 mgr.y (mgr.14152) 216 : cluster [DBG] pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:29.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:29 smithi175 bash[21527]: cluster 2024-03-26T03:40:28.249381+0000 mgr.y (mgr.14152) 216 : cluster [DBG] pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:29.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:29 smithi175 bash[29284]: cluster 2024-03-26T03:40:28.249381+0000 mgr.y (mgr.14152) 216 : cluster [DBG] pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:31.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:31 smithi175 bash[29284]: cluster 2024-03-26T03:40:30.249930+0000 mgr.y (mgr.14152) 217 : cluster [DBG] pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:31.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:31 smithi175 bash[21527]: cluster 2024-03-26T03:40:30.249930+0000 mgr.y (mgr.14152) 217 : cluster [DBG] pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:31.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:31 smithi190 bash[23392]: cluster 2024-03-26T03:40:30.249930+0000 mgr.y (mgr.14152) 217 : cluster [DBG] pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:33.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:33 smithi175 bash[21527]: cluster 2024-03-26T03:40:32.250477+0000 mgr.y (mgr.14152) 218 : cluster [DBG] pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:33.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:33 smithi175 bash[29284]: cluster 2024-03-26T03:40:32.250477+0000 mgr.y (mgr.14152) 218 : cluster [DBG] pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:33.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:33 smithi190 bash[23392]: cluster 2024-03-26T03:40:32.250477+0000 mgr.y (mgr.14152) 218 : cluster [DBG] pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:35.580 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:35 smithi190 bash[23392]: cluster 2024-03-26T03:40:34.251035+0000 mgr.y (mgr.14152) 219 : cluster [DBG] pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:35.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:35 smithi175 bash[21527]: cluster 2024-03-26T03:40:34.251035+0000 mgr.y (mgr.14152) 219 : cluster [DBG] pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:35.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:35 smithi175 bash[29284]: cluster 2024-03-26T03:40:34.251035+0000 mgr.y (mgr.14152) 219 : cluster [DBG] pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:37.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:37 smithi175 bash[29284]: cluster 2024-03-26T03:40:36.251612+0000 mgr.y (mgr.14152) 220 : cluster [DBG] pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:37.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:37 smithi175 bash[21527]: cluster 2024-03-26T03:40:36.251612+0000 mgr.y (mgr.14152) 220 : cluster [DBG] pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:37.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:37 smithi190 bash[23392]: cluster 2024-03-26T03:40:36.251612+0000 mgr.y (mgr.14152) 220 : cluster [DBG] pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:38.628 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:38 smithi190 bash[23392]: audit 2024-03-26T03:40:38.203710+0000 mon.a (mon.0) 492 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-26T03:40:38.629 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:38 smithi190 bash[23392]: audit 2024-03-26T03:40:38.205821+0000 mon.a (mon.0) 493 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:40:38.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:38 smithi175 bash[29284]: audit 2024-03-26T03:40:38.203710+0000 mon.a (mon.0) 492 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-26T03:40:38.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:38 smithi175 bash[29284]: audit 2024-03-26T03:40:38.205821+0000 mon.a (mon.0) 493 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:40:38.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:38 smithi175 bash[21527]: audit 2024-03-26T03:40:38.203710+0000 mon.a (mon.0) 492 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-26T03:40:38.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:38 smithi175 bash[21527]: audit 2024-03-26T03:40:38.205821+0000 mon.a (mon.0) 493 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:40:39.609 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:39 smithi190 bash[23392]: cephadm 2024-03-26T03:40:38.207821+0000 mgr.y (mgr.14152) 221 : cephadm [INF] Deploying daemon osd.4 on smithi190 2024-03-26T03:40:39.609 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:39 smithi190 bash[23392]: cluster 2024-03-26T03:40:38.252113+0000 mgr.y (mgr.14152) 222 : cluster [DBG] pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:39.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:39 smithi175 bash[21527]: cephadm 2024-03-26T03:40:38.207821+0000 mgr.y (mgr.14152) 221 : cephadm [INF] Deploying daemon osd.4 on smithi190 2024-03-26T03:40:39.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:39 smithi175 bash[21527]: cluster 2024-03-26T03:40:38.252113+0000 mgr.y (mgr.14152) 222 : cluster [DBG] pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:39.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:39 smithi175 bash[29284]: cephadm 2024-03-26T03:40:38.207821+0000 mgr.y (mgr.14152) 221 : cephadm [INF] Deploying daemon osd.4 on smithi190 2024-03-26T03:40:39.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:39 smithi175 bash[29284]: cluster 2024-03-26T03:40:38.252113+0000 mgr.y (mgr.14152) 222 : cluster [DBG] pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:41.634 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:41 smithi190 bash[23392]: cluster 2024-03-26T03:40:40.252667+0000 mgr.y (mgr.14152) 223 : cluster [DBG] pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:41.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:41 smithi175 bash[29284]: cluster 2024-03-26T03:40:40.252667+0000 mgr.y (mgr.14152) 223 : cluster [DBG] pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:41.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:41 smithi175 bash[21527]: cluster 2024-03-26T03:40:40.252667+0000 mgr.y (mgr.14152) 223 : cluster [DBG] pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:42.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:42 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:40:42.493 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:40:42 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:40:42.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:42 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:40:42.747 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:40:42 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:40:43.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:43 smithi190 bash[23392]: cluster 2024-03-26T03:40:42.253251+0000 mgr.y (mgr.14152) 224 : cluster [DBG] pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:43.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:43 smithi190 bash[23392]: audit 2024-03-26T03:40:42.768235+0000 mon.a (mon.0) 494 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:43.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:43 smithi190 bash[23392]: audit 2024-03-26T03:40:42.773666+0000 mon.a (mon.0) 495 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:40:43.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:43 smithi190 bash[23392]: audit 2024-03-26T03:40:42.779782+0000 mon.a (mon.0) 496 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:44.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:43 smithi175 bash[29284]: cluster 2024-03-26T03:40:42.253251+0000 mgr.y (mgr.14152) 224 : cluster [DBG] pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:44.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:43 smithi175 bash[29284]: audit 2024-03-26T03:40:42.768235+0000 mon.a (mon.0) 494 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:44.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:43 smithi175 bash[29284]: audit 2024-03-26T03:40:42.773666+0000 mon.a (mon.0) 495 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:40:44.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:43 smithi175 bash[29284]: audit 2024-03-26T03:40:42.779782+0000 mon.a (mon.0) 496 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:44.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:43 smithi175 bash[21527]: cluster 2024-03-26T03:40:42.253251+0000 mgr.y (mgr.14152) 224 : cluster [DBG] pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:44.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:43 smithi175 bash[21527]: audit 2024-03-26T03:40:42.768235+0000 mon.a (mon.0) 494 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:44.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:43 smithi175 bash[21527]: audit 2024-03-26T03:40:42.773666+0000 mon.a (mon.0) 495 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:40:44.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:43 smithi175 bash[21527]: audit 2024-03-26T03:40:42.779782+0000 mon.a (mon.0) 496 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:45.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:45 smithi190 bash[23392]: cluster 2024-03-26T03:40:44.253941+0000 mgr.y (mgr.14152) 225 : cluster [DBG] pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:46.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:45 smithi175 bash[29284]: cluster 2024-03-26T03:40:44.253941+0000 mgr.y (mgr.14152) 225 : cluster [DBG] pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:46.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:45 smithi175 bash[21527]: cluster 2024-03-26T03:40:44.253941+0000 mgr.y (mgr.14152) 225 : cluster [DBG] pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:48.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:47 smithi175 bash[29284]: cluster 2024-03-26T03:40:46.254613+0000 mgr.y (mgr.14152) 226 : cluster [DBG] pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:48.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:47 smithi175 bash[21527]: cluster 2024-03-26T03:40:46.254613+0000 mgr.y (mgr.14152) 226 : cluster [DBG] pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:48.246 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:47 smithi190 bash[23392]: cluster 2024-03-26T03:40:46.254613+0000 mgr.y (mgr.14152) 226 : cluster [DBG] pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:50.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:49 smithi175 bash[21527]: cluster 2024-03-26T03:40:48.255271+0000 mgr.y (mgr.14152) 227 : cluster [DBG] pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:50.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:49 smithi175 bash[21527]: audit 2024-03-26T03:40:49.275041+0000 mon.a (mon.0) 497 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T03:40:50.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:49 smithi175 bash[21527]: audit 2024-03-26T03:40:49.275658+0000 mon.b (mon.1) 12 : audit [INF] from='osd.4 [v2:172.21.15.190:6800/874343325,v1:172.21.15.190:6801/874343325]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T03:40:50.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:49 smithi175 bash[21527]: audit 2024-03-26T03:40:49.772013+0000 mon.a (mon.0) 498 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:50.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:49 smithi175 bash[29284]: cluster 2024-03-26T03:40:48.255271+0000 mgr.y (mgr.14152) 227 : cluster [DBG] pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:50.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:49 smithi175 bash[29284]: audit 2024-03-26T03:40:49.275041+0000 mon.a (mon.0) 497 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T03:40:50.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:49 smithi175 bash[29284]: audit 2024-03-26T03:40:49.275658+0000 mon.b (mon.1) 12 : audit [INF] from='osd.4 [v2:172.21.15.190:6800/874343325,v1:172.21.15.190:6801/874343325]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T03:40:50.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:49 smithi175 bash[29284]: audit 2024-03-26T03:40:49.772013+0000 mon.a (mon.0) 498 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:50.246 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:49 smithi190 bash[23392]: cluster 2024-03-26T03:40:48.255271+0000 mgr.y (mgr.14152) 227 : cluster [DBG] pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:50.246 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:49 smithi190 bash[23392]: audit 2024-03-26T03:40:49.275041+0000 mon.a (mon.0) 497 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T03:40:50.246 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:49 smithi190 bash[23392]: audit 2024-03-26T03:40:49.275658+0000 mon.b (mon.1) 12 : audit [INF] from='osd.4 [v2:172.21.15.190:6800/874343325,v1:172.21.15.190:6801/874343325]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T03:40:50.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:49 smithi190 bash[23392]: audit 2024-03-26T03:40:49.772013+0000 mon.a (mon.0) 498 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:50.836 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:50 smithi190 bash[23392]: audit 2024-03-26T03:40:49.781167+0000 mon.a (mon.0) 499 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:50.836 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:50 smithi190 bash[23392]: audit 2024-03-26T03:40:49.784225+0000 mon.a (mon.0) 500 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-26T03:40:50.836 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:50 smithi190 bash[23392]: audit 2024-03-26T03:40:49.791473+0000 mon.b (mon.1) 13 : audit [INF] from='osd.4 [v2:172.21.15.190:6800/874343325,v1:172.21.15.190:6801/874343325]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-03-26T03:40:50.837 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:50 smithi190 bash[23392]: cluster 2024-03-26T03:40:49.792427+0000 mon.a (mon.0) 501 : cluster [DBG] osdmap e28: 5 total, 4 up, 5 in 2024-03-26T03:40:50.837 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:50 smithi190 bash[23392]: audit 2024-03-26T03:40:49.793109+0000 mon.a (mon.0) 502 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T03:40:50.837 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:50 smithi190 bash[23392]: audit 2024-03-26T03:40:49.793732+0000 mon.a (mon.0) 503 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-03-26T03:40:50.837 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:50 smithi190 bash[23392]: audit 2024-03-26T03:40:50.465266+0000 mon.a (mon.0) 504 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:40:50.837 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:50 smithi190 bash[23392]: audit 2024-03-26T03:40:50.467077+0000 mon.a (mon.0) 505 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:40:50.837 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:50 smithi190 bash[23392]: audit 2024-03-26T03:40:50.476584+0000 mon.a (mon.0) 506 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:51.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:50 smithi175 bash[21527]: audit 2024-03-26T03:40:49.781167+0000 mon.a (mon.0) 499 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:51.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:50 smithi175 bash[21527]: audit 2024-03-26T03:40:49.784225+0000 mon.a (mon.0) 500 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-26T03:40:51.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:50 smithi175 bash[21527]: audit 2024-03-26T03:40:49.791473+0000 mon.b (mon.1) 13 : audit [INF] from='osd.4 [v2:172.21.15.190:6800/874343325,v1:172.21.15.190:6801/874343325]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-03-26T03:40:51.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:50 smithi175 bash[21527]: cluster 2024-03-26T03:40:49.792427+0000 mon.a (mon.0) 501 : cluster [DBG] osdmap e28: 5 total, 4 up, 5 in 2024-03-26T03:40:51.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:50 smithi175 bash[21527]: audit 2024-03-26T03:40:49.793109+0000 mon.a (mon.0) 502 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T03:40:51.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:50 smithi175 bash[21527]: audit 2024-03-26T03:40:49.793732+0000 mon.a (mon.0) 503 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-03-26T03:40:51.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:50 smithi175 bash[21527]: audit 2024-03-26T03:40:50.465266+0000 mon.a (mon.0) 504 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:40:51.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:50 smithi175 bash[21527]: audit 2024-03-26T03:40:50.467077+0000 mon.a (mon.0) 505 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:40:51.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:50 smithi175 bash[21527]: audit 2024-03-26T03:40:50.476584+0000 mon.a (mon.0) 506 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:51.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:50 smithi175 bash[29284]: audit 2024-03-26T03:40:49.781167+0000 mon.a (mon.0) 499 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:51.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:50 smithi175 bash[29284]: audit 2024-03-26T03:40:49.784225+0000 mon.a (mon.0) 500 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-26T03:40:51.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:50 smithi175 bash[29284]: audit 2024-03-26T03:40:49.791473+0000 mon.b (mon.1) 13 : audit [INF] from='osd.4 [v2:172.21.15.190:6800/874343325,v1:172.21.15.190:6801/874343325]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-03-26T03:40:51.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:50 smithi175 bash[29284]: cluster 2024-03-26T03:40:49.792427+0000 mon.a (mon.0) 501 : cluster [DBG] osdmap e28: 5 total, 4 up, 5 in 2024-03-26T03:40:51.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:50 smithi175 bash[29284]: audit 2024-03-26T03:40:49.793109+0000 mon.a (mon.0) 502 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T03:40:51.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:50 smithi175 bash[29284]: audit 2024-03-26T03:40:49.793732+0000 mon.a (mon.0) 503 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-03-26T03:40:51.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:50 smithi175 bash[29284]: audit 2024-03-26T03:40:50.465266+0000 mon.a (mon.0) 504 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:40:51.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:50 smithi175 bash[29284]: audit 2024-03-26T03:40:50.467077+0000 mon.a (mon.0) 505 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:40:51.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:50 smithi175 bash[29284]: audit 2024-03-26T03:40:50.476584+0000 mon.a (mon.0) 506 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:52.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:51 smithi175 bash[21527]: cluster 2024-03-26T03:40:50.255782+0000 mgr.y (mgr.14152) 228 : cluster [DBG] pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:52.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:51 smithi175 bash[21527]: audit 2024-03-26T03:40:50.786950+0000 mon.a (mon.0) 507 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-03-26T03:40:52.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:51 smithi175 bash[21527]: cluster 2024-03-26T03:40:50.793437+0000 mon.a (mon.0) 508 : cluster [DBG] osdmap e29: 5 total, 4 up, 5 in 2024-03-26T03:40:52.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:51 smithi175 bash[21527]: audit 2024-03-26T03:40:50.794749+0000 mon.a (mon.0) 509 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T03:40:52.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:51 smithi175 bash[21527]: audit 2024-03-26T03:40:50.806001+0000 mon.a (mon.0) 510 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T03:40:52.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:51 smithi175 bash[21527]: cluster 2024-03-26T03:40:51.231953+0000 mon.a (mon.0) 511 : cluster [INF] osd.4 [v2:172.21.15.190:6800/874343325,v1:172.21.15.190:6801/874343325] boot 2024-03-26T03:40:52.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:51 smithi175 bash[21527]: cluster 2024-03-26T03:40:51.232071+0000 mon.a (mon.0) 512 : cluster [DBG] osdmap e30: 5 total, 5 up, 5 in 2024-03-26T03:40:52.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:51 smithi175 bash[21527]: audit 2024-03-26T03:40:51.232414+0000 mon.a (mon.0) 513 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T03:40:52.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:51 smithi175 bash[29284]: cluster 2024-03-26T03:40:50.255782+0000 mgr.y (mgr.14152) 228 : cluster [DBG] pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:52.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:51 smithi175 bash[29284]: audit 2024-03-26T03:40:50.786950+0000 mon.a (mon.0) 507 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-03-26T03:40:52.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:51 smithi175 bash[29284]: cluster 2024-03-26T03:40:50.793437+0000 mon.a (mon.0) 508 : cluster [DBG] osdmap e29: 5 total, 4 up, 5 in 2024-03-26T03:40:52.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:51 smithi175 bash[29284]: audit 2024-03-26T03:40:50.794749+0000 mon.a (mon.0) 509 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T03:40:52.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:51 smithi175 bash[29284]: audit 2024-03-26T03:40:50.806001+0000 mon.a (mon.0) 510 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T03:40:52.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:51 smithi175 bash[29284]: cluster 2024-03-26T03:40:51.231953+0000 mon.a (mon.0) 511 : cluster [INF] osd.4 [v2:172.21.15.190:6800/874343325,v1:172.21.15.190:6801/874343325] boot 2024-03-26T03:40:52.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:51 smithi175 bash[29284]: cluster 2024-03-26T03:40:51.232071+0000 mon.a (mon.0) 512 : cluster [DBG] osdmap e30: 5 total, 5 up, 5 in 2024-03-26T03:40:52.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:51 smithi175 bash[29284]: audit 2024-03-26T03:40:51.232414+0000 mon.a (mon.0) 513 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T03:40:52.246 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:51 smithi190 bash[23392]: cluster 2024-03-26T03:40:50.255782+0000 mgr.y (mgr.14152) 228 : cluster [DBG] pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-26T03:40:52.246 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:51 smithi190 bash[23392]: audit 2024-03-26T03:40:50.786950+0000 mon.a (mon.0) 507 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-03-26T03:40:52.246 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:51 smithi190 bash[23392]: cluster 2024-03-26T03:40:50.793437+0000 mon.a (mon.0) 508 : cluster [DBG] osdmap e29: 5 total, 4 up, 5 in 2024-03-26T03:40:52.246 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:51 smithi190 bash[23392]: audit 2024-03-26T03:40:50.794749+0000 mon.a (mon.0) 509 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T03:40:52.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:51 smithi190 bash[23392]: audit 2024-03-26T03:40:50.806001+0000 mon.a (mon.0) 510 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T03:40:52.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:51 smithi190 bash[23392]: cluster 2024-03-26T03:40:51.231953+0000 mon.a (mon.0) 511 : cluster [INF] osd.4 [v2:172.21.15.190:6800/874343325,v1:172.21.15.190:6801/874343325] boot 2024-03-26T03:40:52.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:51 smithi190 bash[23392]: cluster 2024-03-26T03:40:51.232071+0000 mon.a (mon.0) 512 : cluster [DBG] osdmap e30: 5 total, 5 up, 5 in 2024-03-26T03:40:52.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:51 smithi190 bash[23392]: audit 2024-03-26T03:40:51.232414+0000 mon.a (mon.0) 513 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T03:40:53.126 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:53.123+0000 7fe33f7fe700 1 -- 172.21.15.190:0/1625985311 <== mgr.14152 v2:172.21.15.175:6800/2826039805 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe348062a80 con 0x7fe33406e6f0 2024-03-26T03:40:53.126 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 4 on host 'smithi190' 2024-03-26T03:40:53.130 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:53.127+0000 7fe34cdb2700 1 -- 172.21.15.190:0/1625985311 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fe33406e6f0 msgr2=0x7fe334070bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:40:53.130 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:53.127+0000 7fe34cdb2700 1 --2- 172.21.15.190:0/1625985311 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fe33406e6f0 0x7fe334070bb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe348109d40 tx=0x7fe340009040 comp rx=0 tx=0).stop 2024-03-26T03:40:53.130 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:53.127+0000 7fe34cdb2700 1 -- 172.21.15.190:0/1625985311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe34810a880 msgr2=0x7fe3481a59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:40:53.130 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:53.127+0000 7fe34cdb2700 1 --2- 172.21.15.190:0/1625985311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe34810a880 0x7fe3481a59b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe3380047b0 tx=0x7fe338009580 comp rx=0 tx=0).stop 2024-03-26T03:40:53.130 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:53.127+0000 7fe34cdb2700 1 -- 172.21.15.190:0/1625985311 shutdown_connections 2024-03-26T03:40:53.130 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:53.127+0000 7fe34cdb2700 1 --2- 172.21.15.190:0/1625985311 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fe33406e6f0 0x7fe334070bb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:40:53.130 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:53.127+0000 7fe34cdb2700 1 --2- 172.21.15.190:0/1625985311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe34810a880 0x7fe3481a59b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:40:53.130 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:53.127+0000 7fe34cdb2700 1 --2- 172.21.15.190:0/1625985311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe348108ce0 0x7fe34819e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:40:53.130 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:53.127+0000 7fe34cdb2700 1 --2- 172.21.15.190:0/1625985311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe348109ee0 0x7fe34819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:40:53.130 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:53.127+0000 7fe34cdb2700 1 -- 172.21.15.190:0/1625985311 >> 172.21.15.190:0/1625985311 conn(0x7fe348077cd0 msgr2=0x7fe3480797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:40:53.130 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:53.127+0000 7fe34cdb2700 1 -- 172.21.15.190:0/1625985311 shutdown_connections 2024-03-26T03:40:53.130 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:40:53.127+0000 7fe34cdb2700 1 -- 172.21.15.190:0/1625985311 wait complete. 2024-03-26T03:40:53.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:53 smithi175 bash[21527]: cluster 2024-03-26T03:40:50.284194+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2024-03-26T03:40:53.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:53 smithi175 bash[21527]: cluster 2024-03-26T03:40:50.284251+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2024-03-26T03:40:53.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:53 smithi175 bash[21527]: cluster 2024-03-26T03:40:52.231355+0000 mon.a (mon.0) 514 : cluster [DBG] osdmap e31: 5 total, 5 up, 5 in 2024-03-26T03:40:53.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:53 smithi175 bash[21527]: cluster 2024-03-26T03:40:52.256256+0000 mgr.y (mgr.14152) 229 : cluster [DBG] pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:40:53.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:53 smithi175 bash[21527]: audit 2024-03-26T03:40:53.065321+0000 mon.a (mon.0) 515 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:53.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:53 smithi175 bash[21527]: audit 2024-03-26T03:40:53.069416+0000 mon.a (mon.0) 516 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:40:53.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:53 smithi175 bash[21527]: audit 2024-03-26T03:40:53.076086+0000 mon.a (mon.0) 517 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:53.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:53 smithi175 bash[29284]: cluster 2024-03-26T03:40:50.284194+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2024-03-26T03:40:53.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:53 smithi175 bash[29284]: cluster 2024-03-26T03:40:50.284251+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2024-03-26T03:40:53.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:53 smithi175 bash[29284]: cluster 2024-03-26T03:40:52.231355+0000 mon.a (mon.0) 514 : cluster [DBG] osdmap e31: 5 total, 5 up, 5 in 2024-03-26T03:40:53.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:53 smithi175 bash[29284]: cluster 2024-03-26T03:40:52.256256+0000 mgr.y (mgr.14152) 229 : cluster [DBG] pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:40:53.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:53 smithi175 bash[29284]: audit 2024-03-26T03:40:53.065321+0000 mon.a (mon.0) 515 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:53.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:53 smithi175 bash[29284]: audit 2024-03-26T03:40:53.069416+0000 mon.a (mon.0) 516 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:40:53.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:53 smithi175 bash[29284]: audit 2024-03-26T03:40:53.076086+0000 mon.a (mon.0) 517 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:53.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:53 smithi190 bash[23392]: cluster 2024-03-26T03:40:50.284194+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2024-03-26T03:40:53.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:53 smithi190 bash[23392]: cluster 2024-03-26T03:40:50.284251+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2024-03-26T03:40:53.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:53 smithi190 bash[23392]: cluster 2024-03-26T03:40:52.231355+0000 mon.a (mon.0) 514 : cluster [DBG] osdmap e31: 5 total, 5 up, 5 in 2024-03-26T03:40:53.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:53 smithi190 bash[23392]: cluster 2024-03-26T03:40:52.256256+0000 mgr.y (mgr.14152) 229 : cluster [DBG] pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:40:53.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:53 smithi190 bash[23392]: audit 2024-03-26T03:40:53.065321+0000 mon.a (mon.0) 515 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:53.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:53 smithi190 bash[23392]: audit 2024-03-26T03:40:53.069416+0000 mon.a (mon.0) 516 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:40:53.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:53 smithi190 bash[23392]: audit 2024-03-26T03:40:53.076086+0000 mon.a (mon.0) 517 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:40:53.793 DEBUG:teuthology.orchestra.run.smithi190:osd.4> sudo journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@osd.4.service 2024-03-26T03:40:53.795 INFO:tasks.cephadm:Deploying osd.5 on smithi190 with /dev/vg_nvme/lv_3... 2024-03-26T03:40:53.796 DEBUG:teuthology.orchestra.run.smithi190:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-26T03:40:54.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:54 smithi175 bash[21527]: cluster 2024-03-26T03:40:53.239914+0000 mon.a (mon.0) 518 : cluster [DBG] osdmap e32: 5 total, 5 up, 5 in 2024-03-26T03:40:54.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:54 smithi175 bash[29284]: cluster 2024-03-26T03:40:53.239914+0000 mon.a (mon.0) 518 : cluster [DBG] osdmap e32: 5 total, 5 up, 5 in 2024-03-26T03:40:54.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:54 smithi190 bash[23392]: cluster 2024-03-26T03:40:53.239914+0000 mon.a (mon.0) 518 : cluster [DBG] osdmap e32: 5 total, 5 up, 5 in 2024-03-26T03:40:55.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:55 smithi175 bash[21527]: cluster 2024-03-26T03:40:54.241051+0000 mon.a (mon.0) 519 : cluster [DBG] osdmap e33: 5 total, 5 up, 5 in 2024-03-26T03:40:55.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:55 smithi175 bash[21527]: cluster 2024-03-26T03:40:54.256780+0000 mgr.y (mgr.14152) 230 : cluster [DBG] pgmap v204: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 166 KiB/s, 0 objects/s recovering 2024-03-26T03:40:55.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:55 smithi175 bash[29284]: cluster 2024-03-26T03:40:54.241051+0000 mon.a (mon.0) 519 : cluster [DBG] osdmap e33: 5 total, 5 up, 5 in 2024-03-26T03:40:55.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:55 smithi175 bash[29284]: cluster 2024-03-26T03:40:54.256780+0000 mgr.y (mgr.14152) 230 : cluster [DBG] pgmap v204: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 166 KiB/s, 0 objects/s recovering 2024-03-26T03:40:55.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:55 smithi190 bash[23392]: cluster 2024-03-26T03:40:54.241051+0000 mon.a (mon.0) 519 : cluster [DBG] osdmap e33: 5 total, 5 up, 5 in 2024-03-26T03:40:55.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:55 smithi190 bash[23392]: cluster 2024-03-26T03:40:54.256780+0000 mgr.y (mgr.14152) 230 : cluster [DBG] pgmap v204: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 166 KiB/s, 0 objects/s recovering 2024-03-26T03:40:56.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:56 smithi175 bash[21527]: cluster 2024-03-26T03:40:55.242076+0000 mon.a (mon.0) 520 : cluster [DBG] osdmap e34: 5 total, 5 up, 5 in 2024-03-26T03:40:56.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:56 smithi175 bash[29284]: cluster 2024-03-26T03:40:55.242076+0000 mon.a (mon.0) 520 : cluster [DBG] osdmap e34: 5 total, 5 up, 5 in 2024-03-26T03:40:56.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:56 smithi190 bash[23392]: cluster 2024-03-26T03:40:55.242076+0000 mon.a (mon.0) 520 : cluster [DBG] osdmap e34: 5 total, 5 up, 5 in 2024-03-26T03:40:57.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:57 smithi175 bash[21527]: cluster 2024-03-26T03:40:56.257365+0000 mgr.y (mgr.14152) 231 : cluster [DBG] pgmap v206: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 143 KiB/s, 0 objects/s recovering 2024-03-26T03:40:57.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:57 smithi175 bash[29284]: cluster 2024-03-26T03:40:56.257365+0000 mgr.y (mgr.14152) 231 : cluster [DBG] pgmap v206: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 143 KiB/s, 0 objects/s recovering 2024-03-26T03:40:57.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:57 smithi190 bash[23392]: cluster 2024-03-26T03:40:56.257365+0000 mgr.y (mgr.14152) 231 : cluster [DBG] pgmap v206: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 143 KiB/s, 0 objects/s recovering 2024-03-26T03:40:58.050 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T03:40:59.731 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:40:59 smithi190 bash[23392]: cluster 2024-03-26T03:40:58.257938+0000 mgr.y (mgr.14152) 232 : cluster [DBG] pgmap v207: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 144 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T03:40:59.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:40:59 smithi175 bash[21527]: cluster 2024-03-26T03:40:58.257938+0000 mgr.y (mgr.14152) 232 : cluster [DBG] pgmap v207: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 144 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T03:40:59.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:40:59 smithi175 bash[29284]: cluster 2024-03-26T03:40:58.257938+0000 mgr.y (mgr.14152) 232 : cluster [DBG] pgmap v207: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 144 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T03:41:01.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:01 smithi175 bash[21527]: cluster 2024-03-26T03:41:00.258522+0000 mgr.y (mgr.14152) 233 : cluster [DBG] pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-26T03:41:01.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:01 smithi175 bash[29284]: cluster 2024-03-26T03:41:00.258522+0000 mgr.y (mgr.14152) 233 : cluster [DBG] pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-26T03:41:01.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:01 smithi190 bash[23392]: cluster 2024-03-26T03:41:00.258522+0000 mgr.y (mgr.14152) 233 : cluster [DBG] pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-26T03:41:02.624 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-26T03:41:02.624 INFO:teuthology.orchestra.run.smithi190.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-26T03:41:02.624 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10+0 records in 2024-03-26T03:41:02.624 INFO:teuthology.orchestra.run.smithi190.stderr:10+0 records out 2024-03-26T03:41:02.624 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0137328 s, 764 MB/s 2024-03-26T03:41:02.624 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping successful for: 2024-03-26T03:41:03.520 DEBUG:teuthology.orchestra.run.smithi190:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi190:vg_nvme/lv_3 2024-03-26T03:41:03.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:03 smithi175 bash[21527]: cluster 2024-03-26T03:41:02.259092+0000 mgr.y (mgr.14152) 234 : cluster [DBG] pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:03.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:03 smithi175 bash[29284]: cluster 2024-03-26T03:41:02.259092+0000 mgr.y (mgr.14152) 234 : cluster [DBG] pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:03.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:03 smithi190 bash[23392]: cluster 2024-03-26T03:41:02.259092+0000 mgr.y (mgr.14152) 234 : cluster [DBG] pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:05.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:05 smithi175 bash[21527]: cephadm 2024-03-26T03:41:04.016732+0000 mgr.y (mgr.14152) 235 : cephadm [INF] Detected new or changed devices on smithi190 2024-03-26T03:41:05.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:05 smithi175 bash[21527]: audit 2024-03-26T03:41:04.025802+0000 mon.a (mon.0) 521 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:05.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:05 smithi175 bash[21527]: audit 2024-03-26T03:41:04.034824+0000 mon.a (mon.0) 522 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:05.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:05 smithi175 bash[21527]: audit 2024-03-26T03:41:04.036954+0000 mon.a (mon.0) 523 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:41:05.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:05 smithi175 bash[21527]: cephadm 2024-03-26T03:41:04.038512+0000 mgr.y (mgr.14152) 236 : cephadm [INF] Adjusting osd_memory_target on smithi190 to 17265M 2024-03-26T03:41:05.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:05 smithi175 bash[21527]: audit 2024-03-26T03:41:04.044673+0000 mon.a (mon.0) 524 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:05.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:05 smithi175 bash[21527]: audit 2024-03-26T03:41:04.049041+0000 mon.a (mon.0) 525 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:41:05.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:05 smithi175 bash[21527]: audit 2024-03-26T03:41:04.050778+0000 mon.a (mon.0) 526 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:41:05.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:05 smithi175 bash[21527]: audit 2024-03-26T03:41:04.060671+0000 mon.a (mon.0) 527 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:05.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:05 smithi175 bash[29284]: cephadm 2024-03-26T03:41:04.016732+0000 mgr.y (mgr.14152) 235 : cephadm [INF] Detected new or changed devices on smithi190 2024-03-26T03:41:05.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:05 smithi175 bash[29284]: audit 2024-03-26T03:41:04.025802+0000 mon.a (mon.0) 521 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:05.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:05 smithi175 bash[29284]: audit 2024-03-26T03:41:04.034824+0000 mon.a (mon.0) 522 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:05.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:05 smithi175 bash[29284]: audit 2024-03-26T03:41:04.036954+0000 mon.a (mon.0) 523 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:41:05.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:05 smithi175 bash[29284]: cephadm 2024-03-26T03:41:04.038512+0000 mgr.y (mgr.14152) 236 : cephadm [INF] Adjusting osd_memory_target on smithi190 to 17265M 2024-03-26T03:41:05.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:05 smithi175 bash[29284]: audit 2024-03-26T03:41:04.044673+0000 mon.a (mon.0) 524 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:05.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:05 smithi175 bash[29284]: audit 2024-03-26T03:41:04.049041+0000 mon.a (mon.0) 525 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:41:05.495 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:05 smithi175 bash[29284]: audit 2024-03-26T03:41:04.050778+0000 mon.a (mon.0) 526 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:41:05.495 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:05 smithi175 bash[29284]: audit 2024-03-26T03:41:04.060671+0000 mon.a (mon.0) 527 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:05.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:05 smithi190 bash[23392]: cephadm 2024-03-26T03:41:04.016732+0000 mgr.y (mgr.14152) 235 : cephadm [INF] Detected new or changed devices on smithi190 2024-03-26T03:41:05.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:05 smithi190 bash[23392]: audit 2024-03-26T03:41:04.025802+0000 mon.a (mon.0) 521 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:05.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:05 smithi190 bash[23392]: audit 2024-03-26T03:41:04.034824+0000 mon.a (mon.0) 522 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:05.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:05 smithi190 bash[23392]: audit 2024-03-26T03:41:04.036954+0000 mon.a (mon.0) 523 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:41:05.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:05 smithi190 bash[23392]: cephadm 2024-03-26T03:41:04.038512+0000 mgr.y (mgr.14152) 236 : cephadm [INF] Adjusting osd_memory_target on smithi190 to 17265M 2024-03-26T03:41:05.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:05 smithi190 bash[23392]: audit 2024-03-26T03:41:04.044673+0000 mon.a (mon.0) 524 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:05.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:05 smithi190 bash[23392]: audit 2024-03-26T03:41:04.049041+0000 mon.a (mon.0) 525 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:41:05.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:05 smithi190 bash[23392]: audit 2024-03-26T03:41:04.050778+0000 mon.a (mon.0) 526 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:41:05.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:05 smithi190 bash[23392]: audit 2024-03-26T03:41:04.060671+0000 mon.a (mon.0) 527 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:06.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:06 smithi175 bash[21527]: cluster 2024-03-26T03:41:04.259641+0000 mgr.y (mgr.14152) 237 : cluster [DBG] pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:06.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:06 smithi175 bash[29284]: cluster 2024-03-26T03:41:04.259641+0000 mgr.y (mgr.14152) 237 : cluster [DBG] pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:06.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:06 smithi190 bash[23392]: cluster 2024-03-26T03:41:04.259641+0000 mgr.y (mgr.14152) 237 : cluster [DBG] pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:08.242 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T03:41:08.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:08 smithi175 bash[21527]: cluster 2024-03-26T03:41:06.260139+0000 mgr.y (mgr.14152) 238 : cluster [DBG] pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:08.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:08 smithi175 bash[29284]: cluster 2024-03-26T03:41:06.260139+0000 mgr.y (mgr.14152) 238 : cluster [DBG] pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:08.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:08 smithi190 bash[23392]: cluster 2024-03-26T03:41:06.260139+0000 mgr.y (mgr.14152) 238 : cluster [DBG] pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:10.246 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:10 smithi190 bash[23392]: cluster 2024-03-26T03:41:08.260695+0000 mgr.y (mgr.14152) 239 : cluster [DBG] pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:10.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:10 smithi175 bash[21527]: cluster 2024-03-26T03:41:08.260695+0000 mgr.y (mgr.14152) 239 : cluster [DBG] pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:10.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:10 smithi175 bash[29284]: cluster 2024-03-26T03:41:08.260695+0000 mgr.y (mgr.14152) 239 : cluster [DBG] pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:10.881 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc71650a700 1 -- 172.21.15.190:0/1411675283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc710104280 msgr2=0x7fc71010ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:41:10.881 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc71650a700 1 --2- 172.21.15.190:0/1411675283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc710104280 0x7fc71010ab30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc70800b0d0 tx=0x7fc70800b3e0 comp rx=0 tx=0).stop 2024-03-26T03:41:10.881 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc71650a700 1 -- 172.21.15.190:0/1411675283 shutdown_connections 2024-03-26T03:41:10.881 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc71650a700 1 --2- 172.21.15.190:0/1411675283 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7101026e0 0x7fc710102b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:41:10.881 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc71650a700 1 --2- 172.21.15.190:0/1411675283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc710104280 0x7fc71010ab30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:41:10.882 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc71650a700 1 --2- 172.21.15.190:0/1411675283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7101038e0 0x7fc710103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:41:10.882 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc71650a700 1 -- 172.21.15.190:0/1411675283 >> 172.21.15.190:0/1411675283 conn(0x7fc7100fdca0 msgr2=0x7fc7101000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:41:10.882 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc71650a700 1 -- 172.21.15.190:0/1411675283 shutdown_connections 2024-03-26T03:41:10.882 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc71650a700 1 -- 172.21.15.190:0/1411675283 wait complete. 2024-03-26T03:41:10.882 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc71650a700 1 Processor -- start 2024-03-26T03:41:10.882 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc71650a700 1 -- start start 2024-03-26T03:41:10.883 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc71650a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7101026e0 0x7fc710197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:41:10.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc71650a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7101038e0 0x7fc7101982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:41:10.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc71650a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc710104280 0x7fc71019f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:41:10.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc71650a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc71010d7c0 con 0x7fc7101038e0 2024-03-26T03:41:10.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc71650a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc71010d640 con 0x7fc710104280 2024-03-26T03:41:10.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc71650a700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc71010d940 con 0x7fc7101026e0 2024-03-26T03:41:10.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc714aa7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc710104280 0x7fc71019f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:41:10.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc714aa7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc710104280 0x7fc71019f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:55994/0 (socket says 172.21.15.190:55994) 2024-03-26T03:41:10.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc714aa7700 1 -- 172.21.15.190:0/1970491298 learned_addr learned my addr 172.21.15.190:0/1970491298 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-03-26T03:41:10.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc70f7fe700 1 --2- 172.21.15.190:0/1970491298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7101038e0 0x7fc7101982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:41:10.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc70ffff700 1 --2- 172.21.15.190:0/1970491298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7101026e0 0x7fc710197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:41:10.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc714aa7700 1 -- 172.21.15.190:0/1970491298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7101026e0 msgr2=0x7fc710197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:41:10.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc714aa7700 1 --2- 172.21.15.190:0/1970491298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7101026e0 0x7fc710197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:41:10.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc714aa7700 1 -- 172.21.15.190:0/1970491298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7101038e0 msgr2=0x7fc7101982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:41:10.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc714aa7700 1 --2- 172.21.15.190:0/1970491298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7101038e0 0x7fc7101982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:41:10.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc714aa7700 1 -- 172.21.15.190:0/1970491298 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71019fa10 con 0x7fc710104280 2024-03-26T03:41:10.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc70f7fe700 1 --2- 172.21.15.190:0/1970491298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7101038e0 0x7fc7101982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T03:41:10.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc714aa7700 1 --2- 172.21.15.190:0/1970491298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc710104280 0x7fc71019f370 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc708007d10 tx=0x7fc708009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:41:10.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc70d7fa700 1 -- 172.21.15.190:0/1970491298 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708022070 con 0x7fc710104280 2024-03-26T03:41:10.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc70d7fa700 1 -- 172.21.15.190:0/1970491298 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fc708013d10 con 0x7fc710104280 2024-03-26T03:41:10.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc71650a700 1 -- 172.21.15.190:0/1970491298 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc71019fca0 con 0x7fc710104280 2024-03-26T03:41:10.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.879+0000 7fc70d7fa700 1 -- 172.21.15.190:0/1970491298 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70801bd10 con 0x7fc710104280 2024-03-26T03:41:10.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.883+0000 7fc71650a700 1 -- 172.21.15.190:0/1970491298 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7101a00f0 con 0x7fc710104280 2024-03-26T03:41:10.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.883+0000 7fc71650a700 1 -- 172.21.15.190:0/1970491298 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc710050300 con 0x7fc710104280 2024-03-26T03:41:10.886 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.883+0000 7fc70d7fa700 1 -- 172.21.15.190:0/1970491298 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 15) ==== 92475+0+0 (secure 0 0 0) 0x7fc708025070 con 0x7fc710104280 2024-03-26T03:41:10.886 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.883+0000 7fc70d7fa700 1 --2- 172.21.15.190:0/1970491298 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fc6fc06e6f0 0x7fc6fc070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:41:10.886 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.883+0000 7fc70d7fa700 1 -- 172.21.15.190:0/1970491298 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 3991+0+0 (secure 0 0 0) 0x7fc70800f020 con 0x7fc710104280 2024-03-26T03:41:10.886 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.883+0000 7fc70ffff700 1 --2- 172.21.15.190:0/1970491298 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fc6fc06e6f0 0x7fc6fc070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:41:10.886 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.883+0000 7fc70ffff700 1 --2- 172.21.15.190:0/1970491298 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fc6fc06e6f0 0x7fc6fc070bb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc700005950 tx=0x7fc7000058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:41:10.890 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:10.887+0000 7fc70d7fa700 1 -- 172.21.15.190:0/1970491298 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7fc70802c020 con 0x7fc710104280 2024-03-26T03:41:11.108 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:11.107+0000 7fc71650a700 1 -- 172.21.15.190:0/1970491298 --> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc710062a80 con 0x7fc6fc06e6f0 2024-03-26T03:41:12.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:12 smithi175 bash[21527]: cluster 2024-03-26T03:41:10.261233+0000 mgr.y (mgr.14152) 240 : cluster [DBG] pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:12.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:12 smithi175 bash[21527]: audit 2024-03-26T03:41:11.112087+0000 mgr.y (mgr.14152) 241 : audit [DBG] from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:41:12.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:12 smithi175 bash[21527]: audit 2024-03-26T03:41:11.119150+0000 mon.a (mon.0) 528 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T03:41:12.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:12 smithi175 bash[21527]: audit 2024-03-26T03:41:11.127657+0000 mon.a (mon.0) 529 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T03:41:12.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:12 smithi175 bash[21527]: audit 2024-03-26T03:41:11.129321+0000 mon.a (mon.0) 530 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:41:12.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:12 smithi175 bash[29284]: cluster 2024-03-26T03:41:10.261233+0000 mgr.y (mgr.14152) 240 : cluster [DBG] pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:12.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:12 smithi175 bash[29284]: audit 2024-03-26T03:41:11.112087+0000 mgr.y (mgr.14152) 241 : audit [DBG] from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:41:12.495 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:12 smithi175 bash[29284]: audit 2024-03-26T03:41:11.119150+0000 mon.a (mon.0) 528 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T03:41:12.495 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:12 smithi175 bash[29284]: audit 2024-03-26T03:41:11.127657+0000 mon.a (mon.0) 529 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T03:41:12.495 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:12 smithi175 bash[29284]: audit 2024-03-26T03:41:11.129321+0000 mon.a (mon.0) 530 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:41:12.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:12 smithi190 bash[23392]: cluster 2024-03-26T03:41:10.261233+0000 mgr.y (mgr.14152) 240 : cluster [DBG] pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:12.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:12 smithi190 bash[23392]: audit 2024-03-26T03:41:11.112087+0000 mgr.y (mgr.14152) 241 : audit [DBG] from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:41:12.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:12 smithi190 bash[23392]: audit 2024-03-26T03:41:11.119150+0000 mon.a (mon.0) 528 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T03:41:12.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:12 smithi190 bash[23392]: audit 2024-03-26T03:41:11.127657+0000 mon.a (mon.0) 529 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T03:41:12.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:12 smithi190 bash[23392]: audit 2024-03-26T03:41:11.129321+0000 mon.a (mon.0) 530 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:41:14.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:14 smithi175 bash[21527]: cluster 2024-03-26T03:41:12.261801+0000 mgr.y (mgr.14152) 242 : cluster [DBG] pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:14.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:14 smithi175 bash[29284]: cluster 2024-03-26T03:41:12.261801+0000 mgr.y (mgr.14152) 242 : cluster [DBG] pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:14.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:14 smithi190 bash[23392]: cluster 2024-03-26T03:41:12.261801+0000 mgr.y (mgr.14152) 242 : cluster [DBG] pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:16.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:16 smithi175 bash[21527]: cluster 2024-03-26T03:41:14.262395+0000 mgr.y (mgr.14152) 243 : cluster [DBG] pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:16.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:16 smithi175 bash[29284]: cluster 2024-03-26T03:41:14.262395+0000 mgr.y (mgr.14152) 243 : cluster [DBG] pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:16.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:16 smithi190 bash[23392]: cluster 2024-03-26T03:41:14.262395+0000 mgr.y (mgr.14152) 243 : cluster [DBG] pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:18.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:18 smithi190 bash[23392]: cluster 2024-03-26T03:41:16.262941+0000 mgr.y (mgr.14152) 244 : cluster [DBG] pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:18.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:18 smithi175 bash[21527]: cluster 2024-03-26T03:41:16.262941+0000 mgr.y (mgr.14152) 244 : cluster [DBG] pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:18.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:18 smithi175 bash[29284]: cluster 2024-03-26T03:41:16.262941+0000 mgr.y (mgr.14152) 244 : cluster [DBG] pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:20.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:20 smithi175 bash[21527]: cluster 2024-03-26T03:41:18.263590+0000 mgr.y (mgr.14152) 245 : cluster [DBG] pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:20.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:20 smithi175 bash[21527]: audit 2024-03-26T03:41:19.646554+0000 mon.a (mon.0) 531 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1eb5f92-1729-48c7-a89c-3265609fe40d"}]: dispatch 2024-03-26T03:41:20.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:20 smithi175 bash[21527]: audit 2024-03-26T03:41:19.647616+0000 mon.b (mon.1) 14 : audit [INF] from='client.? 172.21.15.190:0/2271377626' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1eb5f92-1729-48c7-a89c-3265609fe40d"}]: dispatch 2024-03-26T03:41:20.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:20 smithi175 bash[21527]: audit 2024-03-26T03:41:19.650044+0000 mon.a (mon.0) 532 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1eb5f92-1729-48c7-a89c-3265609fe40d"}]': finished 2024-03-26T03:41:20.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:20 smithi175 bash[21527]: cluster 2024-03-26T03:41:19.655903+0000 mon.a (mon.0) 533 : cluster [DBG] osdmap e35: 6 total, 5 up, 6 in 2024-03-26T03:41:20.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:20 smithi175 bash[21527]: audit 2024-03-26T03:41:19.656340+0000 mon.a (mon.0) 534 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T03:41:20.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:20 smithi175 bash[29284]: cluster 2024-03-26T03:41:18.263590+0000 mgr.y (mgr.14152) 245 : cluster [DBG] pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:20.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:20 smithi175 bash[29284]: audit 2024-03-26T03:41:19.646554+0000 mon.a (mon.0) 531 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1eb5f92-1729-48c7-a89c-3265609fe40d"}]: dispatch 2024-03-26T03:41:20.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:20 smithi175 bash[29284]: audit 2024-03-26T03:41:19.647616+0000 mon.b (mon.1) 14 : audit [INF] from='client.? 172.21.15.190:0/2271377626' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1eb5f92-1729-48c7-a89c-3265609fe40d"}]: dispatch 2024-03-26T03:41:20.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:20 smithi175 bash[29284]: audit 2024-03-26T03:41:19.650044+0000 mon.a (mon.0) 532 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1eb5f92-1729-48c7-a89c-3265609fe40d"}]': finished 2024-03-26T03:41:20.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:20 smithi175 bash[29284]: cluster 2024-03-26T03:41:19.655903+0000 mon.a (mon.0) 533 : cluster [DBG] osdmap e35: 6 total, 5 up, 6 in 2024-03-26T03:41:20.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:20 smithi175 bash[29284]: audit 2024-03-26T03:41:19.656340+0000 mon.a (mon.0) 534 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T03:41:20.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:20 smithi190 bash[23392]: cluster 2024-03-26T03:41:18.263590+0000 mgr.y (mgr.14152) 245 : cluster [DBG] pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:20.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:20 smithi190 bash[23392]: audit 2024-03-26T03:41:19.646554+0000 mon.a (mon.0) 531 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1eb5f92-1729-48c7-a89c-3265609fe40d"}]: dispatch 2024-03-26T03:41:20.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:20 smithi190 bash[23392]: audit 2024-03-26T03:41:19.647616+0000 mon.b (mon.1) 14 : audit [INF] from='client.? 172.21.15.190:0/2271377626' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1eb5f92-1729-48c7-a89c-3265609fe40d"}]: dispatch 2024-03-26T03:41:20.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:20 smithi190 bash[23392]: audit 2024-03-26T03:41:19.650044+0000 mon.a (mon.0) 532 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1eb5f92-1729-48c7-a89c-3265609fe40d"}]': finished 2024-03-26T03:41:20.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:20 smithi190 bash[23392]: cluster 2024-03-26T03:41:19.655903+0000 mon.a (mon.0) 533 : cluster [DBG] osdmap e35: 6 total, 5 up, 6 in 2024-03-26T03:41:20.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:20 smithi190 bash[23392]: audit 2024-03-26T03:41:19.656340+0000 mon.a (mon.0) 534 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T03:41:21.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:21 smithi175 bash[29284]: audit 2024-03-26T03:41:20.399820+0000 mon.b (mon.1) 15 : audit [DBG] from='client.? 172.21.15.190:0/3075529786' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T03:41:21.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:21 smithi175 bash[21527]: audit 2024-03-26T03:41:20.399820+0000 mon.b (mon.1) 15 : audit [DBG] from='client.? 172.21.15.190:0/3075529786' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T03:41:21.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:21 smithi190 bash[23392]: audit 2024-03-26T03:41:20.399820+0000 mon.b (mon.1) 15 : audit [DBG] from='client.? 172.21.15.190:0/3075529786' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T03:41:22.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:22 smithi175 bash[21527]: cluster 2024-03-26T03:41:20.264215+0000 mgr.y (mgr.14152) 246 : cluster [DBG] pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:22.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:22 smithi175 bash[29284]: cluster 2024-03-26T03:41:20.264215+0000 mgr.y (mgr.14152) 246 : cluster [DBG] pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:22.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:22 smithi190 bash[23392]: cluster 2024-03-26T03:41:20.264215+0000 mgr.y (mgr.14152) 246 : cluster [DBG] pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:24.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:24 smithi175 bash[21527]: cluster 2024-03-26T03:41:22.264821+0000 mgr.y (mgr.14152) 247 : cluster [DBG] pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:24.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:24 smithi175 bash[29284]: cluster 2024-03-26T03:41:22.264821+0000 mgr.y (mgr.14152) 247 : cluster [DBG] pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:24.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:24 smithi190 bash[23392]: cluster 2024-03-26T03:41:22.264821+0000 mgr.y (mgr.14152) 247 : cluster [DBG] pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:26.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:26 smithi175 bash[21527]: cluster 2024-03-26T03:41:24.265401+0000 mgr.y (mgr.14152) 248 : cluster [DBG] pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:26.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:26 smithi175 bash[29284]: cluster 2024-03-26T03:41:24.265401+0000 mgr.y (mgr.14152) 248 : cluster [DBG] pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:26.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:26 smithi190 bash[23392]: cluster 2024-03-26T03:41:24.265401+0000 mgr.y (mgr.14152) 248 : cluster [DBG] pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:28.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:28 smithi175 bash[21527]: cluster 2024-03-26T03:41:26.265955+0000 mgr.y (mgr.14152) 249 : cluster [DBG] pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:28.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:28 smithi175 bash[29284]: cluster 2024-03-26T03:41:26.265955+0000 mgr.y (mgr.14152) 249 : cluster [DBG] pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:28.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:28 smithi190 bash[23392]: cluster 2024-03-26T03:41:26.265955+0000 mgr.y (mgr.14152) 249 : cluster [DBG] pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:30.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:30 smithi175 bash[21527]: cluster 2024-03-26T03:41:28.266496+0000 mgr.y (mgr.14152) 250 : cluster [DBG] pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:30.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:30 smithi175 bash[29284]: cluster 2024-03-26T03:41:28.266496+0000 mgr.y (mgr.14152) 250 : cluster [DBG] pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:30.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:30 smithi190 bash[23392]: cluster 2024-03-26T03:41:28.266496+0000 mgr.y (mgr.14152) 250 : cluster [DBG] pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:32.246 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:32 smithi190 bash[23392]: cluster 2024-03-26T03:41:30.267051+0000 mgr.y (mgr.14152) 251 : cluster [DBG] pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:32.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:32 smithi175 bash[21527]: cluster 2024-03-26T03:41:30.267051+0000 mgr.y (mgr.14152) 251 : cluster [DBG] pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:32.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:32 smithi175 bash[29284]: cluster 2024-03-26T03:41:30.267051+0000 mgr.y (mgr.14152) 251 : cluster [DBG] pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:33.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:33 smithi175 bash[21527]: audit 2024-03-26T03:41:32.432564+0000 mon.a (mon.0) 535 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-26T03:41:33.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:33 smithi175 bash[21527]: audit 2024-03-26T03:41:32.434320+0000 mon.a (mon.0) 536 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:41:33.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:33 smithi175 bash[29284]: audit 2024-03-26T03:41:32.432564+0000 mon.a (mon.0) 535 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-26T03:41:33.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:33 smithi175 bash[29284]: audit 2024-03-26T03:41:32.434320+0000 mon.a (mon.0) 536 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:41:33.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:33 smithi190 bash[23392]: audit 2024-03-26T03:41:32.432564+0000 mon.a (mon.0) 535 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-26T03:41:33.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:33 smithi190 bash[23392]: audit 2024-03-26T03:41:32.434320+0000 mon.a (mon.0) 536 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:41:34.468 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:34 smithi190 bash[23392]: cluster 2024-03-26T03:41:32.267638+0000 mgr.y (mgr.14152) 252 : cluster [DBG] pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:34.468 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:34 smithi190 bash[23392]: cephadm 2024-03-26T03:41:32.435930+0000 mgr.y (mgr.14152) 253 : cephadm [INF] Deploying daemon osd.5 on smithi190 2024-03-26T03:41:34.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:34 smithi175 bash[21527]: cluster 2024-03-26T03:41:32.267638+0000 mgr.y (mgr.14152) 252 : cluster [DBG] pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:34.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:34 smithi175 bash[21527]: cephadm 2024-03-26T03:41:32.435930+0000 mgr.y (mgr.14152) 253 : cephadm [INF] Deploying daemon osd.5 on smithi190 2024-03-26T03:41:34.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:34 smithi175 bash[29284]: cluster 2024-03-26T03:41:32.267638+0000 mgr.y (mgr.14152) 252 : cluster [DBG] pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:34.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:34 smithi175 bash[29284]: cephadm 2024-03-26T03:41:32.435930+0000 mgr.y (mgr.14152) 253 : cephadm [INF] Deploying daemon osd.5 on smithi190 2024-03-26T03:41:36.409 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:36 smithi190 bash[23392]: cluster 2024-03-26T03:41:34.268137+0000 mgr.y (mgr.14152) 254 : cluster [DBG] pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:36.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:36 smithi175 bash[21527]: cluster 2024-03-26T03:41:34.268137+0000 mgr.y (mgr.14152) 254 : cluster [DBG] pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:36.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:36 smithi175 bash[29284]: cluster 2024-03-26T03:41:34.268137+0000 mgr.y (mgr.14152) 254 : cluster [DBG] pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:37.372 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:41:37 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:41:37.372 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 03:41:37 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:41:37.372 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:37 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:41:37.652 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:37 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:41:37.652 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:41:37 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:41:37.652 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 03:41:37 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:41:38.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:38 smithi175 bash[21527]: cluster 2024-03-26T03:41:36.268683+0000 mgr.y (mgr.14152) 255 : cluster [DBG] pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:38.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:38 smithi175 bash[21527]: audit 2024-03-26T03:41:37.653035+0000 mon.a (mon.0) 537 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:38.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:38 smithi175 bash[21527]: audit 2024-03-26T03:41:37.662068+0000 mon.a (mon.0) 538 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:38.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:38 smithi175 bash[21527]: audit 2024-03-26T03:41:37.682055+0000 mon.a (mon.0) 539 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:41:38.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:38 smithi175 bash[29284]: cluster 2024-03-26T03:41:36.268683+0000 mgr.y (mgr.14152) 255 : cluster [DBG] pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:38.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:38 smithi175 bash[29284]: audit 2024-03-26T03:41:37.653035+0000 mon.a (mon.0) 537 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:38.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:38 smithi175 bash[29284]: audit 2024-03-26T03:41:37.662068+0000 mon.a (mon.0) 538 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:38.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:38 smithi175 bash[29284]: audit 2024-03-26T03:41:37.682055+0000 mon.a (mon.0) 539 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:41:38.495 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:38 smithi190 bash[23392]: cluster 2024-03-26T03:41:36.268683+0000 mgr.y (mgr.14152) 255 : cluster [DBG] pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:38.495 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:38 smithi190 bash[23392]: audit 2024-03-26T03:41:37.653035+0000 mon.a (mon.0) 537 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:38.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:38 smithi190 bash[23392]: audit 2024-03-26T03:41:37.662068+0000 mon.a (mon.0) 538 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:38.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:38 smithi190 bash[23392]: audit 2024-03-26T03:41:37.682055+0000 mon.a (mon.0) 539 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:41:40.246 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:40 smithi190 bash[23392]: cluster 2024-03-26T03:41:38.269223+0000 mgr.y (mgr.14152) 256 : cluster [DBG] pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:40.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:40 smithi175 bash[21527]: cluster 2024-03-26T03:41:38.269223+0000 mgr.y (mgr.14152) 256 : cluster [DBG] pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:40.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:40 smithi175 bash[29284]: cluster 2024-03-26T03:41:38.269223+0000 mgr.y (mgr.14152) 256 : cluster [DBG] pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:42.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:42 smithi175 bash[29284]: cluster 2024-03-26T03:41:40.269819+0000 mgr.y (mgr.14152) 257 : cluster [DBG] pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:42.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:42 smithi175 bash[21527]: cluster 2024-03-26T03:41:40.269819+0000 mgr.y (mgr.14152) 257 : cluster [DBG] pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:42.495 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:42 smithi190 bash[23392]: cluster 2024-03-26T03:41:40.269819+0000 mgr.y (mgr.14152) 257 : cluster [DBG] pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:44.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:43 smithi175 bash[21527]: cluster 2024-03-26T03:41:42.270319+0000 mgr.y (mgr.14152) 258 : cluster [DBG] pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:44.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:43 smithi175 bash[21527]: audit 2024-03-26T03:41:42.976037+0000 mon.a (mon.0) 540 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:44.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:43 smithi175 bash[21527]: audit 2024-03-26T03:41:42.985678+0000 mon.a (mon.0) 541 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:44.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:43 smithi175 bash[21527]: audit 2024-03-26T03:41:42.987916+0000 mon.a (mon.0) 542 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:41:44.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:43 smithi175 bash[21527]: audit 2024-03-26T03:41:42.989827+0000 mon.a (mon.0) 543 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:41:44.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:43 smithi175 bash[21527]: audit 2024-03-26T03:41:42.999905+0000 mon.a (mon.0) 544 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:44.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:43 smithi175 bash[29284]: cluster 2024-03-26T03:41:42.270319+0000 mgr.y (mgr.14152) 258 : cluster [DBG] pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:44.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:43 smithi175 bash[29284]: audit 2024-03-26T03:41:42.976037+0000 mon.a (mon.0) 540 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:44.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:43 smithi175 bash[29284]: audit 2024-03-26T03:41:42.985678+0000 mon.a (mon.0) 541 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:44.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:43 smithi175 bash[29284]: audit 2024-03-26T03:41:42.987916+0000 mon.a (mon.0) 542 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:41:44.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:43 smithi175 bash[29284]: audit 2024-03-26T03:41:42.989827+0000 mon.a (mon.0) 543 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:41:44.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:43 smithi175 bash[29284]: audit 2024-03-26T03:41:42.999905+0000 mon.a (mon.0) 544 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:44.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:43 smithi190 bash[23392]: cluster 2024-03-26T03:41:42.270319+0000 mgr.y (mgr.14152) 258 : cluster [DBG] pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:44.246 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:43 smithi190 bash[23392]: audit 2024-03-26T03:41:42.976037+0000 mon.a (mon.0) 540 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:44.246 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:43 smithi190 bash[23392]: audit 2024-03-26T03:41:42.985678+0000 mon.a (mon.0) 541 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:44.246 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:43 smithi190 bash[23392]: audit 2024-03-26T03:41:42.987916+0000 mon.a (mon.0) 542 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:41:44.246 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:43 smithi190 bash[23392]: audit 2024-03-26T03:41:42.989827+0000 mon.a (mon.0) 543 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:41:44.246 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:43 smithi190 bash[23392]: audit 2024-03-26T03:41:42.999905+0000 mon.a (mon.0) 544 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:45.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:44 smithi175 bash[21527]: audit 2024-03-26T03:41:44.372707+0000 mon.a (mon.0) 545 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T03:41:45.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:44 smithi175 bash[21527]: audit 2024-03-26T03:41:44.374257+0000 mon.b (mon.1) 16 : audit [INF] from='osd.5 [v2:172.21.15.190:6808/4089468657,v1:172.21.15.190:6809/4089468657]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T03:41:45.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:44 smithi175 bash[29284]: audit 2024-03-26T03:41:44.372707+0000 mon.a (mon.0) 545 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T03:41:45.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:44 smithi175 bash[29284]: audit 2024-03-26T03:41:44.374257+0000 mon.b (mon.1) 16 : audit [INF] from='osd.5 [v2:172.21.15.190:6808/4089468657,v1:172.21.15.190:6809/4089468657]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T03:41:45.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:44 smithi190 bash[23392]: audit 2024-03-26T03:41:44.372707+0000 mon.a (mon.0) 545 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T03:41:45.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:44 smithi190 bash[23392]: audit 2024-03-26T03:41:44.374257+0000 mon.b (mon.1) 16 : audit [INF] from='osd.5 [v2:172.21.15.190:6808/4089468657,v1:172.21.15.190:6809/4089468657]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T03:41:46.246 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:46 smithi190 bash[23392]: cluster 2024-03-26T03:41:44.270907+0000 mgr.y (mgr.14152) 259 : cluster [DBG] pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:46.246 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:46 smithi190 bash[23392]: audit 2024-03-26T03:41:44.983219+0000 mon.a (mon.0) 546 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-26T03:41:46.246 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:46 smithi190 bash[23392]: cluster 2024-03-26T03:41:44.988528+0000 mon.a (mon.0) 547 : cluster [DBG] osdmap e36: 6 total, 5 up, 6 in 2024-03-26T03:41:46.246 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:46 smithi190 bash[23392]: audit 2024-03-26T03:41:44.989218+0000 mon.a (mon.0) 548 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T03:41:46.246 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:46 smithi190 bash[23392]: audit 2024-03-26T03:41:44.990413+0000 mon.a (mon.0) 549 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-03-26T03:41:46.246 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:46 smithi190 bash[23392]: audit 2024-03-26T03:41:44.990988+0000 mon.b (mon.1) 17 : audit [INF] from='osd.5 [v2:172.21.15.190:6808/4089468657,v1:172.21.15.190:6809/4089468657]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-03-26T03:41:46.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:46 smithi175 bash[21527]: cluster 2024-03-26T03:41:44.270907+0000 mgr.y (mgr.14152) 259 : cluster [DBG] pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:46.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:46 smithi175 bash[21527]: audit 2024-03-26T03:41:44.983219+0000 mon.a (mon.0) 546 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-26T03:41:46.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:46 smithi175 bash[21527]: cluster 2024-03-26T03:41:44.988528+0000 mon.a (mon.0) 547 : cluster [DBG] osdmap e36: 6 total, 5 up, 6 in 2024-03-26T03:41:46.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:46 smithi175 bash[21527]: audit 2024-03-26T03:41:44.989218+0000 mon.a (mon.0) 548 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T03:41:46.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:46 smithi175 bash[21527]: audit 2024-03-26T03:41:44.990413+0000 mon.a (mon.0) 549 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-03-26T03:41:46.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:46 smithi175 bash[21527]: audit 2024-03-26T03:41:44.990988+0000 mon.b (mon.1) 17 : audit [INF] from='osd.5 [v2:172.21.15.190:6808/4089468657,v1:172.21.15.190:6809/4089468657]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-03-26T03:41:46.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:46 smithi175 bash[29284]: cluster 2024-03-26T03:41:44.270907+0000 mgr.y (mgr.14152) 259 : cluster [DBG] pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:46.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:46 smithi175 bash[29284]: audit 2024-03-26T03:41:44.983219+0000 mon.a (mon.0) 546 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-26T03:41:46.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:46 smithi175 bash[29284]: cluster 2024-03-26T03:41:44.988528+0000 mon.a (mon.0) 547 : cluster [DBG] osdmap e36: 6 total, 5 up, 6 in 2024-03-26T03:41:46.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:46 smithi175 bash[29284]: audit 2024-03-26T03:41:44.989218+0000 mon.a (mon.0) 548 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T03:41:46.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:46 smithi175 bash[29284]: audit 2024-03-26T03:41:44.990413+0000 mon.a (mon.0) 549 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-03-26T03:41:46.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:46 smithi175 bash[29284]: audit 2024-03-26T03:41:44.990988+0000 mon.b (mon.1) 17 : audit [INF] from='osd.5 [v2:172.21.15.190:6808/4089468657,v1:172.21.15.190:6809/4089468657]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-03-26T03:41:46.705 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:46.706+0000 7fc70d7fa700 1 -- 172.21.15.190:0/1970491298 <== mgr.14152 v2:172.21.15.175:6800/2826039805 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc710062a80 con 0x7fc6fc06e6f0 2024-03-26T03:41:46.706 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 5 on host 'smithi190' 2024-03-26T03:41:46.709 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:46.710+0000 7fc71650a700 1 -- 172.21.15.190:0/1970491298 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fc6fc06e6f0 msgr2=0x7fc6fc070bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:41:46.709 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:46.710+0000 7fc71650a700 1 --2- 172.21.15.190:0/1970491298 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fc6fc06e6f0 0x7fc6fc070bb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc700005950 tx=0x7fc7000058e0 comp rx=0 tx=0).stop 2024-03-26T03:41:46.709 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:46.710+0000 7fc71650a700 1 -- 172.21.15.190:0/1970491298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc710104280 msgr2=0x7fc71019f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:41:46.709 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:46.710+0000 7fc71650a700 1 --2- 172.21.15.190:0/1970491298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc710104280 0x7fc71019f370 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc708007d10 tx=0x7fc708009bf0 comp rx=0 tx=0).stop 2024-03-26T03:41:46.709 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:46.710+0000 7fc71650a700 1 -- 172.21.15.190:0/1970491298 shutdown_connections 2024-03-26T03:41:46.709 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:46.710+0000 7fc71650a700 1 --2- 172.21.15.190:0/1970491298 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fc6fc06e6f0 0x7fc6fc070bb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:41:46.709 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:46.710+0000 7fc71650a700 1 --2- 172.21.15.190:0/1970491298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7101026e0 0x7fc710197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:41:46.709 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:46.710+0000 7fc71650a700 1 --2- 172.21.15.190:0/1970491298 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc710104280 0x7fc71019f370 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:41:46.709 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:46.710+0000 7fc71650a700 1 --2- 172.21.15.190:0/1970491298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7101038e0 0x7fc7101982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:41:46.710 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:46.710+0000 7fc71650a700 1 -- 172.21.15.190:0/1970491298 >> 172.21.15.190:0/1970491298 conn(0x7fc7100fdca0 msgr2=0x7fc7100ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:41:46.710 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:46.710+0000 7fc71650a700 1 -- 172.21.15.190:0/1970491298 shutdown_connections 2024-03-26T03:41:46.710 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:41:46.710+0000 7fc71650a700 1 -- 172.21.15.190:0/1970491298 wait complete. 2024-03-26T03:41:47.341 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:47 smithi190 bash[23392]: audit 2024-03-26T03:41:45.985776+0000 mon.a (mon.0) 550 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-03-26T03:41:47.342 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:47 smithi190 bash[23392]: cluster 2024-03-26T03:41:45.993896+0000 mon.a (mon.0) 551 : cluster [DBG] osdmap e37: 6 total, 5 up, 6 in 2024-03-26T03:41:47.342 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:47 smithi190 bash[23392]: audit 2024-03-26T03:41:45.995154+0000 mon.a (mon.0) 552 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T03:41:47.342 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:47 smithi190 bash[23392]: cluster 2024-03-26T03:41:46.279361+0000 mon.a (mon.0) 553 : cluster [DBG] osdmap e38: 6 total, 5 up, 6 in 2024-03-26T03:41:47.342 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:47 smithi190 bash[23392]: audit 2024-03-26T03:41:46.281350+0000 mon.a (mon.0) 554 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T03:41:47.342 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:47 smithi190 bash[23392]: audit 2024-03-26T03:41:46.475216+0000 mon.a (mon.0) 555 : audit [INF] from='osd.5 ' entity='osd.5' 2024-03-26T03:41:47.342 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:47 smithi190 bash[23392]: audit 2024-03-26T03:41:46.681641+0000 mon.a (mon.0) 556 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:47.342 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:47 smithi190 bash[23392]: audit 2024-03-26T03:41:46.693044+0000 mon.a (mon.0) 557 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:47.342 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:47 smithi190 bash[23392]: audit 2024-03-26T03:41:46.696008+0000 mon.a (mon.0) 558 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:41:47.342 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:47 smithi190 bash[23392]: audit 2024-03-26T03:41:46.993602+0000 mon.a (mon.0) 559 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T03:41:47.344 DEBUG:teuthology.orchestra.run.smithi190:osd.5> sudo journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@osd.5.service 2024-03-26T03:41:47.347 INFO:tasks.cephadm:Deploying osd.6 on smithi190 with /dev/vg_nvme/lv_2... 2024-03-26T03:41:47.347 DEBUG:teuthology.orchestra.run.smithi190:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-26T03:41:47.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:47 smithi175 bash[21527]: audit 2024-03-26T03:41:45.985776+0000 mon.a (mon.0) 550 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-03-26T03:41:47.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:47 smithi175 bash[21527]: cluster 2024-03-26T03:41:45.993896+0000 mon.a (mon.0) 551 : cluster [DBG] osdmap e37: 6 total, 5 up, 6 in 2024-03-26T03:41:47.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:47 smithi175 bash[21527]: audit 2024-03-26T03:41:45.995154+0000 mon.a (mon.0) 552 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T03:41:47.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:47 smithi175 bash[21527]: cluster 2024-03-26T03:41:46.279361+0000 mon.a (mon.0) 553 : cluster [DBG] osdmap e38: 6 total, 5 up, 6 in 2024-03-26T03:41:47.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:47 smithi175 bash[21527]: audit 2024-03-26T03:41:46.281350+0000 mon.a (mon.0) 554 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T03:41:47.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:47 smithi175 bash[21527]: audit 2024-03-26T03:41:46.475216+0000 mon.a (mon.0) 555 : audit [INF] from='osd.5 ' entity='osd.5' 2024-03-26T03:41:47.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:47 smithi175 bash[21527]: audit 2024-03-26T03:41:46.681641+0000 mon.a (mon.0) 556 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:47.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:47 smithi175 bash[21527]: audit 2024-03-26T03:41:46.693044+0000 mon.a (mon.0) 557 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:47.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:47 smithi175 bash[21527]: audit 2024-03-26T03:41:46.696008+0000 mon.a (mon.0) 558 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:41:47.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:47 smithi175 bash[21527]: audit 2024-03-26T03:41:46.993602+0000 mon.a (mon.0) 559 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T03:41:47.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:47 smithi175 bash[29284]: audit 2024-03-26T03:41:45.985776+0000 mon.a (mon.0) 550 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-03-26T03:41:47.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:47 smithi175 bash[29284]: cluster 2024-03-26T03:41:45.993896+0000 mon.a (mon.0) 551 : cluster [DBG] osdmap e37: 6 total, 5 up, 6 in 2024-03-26T03:41:47.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:47 smithi175 bash[29284]: audit 2024-03-26T03:41:45.995154+0000 mon.a (mon.0) 552 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T03:41:47.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:47 smithi175 bash[29284]: cluster 2024-03-26T03:41:46.279361+0000 mon.a (mon.0) 553 : cluster [DBG] osdmap e38: 6 total, 5 up, 6 in 2024-03-26T03:41:47.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:47 smithi175 bash[29284]: audit 2024-03-26T03:41:46.281350+0000 mon.a (mon.0) 554 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T03:41:47.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:47 smithi175 bash[29284]: audit 2024-03-26T03:41:46.475216+0000 mon.a (mon.0) 555 : audit [INF] from='osd.5 ' entity='osd.5' 2024-03-26T03:41:47.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:47 smithi175 bash[29284]: audit 2024-03-26T03:41:46.681641+0000 mon.a (mon.0) 556 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:47.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:47 smithi175 bash[29284]: audit 2024-03-26T03:41:46.693044+0000 mon.a (mon.0) 557 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:47.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:47 smithi175 bash[29284]: audit 2024-03-26T03:41:46.696008+0000 mon.a (mon.0) 558 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:41:47.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:47 smithi175 bash[29284]: audit 2024-03-26T03:41:46.993602+0000 mon.a (mon.0) 559 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T03:41:48.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:48 smithi175 bash[21527]: cluster 2024-03-26T03:41:45.327455+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2024-03-26T03:41:48.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:48 smithi175 bash[21527]: cluster 2024-03-26T03:41:45.327527+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2024-03-26T03:41:48.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:48 smithi175 bash[21527]: cluster 2024-03-26T03:41:46.271589+0000 mgr.y (mgr.14152) 260 : cluster [DBG] pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:48.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:48 smithi175 bash[21527]: cluster 2024-03-26T03:41:47.371078+0000 mon.a (mon.0) 560 : cluster [INF] osd.5 [v2:172.21.15.190:6808/4089468657,v1:172.21.15.190:6809/4089468657] boot 2024-03-26T03:41:48.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:48 smithi175 bash[21527]: cluster 2024-03-26T03:41:47.371230+0000 mon.a (mon.0) 561 : cluster [DBG] osdmap e39: 6 total, 6 up, 6 in 2024-03-26T03:41:48.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:48 smithi175 bash[21527]: audit 2024-03-26T03:41:47.371580+0000 mon.a (mon.0) 562 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T03:41:48.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:48 smithi175 bash[29284]: cluster 2024-03-26T03:41:45.327455+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2024-03-26T03:41:48.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:48 smithi175 bash[29284]: cluster 2024-03-26T03:41:45.327527+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2024-03-26T03:41:48.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:48 smithi175 bash[29284]: cluster 2024-03-26T03:41:46.271589+0000 mgr.y (mgr.14152) 260 : cluster [DBG] pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:48.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:48 smithi175 bash[29284]: cluster 2024-03-26T03:41:47.371078+0000 mon.a (mon.0) 560 : cluster [INF] osd.5 [v2:172.21.15.190:6808/4089468657,v1:172.21.15.190:6809/4089468657] boot 2024-03-26T03:41:48.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:48 smithi175 bash[29284]: cluster 2024-03-26T03:41:47.371230+0000 mon.a (mon.0) 561 : cluster [DBG] osdmap e39: 6 total, 6 up, 6 in 2024-03-26T03:41:48.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:48 smithi175 bash[29284]: audit 2024-03-26T03:41:47.371580+0000 mon.a (mon.0) 562 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T03:41:48.495 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:48 smithi190 bash[23392]: cluster 2024-03-26T03:41:45.327455+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2024-03-26T03:41:48.495 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:48 smithi190 bash[23392]: cluster 2024-03-26T03:41:45.327527+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2024-03-26T03:41:48.495 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:48 smithi190 bash[23392]: cluster 2024-03-26T03:41:46.271589+0000 mgr.y (mgr.14152) 260 : cluster [DBG] pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T03:41:48.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:48 smithi190 bash[23392]: cluster 2024-03-26T03:41:47.371078+0000 mon.a (mon.0) 560 : cluster [INF] osd.5 [v2:172.21.15.190:6808/4089468657,v1:172.21.15.190:6809/4089468657] boot 2024-03-26T03:41:48.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:48 smithi190 bash[23392]: cluster 2024-03-26T03:41:47.371230+0000 mon.a (mon.0) 561 : cluster [DBG] osdmap e39: 6 total, 6 up, 6 in 2024-03-26T03:41:48.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:48 smithi190 bash[23392]: audit 2024-03-26T03:41:47.371580+0000 mon.a (mon.0) 562 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T03:41:49.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:49 smithi175 bash[21527]: cluster 2024-03-26T03:41:48.272131+0000 mgr.y (mgr.14152) 261 : cluster [DBG] pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:41:49.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:49 smithi175 bash[21527]: cluster 2024-03-26T03:41:48.378917+0000 mon.a (mon.0) 563 : cluster [DBG] osdmap e40: 6 total, 6 up, 6 in 2024-03-26T03:41:49.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:49 smithi175 bash[29284]: cluster 2024-03-26T03:41:48.272131+0000 mgr.y (mgr.14152) 261 : cluster [DBG] pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:41:49.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:49 smithi175 bash[29284]: cluster 2024-03-26T03:41:48.378917+0000 mon.a (mon.0) 563 : cluster [DBG] osdmap e40: 6 total, 6 up, 6 in 2024-03-26T03:41:49.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:49 smithi190 bash[23392]: cluster 2024-03-26T03:41:48.272131+0000 mgr.y (mgr.14152) 261 : cluster [DBG] pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:41:49.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:49 smithi190 bash[23392]: cluster 2024-03-26T03:41:48.378917+0000 mon.a (mon.0) 563 : cluster [DBG] osdmap e40: 6 total, 6 up, 6 in 2024-03-26T03:41:50.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:50 smithi175 bash[21527]: cluster 2024-03-26T03:41:49.376909+0000 mon.a (mon.0) 564 : cluster [DBG] osdmap e41: 6 total, 6 up, 6 in 2024-03-26T03:41:50.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:50 smithi175 bash[29284]: cluster 2024-03-26T03:41:49.376909+0000 mon.a (mon.0) 564 : cluster [DBG] osdmap e41: 6 total, 6 up, 6 in 2024-03-26T03:41:50.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:50 smithi190 bash[23392]: cluster 2024-03-26T03:41:49.376909+0000 mon.a (mon.0) 564 : cluster [DBG] osdmap e41: 6 total, 6 up, 6 in 2024-03-26T03:41:51.703 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T03:41:51.724 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:51 smithi190 bash[23392]: cluster 2024-03-26T03:41:50.272678+0000 mgr.y (mgr.14152) 262 : cluster [DBG] pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:41:51.724 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:51 smithi190 bash[23392]: cluster 2024-03-26T03:41:50.400315+0000 mon.a (mon.0) 565 : cluster [DBG] osdmap e42: 6 total, 6 up, 6 in 2024-03-26T03:41:51.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:51 smithi175 bash[21527]: cluster 2024-03-26T03:41:50.272678+0000 mgr.y (mgr.14152) 262 : cluster [DBG] pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:41:51.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:51 smithi175 bash[21527]: cluster 2024-03-26T03:41:50.400315+0000 mon.a (mon.0) 565 : cluster [DBG] osdmap e42: 6 total, 6 up, 6 in 2024-03-26T03:41:51.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:51 smithi175 bash[29284]: cluster 2024-03-26T03:41:50.272678+0000 mgr.y (mgr.14152) 262 : cluster [DBG] pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:41:51.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:51 smithi175 bash[29284]: cluster 2024-03-26T03:41:50.400315+0000 mon.a (mon.0) 565 : cluster [DBG] osdmap e42: 6 total, 6 up, 6 in 2024-03-26T03:41:52.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:52 smithi175 bash[21527]: cluster 2024-03-26T03:41:51.407284+0000 mon.a (mon.0) 566 : cluster [DBG] osdmap e43: 6 total, 6 up, 6 in 2024-03-26T03:41:52.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:52 smithi175 bash[29284]: cluster 2024-03-26T03:41:51.407284+0000 mon.a (mon.0) 566 : cluster [DBG] osdmap e43: 6 total, 6 up, 6 in 2024-03-26T03:41:52.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:52 smithi190 bash[23392]: cluster 2024-03-26T03:41:51.407284+0000 mon.a (mon.0) 566 : cluster [DBG] osdmap e43: 6 total, 6 up, 6 in 2024-03-26T03:41:53.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:53 smithi175 bash[21527]: cluster 2024-03-26T03:41:52.273220+0000 mgr.y (mgr.14152) 263 : cluster [DBG] pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:41:53.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:53 smithi175 bash[29284]: cluster 2024-03-26T03:41:52.273220+0000 mgr.y (mgr.14152) 263 : cluster [DBG] pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:41:53.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:53 smithi190 bash[23392]: cluster 2024-03-26T03:41:52.273220+0000 mgr.y (mgr.14152) 263 : cluster [DBG] pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:41:55.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:55 smithi175 bash[21527]: cluster 2024-03-26T03:41:54.273800+0000 mgr.y (mgr.14152) 264 : cluster [DBG] pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:41:55.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:55 smithi175 bash[29284]: cluster 2024-03-26T03:41:54.273800+0000 mgr.y (mgr.14152) 264 : cluster [DBG] pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:41:55.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:55 smithi190 bash[23392]: cluster 2024-03-26T03:41:54.273800+0000 mgr.y (mgr.14152) 264 : cluster [DBG] pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:41:56.069 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-26T03:41:56.069 INFO:teuthology.orchestra.run.smithi190.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-26T03:41:56.069 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10+0 records in 2024-03-26T03:41:56.069 INFO:teuthology.orchestra.run.smithi190.stderr:10+0 records out 2024-03-26T03:41:56.069 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0142658 s, 735 MB/s 2024-03-26T03:41:56.070 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping successful for: 2024-03-26T03:41:56.941 DEBUG:teuthology.orchestra.run.smithi190:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi190:vg_nvme/lv_2 2024-03-26T03:41:57.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:57 smithi175 bash[21527]: cluster 2024-03-26T03:41:56.274330+0000 mgr.y (mgr.14152) 265 : cluster [DBG] pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:41:57.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:57 smithi175 bash[21527]: audit 2024-03-26T03:41:57.357536+0000 mon.a (mon.0) 567 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:57.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:57 smithi175 bash[21527]: audit 2024-03-26T03:41:57.367082+0000 mon.a (mon.0) 568 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:57.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:57 smithi175 bash[21527]: audit 2024-03-26T03:41:57.369499+0000 mon.a (mon.0) 569 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:41:57.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:57 smithi175 bash[21527]: audit 2024-03-26T03:41:57.371416+0000 mon.a (mon.0) 570 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:41:57.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:57 smithi175 bash[21527]: audit 2024-03-26T03:41:57.379233+0000 mon.a (mon.0) 571 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:57.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:57 smithi175 bash[21527]: audit 2024-03-26T03:41:57.383558+0000 mon.a (mon.0) 572 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:41:57.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:57 smithi175 bash[21527]: audit 2024-03-26T03:41:57.385371+0000 mon.a (mon.0) 573 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:41:57.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:57 smithi175 bash[21527]: audit 2024-03-26T03:41:57.395218+0000 mon.a (mon.0) 574 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:57.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:57 smithi175 bash[29284]: cluster 2024-03-26T03:41:56.274330+0000 mgr.y (mgr.14152) 265 : cluster [DBG] pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:41:57.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:57 smithi175 bash[29284]: audit 2024-03-26T03:41:57.357536+0000 mon.a (mon.0) 567 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:57.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:57 smithi175 bash[29284]: audit 2024-03-26T03:41:57.367082+0000 mon.a (mon.0) 568 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:57.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:57 smithi175 bash[29284]: audit 2024-03-26T03:41:57.369499+0000 mon.a (mon.0) 569 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:41:57.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:57 smithi175 bash[29284]: audit 2024-03-26T03:41:57.371416+0000 mon.a (mon.0) 570 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:41:57.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:57 smithi175 bash[29284]: audit 2024-03-26T03:41:57.379233+0000 mon.a (mon.0) 571 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:57.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:57 smithi175 bash[29284]: audit 2024-03-26T03:41:57.383558+0000 mon.a (mon.0) 572 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:41:57.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:57 smithi175 bash[29284]: audit 2024-03-26T03:41:57.385371+0000 mon.a (mon.0) 573 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:41:57.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:57 smithi175 bash[29284]: audit 2024-03-26T03:41:57.395218+0000 mon.a (mon.0) 574 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:57.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:57 smithi190 bash[23392]: cluster 2024-03-26T03:41:56.274330+0000 mgr.y (mgr.14152) 265 : cluster [DBG] pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:41:57.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:57 smithi190 bash[23392]: audit 2024-03-26T03:41:57.357536+0000 mon.a (mon.0) 567 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:57.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:57 smithi190 bash[23392]: audit 2024-03-26T03:41:57.367082+0000 mon.a (mon.0) 568 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:57.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:57 smithi190 bash[23392]: audit 2024-03-26T03:41:57.369499+0000 mon.a (mon.0) 569 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:41:57.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:57 smithi190 bash[23392]: audit 2024-03-26T03:41:57.371416+0000 mon.a (mon.0) 570 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:41:57.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:57 smithi190 bash[23392]: audit 2024-03-26T03:41:57.379233+0000 mon.a (mon.0) 571 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:57.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:57 smithi190 bash[23392]: audit 2024-03-26T03:41:57.383558+0000 mon.a (mon.0) 572 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:41:57.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:57 smithi190 bash[23392]: audit 2024-03-26T03:41:57.385371+0000 mon.a (mon.0) 573 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:41:57.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:57 smithi190 bash[23392]: audit 2024-03-26T03:41:57.395218+0000 mon.a (mon.0) 574 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:41:58.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:58 smithi175 bash[21527]: cephadm 2024-03-26T03:41:57.348561+0000 mgr.y (mgr.14152) 266 : cephadm [INF] Detected new or changed devices on smithi190 2024-03-26T03:41:58.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:58 smithi175 bash[21527]: cephadm 2024-03-26T03:41:57.372749+0000 mgr.y (mgr.14152) 267 : cephadm [INF] Adjusting osd_memory_target on smithi190 to 8632M 2024-03-26T03:41:58.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:58 smithi175 bash[29284]: cephadm 2024-03-26T03:41:57.348561+0000 mgr.y (mgr.14152) 266 : cephadm [INF] Detected new or changed devices on smithi190 2024-03-26T03:41:58.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:58 smithi175 bash[29284]: cephadm 2024-03-26T03:41:57.372749+0000 mgr.y (mgr.14152) 267 : cephadm [INF] Adjusting osd_memory_target on smithi190 to 8632M 2024-03-26T03:41:58.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:58 smithi190 bash[23392]: cephadm 2024-03-26T03:41:57.348561+0000 mgr.y (mgr.14152) 266 : cephadm [INF] Detected new or changed devices on smithi190 2024-03-26T03:41:58.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:58 smithi190 bash[23392]: cephadm 2024-03-26T03:41:57.372749+0000 mgr.y (mgr.14152) 267 : cephadm [INF] Adjusting osd_memory_target on smithi190 to 8632M 2024-03-26T03:41:59.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:41:59 smithi175 bash[29284]: cluster 2024-03-26T03:41:58.274869+0000 mgr.y (mgr.14152) 268 : cluster [DBG] pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:41:59.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:41:59 smithi175 bash[21527]: cluster 2024-03-26T03:41:58.274869+0000 mgr.y (mgr.14152) 268 : cluster [DBG] pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:41:59.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:41:59 smithi190 bash[23392]: cluster 2024-03-26T03:41:58.274869+0000 mgr.y (mgr.14152) 268 : cluster [DBG] pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:01.672 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T03:42:01.695 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:01 smithi190 bash[23392]: cluster 2024-03-26T03:42:00.275459+0000 mgr.y (mgr.14152) 269 : cluster [DBG] pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:01.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:01 smithi175 bash[29284]: cluster 2024-03-26T03:42:00.275459+0000 mgr.y (mgr.14152) 269 : cluster [DBG] pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:01.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:01 smithi175 bash[21527]: cluster 2024-03-26T03:42:00.275459+0000 mgr.y (mgr.14152) 269 : cluster [DBG] pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:03.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:03 smithi175 bash[21527]: cluster 2024-03-26T03:42:02.275956+0000 mgr.y (mgr.14152) 270 : cluster [DBG] pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:03.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:03 smithi175 bash[29284]: cluster 2024-03-26T03:42:02.275956+0000 mgr.y (mgr.14152) 270 : cluster [DBG] pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:03.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:03 smithi190 bash[23392]: cluster 2024-03-26T03:42:02.275956+0000 mgr.y (mgr.14152) 270 : cluster [DBG] pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:04.285 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.286+0000 7f258b687700 1 -- 172.21.15.190:0/3761317001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2584103160 msgr2=0x7f2584105550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:42:04.285 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.286+0000 7f258b687700 1 --2- 172.21.15.190:0/3761317001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2584103160 0x7f2584105550 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2574009990 tx=0x7f2574009ca0 comp rx=0 tx=0).stop 2024-03-26T03:42:04.285 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.286+0000 7f258b687700 1 -- 172.21.15.190:0/3761317001 shutdown_connections 2024-03-26T03:42:04.285 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.286+0000 7f258b687700 1 --2- 172.21.15.190:0/3761317001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25841083c0 0x7f258410a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:42:04.285 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.286+0000 7f258b687700 1 --2- 172.21.15.190:0/3761317001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2584103160 0x7f2584105550 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:42:04.285 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.286+0000 7f258b687700 1 --2- 172.21.15.190:0/3761317001 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2584105a90 0x7f2584107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:42:04.285 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.286+0000 7f258b687700 1 -- 172.21.15.190:0/3761317001 >> 172.21.15.190:0/3761317001 conn(0x7f25840fcb70 msgr2=0x7f25840fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:42:04.285 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.286+0000 7f258b687700 1 -- 172.21.15.190:0/3761317001 shutdown_connections 2024-03-26T03:42:04.285 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.286+0000 7f258b687700 1 -- 172.21.15.190:0/3761317001 wait complete. 2024-03-26T03:42:04.287 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.290+0000 7f258b687700 1 Processor -- start 2024-03-26T03:42:04.287 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.290+0000 7f258b687700 1 -- start start 2024-03-26T03:42:04.287 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.290+0000 7f258b687700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2584103160 0x7f2584197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:42:04.288 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.290+0000 7f258b687700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2584105a90 0x7f2584198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:42:04.288 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.290+0000 7f258b687700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25841083c0 0x7f258419f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:42:04.288 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.290+0000 7f258b687700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f258410d7b0 con 0x7f2584105a90 2024-03-26T03:42:04.288 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.290+0000 7f258b687700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f258410d630 con 0x7f2584103160 2024-03-26T03:42:04.288 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.290+0000 7f258b687700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f258410d930 con 0x7f25841083c0 2024-03-26T03:42:04.288 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.290+0000 7f2589423700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2584103160 0x7f2584197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:42:04.288 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.290+0000 7f2589423700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2584103160 0x7f2584197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:56848/0 (socket says 172.21.15.190:56848) 2024-03-26T03:42:04.288 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.290+0000 7f2589423700 1 -- 172.21.15.190:0/2901244514 learned_addr learned my addr 172.21.15.190:0/2901244514 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-03-26T03:42:04.288 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.290+0000 7f2589423700 1 -- 172.21.15.190:0/2901244514 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25841083c0 msgr2=0x7f258419f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:42:04.288 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.290+0000 7f2588c22700 1 --2- 172.21.15.190:0/2901244514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2584105a90 0x7f2584198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:42:04.288 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.290+0000 7f2589c24700 1 --2- 172.21.15.190:0/2901244514 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25841083c0 0x7f258419f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:42:04.288 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.290+0000 7f2589423700 1 --2- 172.21.15.190:0/2901244514 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25841083c0 0x7f258419f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:42:04.288 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.290+0000 7f2589423700 1 -- 172.21.15.190:0/2901244514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2584105a90 msgr2=0x7f2584198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:42:04.288 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.290+0000 7f2589423700 1 --2- 172.21.15.190:0/2901244514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2584105a90 0x7f2584198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:42:04.288 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.290+0000 7f2589423700 1 -- 172.21.15.190:0/2901244514 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f258419fa50 con 0x7f2584103160 2024-03-26T03:42:04.289 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.290+0000 7f2589c24700 1 --2- 172.21.15.190:0/2901244514 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25841083c0 0x7f258419f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T03:42:04.289 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.290+0000 7f2588c22700 1 --2- 172.21.15.190:0/2901244514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2584105a90 0x7f2584198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T03:42:04.289 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.290+0000 7f2589423700 1 --2- 172.21.15.190:0/2901244514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2584103160 0x7f2584197df0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f25740047b0 tx=0x7f25740058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:42:04.289 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.290+0000 7f25827fc700 1 -- 172.21.15.190:0/2901244514 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2574026070 con 0x7f2584103160 2024-03-26T03:42:04.289 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.290+0000 7f25827fc700 1 -- 172.21.15.190:0/2901244514 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f25740040b0 con 0x7f2584103160 2024-03-26T03:42:04.289 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.290+0000 7f258b687700 1 -- 172.21.15.190:0/2901244514 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f258419fce0 con 0x7f2584103160 2024-03-26T03:42:04.289 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.290+0000 7f25827fc700 1 -- 172.21.15.190:0/2901244514 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2574005290 con 0x7f2584103160 2024-03-26T03:42:04.289 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.290+0000 7f258b687700 1 -- 172.21.15.190:0/2901244514 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25841a0190 con 0x7f2584103160 2024-03-26T03:42:04.290 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.290+0000 7f258b687700 1 -- 172.21.15.190:0/2901244514 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2584050300 con 0x7f2584103160 2024-03-26T03:42:04.290 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.290+0000 7f25827fc700 1 -- 172.21.15.190:0/2901244514 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 15) ==== 92475+0+0 (secure 0 0 0) 0x7f2574029070 con 0x7f2584103160 2024-03-26T03:42:04.290 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.290+0000 7f25827fc700 1 --2- 172.21.15.190:0/2901244514 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f256c06e6f0 0x7f256c070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:42:04.290 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.290+0000 7f25827fc700 1 -- 172.21.15.190:0/2901244514 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4423+0+0 (secure 0 0 0) 0x7f257408eac0 con 0x7f2584103160 2024-03-26T03:42:04.290 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.294+0000 7f2588c22700 1 --2- 172.21.15.190:0/2901244514 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f256c06e6f0 0x7f256c070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:42:04.290 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.294+0000 7f2588c22700 1 --2- 172.21.15.190:0/2901244514 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f256c06e6f0 0x7f256c070bb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2578005fd0 tx=0x7f2578005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:42:04.294 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.298+0000 7f25827fc700 1 -- 172.21.15.190:0/2901244514 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f2574013070 con 0x7f2584103160 2024-03-26T03:42:04.504 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:04.506+0000 7f258b687700 1 -- 172.21.15.190:0/2901244514 --> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f2584062a80 con 0x7f256c06e6f0 2024-03-26T03:42:05.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:05 smithi175 bash[21527]: cluster 2024-03-26T03:42:04.276513+0000 mgr.y (mgr.14152) 271 : cluster [DBG] pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:05.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:05 smithi175 bash[21527]: audit 2024-03-26T03:42:04.508862+0000 mgr.y (mgr.14152) 272 : audit [DBG] from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:42:05.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:05 smithi175 bash[21527]: audit 2024-03-26T03:42:04.515951+0000 mon.a (mon.0) 575 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T03:42:05.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:05 smithi175 bash[21527]: audit 2024-03-26T03:42:04.523245+0000 mon.a (mon.0) 576 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T03:42:05.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:05 smithi175 bash[21527]: audit 2024-03-26T03:42:04.524781+0000 mon.a (mon.0) 577 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:42:05.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:05 smithi175 bash[29284]: cluster 2024-03-26T03:42:04.276513+0000 mgr.y (mgr.14152) 271 : cluster [DBG] pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:05.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:05 smithi175 bash[29284]: audit 2024-03-26T03:42:04.508862+0000 mgr.y (mgr.14152) 272 : audit [DBG] from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:42:05.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:05 smithi175 bash[29284]: audit 2024-03-26T03:42:04.515951+0000 mon.a (mon.0) 575 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T03:42:05.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:05 smithi175 bash[29284]: audit 2024-03-26T03:42:04.523245+0000 mon.a (mon.0) 576 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T03:42:05.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:05 smithi175 bash[29284]: audit 2024-03-26T03:42:04.524781+0000 mon.a (mon.0) 577 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:42:05.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:05 smithi190 bash[23392]: cluster 2024-03-26T03:42:04.276513+0000 mgr.y (mgr.14152) 271 : cluster [DBG] pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:05.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:05 smithi190 bash[23392]: audit 2024-03-26T03:42:04.508862+0000 mgr.y (mgr.14152) 272 : audit [DBG] from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:42:05.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:05 smithi190 bash[23392]: audit 2024-03-26T03:42:04.515951+0000 mon.a (mon.0) 575 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T03:42:05.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:05 smithi190 bash[23392]: audit 2024-03-26T03:42:04.523245+0000 mon.a (mon.0) 576 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T03:42:05.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:05 smithi190 bash[23392]: audit 2024-03-26T03:42:04.524781+0000 mon.a (mon.0) 577 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:42:07.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:07 smithi175 bash[21527]: cluster 2024-03-26T03:42:06.277064+0000 mgr.y (mgr.14152) 273 : cluster [DBG] pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:07.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:07 smithi175 bash[29284]: cluster 2024-03-26T03:42:06.277064+0000 mgr.y (mgr.14152) 273 : cluster [DBG] pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:07.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:07 smithi190 bash[23392]: cluster 2024-03-26T03:42:06.277064+0000 mgr.y (mgr.14152) 273 : cluster [DBG] pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:09.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:09 smithi175 bash[21527]: cluster 2024-03-26T03:42:08.277712+0000 mgr.y (mgr.14152) 274 : cluster [DBG] pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:09.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:09 smithi175 bash[29284]: cluster 2024-03-26T03:42:08.277712+0000 mgr.y (mgr.14152) 274 : cluster [DBG] pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:09.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:09 smithi190 bash[23392]: cluster 2024-03-26T03:42:08.277712+0000 mgr.y (mgr.14152) 274 : cluster [DBG] pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:11.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:11 smithi175 bash[21527]: cluster 2024-03-26T03:42:10.278297+0000 mgr.y (mgr.14152) 275 : cluster [DBG] pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:11.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:11 smithi175 bash[29284]: cluster 2024-03-26T03:42:10.278297+0000 mgr.y (mgr.14152) 275 : cluster [DBG] pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:11.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:11 smithi190 bash[23392]: cluster 2024-03-26T03:42:10.278297+0000 mgr.y (mgr.14152) 275 : cluster [DBG] pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:13.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:13 smithi175 bash[21527]: cluster 2024-03-26T03:42:12.278930+0000 mgr.y (mgr.14152) 276 : cluster [DBG] pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:13.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:13 smithi175 bash[21527]: audit 2024-03-26T03:42:12.977760+0000 mon.a (mon.0) 578 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01cebff3-5246-49f3-893d-706cbfd9d829"}]: dispatch 2024-03-26T03:42:13.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:13 smithi175 bash[21527]: audit 2024-03-26T03:42:12.979512+0000 mon.b (mon.1) 18 : audit [INF] from='client.? 172.21.15.190:0/1249617872' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01cebff3-5246-49f3-893d-706cbfd9d829"}]: dispatch 2024-03-26T03:42:13.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:13 smithi175 bash[21527]: audit 2024-03-26T03:42:12.981342+0000 mon.a (mon.0) 579 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01cebff3-5246-49f3-893d-706cbfd9d829"}]': finished 2024-03-26T03:42:13.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:13 smithi175 bash[21527]: cluster 2024-03-26T03:42:12.989508+0000 mon.a (mon.0) 580 : cluster [DBG] osdmap e44: 7 total, 6 up, 7 in 2024-03-26T03:42:13.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:13 smithi175 bash[21527]: audit 2024-03-26T03:42:12.990222+0000 mon.a (mon.0) 581 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T03:42:13.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:13 smithi175 bash[29284]: cluster 2024-03-26T03:42:12.278930+0000 mgr.y (mgr.14152) 276 : cluster [DBG] pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:13.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:13 smithi175 bash[29284]: audit 2024-03-26T03:42:12.977760+0000 mon.a (mon.0) 578 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01cebff3-5246-49f3-893d-706cbfd9d829"}]: dispatch 2024-03-26T03:42:13.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:13 smithi175 bash[29284]: audit 2024-03-26T03:42:12.979512+0000 mon.b (mon.1) 18 : audit [INF] from='client.? 172.21.15.190:0/1249617872' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01cebff3-5246-49f3-893d-706cbfd9d829"}]: dispatch 2024-03-26T03:42:13.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:13 smithi175 bash[29284]: audit 2024-03-26T03:42:12.981342+0000 mon.a (mon.0) 579 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01cebff3-5246-49f3-893d-706cbfd9d829"}]': finished 2024-03-26T03:42:13.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:13 smithi175 bash[29284]: cluster 2024-03-26T03:42:12.989508+0000 mon.a (mon.0) 580 : cluster [DBG] osdmap e44: 7 total, 6 up, 7 in 2024-03-26T03:42:13.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:13 smithi175 bash[29284]: audit 2024-03-26T03:42:12.990222+0000 mon.a (mon.0) 581 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T03:42:13.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:13 smithi190 bash[23392]: cluster 2024-03-26T03:42:12.278930+0000 mgr.y (mgr.14152) 276 : cluster [DBG] pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:13.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:13 smithi190 bash[23392]: audit 2024-03-26T03:42:12.977760+0000 mon.a (mon.0) 578 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01cebff3-5246-49f3-893d-706cbfd9d829"}]: dispatch 2024-03-26T03:42:13.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:13 smithi190 bash[23392]: audit 2024-03-26T03:42:12.979512+0000 mon.b (mon.1) 18 : audit [INF] from='client.? 172.21.15.190:0/1249617872' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01cebff3-5246-49f3-893d-706cbfd9d829"}]: dispatch 2024-03-26T03:42:13.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:13 smithi190 bash[23392]: audit 2024-03-26T03:42:12.981342+0000 mon.a (mon.0) 579 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01cebff3-5246-49f3-893d-706cbfd9d829"}]': finished 2024-03-26T03:42:13.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:13 smithi190 bash[23392]: cluster 2024-03-26T03:42:12.989508+0000 mon.a (mon.0) 580 : cluster [DBG] osdmap e44: 7 total, 6 up, 7 in 2024-03-26T03:42:13.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:13 smithi190 bash[23392]: audit 2024-03-26T03:42:12.990222+0000 mon.a (mon.0) 581 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T03:42:14.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:14 smithi175 bash[21527]: audit 2024-03-26T03:42:13.750615+0000 mon.b (mon.1) 19 : audit [DBG] from='client.? 172.21.15.190:0/1930392535' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T03:42:14.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:14 smithi175 bash[29284]: audit 2024-03-26T03:42:13.750615+0000 mon.b (mon.1) 19 : audit [DBG] from='client.? 172.21.15.190:0/1930392535' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T03:42:14.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:14 smithi190 bash[23392]: audit 2024-03-26T03:42:13.750615+0000 mon.b (mon.1) 19 : audit [DBG] from='client.? 172.21.15.190:0/1930392535' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T03:42:15.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:15 smithi175 bash[21527]: cluster 2024-03-26T03:42:14.279538+0000 mgr.y (mgr.14152) 277 : cluster [DBG] pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:15.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:15 smithi175 bash[29284]: cluster 2024-03-26T03:42:14.279538+0000 mgr.y (mgr.14152) 277 : cluster [DBG] pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:15.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:15 smithi190 bash[23392]: cluster 2024-03-26T03:42:14.279538+0000 mgr.y (mgr.14152) 277 : cluster [DBG] pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:17.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:17 smithi175 bash[21527]: cluster 2024-03-26T03:42:16.280033+0000 mgr.y (mgr.14152) 278 : cluster [DBG] pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:17.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:17 smithi175 bash[29284]: cluster 2024-03-26T03:42:16.280033+0000 mgr.y (mgr.14152) 278 : cluster [DBG] pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:17.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:17 smithi190 bash[23392]: cluster 2024-03-26T03:42:16.280033+0000 mgr.y (mgr.14152) 278 : cluster [DBG] pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:19.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:19 smithi175 bash[21527]: cluster 2024-03-26T03:42:18.280594+0000 mgr.y (mgr.14152) 279 : cluster [DBG] pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:19.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:19 smithi175 bash[29284]: cluster 2024-03-26T03:42:18.280594+0000 mgr.y (mgr.14152) 279 : cluster [DBG] pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:19.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:19 smithi190 bash[23392]: cluster 2024-03-26T03:42:18.280594+0000 mgr.y (mgr.14152) 279 : cluster [DBG] pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:21.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:21 smithi175 bash[21527]: cluster 2024-03-26T03:42:20.281110+0000 mgr.y (mgr.14152) 280 : cluster [DBG] pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:21.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:21 smithi175 bash[29284]: cluster 2024-03-26T03:42:20.281110+0000 mgr.y (mgr.14152) 280 : cluster [DBG] pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:21.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:21 smithi190 bash[23392]: cluster 2024-03-26T03:42:20.281110+0000 mgr.y (mgr.14152) 280 : cluster [DBG] pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:23.737 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:23 smithi190 bash[23392]: cluster 2024-03-26T03:42:22.281608+0000 mgr.y (mgr.14152) 281 : cluster [DBG] pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:23.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:23 smithi175 bash[21527]: cluster 2024-03-26T03:42:22.281608+0000 mgr.y (mgr.14152) 281 : cluster [DBG] pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:23.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:23 smithi175 bash[29284]: cluster 2024-03-26T03:42:22.281608+0000 mgr.y (mgr.14152) 281 : cluster [DBG] pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:25.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:25 smithi175 bash[21527]: cluster 2024-03-26T03:42:24.282158+0000 mgr.y (mgr.14152) 282 : cluster [DBG] pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:25.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:25 smithi175 bash[29284]: cluster 2024-03-26T03:42:24.282158+0000 mgr.y (mgr.14152) 282 : cluster [DBG] pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:25.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:25 smithi190 bash[23392]: cluster 2024-03-26T03:42:24.282158+0000 mgr.y (mgr.14152) 282 : cluster [DBG] pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:26.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:26 smithi175 bash[21527]: audit 2024-03-26T03:42:25.757728+0000 mon.a (mon.0) 582 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-26T03:42:26.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:26 smithi175 bash[21527]: audit 2024-03-26T03:42:25.759521+0000 mon.a (mon.0) 583 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:42:26.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:26 smithi175 bash[21527]: cephadm 2024-03-26T03:42:25.761132+0000 mgr.y (mgr.14152) 283 : cephadm [INF] Deploying daemon osd.6 on smithi190 2024-03-26T03:42:26.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:26 smithi175 bash[29284]: audit 2024-03-26T03:42:25.757728+0000 mon.a (mon.0) 582 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-26T03:42:26.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:26 smithi175 bash[29284]: audit 2024-03-26T03:42:25.759521+0000 mon.a (mon.0) 583 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:42:26.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:26 smithi175 bash[29284]: cephadm 2024-03-26T03:42:25.761132+0000 mgr.y (mgr.14152) 283 : cephadm [INF] Deploying daemon osd.6 on smithi190 2024-03-26T03:42:26.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:26 smithi190 bash[23392]: audit 2024-03-26T03:42:25.757728+0000 mon.a (mon.0) 582 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-26T03:42:26.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:26 smithi190 bash[23392]: audit 2024-03-26T03:42:25.759521+0000 mon.a (mon.0) 583 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:42:26.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:26 smithi190 bash[23392]: cephadm 2024-03-26T03:42:25.761132+0000 mgr.y (mgr.14152) 283 : cephadm [INF] Deploying daemon osd.6 on smithi190 2024-03-26T03:42:27.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:27 smithi190 bash[23392]: cluster 2024-03-26T03:42:26.282618+0000 mgr.y (mgr.14152) 284 : cluster [DBG] pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:27 smithi175 bash[21527]: cluster 2024-03-26T03:42:26.282618+0000 mgr.y (mgr.14152) 284 : cluster [DBG] pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:27.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:27 smithi175 bash[29284]: cluster 2024-03-26T03:42:26.282618+0000 mgr.y (mgr.14152) 284 : cluster [DBG] pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:29.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:29 smithi175 bash[21527]: cluster 2024-03-26T03:42:28.283236+0000 mgr.y (mgr.14152) 285 : cluster [DBG] pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:29.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:29 smithi175 bash[29284]: cluster 2024-03-26T03:42:28.283236+0000 mgr.y (mgr.14152) 285 : cluster [DBG] pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:29.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:29 smithi190 bash[23392]: cluster 2024-03-26T03:42:28.283236+0000 mgr.y (mgr.14152) 285 : cluster [DBG] pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:30.396 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:30 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:42:30.396 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:42:30 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:42:30.396 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 03:42:30 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:42:30.397 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 03:42:30 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:42:30.744 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:42:30 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:42:30.745 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 03:42:30 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:42:30.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:30 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:42:30.745 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 03:42:30 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:42:31.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:31 smithi190 bash[23392]: cluster 2024-03-26T03:42:30.283914+0000 mgr.y (mgr.14152) 286 : cluster [DBG] pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:31.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:31 smithi190 bash[23392]: audit 2024-03-26T03:42:30.555106+0000 mon.a (mon.0) 584 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:31.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:31 smithi190 bash[23392]: audit 2024-03-26T03:42:30.576152+0000 mon.a (mon.0) 585 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:31.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:31 smithi190 bash[23392]: audit 2024-03-26T03:42:30.577378+0000 mon.a (mon.0) 586 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:42:31.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:31 smithi175 bash[21527]: cluster 2024-03-26T03:42:30.283914+0000 mgr.y (mgr.14152) 286 : cluster [DBG] pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:31.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:31 smithi175 bash[21527]: audit 2024-03-26T03:42:30.555106+0000 mon.a (mon.0) 584 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:31.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:31 smithi175 bash[21527]: audit 2024-03-26T03:42:30.576152+0000 mon.a (mon.0) 585 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:31.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:31 smithi175 bash[21527]: audit 2024-03-26T03:42:30.577378+0000 mon.a (mon.0) 586 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:42:31.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:31 smithi175 bash[29284]: cluster 2024-03-26T03:42:30.283914+0000 mgr.y (mgr.14152) 286 : cluster [DBG] pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:31.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:31 smithi175 bash[29284]: audit 2024-03-26T03:42:30.555106+0000 mon.a (mon.0) 584 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:31.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:31 smithi175 bash[29284]: audit 2024-03-26T03:42:30.576152+0000 mon.a (mon.0) 585 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:31.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:31 smithi175 bash[29284]: audit 2024-03-26T03:42:30.577378+0000 mon.a (mon.0) 586 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:42:33.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:33 smithi190 bash[23392]: cluster 2024-03-26T03:42:32.284490+0000 mgr.y (mgr.14152) 287 : cluster [DBG] pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:33.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:33 smithi175 bash[21527]: cluster 2024-03-26T03:42:32.284490+0000 mgr.y (mgr.14152) 287 : cluster [DBG] pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:33.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:33 smithi175 bash[29284]: cluster 2024-03-26T03:42:32.284490+0000 mgr.y (mgr.14152) 287 : cluster [DBG] pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:35.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:35 smithi175 bash[21527]: cluster 2024-03-26T03:42:34.285017+0000 mgr.y (mgr.14152) 288 : cluster [DBG] pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:35.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:35 smithi175 bash[29284]: cluster 2024-03-26T03:42:34.285017+0000 mgr.y (mgr.14152) 288 : cluster [DBG] pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:35.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:35 smithi190 bash[23392]: cluster 2024-03-26T03:42:34.285017+0000 mgr.y (mgr.14152) 288 : cluster [DBG] pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:37.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:36 smithi175 bash[21527]: audit 2024-03-26T03:42:35.871006+0000 mon.a (mon.0) 587 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:37.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:36 smithi175 bash[21527]: audit 2024-03-26T03:42:35.881048+0000 mon.a (mon.0) 588 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:37.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:36 smithi175 bash[21527]: audit 2024-03-26T03:42:36.566188+0000 mon.a (mon.0) 589 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:42:37.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:36 smithi175 bash[21527]: audit 2024-03-26T03:42:36.568264+0000 mon.a (mon.0) 590 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:42:37.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:36 smithi175 bash[21527]: audit 2024-03-26T03:42:36.580462+0000 mon.a (mon.0) 591 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:37.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:36 smithi175 bash[29284]: audit 2024-03-26T03:42:35.871006+0000 mon.a (mon.0) 587 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:37.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:36 smithi175 bash[29284]: audit 2024-03-26T03:42:35.881048+0000 mon.a (mon.0) 588 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:37.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:36 smithi175 bash[29284]: audit 2024-03-26T03:42:36.566188+0000 mon.a (mon.0) 589 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:42:37.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:36 smithi175 bash[29284]: audit 2024-03-26T03:42:36.568264+0000 mon.a (mon.0) 590 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:42:37.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:36 smithi175 bash[29284]: audit 2024-03-26T03:42:36.580462+0000 mon.a (mon.0) 591 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:37.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:36 smithi190 bash[23392]: audit 2024-03-26T03:42:35.871006+0000 mon.a (mon.0) 587 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:37.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:36 smithi190 bash[23392]: audit 2024-03-26T03:42:35.881048+0000 mon.a (mon.0) 588 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:37.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:36 smithi190 bash[23392]: audit 2024-03-26T03:42:36.566188+0000 mon.a (mon.0) 589 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:42:37.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:36 smithi190 bash[23392]: audit 2024-03-26T03:42:36.568264+0000 mon.a (mon.0) 590 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:42:37.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:36 smithi190 bash[23392]: audit 2024-03-26T03:42:36.580462+0000 mon.a (mon.0) 591 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:38.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:37 smithi175 bash[21527]: cluster 2024-03-26T03:42:36.285593+0000 mgr.y (mgr.14152) 289 : cluster [DBG] pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:38.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:37 smithi175 bash[21527]: audit 2024-03-26T03:42:37.078310+0000 mon.a (mon.0) 592 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-26T03:42:38.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:37 smithi175 bash[21527]: audit 2024-03-26T03:42:37.080554+0000 mon.b (mon.1) 20 : audit [INF] from='osd.6 [v2:172.21.15.190:6816/4077005791,v1:172.21.15.190:6817/4077005791]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-26T03:42:38.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:37 smithi175 bash[29284]: cluster 2024-03-26T03:42:36.285593+0000 mgr.y (mgr.14152) 289 : cluster [DBG] pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:38.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:37 smithi175 bash[29284]: audit 2024-03-26T03:42:37.078310+0000 mon.a (mon.0) 592 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-26T03:42:38.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:37 smithi175 bash[29284]: audit 2024-03-26T03:42:37.080554+0000 mon.b (mon.1) 20 : audit [INF] from='osd.6 [v2:172.21.15.190:6816/4077005791,v1:172.21.15.190:6817/4077005791]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-26T03:42:38.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:37 smithi190 bash[23392]: cluster 2024-03-26T03:42:36.285593+0000 mgr.y (mgr.14152) 289 : cluster [DBG] pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:38.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:37 smithi190 bash[23392]: audit 2024-03-26T03:42:37.078310+0000 mon.a (mon.0) 592 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-26T03:42:38.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:37 smithi190 bash[23392]: audit 2024-03-26T03:42:37.080554+0000 mon.b (mon.1) 20 : audit [INF] from='osd.6 [v2:172.21.15.190:6816/4077005791,v1:172.21.15.190:6817/4077005791]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-26T03:42:39.166 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:38 smithi190 bash[23392]: audit 2024-03-26T03:42:37.888897+0000 mon.a (mon.0) 593 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-26T03:42:39.166 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:38 smithi190 bash[23392]: cluster 2024-03-26T03:42:37.894270+0000 mon.a (mon.0) 594 : cluster [DBG] osdmap e45: 7 total, 6 up, 7 in 2024-03-26T03:42:39.166 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:38 smithi190 bash[23392]: audit 2024-03-26T03:42:37.894963+0000 mon.a (mon.0) 595 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T03:42:39.166 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:38 smithi190 bash[23392]: audit 2024-03-26T03:42:37.895701+0000 mon.a (mon.0) 596 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-03-26T03:42:39.166 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:38 smithi190 bash[23392]: audit 2024-03-26T03:42:37.897899+0000 mon.b (mon.1) 21 : audit [INF] from='osd.6 [v2:172.21.15.190:6816/4077005791,v1:172.21.15.190:6817/4077005791]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-03-26T03:42:39.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:38 smithi175 bash[21527]: audit 2024-03-26T03:42:37.888897+0000 mon.a (mon.0) 593 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-26T03:42:39.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:38 smithi175 bash[21527]: cluster 2024-03-26T03:42:37.894270+0000 mon.a (mon.0) 594 : cluster [DBG] osdmap e45: 7 total, 6 up, 7 in 2024-03-26T03:42:39.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:38 smithi175 bash[21527]: audit 2024-03-26T03:42:37.894963+0000 mon.a (mon.0) 595 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T03:42:39.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:38 smithi175 bash[21527]: audit 2024-03-26T03:42:37.895701+0000 mon.a (mon.0) 596 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-03-26T03:42:39.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:38 smithi175 bash[21527]: audit 2024-03-26T03:42:37.897899+0000 mon.b (mon.1) 21 : audit [INF] from='osd.6 [v2:172.21.15.190:6816/4077005791,v1:172.21.15.190:6817/4077005791]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-03-26T03:42:39.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:38 smithi175 bash[29284]: audit 2024-03-26T03:42:37.888897+0000 mon.a (mon.0) 593 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-26T03:42:39.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:38 smithi175 bash[29284]: cluster 2024-03-26T03:42:37.894270+0000 mon.a (mon.0) 594 : cluster [DBG] osdmap e45: 7 total, 6 up, 7 in 2024-03-26T03:42:39.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:38 smithi175 bash[29284]: audit 2024-03-26T03:42:37.894963+0000 mon.a (mon.0) 595 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T03:42:39.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:38 smithi175 bash[29284]: audit 2024-03-26T03:42:37.895701+0000 mon.a (mon.0) 596 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-03-26T03:42:39.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:38 smithi175 bash[29284]: audit 2024-03-26T03:42:37.897899+0000 mon.b (mon.1) 21 : audit [INF] from='osd.6 [v2:172.21.15.190:6816/4077005791,v1:172.21.15.190:6817/4077005791]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-03-26T03:42:39.737 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:39.738+0000 7f25827fc700 1 -- 172.21.15.190:0/2901244514 <== mgr.14152 v2:172.21.15.175:6800/2826039805 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2584062a80 con 0x7f256c06e6f0 2024-03-26T03:42:39.737 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 6 on host 'smithi190' 2024-03-26T03:42:39.740 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:39.742+0000 7f258b687700 1 -- 172.21.15.190:0/2901244514 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f256c06e6f0 msgr2=0x7f256c070bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:42:39.740 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:39.742+0000 7f258b687700 1 --2- 172.21.15.190:0/2901244514 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f256c06e6f0 0x7f256c070bb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2578005fd0 tx=0x7f2578005ee0 comp rx=0 tx=0).stop 2024-03-26T03:42:39.740 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:39.742+0000 7f258b687700 1 -- 172.21.15.190:0/2901244514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2584103160 msgr2=0x7f2584197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:42:39.741 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:39.742+0000 7f258b687700 1 --2- 172.21.15.190:0/2901244514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2584103160 0x7f2584197df0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f25740047b0 tx=0x7f25740058e0 comp rx=0 tx=0).stop 2024-03-26T03:42:39.741 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:39.742+0000 7f258b687700 1 -- 172.21.15.190:0/2901244514 shutdown_connections 2024-03-26T03:42:39.741 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:39.742+0000 7f258b687700 1 --2- 172.21.15.190:0/2901244514 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f256c06e6f0 0x7f256c070bb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:42:39.741 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:39.742+0000 7f258b687700 1 --2- 172.21.15.190:0/2901244514 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25841083c0 0x7f258419f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:42:39.741 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:39.742+0000 7f258b687700 1 --2- 172.21.15.190:0/2901244514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2584103160 0x7f2584197df0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:42:39.741 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:39.742+0000 7f258b687700 1 --2- 172.21.15.190:0/2901244514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2584105a90 0x7f2584198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:42:39.741 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:39.742+0000 7f258b687700 1 -- 172.21.15.190:0/2901244514 >> 172.21.15.190:0/2901244514 conn(0x7f25840fcb70 msgr2=0x7f2584103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:42:39.741 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:39.742+0000 7f258b687700 1 -- 172.21.15.190:0/2901244514 shutdown_connections 2024-03-26T03:42:39.741 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:39.742+0000 7f258b687700 1 -- 172.21.15.190:0/2901244514 wait complete. 2024-03-26T03:42:40.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:39 smithi175 bash[21527]: cluster 2024-03-26T03:42:38.286179+0000 mgr.y (mgr.14152) 290 : cluster [DBG] pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:40.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:39 smithi175 bash[21527]: audit 2024-03-26T03:42:38.891247+0000 mon.a (mon.0) 597 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-03-26T03:42:40.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:39 smithi175 bash[21527]: cluster 2024-03-26T03:42:38.897199+0000 mon.a (mon.0) 598 : cluster [DBG] osdmap e46: 7 total, 6 up, 7 in 2024-03-26T03:42:40.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:39 smithi175 bash[21527]: audit 2024-03-26T03:42:38.898766+0000 mon.a (mon.0) 599 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T03:42:40.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:39 smithi175 bash[21527]: audit 2024-03-26T03:42:38.907701+0000 mon.a (mon.0) 600 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T03:42:40.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:39 smithi175 bash[21527]: audit 2024-03-26T03:42:39.707721+0000 mon.a (mon.0) 601 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:40.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:39 smithi175 bash[21527]: audit 2024-03-26T03:42:39.717414+0000 mon.a (mon.0) 602 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:40.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:39 smithi175 bash[21527]: audit 2024-03-26T03:42:39.724365+0000 mon.a (mon.0) 603 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:42:40.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:39 smithi175 bash[29284]: cluster 2024-03-26T03:42:38.286179+0000 mgr.y (mgr.14152) 290 : cluster [DBG] pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:40.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:39 smithi175 bash[29284]: audit 2024-03-26T03:42:38.891247+0000 mon.a (mon.0) 597 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-03-26T03:42:40.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:39 smithi175 bash[29284]: cluster 2024-03-26T03:42:38.897199+0000 mon.a (mon.0) 598 : cluster [DBG] osdmap e46: 7 total, 6 up, 7 in 2024-03-26T03:42:40.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:39 smithi175 bash[29284]: audit 2024-03-26T03:42:38.898766+0000 mon.a (mon.0) 599 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T03:42:40.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:39 smithi175 bash[29284]: audit 2024-03-26T03:42:38.907701+0000 mon.a (mon.0) 600 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T03:42:40.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:39 smithi175 bash[29284]: audit 2024-03-26T03:42:39.707721+0000 mon.a (mon.0) 601 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:40.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:39 smithi175 bash[29284]: audit 2024-03-26T03:42:39.717414+0000 mon.a (mon.0) 602 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:40.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:39 smithi175 bash[29284]: audit 2024-03-26T03:42:39.724365+0000 mon.a (mon.0) 603 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:42:40.246 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:39 smithi190 bash[23392]: cluster 2024-03-26T03:42:38.286179+0000 mgr.y (mgr.14152) 290 : cluster [DBG] pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:40.246 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:39 smithi190 bash[23392]: audit 2024-03-26T03:42:38.891247+0000 mon.a (mon.0) 597 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-03-26T03:42:40.246 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:39 smithi190 bash[23392]: cluster 2024-03-26T03:42:38.897199+0000 mon.a (mon.0) 598 : cluster [DBG] osdmap e46: 7 total, 6 up, 7 in 2024-03-26T03:42:40.246 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:39 smithi190 bash[23392]: audit 2024-03-26T03:42:38.898766+0000 mon.a (mon.0) 599 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T03:42:40.246 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:39 smithi190 bash[23392]: audit 2024-03-26T03:42:38.907701+0000 mon.a (mon.0) 600 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T03:42:40.246 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:39 smithi190 bash[23392]: audit 2024-03-26T03:42:39.707721+0000 mon.a (mon.0) 601 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:40.246 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:39 smithi190 bash[23392]: audit 2024-03-26T03:42:39.717414+0000 mon.a (mon.0) 602 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:40.246 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:39 smithi190 bash[23392]: audit 2024-03-26T03:42:39.724365+0000 mon.a (mon.0) 603 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:42:40.524 DEBUG:teuthology.orchestra.run.smithi190:osd.6> sudo journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@osd.6.service 2024-03-26T03:42:40.527 INFO:tasks.cephadm:Deploying osd.7 on smithi190 with /dev/vg_nvme/lv_1... 2024-03-26T03:42:40.527 DEBUG:teuthology.orchestra.run.smithi190:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-26T03:42:41.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:40 smithi175 bash[21527]: cluster 2024-03-26T03:42:38.088976+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2024-03-26T03:42:41.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:40 smithi175 bash[21527]: cluster 2024-03-26T03:42:38.089065+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2024-03-26T03:42:41.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:40 smithi175 bash[21527]: audit 2024-03-26T03:42:39.899712+0000 mon.a (mon.0) 604 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T03:42:41.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:40 smithi175 bash[21527]: cluster 2024-03-26T03:42:39.914087+0000 mon.a (mon.0) 605 : cluster [INF] osd.6 [v2:172.21.15.190:6816/4077005791,v1:172.21.15.190:6817/4077005791] boot 2024-03-26T03:42:41.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:40 smithi175 bash[21527]: cluster 2024-03-26T03:42:39.914203+0000 mon.a (mon.0) 606 : cluster [DBG] osdmap e47: 7 total, 7 up, 7 in 2024-03-26T03:42:41.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:40 smithi175 bash[21527]: audit 2024-03-26T03:42:39.915154+0000 mon.a (mon.0) 607 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T03:42:41.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:40 smithi175 bash[29284]: cluster 2024-03-26T03:42:38.088976+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2024-03-26T03:42:41.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:40 smithi175 bash[29284]: cluster 2024-03-26T03:42:38.089065+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2024-03-26T03:42:41.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:40 smithi175 bash[29284]: audit 2024-03-26T03:42:39.899712+0000 mon.a (mon.0) 604 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T03:42:41.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:40 smithi175 bash[29284]: cluster 2024-03-26T03:42:39.914087+0000 mon.a (mon.0) 605 : cluster [INF] osd.6 [v2:172.21.15.190:6816/4077005791,v1:172.21.15.190:6817/4077005791] boot 2024-03-26T03:42:41.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:40 smithi175 bash[29284]: cluster 2024-03-26T03:42:39.914203+0000 mon.a (mon.0) 606 : cluster [DBG] osdmap e47: 7 total, 7 up, 7 in 2024-03-26T03:42:41.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:40 smithi175 bash[29284]: audit 2024-03-26T03:42:39.915154+0000 mon.a (mon.0) 607 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T03:42:41.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:40 smithi190 bash[23392]: cluster 2024-03-26T03:42:38.088976+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2024-03-26T03:42:41.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:40 smithi190 bash[23392]: cluster 2024-03-26T03:42:38.089065+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2024-03-26T03:42:41.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:40 smithi190 bash[23392]: audit 2024-03-26T03:42:39.899712+0000 mon.a (mon.0) 604 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T03:42:41.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:40 smithi190 bash[23392]: cluster 2024-03-26T03:42:39.914087+0000 mon.a (mon.0) 605 : cluster [INF] osd.6 [v2:172.21.15.190:6816/4077005791,v1:172.21.15.190:6817/4077005791] boot 2024-03-26T03:42:41.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:40 smithi190 bash[23392]: cluster 2024-03-26T03:42:39.914203+0000 mon.a (mon.0) 606 : cluster [DBG] osdmap e47: 7 total, 7 up, 7 in 2024-03-26T03:42:41.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:40 smithi190 bash[23392]: audit 2024-03-26T03:42:39.915154+0000 mon.a (mon.0) 607 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T03:42:42.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:41 smithi175 bash[21527]: cluster 2024-03-26T03:42:40.287273+0000 mgr.y (mgr.14152) 291 : cluster [DBG] pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:42.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:41 smithi175 bash[21527]: cluster 2024-03-26T03:42:40.939005+0000 mon.a (mon.0) 608 : cluster [DBG] osdmap e48: 7 total, 7 up, 7 in 2024-03-26T03:42:42.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:41 smithi175 bash[29284]: cluster 2024-03-26T03:42:40.287273+0000 mgr.y (mgr.14152) 291 : cluster [DBG] pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:42.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:41 smithi175 bash[29284]: cluster 2024-03-26T03:42:40.939005+0000 mon.a (mon.0) 608 : cluster [DBG] osdmap e48: 7 total, 7 up, 7 in 2024-03-26T03:42:42.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:41 smithi190 bash[23392]: cluster 2024-03-26T03:42:40.287273+0000 mgr.y (mgr.14152) 291 : cluster [DBG] pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-26T03:42:42.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:41 smithi190 bash[23392]: cluster 2024-03-26T03:42:40.939005+0000 mon.a (mon.0) 608 : cluster [DBG] osdmap e48: 7 total, 7 up, 7 in 2024-03-26T03:42:43.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:42 smithi175 bash[21527]: cluster 2024-03-26T03:42:41.938994+0000 mon.a (mon.0) 609 : cluster [DBG] osdmap e49: 7 total, 7 up, 7 in 2024-03-26T03:42:43.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:42 smithi175 bash[29284]: cluster 2024-03-26T03:42:41.938994+0000 mon.a (mon.0) 609 : cluster [DBG] osdmap e49: 7 total, 7 up, 7 in 2024-03-26T03:42:43.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:42 smithi190 bash[23392]: cluster 2024-03-26T03:42:41.938994+0000 mon.a (mon.0) 609 : cluster [DBG] osdmap e49: 7 total, 7 up, 7 in 2024-03-26T03:42:44.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:43 smithi175 bash[21527]: cluster 2024-03-26T03:42:42.287830+0000 mgr.y (mgr.14152) 292 : cluster [DBG] pgmap v274: 1 pgs: 1 remapped; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:42:44.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:43 smithi175 bash[21527]: cluster 2024-03-26T03:42:42.960659+0000 mon.a (mon.0) 610 : cluster [DBG] osdmap e50: 7 total, 7 up, 7 in 2024-03-26T03:42:44.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:43 smithi175 bash[29284]: cluster 2024-03-26T03:42:42.287830+0000 mgr.y (mgr.14152) 292 : cluster [DBG] pgmap v274: 1 pgs: 1 remapped; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:42:44.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:43 smithi175 bash[29284]: cluster 2024-03-26T03:42:42.960659+0000 mon.a (mon.0) 610 : cluster [DBG] osdmap e50: 7 total, 7 up, 7 in 2024-03-26T03:42:44.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:43 smithi190 bash[23392]: cluster 2024-03-26T03:42:42.287830+0000 mgr.y (mgr.14152) 292 : cluster [DBG] pgmap v274: 1 pgs: 1 remapped; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:42:44.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:43 smithi190 bash[23392]: cluster 2024-03-26T03:42:42.960659+0000 mon.a (mon.0) 610 : cluster [DBG] osdmap e50: 7 total, 7 up, 7 in 2024-03-26T03:42:44.685 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T03:42:45.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:44 smithi175 bash[21527]: cluster 2024-03-26T03:42:43.957812+0000 mon.a (mon.0) 611 : cluster [DBG] osdmap e51: 7 total, 7 up, 7 in 2024-03-26T03:42:45.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:44 smithi175 bash[29284]: cluster 2024-03-26T03:42:43.957812+0000 mon.a (mon.0) 611 : cluster [DBG] osdmap e51: 7 total, 7 up, 7 in 2024-03-26T03:42:45.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:44 smithi190 bash[23392]: cluster 2024-03-26T03:42:43.957812+0000 mon.a (mon.0) 611 : cluster [DBG] osdmap e51: 7 total, 7 up, 7 in 2024-03-26T03:42:46.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:45 smithi175 bash[29284]: cluster 2024-03-26T03:42:44.288366+0000 mgr.y (mgr.14152) 293 : cluster [DBG] pgmap v277: 1 pgs: 1 remapped; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:42:46.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:45 smithi175 bash[21527]: cluster 2024-03-26T03:42:44.288366+0000 mgr.y (mgr.14152) 293 : cluster [DBG] pgmap v277: 1 pgs: 1 remapped; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:42:46.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:45 smithi190 bash[23392]: cluster 2024-03-26T03:42:44.288366+0000 mgr.y (mgr.14152) 293 : cluster [DBG] pgmap v277: 1 pgs: 1 remapped; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:42:48.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:47 smithi175 bash[29284]: cluster 2024-03-26T03:42:46.288994+0000 mgr.y (mgr.14152) 294 : cluster [DBG] pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-03-26T03:42:48.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:47 smithi175 bash[21527]: cluster 2024-03-26T03:42:46.288994+0000 mgr.y (mgr.14152) 294 : cluster [DBG] pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-03-26T03:42:48.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:47 smithi190 bash[23392]: cluster 2024-03-26T03:42:46.288994+0000 mgr.y (mgr.14152) 294 : cluster [DBG] pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-03-26T03:42:49.780 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-26T03:42:49.780 INFO:teuthology.orchestra.run.smithi190.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-26T03:42:49.780 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10+0 records in 2024-03-26T03:42:49.780 INFO:teuthology.orchestra.run.smithi190.stderr:10+0 records out 2024-03-26T03:42:49.780 INFO:teuthology.orchestra.run.smithi190.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0130697 s, 802 MB/s 2024-03-26T03:42:49.780 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 2024-03-26T03:42:49.780 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping successful for: 2024-03-26T03:42:50.178 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:49 smithi190 bash[23392]: cluster 2024-03-26T03:42:48.289606+0000 mgr.y (mgr.14152) 295 : cluster [DBG] pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-03-26T03:42:50.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:49 smithi175 bash[21527]: cluster 2024-03-26T03:42:48.289606+0000 mgr.y (mgr.14152) 295 : cluster [DBG] pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-03-26T03:42:50.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:49 smithi175 bash[29284]: cluster 2024-03-26T03:42:48.289606+0000 mgr.y (mgr.14152) 295 : cluster [DBG] pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-03-26T03:42:50.804 DEBUG:teuthology.orchestra.run.smithi190:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi190:vg_nvme/lv_1 2024-03-26T03:42:52.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:51 smithi175 bash[21527]: cluster 2024-03-26T03:42:50.290178+0000 mgr.y (mgr.14152) 296 : cluster [DBG] pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T03:42:52.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:51 smithi175 bash[21527]: audit 2024-03-26T03:42:51.232525+0000 mon.a (mon.0) 612 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:52.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:51 smithi175 bash[21527]: audit 2024-03-26T03:42:51.244389+0000 mon.a (mon.0) 613 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:52.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:51 smithi175 bash[21527]: audit 2024-03-26T03:42:51.247029+0000 mon.a (mon.0) 614 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:42:52.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:51 smithi175 bash[21527]: audit 2024-03-26T03:42:51.249241+0000 mon.a (mon.0) 615 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:42:52.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:51 smithi175 bash[21527]: audit 2024-03-26T03:42:51.251196+0000 mon.a (mon.0) 616 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:42:52.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:51 smithi175 bash[21527]: audit 2024-03-26T03:42:51.261161+0000 mon.a (mon.0) 617 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:52.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:51 smithi175 bash[21527]: audit 2024-03-26T03:42:51.266507+0000 mon.a (mon.0) 618 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:42:52.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:51 smithi175 bash[21527]: audit 2024-03-26T03:42:51.268140+0000 mon.a (mon.0) 619 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:42:52.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:51 smithi175 bash[21527]: audit 2024-03-26T03:42:51.277986+0000 mon.a (mon.0) 620 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:52.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:51 smithi175 bash[29284]: cluster 2024-03-26T03:42:50.290178+0000 mgr.y (mgr.14152) 296 : cluster [DBG] pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T03:42:52.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:51 smithi175 bash[29284]: audit 2024-03-26T03:42:51.232525+0000 mon.a (mon.0) 612 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:52.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:51 smithi175 bash[29284]: audit 2024-03-26T03:42:51.244389+0000 mon.a (mon.0) 613 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:52.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:51 smithi175 bash[29284]: audit 2024-03-26T03:42:51.247029+0000 mon.a (mon.0) 614 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:42:52.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:51 smithi175 bash[29284]: audit 2024-03-26T03:42:51.249241+0000 mon.a (mon.0) 615 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:42:52.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:51 smithi175 bash[29284]: audit 2024-03-26T03:42:51.251196+0000 mon.a (mon.0) 616 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:42:52.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:51 smithi175 bash[29284]: audit 2024-03-26T03:42:51.261161+0000 mon.a (mon.0) 617 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:52.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:51 smithi175 bash[29284]: audit 2024-03-26T03:42:51.266507+0000 mon.a (mon.0) 618 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:42:52.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:51 smithi175 bash[29284]: audit 2024-03-26T03:42:51.268140+0000 mon.a (mon.0) 619 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:42:52.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:51 smithi175 bash[29284]: audit 2024-03-26T03:42:51.277986+0000 mon.a (mon.0) 620 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:52.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:51 smithi190 bash[23392]: cluster 2024-03-26T03:42:50.290178+0000 mgr.y (mgr.14152) 296 : cluster [DBG] pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T03:42:52.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:51 smithi190 bash[23392]: audit 2024-03-26T03:42:51.232525+0000 mon.a (mon.0) 612 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:52.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:51 smithi190 bash[23392]: audit 2024-03-26T03:42:51.244389+0000 mon.a (mon.0) 613 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:52.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:51 smithi190 bash[23392]: audit 2024-03-26T03:42:51.247029+0000 mon.a (mon.0) 614 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:42:52.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:51 smithi190 bash[23392]: audit 2024-03-26T03:42:51.249241+0000 mon.a (mon.0) 615 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:42:52.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:51 smithi190 bash[23392]: audit 2024-03-26T03:42:51.251196+0000 mon.a (mon.0) 616 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:42:52.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:51 smithi190 bash[23392]: audit 2024-03-26T03:42:51.261161+0000 mon.a (mon.0) 617 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:52.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:51 smithi190 bash[23392]: audit 2024-03-26T03:42:51.266507+0000 mon.a (mon.0) 618 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:42:52.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:51 smithi190 bash[23392]: audit 2024-03-26T03:42:51.268140+0000 mon.a (mon.0) 619 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:42:52.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:51 smithi190 bash[23392]: audit 2024-03-26T03:42:51.277986+0000 mon.a (mon.0) 620 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:42:53.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:52 smithi175 bash[21527]: cephadm 2024-03-26T03:42:51.222901+0000 mgr.y (mgr.14152) 297 : cephadm [INF] Detected new or changed devices on smithi190 2024-03-26T03:42:53.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:52 smithi175 bash[21527]: cephadm 2024-03-26T03:42:51.252681+0000 mgr.y (mgr.14152) 298 : cephadm [INF] Adjusting osd_memory_target on smithi190 to 5755M 2024-03-26T03:42:53.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:52 smithi175 bash[29284]: cephadm 2024-03-26T03:42:51.222901+0000 mgr.y (mgr.14152) 297 : cephadm [INF] Detected new or changed devices on smithi190 2024-03-26T03:42:53.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:52 smithi175 bash[29284]: cephadm 2024-03-26T03:42:51.252681+0000 mgr.y (mgr.14152) 298 : cephadm [INF] Adjusting osd_memory_target on smithi190 to 5755M 2024-03-26T03:42:53.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:52 smithi190 bash[23392]: cephadm 2024-03-26T03:42:51.222901+0000 mgr.y (mgr.14152) 297 : cephadm [INF] Detected new or changed devices on smithi190 2024-03-26T03:42:53.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:52 smithi190 bash[23392]: cephadm 2024-03-26T03:42:51.252681+0000 mgr.y (mgr.14152) 298 : cephadm [INF] Adjusting osd_memory_target on smithi190 to 5755M 2024-03-26T03:42:54.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:53 smithi175 bash[21527]: cluster 2024-03-26T03:42:52.290745+0000 mgr.y (mgr.14152) 299 : cluster [DBG] pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-26T03:42:54.266 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:53 smithi175 bash[29284]: cluster 2024-03-26T03:42:52.290745+0000 mgr.y (mgr.14152) 299 : cluster [DBG] pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-26T03:42:54.266 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:53 smithi190 bash[23392]: cluster 2024-03-26T03:42:52.290745+0000 mgr.y (mgr.14152) 299 : cluster [DBG] pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-26T03:42:55.544 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T03:42:56.418 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:56 smithi190 bash[23392]: cluster 2024-03-26T03:42:54.291313+0000 mgr.y (mgr.14152) 300 : cluster [DBG] pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-03-26T03:42:56.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:55 smithi175 bash[21527]: cluster 2024-03-26T03:42:54.291313+0000 mgr.y (mgr.14152) 300 : cluster [DBG] pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-03-26T03:42:56.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:55 smithi175 bash[29284]: cluster 2024-03-26T03:42:54.291313+0000 mgr.y (mgr.14152) 300 : cluster [DBG] pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-03-26T03:42:58.305 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.306+0000 7f5cc93f8700 1 -- 172.21.15.190:0/774123374 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cc41015d0 msgr2=0x7f5cc41019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:42:58.305 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.306+0000 7f5cc93f8700 1 --2- 172.21.15.190:0/774123374 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cc41015d0 0x7f5cc41019f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40099c0 tx=0x7f5cb4009cd0 comp rx=0 tx=0).stop 2024-03-26T03:42:58.305 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.306+0000 7f5cc93f8700 1 -- 172.21.15.190:0/774123374 shutdown_connections 2024-03-26T03:42:58.305 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.306+0000 7f5cc93f8700 1 --2- 172.21.15.190:0/774123374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5cc4103170 0x7f5cc4109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:42:58.305 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.306+0000 7f5cc93f8700 1 --2- 172.21.15.190:0/774123374 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cc41015d0 0x7f5cc41019f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:42:58.305 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.306+0000 7f5cc93f8700 1 --2- 172.21.15.190:0/774123374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5cc41027d0 0x7f5cc4102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:42:58.305 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.306+0000 7f5cc93f8700 1 -- 172.21.15.190:0/774123374 >> 172.21.15.190:0/774123374 conn(0x7f5cc40fcb90 msgr2=0x7f5cc40fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:42:58.305 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.306+0000 7f5cc93f8700 1 -- 172.21.15.190:0/774123374 shutdown_connections 2024-03-26T03:42:58.305 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.306+0000 7f5cc93f8700 1 -- 172.21.15.190:0/774123374 wait complete. 2024-03-26T03:42:58.305 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.306+0000 7f5cc93f8700 1 Processor -- start 2024-03-26T03:42:58.305 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.306+0000 7f5cc93f8700 1 -- start start 2024-03-26T03:42:58.306 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.306+0000 7f5cc93f8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5cc41015d0 0x7f5cc406b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:42:58.306 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.306+0000 7f5cc93f8700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5cc41027d0 0x7f5cc419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:42:58.306 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.306+0000 7f5cc93f8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cc4103170 0x7f5cc419f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:42:58.306 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.306+0000 7f5cc93f8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc410c700 con 0x7f5cc41015d0 2024-03-26T03:42:58.306 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.306+0000 7f5cc93f8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc410c580 con 0x7f5cc4103170 2024-03-26T03:42:58.306 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.306+0000 7f5cc93f8700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5cc410c880 con 0x7f5cc41027d0 2024-03-26T03:42:58.306 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.306+0000 7f5cc37fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cc4103170 0x7f5cc419f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:42:58.306 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.306+0000 7f5cc37fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cc4103170 0x7f5cc419f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:42568/0 (socket says 172.21.15.190:42568) 2024-03-26T03:42:58.306 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.306+0000 7f5cc37fe700 1 -- 172.21.15.190:0/4120517685 learned_addr learned my addr 172.21.15.190:0/4120517685 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-03-26T03:42:58.306 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.306+0000 7f5cc2ffd700 1 --2- 172.21.15.190:0/4120517685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5cc41015d0 0x7f5cc406b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:42:58.306 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.306+0000 7f5cc27fc700 1 --2- 172.21.15.190:0/4120517685 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5cc41027d0 0x7f5cc419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:42:58.306 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.306+0000 7f5cc37fe700 1 -- 172.21.15.190:0/4120517685 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5cc41027d0 msgr2=0x7f5cc419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:42:58.306 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.306+0000 7f5cc37fe700 1 --2- 172.21.15.190:0/4120517685 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5cc41027d0 0x7f5cc419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:42:58.306 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.306+0000 7f5cc37fe700 1 -- 172.21.15.190:0/4120517685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5cc41015d0 msgr2=0x7f5cc406b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:42:58.307 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.306+0000 7f5cc37fe700 1 --2- 172.21.15.190:0/4120517685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5cc41015d0 0x7f5cc406b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:42:58.307 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.306+0000 7f5cc37fe700 1 -- 172.21.15.190:0/4120517685 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc419fc20 con 0x7f5cc4103170 2024-03-26T03:42:58.307 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.306+0000 7f5cc27fc700 1 --2- 172.21.15.190:0/4120517685 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5cc41027d0 0x7f5cc419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T03:42:58.307 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.306+0000 7f5cc37fe700 1 --2- 172.21.15.190:0/4120517685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cc4103170 0x7f5cc419f350 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc00b970 tx=0x7f5cbc00bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:42:58.307 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.310+0000 7f5cabfff700 1 -- 172.21.15.190:0/4120517685 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc019070 con 0x7f5cc4103170 2024-03-26T03:42:58.307 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.310+0000 7f5cabfff700 1 -- 172.21.15.190:0/4120517685 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f5cbc00cdb0 con 0x7f5cc4103170 2024-03-26T03:42:58.307 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.310+0000 7f5cc93f8700 1 -- 172.21.15.190:0/4120517685 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc419ff10 con 0x7f5cc4103170 2024-03-26T03:42:58.307 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.310+0000 7f5cc93f8700 1 -- 172.21.15.190:0/4120517685 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc41a0440 con 0x7f5cc4103170 2024-03-26T03:42:58.307 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.310+0000 7f5cabfff700 1 -- 172.21.15.190:0/4120517685 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc014dc0 con 0x7f5cc4103170 2024-03-26T03:42:58.308 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.310+0000 7f5cc93f8700 1 -- 172.21.15.190:0/4120517685 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cb0005150 con 0x7f5cc4103170 2024-03-26T03:42:58.308 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.310+0000 7f5cabfff700 1 -- 172.21.15.190:0/4120517685 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 15) ==== 92475+0+0 (secure 0 0 0) 0x7f5cbc01c070 con 0x7f5cc4103170 2024-03-26T03:42:58.308 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.310+0000 7f5cabfff700 1 --2- 172.21.15.190:0/4120517685 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f5cac06e6f0 0x7f5cac070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:42:58.308 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.310+0000 7f5cabfff700 1 -- 172.21.15.190:0/4120517685 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4855+0+0 (secure 0 0 0) 0x7f5cbc08d500 con 0x7f5cc4103170 2024-03-26T03:42:58.308 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.310+0000 7f5cc2ffd700 1 --2- 172.21.15.190:0/4120517685 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f5cac06e6f0 0x7f5cac070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:42:58.308 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.310+0000 7f5cc2ffd700 1 --2- 172.21.15.190:0/4120517685 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f5cac06e6f0 0x7f5cac070bb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4000c00 tx=0x7f5cb4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:42:58.312 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.314+0000 7f5cabfff700 1 -- 172.21.15.190:0/4120517685 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f5cbc05a9a0 con 0x7f5cc4103170 2024-03-26T03:42:58.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:58 smithi175 bash[21527]: cluster 2024-03-26T03:42:56.291933+0000 mgr.y (mgr.14152) 301 : cluster [DBG] pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-26T03:42:58.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:58 smithi175 bash[29284]: cluster 2024-03-26T03:42:56.291933+0000 mgr.y (mgr.14152) 301 : cluster [DBG] pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-26T03:42:58.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:58 smithi190 bash[23392]: cluster 2024-03-26T03:42:56.291933+0000 mgr.y (mgr.14152) 301 : cluster [DBG] pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-26T03:42:58.532 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:42:58.534+0000 7f5cc93f8700 1 -- 172.21.15.190:0/4120517685 --> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f5cb0000bf0 con 0x7f5cac06e6f0 2024-03-26T03:42:59.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:59 smithi175 bash[21527]: audit 2024-03-26T03:42:58.544371+0000 mon.a (mon.0) 621 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T03:42:59.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:59 smithi175 bash[21527]: audit 2024-03-26T03:42:58.551005+0000 mon.a (mon.0) 622 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T03:42:59.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:42:59 smithi175 bash[21527]: audit 2024-03-26T03:42:58.552984+0000 mon.a (mon.0) 623 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:42:59.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:59 smithi175 bash[29284]: audit 2024-03-26T03:42:58.544371+0000 mon.a (mon.0) 621 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T03:42:59.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:59 smithi175 bash[29284]: audit 2024-03-26T03:42:58.551005+0000 mon.a (mon.0) 622 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T03:42:59.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:42:59 smithi175 bash[29284]: audit 2024-03-26T03:42:58.552984+0000 mon.a (mon.0) 623 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:42:59.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:59 smithi190 bash[23392]: audit 2024-03-26T03:42:58.544371+0000 mon.a (mon.0) 621 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T03:42:59.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:59 smithi190 bash[23392]: audit 2024-03-26T03:42:58.551005+0000 mon.a (mon.0) 622 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T03:42:59.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:42:59 smithi190 bash[23392]: audit 2024-03-26T03:42:58.552984+0000 mon.a (mon.0) 623 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:43:00.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:00 smithi175 bash[21527]: cluster 2024-03-26T03:42:58.292452+0000 mgr.y (mgr.14152) 302 : cluster [DBG] pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:00.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:00 smithi175 bash[21527]: audit 2024-03-26T03:42:58.537286+0000 mgr.y (mgr.14152) 303 : audit [DBG] from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:43:00.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:00 smithi175 bash[29284]: cluster 2024-03-26T03:42:58.292452+0000 mgr.y (mgr.14152) 302 : cluster [DBG] pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:00.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:00 smithi175 bash[29284]: audit 2024-03-26T03:42:58.537286+0000 mgr.y (mgr.14152) 303 : audit [DBG] from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:43:00.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:00 smithi190 bash[23392]: cluster 2024-03-26T03:42:58.292452+0000 mgr.y (mgr.14152) 302 : cluster [DBG] pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:00.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:00 smithi190 bash[23392]: audit 2024-03-26T03:42:58.537286+0000 mgr.y (mgr.14152) 303 : audit [DBG] from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:43:02.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:02 smithi175 bash[21527]: cluster 2024-03-26T03:43:00.293010+0000 mgr.y (mgr.14152) 304 : cluster [DBG] pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:02.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:02 smithi175 bash[29284]: cluster 2024-03-26T03:43:00.293010+0000 mgr.y (mgr.14152) 304 : cluster [DBG] pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:02.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:02 smithi190 bash[23392]: cluster 2024-03-26T03:43:00.293010+0000 mgr.y (mgr.14152) 304 : cluster [DBG] pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:04.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:04 smithi190 bash[23392]: cluster 2024-03-26T03:43:02.293617+0000 mgr.y (mgr.14152) 305 : cluster [DBG] pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:04.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:04 smithi175 bash[21527]: cluster 2024-03-26T03:43:02.293617+0000 mgr.y (mgr.14152) 305 : cluster [DBG] pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:04.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:04 smithi175 bash[29284]: cluster 2024-03-26T03:43:02.293617+0000 mgr.y (mgr.14152) 305 : cluster [DBG] pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:06.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:06 smithi175 bash[21527]: cluster 2024-03-26T03:43:04.294306+0000 mgr.y (mgr.14152) 306 : cluster [DBG] pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:06.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:06 smithi175 bash[29284]: cluster 2024-03-26T03:43:04.294306+0000 mgr.y (mgr.14152) 306 : cluster [DBG] pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:06.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:06 smithi190 bash[23392]: cluster 2024-03-26T03:43:04.294306+0000 mgr.y (mgr.14152) 306 : cluster [DBG] pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:08.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:08 smithi175 bash[21527]: cluster 2024-03-26T03:43:06.295013+0000 mgr.y (mgr.14152) 307 : cluster [DBG] pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:08.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:08 smithi175 bash[21527]: audit 2024-03-26T03:43:07.106470+0000 mon.a (mon.0) 624 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ee736b1-45ad-4daf-aea6-ae92e1f4c170"}]: dispatch 2024-03-26T03:43:08.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:08 smithi175 bash[21527]: audit 2024-03-26T03:43:07.108963+0000 mon.b (mon.1) 22 : audit [INF] from='client.? 172.21.15.190:0/2863011293' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ee736b1-45ad-4daf-aea6-ae92e1f4c170"}]: dispatch 2024-03-26T03:43:08.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:08 smithi175 bash[21527]: audit 2024-03-26T03:43:07.111955+0000 mon.a (mon.0) 625 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ee736b1-45ad-4daf-aea6-ae92e1f4c170"}]': finished 2024-03-26T03:43:08.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:08 smithi175 bash[21527]: cluster 2024-03-26T03:43:07.117689+0000 mon.a (mon.0) 626 : cluster [DBG] osdmap e52: 8 total, 7 up, 8 in 2024-03-26T03:43:08.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:08 smithi175 bash[21527]: audit 2024-03-26T03:43:07.118303+0000 mon.a (mon.0) 627 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T03:43:08.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:08 smithi175 bash[21527]: audit 2024-03-26T03:43:07.886144+0000 mon.b (mon.1) 23 : audit [DBG] from='client.? 172.21.15.190:0/1214612651' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T03:43:08.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:08 smithi175 bash[29284]: cluster 2024-03-26T03:43:06.295013+0000 mgr.y (mgr.14152) 307 : cluster [DBG] pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:08.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:08 smithi175 bash[29284]: audit 2024-03-26T03:43:07.106470+0000 mon.a (mon.0) 624 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ee736b1-45ad-4daf-aea6-ae92e1f4c170"}]: dispatch 2024-03-26T03:43:08.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:08 smithi175 bash[29284]: audit 2024-03-26T03:43:07.108963+0000 mon.b (mon.1) 22 : audit [INF] from='client.? 172.21.15.190:0/2863011293' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ee736b1-45ad-4daf-aea6-ae92e1f4c170"}]: dispatch 2024-03-26T03:43:08.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:08 smithi175 bash[29284]: audit 2024-03-26T03:43:07.111955+0000 mon.a (mon.0) 625 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ee736b1-45ad-4daf-aea6-ae92e1f4c170"}]': finished 2024-03-26T03:43:08.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:08 smithi175 bash[29284]: cluster 2024-03-26T03:43:07.117689+0000 mon.a (mon.0) 626 : cluster [DBG] osdmap e52: 8 total, 7 up, 8 in 2024-03-26T03:43:08.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:08 smithi175 bash[29284]: audit 2024-03-26T03:43:07.118303+0000 mon.a (mon.0) 627 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T03:43:08.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:08 smithi175 bash[29284]: audit 2024-03-26T03:43:07.886144+0000 mon.b (mon.1) 23 : audit [DBG] from='client.? 172.21.15.190:0/1214612651' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T03:43:08.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:08 smithi190 bash[23392]: cluster 2024-03-26T03:43:06.295013+0000 mgr.y (mgr.14152) 307 : cluster [DBG] pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:08.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:08 smithi190 bash[23392]: audit 2024-03-26T03:43:07.106470+0000 mon.a (mon.0) 624 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ee736b1-45ad-4daf-aea6-ae92e1f4c170"}]: dispatch 2024-03-26T03:43:08.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:08 smithi190 bash[23392]: audit 2024-03-26T03:43:07.108963+0000 mon.b (mon.1) 22 : audit [INF] from='client.? 172.21.15.190:0/2863011293' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ee736b1-45ad-4daf-aea6-ae92e1f4c170"}]: dispatch 2024-03-26T03:43:08.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:08 smithi190 bash[23392]: audit 2024-03-26T03:43:07.111955+0000 mon.a (mon.0) 625 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ee736b1-45ad-4daf-aea6-ae92e1f4c170"}]': finished 2024-03-26T03:43:08.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:08 smithi190 bash[23392]: cluster 2024-03-26T03:43:07.117689+0000 mon.a (mon.0) 626 : cluster [DBG] osdmap e52: 8 total, 7 up, 8 in 2024-03-26T03:43:08.495 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:08 smithi190 bash[23392]: audit 2024-03-26T03:43:07.118303+0000 mon.a (mon.0) 627 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T03:43:08.495 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:08 smithi190 bash[23392]: audit 2024-03-26T03:43:07.886144+0000 mon.b (mon.1) 23 : audit [DBG] from='client.? 172.21.15.190:0/1214612651' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T03:43:10.375 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:10 smithi190 bash[23392]: cluster 2024-03-26T03:43:08.295711+0000 mgr.y (mgr.14152) 308 : cluster [DBG] pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:10.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:10 smithi175 bash[21527]: cluster 2024-03-26T03:43:08.295711+0000 mgr.y (mgr.14152) 308 : cluster [DBG] pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:10.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:10 smithi175 bash[29284]: cluster 2024-03-26T03:43:08.295711+0000 mgr.y (mgr.14152) 308 : cluster [DBG] pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:12.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:12 smithi175 bash[21527]: cluster 2024-03-26T03:43:10.296260+0000 mgr.y (mgr.14152) 309 : cluster [DBG] pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:12.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:12 smithi175 bash[29284]: cluster 2024-03-26T03:43:10.296260+0000 mgr.y (mgr.14152) 309 : cluster [DBG] pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:12.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:12 smithi190 bash[23392]: cluster 2024-03-26T03:43:10.296260+0000 mgr.y (mgr.14152) 309 : cluster [DBG] pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:14.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:14 smithi175 bash[29284]: cluster 2024-03-26T03:43:12.296839+0000 mgr.y (mgr.14152) 310 : cluster [DBG] pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:14.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:14 smithi175 bash[21527]: cluster 2024-03-26T03:43:12.296839+0000 mgr.y (mgr.14152) 310 : cluster [DBG] pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:14.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:14 smithi190 bash[23392]: cluster 2024-03-26T03:43:12.296839+0000 mgr.y (mgr.14152) 310 : cluster [DBG] pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:16.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:16 smithi175 bash[21527]: cluster 2024-03-26T03:43:14.297417+0000 mgr.y (mgr.14152) 311 : cluster [DBG] pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:16.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:16 smithi175 bash[29284]: cluster 2024-03-26T03:43:14.297417+0000 mgr.y (mgr.14152) 311 : cluster [DBG] pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:16.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:16 smithi190 bash[23392]: cluster 2024-03-26T03:43:14.297417+0000 mgr.y (mgr.14152) 311 : cluster [DBG] pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:18.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:18 smithi175 bash[29284]: cluster 2024-03-26T03:43:16.298028+0000 mgr.y (mgr.14152) 312 : cluster [DBG] pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:18.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:18 smithi175 bash[21527]: cluster 2024-03-26T03:43:16.298028+0000 mgr.y (mgr.14152) 312 : cluster [DBG] pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:18.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:18 smithi190 bash[23392]: cluster 2024-03-26T03:43:16.298028+0000 mgr.y (mgr.14152) 312 : cluster [DBG] pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:20.476 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:20 smithi190 bash[23392]: cluster 2024-03-26T03:43:18.298593+0000 mgr.y (mgr.14152) 313 : cluster [DBG] pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:20.477 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:20 smithi190 bash[23392]: audit 2024-03-26T03:43:20.048420+0000 mon.a (mon.0) 628 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-26T03:43:20.477 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:20 smithi190 bash[23392]: audit 2024-03-26T03:43:20.050236+0000 mon.a (mon.0) 629 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:43:20.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:20 smithi175 bash[21527]: cluster 2024-03-26T03:43:18.298593+0000 mgr.y (mgr.14152) 313 : cluster [DBG] pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:20.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:20 smithi175 bash[21527]: audit 2024-03-26T03:43:20.048420+0000 mon.a (mon.0) 628 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-26T03:43:20.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:20 smithi175 bash[21527]: audit 2024-03-26T03:43:20.050236+0000 mon.a (mon.0) 629 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:43:20.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:20 smithi175 bash[29284]: cluster 2024-03-26T03:43:18.298593+0000 mgr.y (mgr.14152) 313 : cluster [DBG] pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:20.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:20 smithi175 bash[29284]: audit 2024-03-26T03:43:20.048420+0000 mon.a (mon.0) 628 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-26T03:43:20.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:20 smithi175 bash[29284]: audit 2024-03-26T03:43:20.050236+0000 mon.a (mon.0) 629 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:43:21.397 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:21 smithi190 bash[23392]: cephadm 2024-03-26T03:43:20.051809+0000 mgr.y (mgr.14152) 314 : cephadm [INF] Deploying daemon osd.7 on smithi190 2024-03-26T03:43:21.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:21 smithi175 bash[21527]: cephadm 2024-03-26T03:43:20.051809+0000 mgr.y (mgr.14152) 314 : cephadm [INF] Deploying daemon osd.7 on smithi190 2024-03-26T03:43:21.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:21 smithi175 bash[29284]: cephadm 2024-03-26T03:43:20.051809+0000 mgr.y (mgr.14152) 314 : cephadm [INF] Deploying daemon osd.7 on smithi190 2024-03-26T03:43:22.383 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:22 smithi190 bash[23392]: cluster 2024-03-26T03:43:20.299152+0000 mgr.y (mgr.14152) 315 : cluster [DBG] pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:22.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:22 smithi175 bash[21527]: cluster 2024-03-26T03:43:20.299152+0000 mgr.y (mgr.14152) 315 : cluster [DBG] pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:22.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:22 smithi175 bash[29284]: cluster 2024-03-26T03:43:20.299152+0000 mgr.y (mgr.14152) 315 : cluster [DBG] pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:24.331 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:24 smithi190 bash[23392]: cluster 2024-03-26T03:43:22.299739+0000 mgr.y (mgr.14152) 316 : cluster [DBG] pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:24.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:24 smithi175 bash[21527]: cluster 2024-03-26T03:43:22.299739+0000 mgr.y (mgr.14152) 316 : cluster [DBG] pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:24.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:24 smithi175 bash[29284]: cluster 2024-03-26T03:43:22.299739+0000 mgr.y (mgr.14152) 316 : cluster [DBG] pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:24.642 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 03:43:24 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:43:24.642 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 03:43:24 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:43:24.643 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:24 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:43:24.643 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 03:43:24 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:43:24.643 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:43:24 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:43:24.892 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:24 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:43:24.893 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 03:43:24 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:43:24.893 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 03:43:24 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:43:24.893 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:43:24 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:43:24.894 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 03:43:24 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:43:26.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:25 smithi175 bash[21527]: cluster 2024-03-26T03:43:24.300273+0000 mgr.y (mgr.14152) 317 : cluster [DBG] pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:26.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:25 smithi175 bash[21527]: audit 2024-03-26T03:43:24.947080+0000 mon.a (mon.0) 630 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:26.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:25 smithi175 bash[21527]: audit 2024-03-26T03:43:24.952730+0000 mon.a (mon.0) 631 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:43:26.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:25 smithi175 bash[21527]: audit 2024-03-26T03:43:24.957972+0000 mon.a (mon.0) 632 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:26.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:25 smithi175 bash[29284]: cluster 2024-03-26T03:43:24.300273+0000 mgr.y (mgr.14152) 317 : cluster [DBG] pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:26.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:25 smithi175 bash[29284]: audit 2024-03-26T03:43:24.947080+0000 mon.a (mon.0) 630 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:26.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:25 smithi175 bash[29284]: audit 2024-03-26T03:43:24.952730+0000 mon.a (mon.0) 631 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:43:26.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:25 smithi175 bash[29284]: audit 2024-03-26T03:43:24.957972+0000 mon.a (mon.0) 632 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:26.243 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:25 smithi190 bash[23392]: cluster 2024-03-26T03:43:24.300273+0000 mgr.y (mgr.14152) 317 : cluster [DBG] pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:26.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:25 smithi190 bash[23392]: audit 2024-03-26T03:43:24.947080+0000 mon.a (mon.0) 630 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:26.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:25 smithi190 bash[23392]: audit 2024-03-26T03:43:24.952730+0000 mon.a (mon.0) 631 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:43:26.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:25 smithi190 bash[23392]: audit 2024-03-26T03:43:24.957972+0000 mon.a (mon.0) 632 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:28.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:27 smithi175 bash[21527]: cluster 2024-03-26T03:43:26.300839+0000 mgr.y (mgr.14152) 318 : cluster [DBG] pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:28.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:27 smithi175 bash[29284]: cluster 2024-03-26T03:43:26.300839+0000 mgr.y (mgr.14152) 318 : cluster [DBG] pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:28.243 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:27 smithi190 bash[23392]: cluster 2024-03-26T03:43:26.300839+0000 mgr.y (mgr.14152) 318 : cluster [DBG] pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:30.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:29 smithi175 bash[21527]: cluster 2024-03-26T03:43:28.301396+0000 mgr.y (mgr.14152) 319 : cluster [DBG] pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:30.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:29 smithi175 bash[29284]: cluster 2024-03-26T03:43:28.301396+0000 mgr.y (mgr.14152) 319 : cluster [DBG] pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:30.243 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:29 smithi190 bash[23392]: cluster 2024-03-26T03:43:28.301396+0000 mgr.y (mgr.14152) 319 : cluster [DBG] pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:32.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:31 smithi175 bash[29284]: cluster 2024-03-26T03:43:30.301944+0000 mgr.y (mgr.14152) 320 : cluster [DBG] pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:32.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:31 smithi175 bash[29284]: audit 2024-03-26T03:43:31.656786+0000 mon.a (mon.0) 633 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T03:43:32.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:31 smithi175 bash[29284]: audit 2024-03-26T03:43:31.659637+0000 mon.b (mon.1) 24 : audit [INF] from='osd.7 [v2:172.21.15.190:6824/3444881554,v1:172.21.15.190:6825/3444881554]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T03:43:32.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:31 smithi175 bash[21527]: cluster 2024-03-26T03:43:30.301944+0000 mgr.y (mgr.14152) 320 : cluster [DBG] pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:32.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:31 smithi175 bash[21527]: audit 2024-03-26T03:43:31.656786+0000 mon.a (mon.0) 633 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T03:43:32.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:31 smithi175 bash[21527]: audit 2024-03-26T03:43:31.659637+0000 mon.b (mon.1) 24 : audit [INF] from='osd.7 [v2:172.21.15.190:6824/3444881554,v1:172.21.15.190:6825/3444881554]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T03:43:32.243 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:31 smithi190 bash[23392]: cluster 2024-03-26T03:43:30.301944+0000 mgr.y (mgr.14152) 320 : cluster [DBG] pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:32.243 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:31 smithi190 bash[23392]: audit 2024-03-26T03:43:31.656786+0000 mon.a (mon.0) 633 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T03:43:32.243 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:31 smithi190 bash[23392]: audit 2024-03-26T03:43:31.659637+0000 mon.b (mon.1) 24 : audit [INF] from='osd.7 [v2:172.21.15.190:6824/3444881554,v1:172.21.15.190:6825/3444881554]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T03:43:33.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:32 smithi175 bash[21527]: audit 2024-03-26T03:43:31.958839+0000 mon.a (mon.0) 634 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-26T03:43:33.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:32 smithi175 bash[21527]: audit 2024-03-26T03:43:31.966202+0000 mon.b (mon.1) 25 : audit [INF] from='osd.7 [v2:172.21.15.190:6824/3444881554,v1:172.21.15.190:6825/3444881554]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-03-26T03:43:33.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:32 smithi175 bash[21527]: cluster 2024-03-26T03:43:31.966552+0000 mon.a (mon.0) 635 : cluster [DBG] osdmap e53: 8 total, 7 up, 8 in 2024-03-26T03:43:33.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:32 smithi175 bash[21527]: audit 2024-03-26T03:43:31.967578+0000 mon.a (mon.0) 636 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T03:43:33.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:32 smithi175 bash[21527]: audit 2024-03-26T03:43:31.968186+0000 mon.a (mon.0) 637 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-03-26T03:43:33.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:32 smithi175 bash[21527]: audit 2024-03-26T03:43:32.114575+0000 mon.a (mon.0) 638 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:33.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:32 smithi175 bash[21527]: audit 2024-03-26T03:43:32.128226+0000 mon.a (mon.0) 639 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:33.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:32 smithi175 bash[21527]: audit 2024-03-26T03:43:32.130773+0000 mon.a (mon.0) 640 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:43:33.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:32 smithi175 bash[21527]: audit 2024-03-26T03:43:32.132689+0000 mon.a (mon.0) 641 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:43:33.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:32 smithi175 bash[21527]: audit 2024-03-26T03:43:32.145574+0000 mon.a (mon.0) 642 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:33.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:32 smithi175 bash[21527]: audit 2024-03-26T03:43:32.963216+0000 mon.a (mon.0) 643 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-03-26T03:43:33.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:32 smithi175 bash[29284]: audit 2024-03-26T03:43:31.958839+0000 mon.a (mon.0) 634 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-26T03:43:33.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:32 smithi175 bash[29284]: audit 2024-03-26T03:43:31.966202+0000 mon.b (mon.1) 25 : audit [INF] from='osd.7 [v2:172.21.15.190:6824/3444881554,v1:172.21.15.190:6825/3444881554]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-03-26T03:43:33.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:32 smithi175 bash[29284]: cluster 2024-03-26T03:43:31.966552+0000 mon.a (mon.0) 635 : cluster [DBG] osdmap e53: 8 total, 7 up, 8 in 2024-03-26T03:43:33.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:32 smithi175 bash[29284]: audit 2024-03-26T03:43:31.967578+0000 mon.a (mon.0) 636 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T03:43:33.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:32 smithi175 bash[29284]: audit 2024-03-26T03:43:31.968186+0000 mon.a (mon.0) 637 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-03-26T03:43:33.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:32 smithi175 bash[29284]: audit 2024-03-26T03:43:32.114575+0000 mon.a (mon.0) 638 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:33.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:32 smithi175 bash[29284]: audit 2024-03-26T03:43:32.128226+0000 mon.a (mon.0) 639 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:33.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:32 smithi175 bash[29284]: audit 2024-03-26T03:43:32.130773+0000 mon.a (mon.0) 640 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:43:33.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:32 smithi175 bash[29284]: audit 2024-03-26T03:43:32.132689+0000 mon.a (mon.0) 641 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:43:33.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:32 smithi175 bash[29284]: audit 2024-03-26T03:43:32.145574+0000 mon.a (mon.0) 642 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:33.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:32 smithi175 bash[29284]: audit 2024-03-26T03:43:32.963216+0000 mon.a (mon.0) 643 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-03-26T03:43:33.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:32 smithi190 bash[23392]: audit 2024-03-26T03:43:31.958839+0000 mon.a (mon.0) 634 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-26T03:43:33.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:32 smithi190 bash[23392]: audit 2024-03-26T03:43:31.966202+0000 mon.b (mon.1) 25 : audit [INF] from='osd.7 [v2:172.21.15.190:6824/3444881554,v1:172.21.15.190:6825/3444881554]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-03-26T03:43:33.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:32 smithi190 bash[23392]: cluster 2024-03-26T03:43:31.966552+0000 mon.a (mon.0) 635 : cluster [DBG] osdmap e53: 8 total, 7 up, 8 in 2024-03-26T03:43:33.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:32 smithi190 bash[23392]: audit 2024-03-26T03:43:31.967578+0000 mon.a (mon.0) 636 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T03:43:33.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:32 smithi190 bash[23392]: audit 2024-03-26T03:43:31.968186+0000 mon.a (mon.0) 637 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2024-03-26T03:43:33.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:32 smithi190 bash[23392]: audit 2024-03-26T03:43:32.114575+0000 mon.a (mon.0) 638 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:33.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:32 smithi190 bash[23392]: audit 2024-03-26T03:43:32.128226+0000 mon.a (mon.0) 639 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:33.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:32 smithi190 bash[23392]: audit 2024-03-26T03:43:32.130773+0000 mon.a (mon.0) 640 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:43:33.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:32 smithi190 bash[23392]: audit 2024-03-26T03:43:32.132689+0000 mon.a (mon.0) 641 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:43:33.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:32 smithi190 bash[23392]: audit 2024-03-26T03:43:32.145574+0000 mon.a (mon.0) 642 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:33.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:32 smithi190 bash[23392]: audit 2024-03-26T03:43:32.963216+0000 mon.a (mon.0) 643 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2024-03-26T03:43:34.243 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:33 smithi190 bash[23392]: cluster 2024-03-26T03:43:32.302612+0000 mgr.y (mgr.14152) 321 : cluster [DBG] pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:34.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:33 smithi190 bash[23392]: cluster 2024-03-26T03:43:32.971707+0000 mon.a (mon.0) 644 : cluster [DBG] osdmap e54: 8 total, 7 up, 8 in 2024-03-26T03:43:34.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:33 smithi190 bash[23392]: audit 2024-03-26T03:43:32.972890+0000 mon.a (mon.0) 645 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T03:43:34.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:33 smithi190 bash[23392]: audit 2024-03-26T03:43:33.972193+0000 mon.a (mon.0) 646 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T03:43:34.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:33 smithi175 bash[29284]: cluster 2024-03-26T03:43:32.302612+0000 mgr.y (mgr.14152) 321 : cluster [DBG] pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:34.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:33 smithi175 bash[29284]: cluster 2024-03-26T03:43:32.971707+0000 mon.a (mon.0) 644 : cluster [DBG] osdmap e54: 8 total, 7 up, 8 in 2024-03-26T03:43:34.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:33 smithi175 bash[29284]: audit 2024-03-26T03:43:32.972890+0000 mon.a (mon.0) 645 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T03:43:34.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:33 smithi175 bash[29284]: audit 2024-03-26T03:43:33.972193+0000 mon.a (mon.0) 646 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T03:43:34.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:33 smithi175 bash[21527]: cluster 2024-03-26T03:43:32.302612+0000 mgr.y (mgr.14152) 321 : cluster [DBG] pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:34.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:33 smithi175 bash[21527]: cluster 2024-03-26T03:43:32.971707+0000 mon.a (mon.0) 644 : cluster [DBG] osdmap e54: 8 total, 7 up, 8 in 2024-03-26T03:43:34.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:33 smithi175 bash[21527]: audit 2024-03-26T03:43:32.972890+0000 mon.a (mon.0) 645 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T03:43:34.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:33 smithi175 bash[21527]: audit 2024-03-26T03:43:33.972193+0000 mon.a (mon.0) 646 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T03:43:35.243 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:35 smithi190 bash[23392]: cluster 2024-03-26T03:43:32.633638+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2024-03-26T03:43:35.243 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:35 smithi190 bash[23392]: cluster 2024-03-26T03:43:32.633736+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2024-03-26T03:43:35.243 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:35 smithi190 bash[23392]: cluster 2024-03-26T03:43:33.987125+0000 mon.a (mon.0) 647 : cluster [INF] osd.7 [v2:172.21.15.190:6824/3444881554,v1:172.21.15.190:6825/3444881554] boot 2024-03-26T03:43:35.243 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:35 smithi190 bash[23392]: cluster 2024-03-26T03:43:33.987278+0000 mon.a (mon.0) 648 : cluster [DBG] osdmap e55: 8 total, 8 up, 8 in 2024-03-26T03:43:35.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:35 smithi190 bash[23392]: audit 2024-03-26T03:43:33.987964+0000 mon.a (mon.0) 649 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T03:43:35.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:35 smithi175 bash[21527]: cluster 2024-03-26T03:43:32.633638+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2024-03-26T03:43:35.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:35 smithi175 bash[21527]: cluster 2024-03-26T03:43:32.633736+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2024-03-26T03:43:35.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:35 smithi175 bash[21527]: cluster 2024-03-26T03:43:33.987125+0000 mon.a (mon.0) 647 : cluster [INF] osd.7 [v2:172.21.15.190:6824/3444881554,v1:172.21.15.190:6825/3444881554] boot 2024-03-26T03:43:35.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:35 smithi175 bash[21527]: cluster 2024-03-26T03:43:33.987278+0000 mon.a (mon.0) 648 : cluster [DBG] osdmap e55: 8 total, 8 up, 8 in 2024-03-26T03:43:35.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:35 smithi175 bash[21527]: audit 2024-03-26T03:43:33.987964+0000 mon.a (mon.0) 649 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T03:43:35.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:35 smithi175 bash[29284]: cluster 2024-03-26T03:43:32.633638+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2024-03-26T03:43:35.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:35 smithi175 bash[29284]: cluster 2024-03-26T03:43:32.633736+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2024-03-26T03:43:35.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:35 smithi175 bash[29284]: cluster 2024-03-26T03:43:33.987125+0000 mon.a (mon.0) 647 : cluster [INF] osd.7 [v2:172.21.15.190:6824/3444881554,v1:172.21.15.190:6825/3444881554] boot 2024-03-26T03:43:35.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:35 smithi175 bash[29284]: cluster 2024-03-26T03:43:33.987278+0000 mon.a (mon.0) 648 : cluster [DBG] osdmap e55: 8 total, 8 up, 8 in 2024-03-26T03:43:35.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:35 smithi175 bash[29284]: audit 2024-03-26T03:43:33.987964+0000 mon.a (mon.0) 649 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T03:43:36.415 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:43:36.417+0000 7f5cabfff700 1 -- 172.21.15.190:0/4120517685 <== mgr.14152 v2:172.21.15.175:6800/2826039805 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5cb0000bf0 con 0x7f5cac06e6f0 2024-03-26T03:43:36.416 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 7 on host 'smithi190' 2024-03-26T03:43:36.419 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:43:36.421+0000 7f5cc93f8700 1 -- 172.21.15.190:0/4120517685 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f5cac06e6f0 msgr2=0x7f5cac070bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:43:36.419 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:43:36.421+0000 7f5cc93f8700 1 --2- 172.21.15.190:0/4120517685 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f5cac06e6f0 0x7f5cac070bb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4000c00 tx=0x7f5cb4005fb0 comp rx=0 tx=0).stop 2024-03-26T03:43:36.419 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:43:36.421+0000 7f5cc93f8700 1 -- 172.21.15.190:0/4120517685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cc4103170 msgr2=0x7f5cc419f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:43:36.419 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:43:36.421+0000 7f5cc93f8700 1 --2- 172.21.15.190:0/4120517685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cc4103170 0x7f5cc419f350 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc00b970 tx=0x7f5cbc00bc80 comp rx=0 tx=0).stop 2024-03-26T03:43:36.419 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:43:36.421+0000 7f5cc93f8700 1 -- 172.21.15.190:0/4120517685 shutdown_connections 2024-03-26T03:43:36.419 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:43:36.421+0000 7f5cc93f8700 1 --2- 172.21.15.190:0/4120517685 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f5cac06e6f0 0x7f5cac070bb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:43:36.419 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:43:36.421+0000 7f5cc93f8700 1 --2- 172.21.15.190:0/4120517685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cc4103170 0x7f5cc419f350 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:43:36.419 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:43:36.421+0000 7f5cc93f8700 1 --2- 172.21.15.190:0/4120517685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5cc41015d0 0x7f5cc406b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:43:36.419 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:43:36.421+0000 7f5cc93f8700 1 --2- 172.21.15.190:0/4120517685 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5cc41027d0 0x7f5cc419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:43:36.419 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:43:36.421+0000 7f5cc93f8700 1 -- 172.21.15.190:0/4120517685 >> 172.21.15.190:0/4120517685 conn(0x7f5cc40fcb90 msgr2=0x7f5cc40fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:43:36.419 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:43:36.421+0000 7f5cc93f8700 1 -- 172.21.15.190:0/4120517685 shutdown_connections 2024-03-26T03:43:36.419 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:43:36.421+0000 7f5cc93f8700 1 -- 172.21.15.190:0/4120517685 wait complete. 2024-03-26T03:43:36.439 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:36 smithi190 bash[23392]: cluster 2024-03-26T03:43:34.303203+0000 mgr.y (mgr.14152) 322 : cluster [DBG] pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:36.439 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:36 smithi190 bash[23392]: cluster 2024-03-26T03:43:35.017282+0000 mon.a (mon.0) 650 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2024-03-26T03:43:36.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:36 smithi175 bash[21527]: cluster 2024-03-26T03:43:34.303203+0000 mgr.y (mgr.14152) 322 : cluster [DBG] pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:36.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:36 smithi175 bash[21527]: cluster 2024-03-26T03:43:35.017282+0000 mon.a (mon.0) 650 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2024-03-26T03:43:36.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:36 smithi175 bash[29284]: cluster 2024-03-26T03:43:34.303203+0000 mgr.y (mgr.14152) 322 : cluster [DBG] pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-03-26T03:43:36.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:36 smithi175 bash[29284]: cluster 2024-03-26T03:43:35.017282+0000 mon.a (mon.0) 650 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2024-03-26T03:43:37.277 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:37 smithi190 bash[23392]: cluster 2024-03-26T03:43:36.013246+0000 mon.a (mon.0) 651 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2024-03-26T03:43:37.277 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:37 smithi190 bash[23392]: audit 2024-03-26T03:43:36.346020+0000 mon.a (mon.0) 652 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:43:37.277 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:37 smithi190 bash[23392]: audit 2024-03-26T03:43:36.355767+0000 mon.a (mon.0) 653 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:37.278 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:37 smithi190 bash[23392]: audit 2024-03-26T03:43:36.367856+0000 mon.a (mon.0) 654 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:37.279 DEBUG:teuthology.orchestra.run.smithi190:osd.7> sudo journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@osd.7.service 2024-03-26T03:43:37.281 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-03-26T03:43:37.281 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph osd stat -f json 2024-03-26T03:43:37.292 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:37 smithi175 bash[21527]: cluster 2024-03-26T03:43:36.013246+0000 mon.a (mon.0) 651 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2024-03-26T03:43:37.292 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:37 smithi175 bash[21527]: audit 2024-03-26T03:43:36.346020+0000 mon.a (mon.0) 652 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:43:37.292 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:37 smithi175 bash[21527]: audit 2024-03-26T03:43:36.355767+0000 mon.a (mon.0) 653 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:37.292 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:37 smithi175 bash[21527]: audit 2024-03-26T03:43:36.367856+0000 mon.a (mon.0) 654 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:37.292 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:37 smithi175 bash[29284]: cluster 2024-03-26T03:43:36.013246+0000 mon.a (mon.0) 651 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2024-03-26T03:43:37.293 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:37 smithi175 bash[29284]: audit 2024-03-26T03:43:36.346020+0000 mon.a (mon.0) 652 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:43:37.293 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:37 smithi175 bash[29284]: audit 2024-03-26T03:43:36.355767+0000 mon.a (mon.0) 653 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:37.293 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:37 smithi175 bash[29284]: audit 2024-03-26T03:43:36.367856+0000 mon.a (mon.0) 654 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:38.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:38 smithi175 bash[21527]: cluster 2024-03-26T03:43:36.303811+0000 mgr.y (mgr.14152) 323 : cluster [DBG] pgmap v309: 1 pgs: 1 remapped+peering; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:43:38.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:38 smithi175 bash[21527]: cluster 2024-03-26T03:43:37.036045+0000 mon.a (mon.0) 655 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2024-03-26T03:43:38.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:38 smithi175 bash[29284]: cluster 2024-03-26T03:43:36.303811+0000 mgr.y (mgr.14152) 323 : cluster [DBG] pgmap v309: 1 pgs: 1 remapped+peering; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:43:38.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:38 smithi175 bash[29284]: cluster 2024-03-26T03:43:37.036045+0000 mon.a (mon.0) 655 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2024-03-26T03:43:38.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:38 smithi190 bash[23392]: cluster 2024-03-26T03:43:36.303811+0000 mgr.y (mgr.14152) 323 : cluster [DBG] pgmap v309: 1 pgs: 1 remapped+peering; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:43:38.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:38 smithi190 bash[23392]: cluster 2024-03-26T03:43:37.036045+0000 mon.a (mon.0) 655 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2024-03-26T03:43:39.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:39 smithi175 bash[21527]: cluster 2024-03-26T03:43:38.035020+0000 mon.a (mon.0) 656 : cluster [DBG] osdmap e59: 8 total, 8 up, 8 in 2024-03-26T03:43:39.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:39 smithi175 bash[29284]: cluster 2024-03-26T03:43:38.035020+0000 mon.a (mon.0) 656 : cluster [DBG] osdmap e59: 8 total, 8 up, 8 in 2024-03-26T03:43:39.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:39 smithi190 bash[23392]: cluster 2024-03-26T03:43:38.035020+0000 mon.a (mon.0) 656 : cluster [DBG] osdmap e59: 8 total, 8 up, 8 in 2024-03-26T03:43:40.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:40 smithi175 bash[21527]: cluster 2024-03-26T03:43:38.304321+0000 mgr.y (mgr.14152) 324 : cluster [DBG] pgmap v312: 1 pgs: 1 remapped+peering; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:43:40.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:40 smithi175 bash[29284]: cluster 2024-03-26T03:43:38.304321+0000 mgr.y (mgr.14152) 324 : cluster [DBG] pgmap v312: 1 pgs: 1 remapped+peering; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:43:40.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:40 smithi190 bash[23392]: cluster 2024-03-26T03:43:38.304321+0000 mgr.y (mgr.14152) 324 : cluster [DBG] pgmap v312: 1 pgs: 1 remapped+peering; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:43:42.040 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:43:42.402 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:42 smithi190 bash[23392]: cluster 2024-03-26T03:43:40.304933+0000 mgr.y (mgr.14152) 325 : cluster [DBG] pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-03-26T03:43:42.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:42 smithi175 bash[21527]: cluster 2024-03-26T03:43:40.304933+0000 mgr.y (mgr.14152) 325 : cluster [DBG] pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-03-26T03:43:42.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:42 smithi175 bash[29284]: cluster 2024-03-26T03:43:40.304933+0000 mgr.y (mgr.14152) 325 : cluster [DBG] pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-03-26T03:43:44.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:44 smithi175 bash[21527]: cluster 2024-03-26T03:43:42.305507+0000 mgr.y (mgr.14152) 326 : cluster [DBG] pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-03-26T03:43:44.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:44 smithi175 bash[29284]: cluster 2024-03-26T03:43:42.305507+0000 mgr.y (mgr.14152) 326 : cluster [DBG] pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-03-26T03:43:44.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:44 smithi190 bash[23392]: cluster 2024-03-26T03:43:42.305507+0000 mgr.y (mgr.14152) 326 : cluster [DBG] pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-03-26T03:43:44.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5ae112700 1 -- 172.21.15.175:0/1074662384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5a81083c0 msgr2=0x7fc5a810a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:43:44.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5ae112700 1 --2- 172.21.15.175:0/1074662384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5a81083c0 0x7fc5a810a830 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0009f90 tx=0x7fc5a000b330 comp rx=0 tx=0).stop 2024-03-26T03:43:44.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5ae112700 1 -- 172.21.15.175:0/1074662384 shutdown_connections 2024-03-26T03:43:44.725 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5ae112700 1 --2- 172.21.15.175:0/1074662384 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5a8103160 0x7fc5a8105550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:43:44.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5ae112700 1 --2- 172.21.15.175:0/1074662384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5a81083c0 0x7fc5a810a830 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:43:44.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5ae112700 1 --2- 172.21.15.175:0/1074662384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5a8105a90 0x7fc5a8107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:43:44.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5ae112700 1 -- 172.21.15.175:0/1074662384 >> 172.21.15.175:0/1074662384 conn(0x7fc5a80fcb70 msgr2=0x7fc5a80fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:43:44.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5ae112700 1 -- 172.21.15.175:0/1074662384 shutdown_connections 2024-03-26T03:43:44.726 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5ae112700 1 -- 172.21.15.175:0/1074662384 wait complete. 2024-03-26T03:43:44.727 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5ae112700 1 Processor -- start 2024-03-26T03:43:44.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5ae112700 1 -- start start 2024-03-26T03:43:44.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5ae112700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5a8103160 0x7fc5a8102ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:43:44.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5ae112700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5a8105a90 0x7fc5a8101110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:43:44.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5ae112700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5a81083c0 0x7fc5a8101650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:43:44.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5ae112700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a810d7b0 con 0x7fc5a81083c0 2024-03-26T03:43:44.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5ae112700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a810d630 con 0x7fc5a8103160 2024-03-26T03:43:44.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5ae112700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc5a810d930 con 0x7fc5a8105a90 2024-03-26T03:43:44.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5a6ffd700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5a8105a90 0x7fc5a8101110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:43:44.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5a7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5a81083c0 0x7fc5a8101650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:43:44.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5a6ffd700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5a8105a90 0x7fc5a8101110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:42208/0 (socket says 172.21.15.175:42208) 2024-03-26T03:43:44.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5a6ffd700 1 -- 172.21.15.175:0/2866306585 learned_addr learned my addr 172.21.15.175:0/2866306585 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:43:44.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5a77fe700 1 --2- 172.21.15.175:0/2866306585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5a8103160 0x7fc5a8102ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:43:44.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5a7fff700 1 -- 172.21.15.175:0/2866306585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5a8105a90 msgr2=0x7fc5a8101110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:43:44.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5a7fff700 1 --2- 172.21.15.175:0/2866306585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5a8105a90 0x7fc5a8101110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:43:44.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5a7fff700 1 -- 172.21.15.175:0/2866306585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5a8103160 msgr2=0x7fc5a8102ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:43:44.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5a7fff700 1 --2- 172.21.15.175:0/2866306585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5a8103160 0x7fc5a8102ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:43:44.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5a7fff700 1 -- 172.21.15.175:0/2866306585 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a8101f00 con 0x7fc5a81083c0 2024-03-26T03:43:44.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5a6ffd700 1 --2- 172.21.15.175:0/2866306585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5a8105a90 0x7fc5a8101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T03:43:44.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5a7fff700 1 --2- 172.21.15.175:0/2866306585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5a81083c0 0x7fc5a8101650 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc5a806be60 tx=0x7fc5a000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:43:44.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5a4ff9700 1 -- 172.21.15.175:0/2866306585 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0021070 con 0x7fc5a81083c0 2024-03-26T03:43:44.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5a4ff9700 1 -- 172.21.15.175:0/2866306585 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fc5a001d450 con 0x7fc5a81083c0 2024-03-26T03:43:44.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5a4ff9700 1 -- 172.21.15.175:0/2866306585 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a00052a0 con 0x7fc5a81083c0 2024-03-26T03:43:44.729 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5ae112700 1 -- 172.21.15.175:0/2866306585 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a81a1f00 con 0x7fc5a81083c0 2024-03-26T03:43:44.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.726+0000 7fc5ae112700 1 -- 172.21.15.175:0/2866306585 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a81a22a0 con 0x7fc5a81083c0 2024-03-26T03:43:44.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.730+0000 7fc5ae112700 1 -- 172.21.15.175:0/2866306585 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5a8050300 con 0x7fc5a81083c0 2024-03-26T03:43:44.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.730+0000 7fc5a4ff9700 1 -- 172.21.15.175:0/2866306585 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 15) ==== 92475+0+0 (secure 0 0 0) 0x7fc5a0024070 con 0x7fc5a81083c0 2024-03-26T03:43:44.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.730+0000 7fc5a4ff9700 1 --2- 172.21.15.175:0/2866306585 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fc59406e6f0 0x7fc594070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:43:44.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.730+0000 7fc5a4ff9700 1 -- 172.21.15.175:0/2866306585 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5287+0+0 (secure 0 0 0) 0x7fc5a000f070 con 0x7fc5a81083c0 2024-03-26T03:43:44.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.730+0000 7fc5a77fe700 1 --2- 172.21.15.175:0/2866306585 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fc59406e6f0 0x7fc594070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:43:44.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.730+0000 7fc5a77fe700 1 --2- 172.21.15.175:0/2866306585 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fc59406e6f0 0x7fc594070bb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc598009750 tx=0x7fc598009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:43:44.735 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.734+0000 7fc5a4ff9700 1 -- 172.21.15.175:0/2866306585 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7fc5a002b020 con 0x7fc5a81083c0 2024-03-26T03:43:44.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.938+0000 7fc5ae112700 1 -- 172.21.15.175:0/2866306585 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fc5a802dd30 con 0x7fc5a81083c0 2024-03-26T03:43:44.939 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.938+0000 7fc5a4ff9700 1 -- 172.21.15.175:0/2866306585 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v59) ==== 74+0+130 (secure 0 0 0) 0x7fc5a005ccd0 con 0x7fc5a81083c0 2024-03-26T03:43:44.939 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-26T03:43:44.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.942+0000 7fc5ae112700 1 -- 172.21.15.175:0/2866306585 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fc59406e6f0 msgr2=0x7fc594070bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:43:44.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.942+0000 7fc5ae112700 1 --2- 172.21.15.175:0/2866306585 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fc59406e6f0 0x7fc594070bb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc598009750 tx=0x7fc598009310 comp rx=0 tx=0).stop 2024-03-26T03:43:44.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.942+0000 7fc5ae112700 1 -- 172.21.15.175:0/2866306585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5a81083c0 msgr2=0x7fc5a8101650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:43:44.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.942+0000 7fc5ae112700 1 --2- 172.21.15.175:0/2866306585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5a81083c0 0x7fc5a8101650 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc5a806be60 tx=0x7fc5a000cc30 comp rx=0 tx=0).stop 2024-03-26T03:43:44.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.942+0000 7fc5ae112700 1 -- 172.21.15.175:0/2866306585 shutdown_connections 2024-03-26T03:43:44.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.942+0000 7fc5ae112700 1 --2- 172.21.15.175:0/2866306585 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fc59406e6f0 0x7fc594070bb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:43:44.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.942+0000 7fc5ae112700 1 --2- 172.21.15.175:0/2866306585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5a81083c0 0x7fc5a8101650 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:43:44.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.942+0000 7fc5ae112700 1 --2- 172.21.15.175:0/2866306585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5a8103160 0x7fc5a8102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:43:44.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.942+0000 7fc5ae112700 1 --2- 172.21.15.175:0/2866306585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5a8105a90 0x7fc5a8101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:43:44.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.942+0000 7fc5ae112700 1 -- 172.21.15.175:0/2866306585 >> 172.21.15.175:0/2866306585 conn(0x7fc5a80fcb70 msgr2=0x7fc5a8103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:43:44.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.942+0000 7fc5ae112700 1 -- 172.21.15.175:0/2866306585 shutdown_connections 2024-03-26T03:43:44.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:44.942+0000 7fc5ae112700 1 -- 172.21.15.175:0/2866306585 wait complete. 2024-03-26T03:43:45.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:45 smithi175 bash[21527]: audit 2024-03-26T03:43:44.942207+0000 mon.a (mon.0) 657 : audit [DBG] from='client.? 172.21.15.175:0/2866306585' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T03:43:45.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:45 smithi175 bash[29284]: audit 2024-03-26T03:43:44.942207+0000 mon.a (mon.0) 657 : audit [DBG] from='client.? 172.21.15.175:0/2866306585' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T03:43:45.466 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:45 smithi190 bash[23392]: audit 2024-03-26T03:43:44.942207+0000 mon.a (mon.0) 657 : audit [DBG] from='client.? 172.21.15.175:0/2866306585' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T03:43:45.710 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":59,"num_osds":8,"num_up_osds":8,"osd_up_since":1711424613,"num_in_osds":8,"osd_in_since":1711424587,"num_remapped_pgs":0} 2024-03-26T03:43:45.711 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-26T03:43:46.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:46 smithi175 bash[21527]: cluster 2024-03-26T03:43:44.306061+0000 mgr.y (mgr.14152) 327 : cluster [DBG] pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T03:43:46.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:46 smithi175 bash[29284]: cluster 2024-03-26T03:43:44.306061+0000 mgr.y (mgr.14152) 327 : cluster [DBG] pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T03:43:46.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:46 smithi190 bash[23392]: cluster 2024-03-26T03:43:44.306061+0000 mgr.y (mgr.14152) 327 : cluster [DBG] pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T03:43:47.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:47 smithi175 bash[21527]: cluster 2024-03-26T03:43:46.306655+0000 mgr.y (mgr.14152) 328 : cluster [DBG] pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-26T03:43:47.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:47 smithi175 bash[21527]: cephadm 2024-03-26T03:43:46.339451+0000 mgr.y (mgr.14152) 329 : cephadm [INF] Detected new or changed devices on smithi190 2024-03-26T03:43:47.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:47 smithi175 bash[21527]: audit 2024-03-26T03:43:46.348647+0000 mon.a (mon.0) 658 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:47.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:47 smithi175 bash[21527]: audit 2024-03-26T03:43:46.358882+0000 mon.a (mon.0) 659 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:47.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:47 smithi175 bash[21527]: audit 2024-03-26T03:43:46.361576+0000 mon.a (mon.0) 660 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:43:47.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:47 smithi175 bash[21527]: audit 2024-03-26T03:43:46.363413+0000 mon.a (mon.0) 661 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:43:47.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:47 smithi175 bash[21527]: audit 2024-03-26T03:43:46.365141+0000 mon.a (mon.0) 662 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:43:47.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:47 smithi175 bash[21527]: audit 2024-03-26T03:43:46.366722+0000 mon.a (mon.0) 663 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:43:47.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:47 smithi175 bash[21527]: cephadm 2024-03-26T03:43:46.367832+0000 mgr.y (mgr.14152) 330 : cephadm [INF] Adjusting osd_memory_target on smithi190 to 4316M 2024-03-26T03:43:47.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:47 smithi175 bash[21527]: audit 2024-03-26T03:43:46.374052+0000 mon.a (mon.0) 664 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:47.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:47 smithi175 bash[21527]: audit 2024-03-26T03:43:46.378292+0000 mon.a (mon.0) 665 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:43:47.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:47 smithi175 bash[21527]: audit 2024-03-26T03:43:46.380098+0000 mon.a (mon.0) 666 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:43:47.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:47 smithi175 bash[21527]: audit 2024-03-26T03:43:46.389957+0000 mon.a (mon.0) 667 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:47.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:47 smithi175 bash[29284]: cluster 2024-03-26T03:43:46.306655+0000 mgr.y (mgr.14152) 328 : cluster [DBG] pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-26T03:43:47.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:47 smithi175 bash[29284]: cephadm 2024-03-26T03:43:46.339451+0000 mgr.y (mgr.14152) 329 : cephadm [INF] Detected new or changed devices on smithi190 2024-03-26T03:43:47.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:47 smithi175 bash[29284]: audit 2024-03-26T03:43:46.348647+0000 mon.a (mon.0) 658 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:47.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:47 smithi175 bash[29284]: audit 2024-03-26T03:43:46.358882+0000 mon.a (mon.0) 659 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:47.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:47 smithi175 bash[29284]: audit 2024-03-26T03:43:46.361576+0000 mon.a (mon.0) 660 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:43:47.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:47 smithi175 bash[29284]: audit 2024-03-26T03:43:46.363413+0000 mon.a (mon.0) 661 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:43:47.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:47 smithi175 bash[29284]: audit 2024-03-26T03:43:46.365141+0000 mon.a (mon.0) 662 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:43:47.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:47 smithi175 bash[29284]: audit 2024-03-26T03:43:46.366722+0000 mon.a (mon.0) 663 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:43:47.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:47 smithi175 bash[29284]: cephadm 2024-03-26T03:43:46.367832+0000 mgr.y (mgr.14152) 330 : cephadm [INF] Adjusting osd_memory_target on smithi190 to 4316M 2024-03-26T03:43:47.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:47 smithi175 bash[29284]: audit 2024-03-26T03:43:46.374052+0000 mon.a (mon.0) 664 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:47.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:47 smithi175 bash[29284]: audit 2024-03-26T03:43:46.378292+0000 mon.a (mon.0) 665 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:43:47.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:47 smithi175 bash[29284]: audit 2024-03-26T03:43:46.380098+0000 mon.a (mon.0) 666 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:43:47.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:47 smithi175 bash[29284]: audit 2024-03-26T03:43:46.389957+0000 mon.a (mon.0) 667 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:47.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:47 smithi190 bash[23392]: cluster 2024-03-26T03:43:46.306655+0000 mgr.y (mgr.14152) 328 : cluster [DBG] pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-26T03:43:47.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:47 smithi190 bash[23392]: cephadm 2024-03-26T03:43:46.339451+0000 mgr.y (mgr.14152) 329 : cephadm [INF] Detected new or changed devices on smithi190 2024-03-26T03:43:47.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:47 smithi190 bash[23392]: audit 2024-03-26T03:43:46.348647+0000 mon.a (mon.0) 658 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:47.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:47 smithi190 bash[23392]: audit 2024-03-26T03:43:46.358882+0000 mon.a (mon.0) 659 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:47.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:47 smithi190 bash[23392]: audit 2024-03-26T03:43:46.361576+0000 mon.a (mon.0) 660 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:43:47.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:47 smithi190 bash[23392]: audit 2024-03-26T03:43:46.363413+0000 mon.a (mon.0) 661 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:43:47.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:47 smithi190 bash[23392]: audit 2024-03-26T03:43:46.365141+0000 mon.a (mon.0) 662 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:43:47.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:47 smithi190 bash[23392]: audit 2024-03-26T03:43:46.366722+0000 mon.a (mon.0) 663 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:43:47.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:47 smithi190 bash[23392]: cephadm 2024-03-26T03:43:46.367832+0000 mgr.y (mgr.14152) 330 : cephadm [INF] Adjusting osd_memory_target on smithi190 to 4316M 2024-03-26T03:43:47.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:47 smithi190 bash[23392]: audit 2024-03-26T03:43:46.374052+0000 mon.a (mon.0) 664 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:47.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:47 smithi190 bash[23392]: audit 2024-03-26T03:43:46.378292+0000 mon.a (mon.0) 665 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:43:47.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:47 smithi190 bash[23392]: audit 2024-03-26T03:43:46.380098+0000 mon.a (mon.0) 666 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:43:47.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:47 smithi190 bash[23392]: audit 2024-03-26T03:43:46.389957+0000 mon.a (mon.0) 667 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:43:49.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:49 smithi175 bash[21527]: cluster 2024-03-26T03:43:48.307181+0000 mgr.y (mgr.14152) 331 : cluster [DBG] pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-03-26T03:43:49.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:49 smithi175 bash[29284]: cluster 2024-03-26T03:43:48.307181+0000 mgr.y (mgr.14152) 331 : cluster [DBG] pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-03-26T03:43:49.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:49 smithi190 bash[23392]: cluster 2024-03-26T03:43:48.307181+0000 mgr.y (mgr.14152) 331 : cluster [DBG] pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-03-26T03:43:50.511 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:43:51.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:51 smithi175 bash[29284]: cluster 2024-03-26T03:43:50.307802+0000 mgr.y (mgr.14152) 332 : cluster [DBG] pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-26T03:43:51.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:51 smithi175 bash[21527]: cluster 2024-03-26T03:43:50.307802+0000 mgr.y (mgr.14152) 332 : cluster [DBG] pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-26T03:43:51.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:51 smithi190 bash[23392]: cluster 2024-03-26T03:43:50.307802+0000 mgr.y (mgr.14152) 332 : cluster [DBG] pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-26T03:43:53.227 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd094aea700 1 -- 172.21.15.175:0/329055980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd090108660 msgr2=0x7fd090108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:43:53.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd094aea700 1 --2- 172.21.15.175:0/329055980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd090108660 0x7fd090108a40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd088009990 tx=0x7fd088009ca0 comp rx=0 tx=0).stop 2024-03-26T03:43:53.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd094aea700 1 -- 172.21.15.175:0/329055980 shutdown_connections 2024-03-26T03:43:53.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd094aea700 1 --2- 172.21.15.175:0/329055980 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd09006b660 0x7fd09010f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:43:53.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd094aea700 1 --2- 172.21.15.175:0/329055980 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd090108660 0x7fd090108a40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:43:53.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd094aea700 1 --2- 172.21.15.175:0/329055980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd09006acc0 0x7fd09006b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:43:53.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd094aea700 1 -- 172.21.15.175:0/329055980 >> 172.21.15.175:0/329055980 conn(0x7fd090078540 msgr2=0x7fd090078950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:43:53.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd094aea700 1 -- 172.21.15.175:0/329055980 shutdown_connections 2024-03-26T03:43:53.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd094aea700 1 -- 172.21.15.175:0/329055980 wait complete. 2024-03-26T03:43:53.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd094aea700 1 Processor -- start 2024-03-26T03:43:53.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd094aea700 1 -- start start 2024-03-26T03:43:53.228 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd094aea700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd09006acc0 0x7fd0901983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:43:53.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd094aea700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd09006b660 0x7fd090198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:43:53.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd094aea700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd090108660 0x7fd09019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:43:53.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd094aea700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd090111e80 con 0x7fd09006acc0 2024-03-26T03:43:53.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd094aea700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd090111d00 con 0x7fd09006b660 2024-03-26T03:43:53.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd094aea700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd090112000 con 0x7fd090108660 2024-03-26T03:43:53.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd08e59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd09006acc0 0x7fd0901983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:43:53.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd08ed9d700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd090108660 0x7fd09019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:43:53.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd08e59c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd09006acc0 0x7fd0901983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:37080/0 (socket says 172.21.15.175:37080) 2024-03-26T03:43:53.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd08e59c700 1 -- 172.21.15.175:0/1865691010 learned_addr learned my addr 172.21.15.175:0/1865691010 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:43:53.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd08dd9b700 1 --2- 172.21.15.175:0/1865691010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd09006b660 0x7fd090198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:43:53.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd08ed9d700 1 -- 172.21.15.175:0/1865691010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd09006b660 msgr2=0x7fd090198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:43:53.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd08ed9d700 1 --2- 172.21.15.175:0/1865691010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd09006b660 0x7fd090198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:43:53.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd08ed9d700 1 -- 172.21.15.175:0/1865691010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd09006acc0 msgr2=0x7fd0901983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:43:53.229 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd08ed9d700 1 --2- 172.21.15.175:0/1865691010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd09006acc0 0x7fd0901983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:43:53.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd08ed9d700 1 -- 172.21.15.175:0/1865691010 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd09019d400 con 0x7fd090108660 2024-03-26T03:43:53.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd08ed9d700 1 --2- 172.21.15.175:0/1865691010 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd090108660 0x7fd09019ccc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd08000efb0 tx=0x7fd08000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:43:53.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd0877fe700 1 -- 172.21.15.175:0/1865691010 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08000ca50 con 0x7fd090108660 2024-03-26T03:43:53.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd0877fe700 1 -- 172.21.15.175:0/1865691010 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fd08000cbf0 con 0x7fd090108660 2024-03-26T03:43:53.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd094aea700 1 -- 172.21.15.175:0/1865691010 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd09019d6f0 con 0x7fd090108660 2024-03-26T03:43:53.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd0877fe700 1 -- 172.21.15.175:0/1865691010 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd080004740 con 0x7fd090108660 2024-03-26T03:43:53.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.226+0000 7fd094aea700 1 -- 172.21.15.175:0/1865691010 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0901a4f70 con 0x7fd090108660 2024-03-26T03:43:53.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.230+0000 7fd094aea700 1 -- 172.21.15.175:0/1865691010 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd090103da0 con 0x7fd090108660 2024-03-26T03:43:53.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.230+0000 7fd0877fe700 1 -- 172.21.15.175:0/1865691010 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 15) ==== 92475+0+0 (secure 0 0 0) 0x7fd080012050 con 0x7fd090108660 2024-03-26T03:43:53.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.230+0000 7fd0877fe700 1 --2- 172.21.15.175:0/1865691010 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fd07c06e6f0 0x7fd07c070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:43:53.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.230+0000 7fd0877fe700 1 -- 172.21.15.175:0/1865691010 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(59..59 src has 1..59) ==== 5287+0+0 (secure 0 0 0) 0x7fd08008de50 con 0x7fd090108660 2024-03-26T03:43:53.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.230+0000 7fd08e59c700 1 --2- 172.21.15.175:0/1865691010 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fd07c06e6f0 0x7fd07c070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:43:53.231 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.230+0000 7fd08e59c700 1 --2- 172.21.15.175:0/1865691010 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fd07c06e6f0 0x7fd07c070bb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd088009750 tx=0x7fd08800b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:43:53.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.234+0000 7fd0877fe700 1 -- 172.21.15.175:0/1865691010 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7fd080022060 con 0x7fd090108660 2024-03-26T03:43:53.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.446+0000 7fd094aea700 1 -- 172.21.15.175:0/1865691010 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd090199750 con 0x7fd090108660 2024-03-26T03:43:53.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.446+0000 7fd0877fe700 1 -- 172.21.15.175:0/1865691010 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+13587 (secure 0 0 0) 0x7fd080010040 con 0x7fd090108660 2024-03-26T03:43:53.447 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-26T03:43:53.448 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":59,"fsid":"762e930c-eb21-11ee-95d0-87774f69a715","created":"2024-03-26T03:33:45.388461+0000","modified":"2024-03-26T03:43:38.027458+0000","last_up_change":"2024-03-26T03:43:33.976035+0000","last_in_change":"2024-03-26T03:43:07.107635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T03:39:08.367712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41a2182b-5eeb-47fa-be41-1d7639df2f35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":241042466},{"type":"v1","addr":"172.21.15.175:6803","nonce":241042466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":241042466},{"type":"v1","addr":"172.21.15.175:6805","nonce":241042466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":241042466},{"type":"v1","addr":"172.21.15.175:6809","nonce":241042466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":241042466},{"type":"v1","addr":"172.21.15.175:6807","nonce":241042466}]},"public_addr":"172.21.15.175:6803/241042466","cluster_addr":"172.21.15.175:6805/241042466","heartbeat_back_addr":"172.21.15.175:6809/241042466","heartbeat_front_addr":"172.21.15.175:6807/241042466","state":["exists","up"]},{"osd":1,"uuid":"8fbe50bf-5723-4174-9c85-6010a32bd55b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":931168929},{"type":"v1","addr":"172.21.15.175:6811","nonce":931168929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":931168929},{"type":"v1","addr":"172.21.15.175:6813","nonce":931168929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":931168929},{"type":"v1","addr":"172.21.15.175:6817","nonce":931168929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":931168929},{"type":"v1","addr":"172.21.15.175:6815","nonce":931168929}]},"public_addr":"172.21.15.175:6811/931168929","cluster_addr":"172.21.15.175:6813/931168929","heartbeat_back_addr":"172.21.15.175:6817/931168929","heartbeat_front_addr":"172.21.15.175:6815/931168929","state":["exists","up"]},{"osd":2,"uuid":"02333e36-f498-4ac2-ab6e-5ae1e1148e97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3775031674},{"type":"v1","addr":"172.21.15.175:6819","nonce":3775031674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":3775031674},{"type":"v1","addr":"172.21.15.175:6821","nonce":3775031674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3775031674},{"type":"v1","addr":"172.21.15.175:6825","nonce":3775031674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":3775031674},{"type":"v1","addr":"172.21.15.175:6823","nonce":3775031674}]},"public_addr":"172.21.15.175:6819/3775031674","cluster_addr":"172.21.15.175:6821/3775031674","heartbeat_back_addr":"172.21.15.175:6825/3775031674","heartbeat_front_addr":"172.21.15.175:6823/3775031674","state":["exists","up"]},{"osd":3,"uuid":"8add17f6-b408-4dc0-af05-b3b64845d8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":672326154},{"type":"v1","addr":"172.21.15.175:6827","nonce":672326154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":672326154},{"type":"v1","addr":"172.21.15.175:6829","nonce":672326154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":672326154},{"type":"v1","addr":"172.21.15.175:6833","nonce":672326154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":672326154},{"type":"v1","addr":"172.21.15.175:6831","nonce":672326154}]},"public_addr":"172.21.15.175:6827/672326154","cluster_addr":"172.21.15.175:6829/672326154","heartbeat_back_addr":"172.21.15.175:6833/672326154","heartbeat_front_addr":"172.21.15.175:6831/672326154","state":["exists","up"]},{"osd":4,"uuid":"54aac0c2-d77e-4bf4-b1e5-856b4e997426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":874343325},{"type":"v1","addr":"172.21.15.190:6801","nonce":874343325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":874343325},{"type":"v1","addr":"172.21.15.190:6803","nonce":874343325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":874343325},{"type":"v1","addr":"172.21.15.190:6807","nonce":874343325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":874343325},{"type":"v1","addr":"172.21.15.190:6805","nonce":874343325}]},"public_addr":"172.21.15.190:6801/874343325","cluster_addr":"172.21.15.190:6803/874343325","heartbeat_back_addr":"172.21.15.190:6807/874343325","heartbeat_front_addr":"172.21.15.190:6805/874343325","state":["exists","up"]},{"osd":5,"uuid":"d1eb5f92-1729-48c7-a89c-3265609fe40d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":4089468657},{"type":"v1","addr":"172.21.15.190:6809","nonce":4089468657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":4089468657},{"type":"v1","addr":"172.21.15.190:6811","nonce":4089468657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":4089468657},{"type":"v1","addr":"172.21.15.190:6815","nonce":4089468657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":4089468657},{"type":"v1","addr":"172.21.15.190:6813","nonce":4089468657}]},"public_addr":"172.21.15.190:6809/4089468657","cluster_addr":"172.21.15.190:6811/4089468657","heartbeat_back_addr":"172.21.15.190:6815/4089468657","heartbeat_front_addr":"172.21.15.190:6813/4089468657","state":["exists","up"]},{"osd":6,"uuid":"01cebff3-5246-49f3-893d-706cbfd9d829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4077005791},{"type":"v1","addr":"172.21.15.190:6817","nonce":4077005791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4077005791},{"type":"v1","addr":"172.21.15.190:6819","nonce":4077005791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4077005791},{"type":"v1","addr":"172.21.15.190:6823","nonce":4077005791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":4077005791},{"type":"v1","addr":"172.21.15.190:6821","nonce":4077005791}]},"public_addr":"172.21.15.190:6817/4077005791","cluster_addr":"172.21.15.190:6819/4077005791","heartbeat_back_addr":"172.21.15.190:6823/4077005791","heartbeat_front_addr":"172.21.15.190:6821/4077005791","state":["exists","up"]},{"osd":7,"uuid":"7ee736b1-45ad-4daf-aea6-ae92e1f4c170","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3444881554},{"type":"v1","addr":"172.21.15.190:6825","nonce":3444881554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3444881554},{"type":"v1","addr":"172.21.15.190:6827","nonce":3444881554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3444881554},{"type":"v1","addr":"172.21.15.190:6831","nonce":3444881554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3444881554},{"type":"v1","addr":"172.21.15.190:6829","nonce":3444881554}]},"public_addr":"172.21.15.190:6825/3444881554","cluster_addr":"172.21.15.190:6827/3444881554","heartbeat_back_addr":"172.21.15.190:6831/3444881554","heartbeat_front_addr":"172.21.15.190:6829/3444881554","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T03:37:17.820423+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T03:38:11.242278+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T03:39:06.127491+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T03:39:59.306276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T03:40:50.284254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T03:41:45.327530+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T03:42:38.089070+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T03:43:32.633741+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:6801/712246762":"2024-03-27T03:34:52.164937+0000","172.21.15.175:6800/712246762":"2024-03-27T03:34:52.164937+0000","172.21.15.175:6800/1363563398":"2024-03-27T03:34:16.885473+0000","172.21.15.175:0/544124769":"2024-03-27T03:34:52.164937+0000","172.21.15.175:6801/1363563398":"2024-03-27T03:34:16.885473+0000","172.21.15.175:0/4149587290":"2024-03-27T03:34:16.885473+0000","172.21.15.175:0/4269788583":"2024-03-27T03:34:16.885473+0000","172.21.15.175:0/2156166628":"2024-03-27T03:34:52.164937+0000","172.21.15.175:0/1112608733":"2024-03-27T03:34:16.885473+0000","172.21.15.175:0/4127351834":"2024-03-27T03:34:52.164937+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T03:43:53.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.450+0000 7fd094aea700 1 -- 172.21.15.175:0/1865691010 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fd07c06e6f0 msgr2=0x7fd07c070bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:43:53.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.450+0000 7fd094aea700 1 --2- 172.21.15.175:0/1865691010 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fd07c06e6f0 0x7fd07c070bb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd088009750 tx=0x7fd08800b610 comp rx=0 tx=0).stop 2024-03-26T03:43:53.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.450+0000 7fd094aea700 1 -- 172.21.15.175:0/1865691010 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd090108660 msgr2=0x7fd09019ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:43:53.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.450+0000 7fd094aea700 1 --2- 172.21.15.175:0/1865691010 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd090108660 0x7fd09019ccc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd08000efb0 tx=0x7fd08000c3a0 comp rx=0 tx=0).stop 2024-03-26T03:43:53.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.450+0000 7fd094aea700 1 -- 172.21.15.175:0/1865691010 shutdown_connections 2024-03-26T03:43:53.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.450+0000 7fd094aea700 1 --2- 172.21.15.175:0/1865691010 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fd07c06e6f0 0x7fd07c070bb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:43:53.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.450+0000 7fd094aea700 1 --2- 172.21.15.175:0/1865691010 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd090108660 0x7fd09019ccc0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:43:53.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.450+0000 7fd094aea700 1 --2- 172.21.15.175:0/1865691010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd09006acc0 0x7fd0901983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:43:53.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.450+0000 7fd094aea700 1 --2- 172.21.15.175:0/1865691010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd09006b660 0x7fd090198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:43:53.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.450+0000 7fd094aea700 1 -- 172.21.15.175:0/1865691010 >> 172.21.15.175:0/1865691010 conn(0x7fd090078540 msgr2=0x7fd0901003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:43:53.450 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.450+0000 7fd094aea700 1 -- 172.21.15.175:0/1865691010 shutdown_connections 2024-03-26T03:43:53.451 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:43:53.450+0000 7fd094aea700 1 -- 172.21.15.175:0/1865691010 wait complete. 2024-03-26T03:43:53.675 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:53 smithi175 bash[21527]: cluster 2024-03-26T03:43:52.308448+0000 mgr.y (mgr.14152) 333 : cluster [DBG] pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:43:53.675 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:53 smithi175 bash[29284]: cluster 2024-03-26T03:43:52.308448+0000 mgr.y (mgr.14152) 333 : cluster [DBG] pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:43:53.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:53 smithi190 bash[23392]: cluster 2024-03-26T03:43:52.308448+0000 mgr.y (mgr.14152) 333 : cluster [DBG] pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:43:54.258 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T03:39:08.367712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T03:43:54.259 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-26T03:43:54.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:54 smithi175 bash[29284]: audit 2024-03-26T03:43:53.448989+0000 mon.c (mon.2) 15 : audit [DBG] from='client.? 172.21.15.175:0/1865691010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T03:43:54.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:54 smithi175 bash[21527]: audit 2024-03-26T03:43:53.448989+0000 mon.c (mon.2) 15 : audit [DBG] from='client.? 172.21.15.175:0/1865691010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T03:43:54.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:54 smithi190 bash[23392]: audit 2024-03-26T03:43:53.448989+0000 mon.c (mon.2) 15 : audit [DBG] from='client.? 172.21.15.175:0/1865691010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T03:43:55.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:55 smithi175 bash[21527]: cluster 2024-03-26T03:43:54.309051+0000 mgr.y (mgr.14152) 334 : cluster [DBG] pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:43:55.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:55 smithi175 bash[29284]: cluster 2024-03-26T03:43:54.309051+0000 mgr.y (mgr.14152) 334 : cluster [DBG] pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:43:55.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:55 smithi190 bash[23392]: cluster 2024-03-26T03:43:54.309051+0000 mgr.y (mgr.14152) 334 : cluster [DBG] pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:43:57.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:57 smithi175 bash[21527]: cluster 2024-03-26T03:43:56.309650+0000 mgr.y (mgr.14152) 335 : cluster [DBG] pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:43:57.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:57 smithi175 bash[29284]: cluster 2024-03-26T03:43:56.309650+0000 mgr.y (mgr.14152) 335 : cluster [DBG] pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:43:57.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:57 smithi190 bash[23392]: cluster 2024-03-26T03:43:56.309650+0000 mgr.y (mgr.14152) 335 : cluster [DBG] pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:43:59.020 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:43:59.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:43:59 smithi175 bash[21527]: cluster 2024-03-26T03:43:58.310238+0000 mgr.y (mgr.14152) 336 : cluster [DBG] pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:43:59.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:43:59 smithi175 bash[29284]: cluster 2024-03-26T03:43:58.310238+0000 mgr.y (mgr.14152) 336 : cluster [DBG] pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:43:59.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:43:59 smithi190 bash[23392]: cluster 2024-03-26T03:43:58.310238+0000 mgr.y (mgr.14152) 336 : cluster [DBG] pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:01.662 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.662+0000 7fd039218700 1 -- 172.21.15.175:0/1864899191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd034104870 msgr2=0x7fd034104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:44:01.662 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.662+0000 7fd039218700 1 --2- 172.21.15.175:0/1864899191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd034104870 0x7fd034104cf0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd024009990 tx=0x7fd024009ca0 comp rx=0 tx=0).stop 2024-03-26T03:44:01.662 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.662+0000 7fd039218700 1 -- 172.21.15.175:0/1864899191 shutdown_connections 2024-03-26T03:44:01.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.662+0000 7fd039218700 1 --2- 172.21.15.175:0/1864899191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd034105230 0x7fd034111740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:01.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.662+0000 7fd039218700 1 --2- 172.21.15.175:0/1864899191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd03410a870 0x7fd03410ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:01.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.662+0000 7fd039218700 1 --2- 172.21.15.175:0/1864899191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd034104870 0x7fd034104cf0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:01.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.662+0000 7fd039218700 1 -- 172.21.15.175:0/1864899191 >> 172.21.15.175:0/1864899191 conn(0x7fd034100360 msgr2=0x7fd034102780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:44:01.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.662+0000 7fd039218700 1 -- 172.21.15.175:0/1864899191 shutdown_connections 2024-03-26T03:44:01.663 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.662+0000 7fd039218700 1 -- 172.21.15.175:0/1864899191 wait complete. 2024-03-26T03:44:01.664 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.662+0000 7fd039218700 1 Processor -- start 2024-03-26T03:44:01.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.662+0000 7fd039218700 1 -- start start 2024-03-26T03:44:01.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.662+0000 7fd039218700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd034104870 0x7fd03419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:44:01.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.662+0000 7fd039218700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd034105230 0x7fd03419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:44:01.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.662+0000 7fd039218700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd03410a870 0x7fd03419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:44:01.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.662+0000 7fd039218700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd0341140d0 con 0x7fd034105230 2024-03-26T03:44:01.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.662+0000 7fd039218700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd034113f50 con 0x7fd03410a870 2024-03-26T03:44:01.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.662+0000 7fd039218700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd034114250 con 0x7fd034104870 2024-03-26T03:44:01.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.662+0000 7fd03259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd034105230 0x7fd03419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:44:01.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.662+0000 7fd032d9d700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd034104870 0x7fd03419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:44:01.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.662+0000 7fd03259c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd034105230 0x7fd03419ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:46300/0 (socket says 172.21.15.175:46300) 2024-03-26T03:44:01.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.662+0000 7fd03259c700 1 -- 172.21.15.175:0/3886936489 learned_addr learned my addr 172.21.15.175:0/3886936489 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:44:01.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.662+0000 7fd03359e700 1 --2- 172.21.15.175:0/3886936489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd03410a870 0x7fd03419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:44:01.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.662+0000 7fd03259c700 1 -- 172.21.15.175:0/3886936489 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd034104870 msgr2=0x7fd03419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:44:01.665 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.662+0000 7fd03259c700 1 --2- 172.21.15.175:0/3886936489 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd034104870 0x7fd03419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:01.666 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.662+0000 7fd03259c700 1 -- 172.21.15.175:0/3886936489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd03410a870 msgr2=0x7fd03419eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:44:01.666 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.662+0000 7fd03259c700 1 --2- 172.21.15.175:0/3886936489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd03410a870 0x7fd03419eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:01.666 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.662+0000 7fd03259c700 1 -- 172.21.15.175:0/3886936489 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03419f5e0 con 0x7fd034105230 2024-03-26T03:44:01.666 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.662+0000 7fd03359e700 1 --2- 172.21.15.175:0/3886936489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd03410a870 0x7fd03419eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T03:44:01.666 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.666+0000 7fd03259c700 1 --2- 172.21.15.175:0/3886936489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd034105230 0x7fd03419ab10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd0240057c0 tx=0x7fd0240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:44:01.666 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.666+0000 7fd02bfff700 1 -- 172.21.15.175:0/3886936489 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd024026070 con 0x7fd034105230 2024-03-26T03:44:01.666 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.666+0000 7fd02bfff700 1 -- 172.21.15.175:0/3886936489 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fd024021410 con 0x7fd034105230 2024-03-26T03:44:01.666 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.666+0000 7fd039218700 1 -- 172.21.15.175:0/3886936489 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd03419f870 con 0x7fd034105230 2024-03-26T03:44:01.666 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.666+0000 7fd02bfff700 1 -- 172.21.15.175:0/3886936489 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0240173f0 con 0x7fd034105230 2024-03-26T03:44:01.666 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.666+0000 7fd039218700 1 -- 172.21.15.175:0/3886936489 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0341a7150 con 0x7fd034105230 2024-03-26T03:44:01.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.666+0000 7fd039218700 1 -- 172.21.15.175:0/3886936489 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd034050300 con 0x7fd034105230 2024-03-26T03:44:01.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.666+0000 7fd02bfff700 1 -- 172.21.15.175:0/3886936489 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 15) ==== 92475+0+0 (secure 0 0 0) 0x7fd024029070 con 0x7fd034105230 2024-03-26T03:44:01.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.666+0000 7fd02bfff700 1 --2- 172.21.15.175:0/3886936489 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fd02006e6f0 0x7fd020070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:44:01.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.666+0000 7fd02bfff700 1 -- 172.21.15.175:0/3886936489 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5287+0+0 (secure 0 0 0) 0x7fd02408f1e0 con 0x7fd034105230 2024-03-26T03:44:01.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.666+0000 7fd032d9d700 1 --2- 172.21.15.175:0/3886936489 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fd02006e6f0 0x7fd020070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:44:01.667 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.666+0000 7fd032d9d700 1 --2- 172.21.15.175:0/3886936489 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fd02006e6f0 0x7fd020070bb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd01c009a10 tx=0x7fd01c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:44:01.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.670+0000 7fd02bfff700 1 -- 172.21.15.175:0/3886936489 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7fd024013070 con 0x7fd034105230 2024-03-26T03:44:01.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:01 smithi175 bash[29284]: cluster 2024-03-26T03:44:00.310826+0000 mgr.y (mgr.14152) 337 : cluster [DBG] pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:01.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:01 smithi175 bash[21527]: cluster 2024-03-26T03:44:00.310826+0000 mgr.y (mgr.14152) 337 : cluster [DBG] pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:01.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:01 smithi190 bash[23392]: cluster 2024-03-26T03:44:00.310826+0000 mgr.y (mgr.14152) 337 : cluster [DBG] pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:01.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.878+0000 7fd039218700 1 -- 172.21.15.175:0/3886936489 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd0341a7320 con 0x7fd034105230 2024-03-26T03:44:01.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.878+0000 7fd02bfff700 1 -- 172.21.15.175:0/3886936489 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) ==== 93+0+10 (secure 0 0 0) 0x7fd02405c450 con 0x7fd034105230 2024-03-26T03:44:01.879 INFO:teuthology.orchestra.run.smithi175.stdout:pg_num: 1 2024-03-26T03:44:01.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.882+0000 7fd039218700 1 -- 172.21.15.175:0/3886936489 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fd02006e6f0 msgr2=0x7fd020070bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:44:01.881 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.882+0000 7fd039218700 1 --2- 172.21.15.175:0/3886936489 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fd02006e6f0 0x7fd020070bb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd01c009a10 tx=0x7fd01c009190 comp rx=0 tx=0).stop 2024-03-26T03:44:01.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.882+0000 7fd039218700 1 -- 172.21.15.175:0/3886936489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd034105230 msgr2=0x7fd03419ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:44:01.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.882+0000 7fd039218700 1 --2- 172.21.15.175:0/3886936489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd034105230 0x7fd03419ab10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd0240057c0 tx=0x7fd0240057f0 comp rx=0 tx=0).stop 2024-03-26T03:44:01.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.882+0000 7fd039218700 1 -- 172.21.15.175:0/3886936489 shutdown_connections 2024-03-26T03:44:01.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.882+0000 7fd039218700 1 --2- 172.21.15.175:0/3886936489 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fd02006e6f0 0x7fd020070bb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:01.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.882+0000 7fd039218700 1 --2- 172.21.15.175:0/3886936489 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd034104870 0x7fd03419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:01.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.882+0000 7fd039218700 1 --2- 172.21.15.175:0/3886936489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd03410a870 0x7fd03419eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:01.882 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.882+0000 7fd039218700 1 --2- 172.21.15.175:0/3886936489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd034105230 0x7fd03419ab10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:01.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.882+0000 7fd039218700 1 -- 172.21.15.175:0/3886936489 >> 172.21.15.175:0/3886936489 conn(0x7fd034100360 msgr2=0x7fd034100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:44:01.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.882+0000 7fd039218700 1 -- 172.21.15.175:0/3886936489 shutdown_connections 2024-03-26T03:44:01.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:44:01.882+0000 7fd039218700 1 -- 172.21.15.175:0/3886936489 wait complete. 2024-03-26T03:44:02.651 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:02 smithi175 bash[29284]: audit 2024-03-26T03:44:01.882621+0000 mon.a (mon.0) 668 : audit [DBG] from='client.? 172.21.15.175:0/3886936489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T03:44:02.652 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:02 smithi175 bash[21527]: audit 2024-03-26T03:44:01.882621+0000 mon.a (mon.0) 668 : audit [DBG] from='client.? 172.21.15.175:0/3886936489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T03:44:02.653 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi175 2024-03-26T03:44:02.653 DEBUG:teuthology.orchestra.run.smithi190:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph orch apply rgw foo.a --placement '1;smithi175=foo.a' 2024-03-26T03:44:02.664 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:02 smithi190 bash[23392]: audit 2024-03-26T03:44:01.882621+0000 mon.a (mon.0) 668 : audit [DBG] from='client.? 172.21.15.175:0/3886936489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T03:44:03.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:03 smithi175 bash[21527]: cluster 2024-03-26T03:44:02.311423+0000 mgr.y (mgr.14152) 338 : cluster [DBG] pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:03.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:03 smithi175 bash[29284]: cluster 2024-03-26T03:44:02.311423+0000 mgr.y (mgr.14152) 338 : cluster [DBG] pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:03.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:03 smithi190 bash[23392]: cluster 2024-03-26T03:44:02.311423+0000 mgr.y (mgr.14152) 338 : cluster [DBG] pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:05.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:05 smithi175 bash[21527]: cluster 2024-03-26T03:44:04.312029+0000 mgr.y (mgr.14152) 339 : cluster [DBG] pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:05.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:05 smithi175 bash[29284]: cluster 2024-03-26T03:44:04.312029+0000 mgr.y (mgr.14152) 339 : cluster [DBG] pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:05.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:05 smithi190 bash[23392]: cluster 2024-03-26T03:44:04.312029+0000 mgr.y (mgr.14152) 339 : cluster [DBG] pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:07.398 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T03:44:07.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:07 smithi175 bash[29284]: cluster 2024-03-26T03:44:06.312557+0000 mgr.y (mgr.14152) 340 : cluster [DBG] pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:07.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:07 smithi175 bash[21527]: cluster 2024-03-26T03:44:06.312557+0000 mgr.y (mgr.14152) 340 : cluster [DBG] pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:07.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:07 smithi190 bash[23392]: cluster 2024-03-26T03:44:06.312557+0000 mgr.y (mgr.14152) 340 : cluster [DBG] pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:09.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:09 smithi175 bash[29284]: cluster 2024-03-26T03:44:08.313115+0000 mgr.y (mgr.14152) 341 : cluster [DBG] pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:09.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:09 smithi175 bash[21527]: cluster 2024-03-26T03:44:08.313115+0000 mgr.y (mgr.14152) 341 : cluster [DBG] pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:09.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:09 smithi190 bash[23392]: cluster 2024-03-26T03:44:08.313115+0000 mgr.y (mgr.14152) 341 : cluster [DBG] pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:10.368 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.369+0000 7fe1c80b3700 1 -- 172.21.15.190:0/485555566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1c0100f50 msgr2=0x7fe1c0101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:44:10.368 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.369+0000 7fe1c80b3700 1 --2- 172.21.15.190:0/485555566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1c0100f50 0x7fe1c0101370 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000b0a0 tx=0x7fe1b000b3b0 comp rx=0 tx=0).stop 2024-03-26T03:44:10.369 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.369+0000 7fe1c80b3700 1 -- 172.21.15.190:0/485555566 shutdown_connections 2024-03-26T03:44:10.369 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.369+0000 7fe1c80b3700 1 --2- 172.21.15.190:0/485555566 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1c0102270 0x7fe1c01026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:10.369 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.369+0000 7fe1c80b3700 1 --2- 172.21.15.190:0/485555566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1c0100f50 0x7fe1c0101370 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:10.369 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.369+0000 7fe1c80b3700 1 --2- 172.21.15.190:0/485555566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1c01018b0 0x7fe1c0101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:10.369 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.369+0000 7fe1c80b3700 1 -- 172.21.15.190:0/485555566 >> 172.21.15.190:0/485555566 conn(0x7fe1c00fcb70 msgr2=0x7fe1c00fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:44:10.369 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.369+0000 7fe1c80b3700 1 -- 172.21.15.190:0/485555566 shutdown_connections 2024-03-26T03:44:10.369 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.369+0000 7fe1c80b3700 1 -- 172.21.15.190:0/485555566 wait complete. 2024-03-26T03:44:10.369 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.373+0000 7fe1c80b3700 1 Processor -- start 2024-03-26T03:44:10.369 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.373+0000 7fe1c80b3700 1 -- start start 2024-03-26T03:44:10.369 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.373+0000 7fe1c80b3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1c0100f50 0x7fe1c0199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:44:10.369 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.373+0000 7fe1c80b3700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1c01018b0 0x7fe1c019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:44:10.369 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.373+0000 7fe1c80b3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1c0102270 0x7fe1c01a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:44:10.369 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.373+0000 7fe1c80b3700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c010f8e0 con 0x7fe1c0100f50 2024-03-26T03:44:10.369 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.373+0000 7fe1c80b3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c010f760 con 0x7fe1c0102270 2024-03-26T03:44:10.370 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.373+0000 7fe1c80b3700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe1c010fa60 con 0x7fe1c01018b0 2024-03-26T03:44:10.370 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.373+0000 7fe1c6650700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1c0102270 0x7fe1c01a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:44:10.370 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.373+0000 7fe1c6650700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1c0102270 0x7fe1c01a1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:45670/0 (socket says 172.21.15.190:45670) 2024-03-26T03:44:10.370 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.373+0000 7fe1c6650700 1 -- 172.21.15.190:0/2438288151 learned_addr learned my addr 172.21.15.190:0/2438288151 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-03-26T03:44:10.370 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.373+0000 7fe1c5e4f700 1 --2- 172.21.15.190:0/2438288151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1c0100f50 0x7fe1c0199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:44:10.370 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.373+0000 7fe1c564e700 1 --2- 172.21.15.190:0/2438288151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1c01018b0 0x7fe1c019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:44:10.370 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.373+0000 7fe1c6650700 1 -- 172.21.15.190:0/2438288151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1c01018b0 msgr2=0x7fe1c019a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:44:10.370 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.373+0000 7fe1c6650700 1 --2- 172.21.15.190:0/2438288151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1c01018b0 0x7fe1c019a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:10.370 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.373+0000 7fe1c6650700 1 -- 172.21.15.190:0/2438288151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1c0100f50 msgr2=0x7fe1c0199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:44:10.370 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.373+0000 7fe1c6650700 1 --2- 172.21.15.190:0/2438288151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1c0100f50 0x7fe1c0199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:10.370 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.373+0000 7fe1c6650700 1 -- 172.21.15.190:0/2438288151 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c01a1bd0 con 0x7fe1c0102270 2024-03-26T03:44:10.370 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.373+0000 7fe1c6650700 1 --2- 172.21.15.190:0/2438288151 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1c0102270 0x7fe1c01a1530 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800c900 tx=0x7fe1b800ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:44:10.370 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.373+0000 7fe1beffd700 1 -- 172.21.15.190:0/2438288151 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8007b00 con 0x7fe1c0102270 2024-03-26T03:44:10.370 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.373+0000 7fe1beffd700 1 -- 172.21.15.190:0/2438288151 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fe1b8007ca0 con 0x7fe1c0102270 2024-03-26T03:44:10.371 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.373+0000 7fe1c80b3700 1 -- 172.21.15.190:0/2438288151 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1c01a1ec0 con 0x7fe1c0102270 2024-03-26T03:44:10.371 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.373+0000 7fe1beffd700 1 -- 172.21.15.190:0/2438288151 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8018660 con 0x7fe1c0102270 2024-03-26T03:44:10.371 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.373+0000 7fe1c80b3700 1 -- 172.21.15.190:0/2438288151 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1c01a2320 con 0x7fe1c0102270 2024-03-26T03:44:10.372 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.373+0000 7fe1beffd700 1 -- 172.21.15.190:0/2438288151 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 15) ==== 92475+0+0 (secure 0 0 0) 0x7fe1b800f450 con 0x7fe1c0102270 2024-03-26T03:44:10.372 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.373+0000 7fe1c80b3700 1 -- 172.21.15.190:0/2438288151 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1c0050300 con 0x7fe1c0102270 2024-03-26T03:44:10.372 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.373+0000 7fe1beffd700 1 --2- 172.21.15.190:0/2438288151 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fe1a806e6f0 0x7fe1a8070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:44:10.372 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.373+0000 7fe1beffd700 1 -- 172.21.15.190:0/2438288151 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5287+0+0 (secure 0 0 0) 0x7fe1b808e5c0 con 0x7fe1c0102270 2024-03-26T03:44:10.372 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.373+0000 7fe1c5e4f700 1 --2- 172.21.15.190:0/2438288151 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fe1a806e6f0 0x7fe1a8070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:44:10.372 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.377+0000 7fe1c5e4f700 1 --2- 172.21.15.190:0/2438288151 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fe1a806e6f0 0x7fe1a8070bb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00062a0 tx=0x7fe1b0006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:44:10.375 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.381+0000 7fe1beffd700 1 -- 172.21.15.190:0/2438288151 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7fe1b8014030 con 0x7fe1c0102270 2024-03-26T03:44:10.589 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.593+0000 7fe1c80b3700 1 -- 172.21.15.190:0/2438288151 --> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] -- mgr_command(tid 0: {"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi175=foo.a", "target": ["mon-mgr", ""]}) -- 0x7fe1c0062a80 con 0x7fe1a806e6f0 2024-03-26T03:44:10.603 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.609+0000 7fe1beffd700 1 -- 172.21.15.190:0/2438288151 <== mgr.14152 v2:172.21.15.175:6800/2826039805 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7fe1c0062a80 con 0x7fe1a806e6f0 2024-03-26T03:44:10.604 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled rgw.foo.a update... 2024-03-26T03:44:10.606 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.609+0000 7fe1c80b3700 1 -- 172.21.15.190:0/2438288151 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fe1a806e6f0 msgr2=0x7fe1a8070bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:44:10.606 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.609+0000 7fe1c80b3700 1 --2- 172.21.15.190:0/2438288151 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fe1a806e6f0 0x7fe1a8070bb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00062a0 tx=0x7fe1b0006210 comp rx=0 tx=0).stop 2024-03-26T03:44:10.606 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.609+0000 7fe1c80b3700 1 -- 172.21.15.190:0/2438288151 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1c0102270 msgr2=0x7fe1c01a1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:44:10.607 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.609+0000 7fe1c80b3700 1 --2- 172.21.15.190:0/2438288151 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1c0102270 0x7fe1c01a1530 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800c900 tx=0x7fe1b800ccc0 comp rx=0 tx=0).stop 2024-03-26T03:44:10.607 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.609+0000 7fe1c80b3700 1 -- 172.21.15.190:0/2438288151 shutdown_connections 2024-03-26T03:44:10.607 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.609+0000 7fe1c80b3700 1 --2- 172.21.15.190:0/2438288151 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fe1a806e6f0 0x7fe1a8070bb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:10.607 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.609+0000 7fe1c80b3700 1 --2- 172.21.15.190:0/2438288151 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1c0102270 0x7fe1c01a1530 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:10.607 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.609+0000 7fe1c80b3700 1 --2- 172.21.15.190:0/2438288151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1c0100f50 0x7fe1c0199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:10.607 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.609+0000 7fe1c80b3700 1 --2- 172.21.15.190:0/2438288151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1c01018b0 0x7fe1c019a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:10.607 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.609+0000 7fe1c80b3700 1 -- 172.21.15.190:0/2438288151 >> 172.21.15.190:0/2438288151 conn(0x7fe1c00fcb70 msgr2=0x7fe1c00fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:44:10.607 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.609+0000 7fe1c80b3700 1 -- 172.21.15.190:0/2438288151 shutdown_connections 2024-03-26T03:44:10.607 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:10.609+0000 7fe1c80b3700 1 -- 172.21.15.190:0/2438288151 wait complete. 2024-03-26T03:44:11.453 DEBUG:teuthology.orchestra.run.smithi175:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@rgw.foo.a.service 2024-03-26T03:44:11.456 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi190 2024-03-26T03:44:11.456 DEBUG:teuthology.orchestra.run.smithi190:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph osd pool create datapool 3 3 replicated 2024-03-26T03:44:11.728 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:11 smithi175 bash[21527]: cluster 2024-03-26T03:44:10.313746+0000 mgr.y (mgr.14152) 342 : cluster [DBG] pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:11.728 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:11 smithi175 bash[21527]: audit 2024-03-26T03:44:10.594482+0000 mgr.y (mgr.14152) 343 : audit [DBG] from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi175=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:44:11.729 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:11 smithi175 bash[21527]: cephadm 2024-03-26T03:44:10.598522+0000 mgr.y (mgr.14152) 344 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi175=foo.a;count:1 2024-03-26T03:44:11.729 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:11 smithi175 bash[21527]: audit 2024-03-26T03:44:10.605579+0000 mon.a (mon.0) 669 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:11.729 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:11 smithi175 bash[21527]: audit 2024-03-26T03:44:10.639667+0000 mon.a (mon.0) 670 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:44:11.729 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:11 smithi175 bash[21527]: audit 2024-03-26T03:44:11.266937+0000 mon.a (mon.0) 671 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:44:11.729 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:11 smithi175 bash[21527]: audit 2024-03-26T03:44:11.268740+0000 mon.a (mon.0) 672 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:44:11.729 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:11 smithi175 bash[21527]: audit 2024-03-26T03:44:11.277947+0000 mon.a (mon.0) 673 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:11.729 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:11 smithi175 bash[21527]: audit 2024-03-26T03:44:11.282914+0000 mon.a (mon.0) 674 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-26T03:44:11.729 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:11 smithi175 bash[21527]: audit 2024-03-26T03:44:11.286075+0000 mon.a (mon.0) 675 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-03-26T03:44:11.729 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:11 smithi175 bash[21527]: audit 2024-03-26T03:44:11.298159+0000 mon.a (mon.0) 676 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:11.729 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:11 smithi175 bash[21527]: audit 2024-03-26T03:44:11.302571+0000 mon.a (mon.0) 677 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:44:11.729 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:11 smithi175 bash[29284]: cluster 2024-03-26T03:44:10.313746+0000 mgr.y (mgr.14152) 342 : cluster [DBG] pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:11.730 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:11 smithi175 bash[29284]: audit 2024-03-26T03:44:10.594482+0000 mgr.y (mgr.14152) 343 : audit [DBG] from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi175=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:44:11.730 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:11 smithi175 bash[29284]: cephadm 2024-03-26T03:44:10.598522+0000 mgr.y (mgr.14152) 344 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi175=foo.a;count:1 2024-03-26T03:44:11.730 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:11 smithi175 bash[29284]: audit 2024-03-26T03:44:10.605579+0000 mon.a (mon.0) 669 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:11.730 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:11 smithi175 bash[29284]: audit 2024-03-26T03:44:10.639667+0000 mon.a (mon.0) 670 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:44:11.730 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:11 smithi175 bash[29284]: audit 2024-03-26T03:44:11.266937+0000 mon.a (mon.0) 671 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:44:11.730 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:11 smithi175 bash[29284]: audit 2024-03-26T03:44:11.268740+0000 mon.a (mon.0) 672 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:44:11.731 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:11 smithi175 bash[29284]: audit 2024-03-26T03:44:11.277947+0000 mon.a (mon.0) 673 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:11.731 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:11 smithi175 bash[29284]: audit 2024-03-26T03:44:11.282914+0000 mon.a (mon.0) 674 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-26T03:44:11.731 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:11 smithi175 bash[29284]: audit 2024-03-26T03:44:11.286075+0000 mon.a (mon.0) 675 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-03-26T03:44:11.731 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:11 smithi175 bash[29284]: audit 2024-03-26T03:44:11.298159+0000 mon.a (mon.0) 676 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:11.731 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:11 smithi175 bash[29284]: audit 2024-03-26T03:44:11.302571+0000 mon.a (mon.0) 677 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:44:11.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:11 smithi190 bash[23392]: cluster 2024-03-26T03:44:10.313746+0000 mgr.y (mgr.14152) 342 : cluster [DBG] pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:11.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:11 smithi190 bash[23392]: audit 2024-03-26T03:44:10.594482+0000 mgr.y (mgr.14152) 343 : audit [DBG] from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi175=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:44:11.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:11 smithi190 bash[23392]: cephadm 2024-03-26T03:44:10.598522+0000 mgr.y (mgr.14152) 344 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi175=foo.a;count:1 2024-03-26T03:44:11.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:11 smithi190 bash[23392]: audit 2024-03-26T03:44:10.605579+0000 mon.a (mon.0) 669 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:11.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:11 smithi190 bash[23392]: audit 2024-03-26T03:44:10.639667+0000 mon.a (mon.0) 670 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:44:11.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:11 smithi190 bash[23392]: audit 2024-03-26T03:44:11.266937+0000 mon.a (mon.0) 671 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:44:11.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:11 smithi190 bash[23392]: audit 2024-03-26T03:44:11.268740+0000 mon.a (mon.0) 672 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:44:11.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:11 smithi190 bash[23392]: audit 2024-03-26T03:44:11.277947+0000 mon.a (mon.0) 673 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:11.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:11 smithi190 bash[23392]: audit 2024-03-26T03:44:11.282914+0000 mon.a (mon.0) 674 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-26T03:44:11.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:11 smithi190 bash[23392]: audit 2024-03-26T03:44:11.286075+0000 mon.a (mon.0) 675 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-03-26T03:44:11.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:11 smithi190 bash[23392]: audit 2024-03-26T03:44:11.298159+0000 mon.a (mon.0) 676 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:11.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:11 smithi190 bash[23392]: audit 2024-03-26T03:44:11.302571+0000 mon.a (mon.0) 677 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:44:12.861 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:12 smithi175 bash[21527]: cephadm 2024-03-26T03:44:11.304558+0000 mgr.y (mgr.14152) 345 : cephadm [INF] Deploying daemon rgw.foo.a on smithi175 2024-03-26T03:44:12.861 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:12 smithi175 bash[29284]: cephadm 2024-03-26T03:44:11.304558+0000 mgr.y (mgr.14152) 345 : cephadm [INF] Deploying daemon rgw.foo.a on smithi175 2024-03-26T03:44:12.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:12 smithi190 bash[23392]: cephadm 2024-03-26T03:44:11.304558+0000 mgr.y (mgr.14152) 345 : cephadm [INF] Deploying daemon rgw.foo.a on smithi175 2024-03-26T03:44:13.898 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:44:13 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:44:13.898 INFO:journalctl@ceph.osd.1.smithi175.stdout:Mar 26 03:44:13 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:44:13.898 INFO:journalctl@ceph.osd.3.smithi175.stdout:Mar 26 03:44:13 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:44:13.899 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:13 smithi175 bash[21527]: cluster 2024-03-26T03:44:12.314329+0000 mgr.y (mgr.14152) 346 : cluster [DBG] pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:13.899 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:13 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:44:13.899 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:13 smithi175 bash[29284]: cluster 2024-03-26T03:44:12.314329+0000 mgr.y (mgr.14152) 346 : cluster [DBG] pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:13.899 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:13 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:44:13.899 INFO:journalctl@ceph.osd.0.smithi175.stdout:Mar 26 03:44:13 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:44:13.899 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 26 03:44:13 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:44:13.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:13 smithi190 bash[23392]: cluster 2024-03-26T03:44:12.314329+0000 mgr.y (mgr.14152) 346 : cluster [DBG] pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:14.187 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:14 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:44:14.187 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:44:14 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:44:14.187 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:14 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:44:14.188 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 26 03:44:14 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:44:14.188 INFO:journalctl@ceph.osd.3.smithi175.stdout:Mar 26 03:44:14 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:44:14.188 INFO:journalctl@ceph.rgw.foo.a.smithi175.stdout:Mar 26 03:44:14 smithi175 systemd[1]: Started Ceph rgw.foo.a for 762e930c-eb21-11ee-95d0-87774f69a715. 2024-03-26T03:44:14.189 INFO:journalctl@ceph.osd.1.smithi175.stdout:Mar 26 03:44:14 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:44:14.189 INFO:journalctl@ceph.osd.0.smithi175.stdout:Mar 26 03:44:14 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:44:15.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:15 smithi175 bash[21527]: audit 2024-03-26T03:44:14.188208+0000 mon.a (mon.0) 678 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:15.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:15 smithi175 bash[21527]: audit 2024-03-26T03:44:14.197352+0000 mon.a (mon.0) 679 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:15.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:15 smithi175 bash[21527]: audit 2024-03-26T03:44:14.229728+0000 mon.a (mon.0) 680 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:15.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:15 smithi175 bash[21527]: audit 2024-03-26T03:44:14.235360+0000 mon.a (mon.0) 681 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:15.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:15 smithi175 bash[21527]: audit 2024-03-26T03:44:14.240948+0000 mon.a (mon.0) 682 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:15.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:15 smithi175 bash[21527]: audit 2024-03-26T03:44:14.273276+0000 mon.a (mon.0) 683 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:44:15.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:15 smithi175 bash[29284]: audit 2024-03-26T03:44:14.188208+0000 mon.a (mon.0) 678 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:15.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:15 smithi175 bash[29284]: audit 2024-03-26T03:44:14.197352+0000 mon.a (mon.0) 679 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:15.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:15 smithi175 bash[29284]: audit 2024-03-26T03:44:14.229728+0000 mon.a (mon.0) 680 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:15.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:15 smithi175 bash[29284]: audit 2024-03-26T03:44:14.235360+0000 mon.a (mon.0) 681 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:15.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:15 smithi175 bash[29284]: audit 2024-03-26T03:44:14.240948+0000 mon.a (mon.0) 682 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:15.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:15 smithi175 bash[29284]: audit 2024-03-26T03:44:14.273276+0000 mon.a (mon.0) 683 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:44:15.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:15 smithi190 bash[23392]: audit 2024-03-26T03:44:14.188208+0000 mon.a (mon.0) 678 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:15.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:15 smithi190 bash[23392]: audit 2024-03-26T03:44:14.197352+0000 mon.a (mon.0) 679 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:15.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:15 smithi190 bash[23392]: audit 2024-03-26T03:44:14.229728+0000 mon.a (mon.0) 680 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:15.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:15 smithi190 bash[23392]: audit 2024-03-26T03:44:14.235360+0000 mon.a (mon.0) 681 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:15.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:15 smithi190 bash[23392]: audit 2024-03-26T03:44:14.240948+0000 mon.a (mon.0) 682 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:15.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:15 smithi190 bash[23392]: audit 2024-03-26T03:44:14.273276+0000 mon.a (mon.0) 683 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:44:16.246 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T03:44:16.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:16 smithi175 bash[21527]: cephadm 2024-03-26T03:44:14.230373+0000 mgr.y (mgr.14152) 347 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi175=foo.a;count:1 2024-03-26T03:44:16.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:16 smithi175 bash[21527]: cluster 2024-03-26T03:44:14.314705+0000 mgr.y (mgr.14152) 348 : cluster [DBG] pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:16.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:16 smithi175 bash[21527]: cluster 2024-03-26T03:44:15.258682+0000 mon.a (mon.0) 684 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2024-03-26T03:44:16.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:16 smithi175 bash[21527]: audit 2024-03-26T03:44:15.266804+0000 mon.a (mon.0) 685 : audit [INF] from='client.? 172.21.15.175:0/1778182885' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-03-26T03:44:16.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:16 smithi175 bash[29284]: cephadm 2024-03-26T03:44:14.230373+0000 mgr.y (mgr.14152) 347 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi175=foo.a;count:1 2024-03-26T03:44:16.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:16 smithi175 bash[29284]: cluster 2024-03-26T03:44:14.314705+0000 mgr.y (mgr.14152) 348 : cluster [DBG] pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:16.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:16 smithi175 bash[29284]: cluster 2024-03-26T03:44:15.258682+0000 mon.a (mon.0) 684 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2024-03-26T03:44:16.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:16 smithi175 bash[29284]: audit 2024-03-26T03:44:15.266804+0000 mon.a (mon.0) 685 : audit [INF] from='client.? 172.21.15.175:0/1778182885' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-03-26T03:44:16.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:16 smithi190 bash[23392]: cephadm 2024-03-26T03:44:14.230373+0000 mgr.y (mgr.14152) 347 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi175=foo.a;count:1 2024-03-26T03:44:16.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:16 smithi190 bash[23392]: cluster 2024-03-26T03:44:14.314705+0000 mgr.y (mgr.14152) 348 : cluster [DBG] pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:16.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:16 smithi190 bash[23392]: cluster 2024-03-26T03:44:15.258682+0000 mon.a (mon.0) 684 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2024-03-26T03:44:16.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:16 smithi190 bash[23392]: audit 2024-03-26T03:44:15.266804+0000 mon.a (mon.0) 685 : audit [INF] from='client.? 172.21.15.175:0/1778182885' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-03-26T03:44:17.684 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:17 smithi190 bash[23392]: audit 2024-03-26T03:44:16.251738+0000 mon.a (mon.0) 686 : audit [INF] from='client.? 172.21.15.175:0/1778182885' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-03-26T03:44:17.684 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:17 smithi190 bash[23392]: cluster 2024-03-26T03:44:16.258725+0000 mon.a (mon.0) 687 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2024-03-26T03:44:17.684 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:17 smithi190 bash[23392]: cluster 2024-03-26T03:44:16.315221+0000 mgr.y (mgr.14152) 349 : cluster [DBG] pgmap v333: 33 pgs: 7 creating+peering, 1 active+clean, 25 unknown; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:17.684 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:17 smithi190 bash[23392]: audit 2024-03-26T03:44:16.900630+0000 mon.a (mon.0) 688 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:17.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:17 smithi175 bash[21527]: audit 2024-03-26T03:44:16.251738+0000 mon.a (mon.0) 686 : audit [INF] from='client.? 172.21.15.175:0/1778182885' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-03-26T03:44:17.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:17 smithi175 bash[21527]: cluster 2024-03-26T03:44:16.258725+0000 mon.a (mon.0) 687 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2024-03-26T03:44:17.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:17 smithi175 bash[21527]: cluster 2024-03-26T03:44:16.315221+0000 mgr.y (mgr.14152) 349 : cluster [DBG] pgmap v333: 33 pgs: 7 creating+peering, 1 active+clean, 25 unknown; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:17.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:17 smithi175 bash[21527]: audit 2024-03-26T03:44:16.900630+0000 mon.a (mon.0) 688 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:17.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:17 smithi175 bash[29284]: audit 2024-03-26T03:44:16.251738+0000 mon.a (mon.0) 686 : audit [INF] from='client.? 172.21.15.175:0/1778182885' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-03-26T03:44:17.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:17 smithi175 bash[29284]: cluster 2024-03-26T03:44:16.258725+0000 mon.a (mon.0) 687 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2024-03-26T03:44:17.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:17 smithi175 bash[29284]: cluster 2024-03-26T03:44:16.315221+0000 mgr.y (mgr.14152) 349 : cluster [DBG] pgmap v333: 33 pgs: 7 creating+peering, 1 active+clean, 25 unknown; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:17.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:17 smithi175 bash[29284]: audit 2024-03-26T03:44:16.900630+0000 mon.a (mon.0) 688 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:18.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:18 smithi175 bash[29284]: cluster 2024-03-26T03:44:17.270017+0000 mon.a (mon.0) 689 : cluster [DBG] osdmap e62: 8 total, 8 up, 8 in 2024-03-26T03:44:18.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:18 smithi175 bash[29284]: audit 2024-03-26T03:44:17.271653+0000 mon.a (mon.0) 690 : audit [INF] from='client.? 172.21.15.175:0/1435275536' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-26T03:44:18.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:18 smithi175 bash[21527]: cluster 2024-03-26T03:44:17.270017+0000 mon.a (mon.0) 689 : cluster [DBG] osdmap e62: 8 total, 8 up, 8 in 2024-03-26T03:44:18.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:18 smithi175 bash[21527]: audit 2024-03-26T03:44:17.271653+0000 mon.a (mon.0) 690 : audit [INF] from='client.? 172.21.15.175:0/1435275536' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-26T03:44:18.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:18 smithi190 bash[23392]: cluster 2024-03-26T03:44:17.270017+0000 mon.a (mon.0) 689 : cluster [DBG] osdmap e62: 8 total, 8 up, 8 in 2024-03-26T03:44:18.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:18 smithi190 bash[23392]: audit 2024-03-26T03:44:17.271653+0000 mon.a (mon.0) 690 : audit [INF] from='client.? 172.21.15.175:0/1435275536' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-26T03:44:19.402 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.405+0000 7f8f97ced700 1 -- 172.21.15.190:0/2354268495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f901083b0 msgr2=0x7f8f9010a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:44:19.402 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.405+0000 7f8f97ced700 1 --2- 172.21.15.190:0/2354268495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f901083b0 0x7f8f9010a820 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8f80009990 tx=0x7f8f80009ca0 comp rx=0 tx=0).stop 2024-03-26T03:44:19.402 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.405+0000 7f8f97ced700 1 -- 172.21.15.190:0/2354268495 shutdown_connections 2024-03-26T03:44:19.402 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.405+0000 7f8f97ced700 1 --2- 172.21.15.190:0/2354268495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f901083b0 0x7f8f9010a820 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:19.402 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.405+0000 7f8f97ced700 1 --2- 172.21.15.190:0/2354268495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f90103150 0x7f8f90105540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:19.402 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.405+0000 7f8f97ced700 1 --2- 172.21.15.190:0/2354268495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f90105a80 0x7f8f90107e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:19.402 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.405+0000 7f8f97ced700 1 -- 172.21.15.190:0/2354268495 >> 172.21.15.190:0/2354268495 conn(0x7f8f900fcb60 msgr2=0x7f8f900fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:44:19.402 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.405+0000 7f8f97ced700 1 -- 172.21.15.190:0/2354268495 shutdown_connections 2024-03-26T03:44:19.403 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.405+0000 7f8f97ced700 1 -- 172.21.15.190:0/2354268495 wait complete. 2024-03-26T03:44:19.404 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.405+0000 7f8f97ced700 1 Processor -- start 2024-03-26T03:44:19.404 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.405+0000 7f8f97ced700 1 -- start start 2024-03-26T03:44:19.404 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.405+0000 7f8f97ced700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f90103150 0x7f8f90102a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:44:19.404 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.405+0000 7f8f97ced700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f90105a80 0x7f8f90101100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:44:19.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.405+0000 7f8f97ced700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f901083b0 0x7f8f90101640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:44:19.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.405+0000 7f8f97ced700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9010d7b0 con 0x7f8f90103150 2024-03-26T03:44:19.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.405+0000 7f8f97ced700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9010d630 con 0x7f8f90105a80 2024-03-26T03:44:19.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.405+0000 7f8f97ced700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8f9010d930 con 0x7f8f901083b0 2024-03-26T03:44:19.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.405+0000 7f8f95288700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f90105a80 0x7f8f90101100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:44:19.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.405+0000 7f8f95288700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f90105a80 0x7f8f90101100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:55584/0 (socket says 172.21.15.190:55584) 2024-03-26T03:44:19.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.405+0000 7f8f95288700 1 -- 172.21.15.190:0/3073639684 learned_addr learned my addr 172.21.15.190:0/3073639684 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-03-26T03:44:19.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.405+0000 7f8f95a89700 1 --2- 172.21.15.190:0/3073639684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f90103150 0x7f8f90102a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:44:19.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.409+0000 7f8f9628a700 1 --2- 172.21.15.190:0/3073639684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f901083b0 0x7f8f90101640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:44:19.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.409+0000 7f8f95288700 1 -- 172.21.15.190:0/3073639684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f901083b0 msgr2=0x7f8f90101640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:44:19.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.409+0000 7f8f95288700 1 --2- 172.21.15.190:0/3073639684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f901083b0 0x7f8f90101640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:19.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.409+0000 7f8f95288700 1 -- 172.21.15.190:0/3073639684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f90103150 msgr2=0x7f8f90102a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:44:19.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.409+0000 7f8f95288700 1 --2- 172.21.15.190:0/3073639684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f90103150 0x7f8f90102a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:19.405 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.409+0000 7f8f95288700 1 -- 172.21.15.190:0/3073639684 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f90101f10 con 0x7f8f90105a80 2024-03-26T03:44:19.406 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.409+0000 7f8f95288700 1 --2- 172.21.15.190:0/3073639684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f90105a80 0x7f8f90101100 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400e9c0 tx=0x7f8f8400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:44:19.406 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.409+0000 7f8f8effd700 1 -- 172.21.15.190:0/3073639684 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8400cbd0 con 0x7f8f90105a80 2024-03-26T03:44:19.406 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.409+0000 7f8f8effd700 1 -- 172.21.15.190:0/3073639684 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f8f8400ee60 con 0x7f8f90105a80 2024-03-26T03:44:19.406 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.409+0000 7f8f8effd700 1 -- 172.21.15.190:0/3073639684 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f84005480 con 0x7f8f90105a80 2024-03-26T03:44:19.406 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.409+0000 7f8f97ced700 1 -- 172.21.15.190:0/3073639684 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f901a1f90 con 0x7f8f90105a80 2024-03-26T03:44:19.406 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.409+0000 7f8f97ced700 1 -- 172.21.15.190:0/3073639684 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f901a2390 con 0x7f8f90105a80 2024-03-26T03:44:19.407 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.409+0000 7f8f97ced700 1 -- 172.21.15.190:0/3073639684 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f90050300 con 0x7f8f90105a80 2024-03-26T03:44:19.407 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.409+0000 7f8f8effd700 1 -- 172.21.15.190:0/3073639684 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 15) ==== 92475+0+0 (secure 0 0 0) 0x7f8f84004750 con 0x7f8f90105a80 2024-03-26T03:44:19.407 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.409+0000 7f8f8effd700 1 --2- 172.21.15.190:0/3073639684 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f8f7806e6f0 0x7f8f78070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:44:19.407 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.409+0000 7f8f8effd700 1 -- 172.21.15.190:0/3073639684 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6369+0+0 (secure 0 0 0) 0x7f8f8408ec00 con 0x7f8f90105a80 2024-03-26T03:44:19.407 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.409+0000 7f8f95a89700 1 --2- 172.21.15.190:0/3073639684 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f8f7806e6f0 0x7f8f78070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:44:19.407 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.409+0000 7f8f95a89700 1 --2- 172.21.15.190:0/3073639684 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f8f7806e6f0 0x7f8f78070bb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800f540 tx=0x7f8f8800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:44:19.411 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.417+0000 7f8f8effd700 1 -- 172.21.15.190:0/3073639684 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f8f84095080 con 0x7f8f90105a80 2024-03-26T03:44:19.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:19 smithi190 bash[23392]: audit 2024-03-26T03:44:18.274315+0000 mon.a (mon.0) 691 : audit [INF] from='client.? 172.21.15.175:0/1435275536' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-03-26T03:44:19.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:19 smithi190 bash[23392]: cluster 2024-03-26T03:44:18.281318+0000 mon.a (mon.0) 692 : cluster [DBG] osdmap e63: 8 total, 8 up, 8 in 2024-03-26T03:44:19.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:19 smithi190 bash[23392]: cluster 2024-03-26T03:44:18.315796+0000 mgr.y (mgr.14152) 350 : cluster [DBG] pgmap v336: 65 pgs: 12 creating+peering, 10 active+clean, 43 unknown; 577 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 767 B/s wr, 1 op/s 2024-03-26T03:44:19.618 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:19.621+0000 7f8f97ced700 1 -- 172.21.15.190:0/3073639684 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} v 0) -- 0x7f8f9002dd30 con 0x7f8f90105a80 2024-03-26T03:44:19.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:19 smithi175 bash[21527]: audit 2024-03-26T03:44:18.274315+0000 mon.a (mon.0) 691 : audit [INF] from='client.? 172.21.15.175:0/1435275536' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-03-26T03:44:19.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:19 smithi175 bash[21527]: cluster 2024-03-26T03:44:18.281318+0000 mon.a (mon.0) 692 : cluster [DBG] osdmap e63: 8 total, 8 up, 8 in 2024-03-26T03:44:19.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:19 smithi175 bash[21527]: cluster 2024-03-26T03:44:18.315796+0000 mgr.y (mgr.14152) 350 : cluster [DBG] pgmap v336: 65 pgs: 12 creating+peering, 10 active+clean, 43 unknown; 577 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 767 B/s wr, 1 op/s 2024-03-26T03:44:19.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:19 smithi175 bash[29284]: audit 2024-03-26T03:44:18.274315+0000 mon.a (mon.0) 691 : audit [INF] from='client.? 172.21.15.175:0/1435275536' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-03-26T03:44:19.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:19 smithi175 bash[29284]: cluster 2024-03-26T03:44:18.281318+0000 mon.a (mon.0) 692 : cluster [DBG] osdmap e63: 8 total, 8 up, 8 in 2024-03-26T03:44:19.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:19 smithi175 bash[29284]: cluster 2024-03-26T03:44:18.315796+0000 mgr.y (mgr.14152) 350 : cluster [DBG] pgmap v336: 65 pgs: 12 creating+peering, 10 active+clean, 43 unknown; 577 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 767 B/s wr, 1 op/s 2024-03-26T03:44:20.292 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:20.293+0000 7f8f8effd700 1 -- 172.21.15.190:0/3073639684 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]=0 pool 'datapool' created v65) ==== 160+0+0 (secure 0 0 0) 0x7f8f8405c100 con 0x7f8f90105a80 2024-03-26T03:44:20.299 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:20.301+0000 7f8f97ced700 1 -- 172.21.15.190:0/3073639684 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f8f7806e6f0 msgr2=0x7f8f78070bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:44:20.299 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:20.301+0000 7f8f97ced700 1 --2- 172.21.15.190:0/3073639684 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f8f7806e6f0 0x7f8f78070bb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800f540 tx=0x7f8f8800b040 comp rx=0 tx=0).stop 2024-03-26T03:44:20.299 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:20.301+0000 7f8f97ced700 1 -- 172.21.15.190:0/3073639684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f90105a80 msgr2=0x7f8f90101100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:44:20.299 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:20.301+0000 7f8f97ced700 1 --2- 172.21.15.190:0/3073639684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f90105a80 0x7f8f90101100 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400e9c0 tx=0x7f8f8400ecd0 comp rx=0 tx=0).stop 2024-03-26T03:44:20.299 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:20.301+0000 7f8f97ced700 1 -- 172.21.15.190:0/3073639684 shutdown_connections 2024-03-26T03:44:20.299 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:20.301+0000 7f8f97ced700 1 --2- 172.21.15.190:0/3073639684 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f8f7806e6f0 0x7f8f78070bb0 secure :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800f540 tx=0x7f8f8800b040 comp rx=0 tx=0).stop 2024-03-26T03:44:20.299 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:20.301+0000 7f8f97ced700 1 --2- 172.21.15.190:0/3073639684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f901083b0 0x7f8f90101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:20.299 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:20.301+0000 7f8f97ced700 1 --2- 172.21.15.190:0/3073639684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f90103150 0x7f8f90102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:20.299 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:20.301+0000 7f8f97ced700 1 --2- 172.21.15.190:0/3073639684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f90105a80 0x7f8f90101100 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:20.299 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:20.301+0000 7f8f97ced700 1 -- 172.21.15.190:0/3073639684 >> 172.21.15.190:0/3073639684 conn(0x7f8f900fcb60 msgr2=0x7f8f90103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:44:20.304 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:20.309+0000 7f8f97ced700 1 -- 172.21.15.190:0/3073639684 shutdown_connections 2024-03-26T03:44:20.304 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:20.309+0000 7f8f97ced700 1 -- 172.21.15.190:0/3073639684 wait complete. 2024-03-26T03:44:20.307 INFO:teuthology.orchestra.run.smithi190.stderr:pool 'datapool' created 2024-03-26T03:44:20.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:20 smithi175 bash[21527]: cluster 2024-03-26T03:44:19.286114+0000 mon.a (mon.0) 693 : cluster [DBG] osdmap e64: 8 total, 8 up, 8 in 2024-03-26T03:44:20.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:20 smithi175 bash[21527]: audit 2024-03-26T03:44:19.291453+0000 mon.a (mon.0) 694 : audit [INF] from='client.? 172.21.15.175:0/1435275536' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-26T03:44:20.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:20 smithi175 bash[21527]: audit 2024-03-26T03:44:19.622810+0000 mon.a (mon.0) 695 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-03-26T03:44:20.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:20 smithi175 bash[21527]: audit 2024-03-26T03:44:19.626486+0000 mon.b (mon.1) 26 : audit [INF] from='client.? 172.21.15.190:0/3073639684' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-03-26T03:44:20.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:20 smithi175 bash[21527]: audit 2024-03-26T03:44:19.659276+0000 mon.a (mon.0) 696 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:20.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:20 smithi175 bash[21527]: audit 2024-03-26T03:44:19.665116+0000 mon.a (mon.0) 697 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:20.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:20 smithi175 bash[21527]: audit 2024-03-26T03:44:19.666175+0000 mon.a (mon.0) 698 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:44:20.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:20 smithi175 bash[21527]: audit 2024-03-26T03:44:19.667002+0000 mon.a (mon.0) 699 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:44:20.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:20 smithi175 bash[21527]: cephadm 2024-03-26T03:44:19.671517+0000 mgr.y (mgr.14152) 351 : cephadm [INF] Checking dashboard <-> RGW credentials 2024-03-26T03:44:20.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:20 smithi175 bash[29284]: cluster 2024-03-26T03:44:19.286114+0000 mon.a (mon.0) 693 : cluster [DBG] osdmap e64: 8 total, 8 up, 8 in 2024-03-26T03:44:20.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:20 smithi175 bash[29284]: audit 2024-03-26T03:44:19.291453+0000 mon.a (mon.0) 694 : audit [INF] from='client.? 172.21.15.175:0/1435275536' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-26T03:44:20.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:20 smithi175 bash[29284]: audit 2024-03-26T03:44:19.622810+0000 mon.a (mon.0) 695 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-03-26T03:44:20.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:20 smithi175 bash[29284]: audit 2024-03-26T03:44:19.626486+0000 mon.b (mon.1) 26 : audit [INF] from='client.? 172.21.15.190:0/3073639684' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-03-26T03:44:20.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:20 smithi175 bash[29284]: audit 2024-03-26T03:44:19.659276+0000 mon.a (mon.0) 696 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:20.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:20 smithi175 bash[29284]: audit 2024-03-26T03:44:19.665116+0000 mon.a (mon.0) 697 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:20.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:20 smithi175 bash[29284]: audit 2024-03-26T03:44:19.666175+0000 mon.a (mon.0) 698 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:44:20.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:20 smithi175 bash[29284]: audit 2024-03-26T03:44:19.667002+0000 mon.a (mon.0) 699 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:44:20.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:20 smithi175 bash[29284]: cephadm 2024-03-26T03:44:19.671517+0000 mgr.y (mgr.14152) 351 : cephadm [INF] Checking dashboard <-> RGW credentials 2024-03-26T03:44:20.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:20 smithi190 bash[23392]: cluster 2024-03-26T03:44:19.286114+0000 mon.a (mon.0) 693 : cluster [DBG] osdmap e64: 8 total, 8 up, 8 in 2024-03-26T03:44:20.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:20 smithi190 bash[23392]: audit 2024-03-26T03:44:19.291453+0000 mon.a (mon.0) 694 : audit [INF] from='client.? 172.21.15.175:0/1435275536' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-26T03:44:20.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:20 smithi190 bash[23392]: audit 2024-03-26T03:44:19.622810+0000 mon.a (mon.0) 695 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-03-26T03:44:20.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:20 smithi190 bash[23392]: audit 2024-03-26T03:44:19.626486+0000 mon.b (mon.1) 26 : audit [INF] from='client.? 172.21.15.190:0/3073639684' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-03-26T03:44:20.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:20 smithi190 bash[23392]: audit 2024-03-26T03:44:19.659276+0000 mon.a (mon.0) 696 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:20.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:20 smithi190 bash[23392]: audit 2024-03-26T03:44:19.665116+0000 mon.a (mon.0) 697 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:20.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:20 smithi190 bash[23392]: audit 2024-03-26T03:44:19.666175+0000 mon.a (mon.0) 698 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:44:20.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:20 smithi190 bash[23392]: audit 2024-03-26T03:44:19.667002+0000 mon.a (mon.0) 699 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:44:20.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:20 smithi190 bash[23392]: cephadm 2024-03-26T03:44:19.671517+0000 mgr.y (mgr.14152) 351 : cephadm [INF] Checking dashboard <-> RGW credentials 2024-03-26T03:44:21.140 DEBUG:teuthology.orchestra.run.smithi190:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- rbd pool init datapool 2024-03-26T03:44:21.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:21 smithi190 bash[23392]: audit 2024-03-26T03:44:20.284425+0000 mon.a (mon.0) 700 : audit [INF] from='client.? 172.21.15.175:0/1435275536' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-03-26T03:44:21.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:21 smithi190 bash[23392]: audit 2024-03-26T03:44:20.284595+0000 mon.a (mon.0) 701 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-03-26T03:44:21.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:21 smithi190 bash[23392]: cluster 2024-03-26T03:44:20.295908+0000 mon.a (mon.0) 702 : cluster [DBG] osdmap e65: 8 total, 8 up, 8 in 2024-03-26T03:44:21.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:21 smithi190 bash[23392]: cluster 2024-03-26T03:44:20.316889+0000 mgr.y (mgr.14152) 352 : cluster [DBG] pgmap v339: 100 pgs: 2 creating+activating, 12 creating+peering, 33 active+clean, 53 unknown; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-26T03:44:21.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:21 smithi190 bash[23392]: cluster 2024-03-26T03:44:21.297576+0000 mon.a (mon.0) 703 : cluster [DBG] osdmap e66: 8 total, 8 up, 8 in 2024-03-26T03:44:21.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:21 smithi190 bash[23392]: audit 2024-03-26T03:44:21.309468+0000 mon.a (mon.0) 704 : audit [INF] from='client.? 172.21.15.175:0/1435275536' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-26T03:44:21.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:21 smithi175 bash[21527]: audit 2024-03-26T03:44:20.284425+0000 mon.a (mon.0) 700 : audit [INF] from='client.? 172.21.15.175:0/1435275536' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-03-26T03:44:21.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:21 smithi175 bash[21527]: audit 2024-03-26T03:44:20.284595+0000 mon.a (mon.0) 701 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-03-26T03:44:21.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:21 smithi175 bash[21527]: cluster 2024-03-26T03:44:20.295908+0000 mon.a (mon.0) 702 : cluster [DBG] osdmap e65: 8 total, 8 up, 8 in 2024-03-26T03:44:21.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:21 smithi175 bash[21527]: cluster 2024-03-26T03:44:20.316889+0000 mgr.y (mgr.14152) 352 : cluster [DBG] pgmap v339: 100 pgs: 2 creating+activating, 12 creating+peering, 33 active+clean, 53 unknown; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-26T03:44:21.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:21 smithi175 bash[21527]: cluster 2024-03-26T03:44:21.297576+0000 mon.a (mon.0) 703 : cluster [DBG] osdmap e66: 8 total, 8 up, 8 in 2024-03-26T03:44:21.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:21 smithi175 bash[21527]: audit 2024-03-26T03:44:21.309468+0000 mon.a (mon.0) 704 : audit [INF] from='client.? 172.21.15.175:0/1435275536' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-26T03:44:21.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:21 smithi175 bash[29284]: audit 2024-03-26T03:44:20.284425+0000 mon.a (mon.0) 700 : audit [INF] from='client.? 172.21.15.175:0/1435275536' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-03-26T03:44:21.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:21 smithi175 bash[29284]: audit 2024-03-26T03:44:20.284595+0000 mon.a (mon.0) 701 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-03-26T03:44:21.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:21 smithi175 bash[29284]: cluster 2024-03-26T03:44:20.295908+0000 mon.a (mon.0) 702 : cluster [DBG] osdmap e65: 8 total, 8 up, 8 in 2024-03-26T03:44:21.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:21 smithi175 bash[29284]: cluster 2024-03-26T03:44:20.316889+0000 mgr.y (mgr.14152) 352 : cluster [DBG] pgmap v339: 100 pgs: 2 creating+activating, 12 creating+peering, 33 active+clean, 53 unknown; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-26T03:44:21.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:21 smithi175 bash[29284]: cluster 2024-03-26T03:44:21.297576+0000 mon.a (mon.0) 703 : cluster [DBG] osdmap e66: 8 total, 8 up, 8 in 2024-03-26T03:44:21.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:21 smithi175 bash[29284]: audit 2024-03-26T03:44:21.309468+0000 mon.a (mon.0) 704 : audit [INF] from='client.? 172.21.15.175:0/1435275536' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-26T03:44:22.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:22 smithi175 bash[21527]: audit 2024-03-26T03:44:21.317391+0000 mon.c (mon.2) 16 : audit [INF] from='client.? 172.21.15.175:0/2067867592' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-26T03:44:22.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:22 smithi175 bash[21527]: cluster 2024-03-26T03:44:21.317440+0000 mon.a (mon.0) 705 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:44:22.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:22 smithi175 bash[21527]: audit 2024-03-26T03:44:21.318661+0000 mon.a (mon.0) 706 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-26T03:44:22.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:22 smithi175 bash[21527]: audit 2024-03-26T03:44:22.290968+0000 mon.a (mon.0) 707 : audit [INF] from='client.? 172.21.15.175:0/1435275536' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-03-26T03:44:22.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:22 smithi175 bash[21527]: audit 2024-03-26T03:44:22.291150+0000 mon.a (mon.0) 708 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-03-26T03:44:22.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:22 smithi175 bash[21527]: cluster 2024-03-26T03:44:22.301166+0000 mon.a (mon.0) 709 : cluster [DBG] osdmap e67: 8 total, 8 up, 8 in 2024-03-26T03:44:22.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:22 smithi175 bash[21527]: audit 2024-03-26T03:44:22.305758+0000 mon.a (mon.0) 710 : audit [INF] from='client.? 172.21.15.175:0/1435275536' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-26T03:44:22.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:22 smithi175 bash[21527]: audit 2024-03-26T03:44:22.314313+0000 mon.c (mon.2) 17 : audit [INF] from='client.? 172.21.15.175:0/2067867592' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-26T03:44:22.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:22 smithi175 bash[21527]: audit 2024-03-26T03:44:22.316374+0000 mon.a (mon.0) 711 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-26T03:44:22.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:22 smithi175 bash[29284]: audit 2024-03-26T03:44:21.317391+0000 mon.c (mon.2) 16 : audit [INF] from='client.? 172.21.15.175:0/2067867592' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-26T03:44:22.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:22 smithi175 bash[29284]: cluster 2024-03-26T03:44:21.317440+0000 mon.a (mon.0) 705 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:44:22.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:22 smithi175 bash[29284]: audit 2024-03-26T03:44:21.318661+0000 mon.a (mon.0) 706 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-26T03:44:22.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:22 smithi175 bash[29284]: audit 2024-03-26T03:44:22.290968+0000 mon.a (mon.0) 707 : audit [INF] from='client.? 172.21.15.175:0/1435275536' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-03-26T03:44:22.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:22 smithi175 bash[29284]: audit 2024-03-26T03:44:22.291150+0000 mon.a (mon.0) 708 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-03-26T03:44:22.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:22 smithi175 bash[29284]: cluster 2024-03-26T03:44:22.301166+0000 mon.a (mon.0) 709 : cluster [DBG] osdmap e67: 8 total, 8 up, 8 in 2024-03-26T03:44:22.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:22 smithi175 bash[29284]: audit 2024-03-26T03:44:22.305758+0000 mon.a (mon.0) 710 : audit [INF] from='client.? 172.21.15.175:0/1435275536' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-26T03:44:22.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:22 smithi175 bash[29284]: audit 2024-03-26T03:44:22.314313+0000 mon.c (mon.2) 17 : audit [INF] from='client.? 172.21.15.175:0/2067867592' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-26T03:44:22.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:22 smithi175 bash[29284]: audit 2024-03-26T03:44:22.316374+0000 mon.a (mon.0) 711 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-26T03:44:22.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:22 smithi190 bash[23392]: audit 2024-03-26T03:44:21.317391+0000 mon.c (mon.2) 16 : audit [INF] from='client.? 172.21.15.175:0/2067867592' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-26T03:44:22.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:22 smithi190 bash[23392]: cluster 2024-03-26T03:44:21.317440+0000 mon.a (mon.0) 705 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:44:22.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:22 smithi190 bash[23392]: audit 2024-03-26T03:44:21.318661+0000 mon.a (mon.0) 706 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-26T03:44:22.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:22 smithi190 bash[23392]: audit 2024-03-26T03:44:22.290968+0000 mon.a (mon.0) 707 : audit [INF] from='client.? 172.21.15.175:0/1435275536' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-03-26T03:44:22.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:22 smithi190 bash[23392]: audit 2024-03-26T03:44:22.291150+0000 mon.a (mon.0) 708 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-03-26T03:44:22.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:22 smithi190 bash[23392]: cluster 2024-03-26T03:44:22.301166+0000 mon.a (mon.0) 709 : cluster [DBG] osdmap e67: 8 total, 8 up, 8 in 2024-03-26T03:44:22.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:22 smithi190 bash[23392]: audit 2024-03-26T03:44:22.305758+0000 mon.a (mon.0) 710 : audit [INF] from='client.? 172.21.15.175:0/1435275536' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-26T03:44:22.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:22 smithi190 bash[23392]: audit 2024-03-26T03:44:22.314313+0000 mon.c (mon.2) 17 : audit [INF] from='client.? 172.21.15.175:0/2067867592' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-26T03:44:22.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:22 smithi190 bash[23392]: audit 2024-03-26T03:44:22.316374+0000 mon.a (mon.0) 711 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-26T03:44:23.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:23 smithi175 bash[29284]: cluster 2024-03-26T03:44:22.318104+0000 mgr.y (mgr.14152) 353 : cluster [DBG] pgmap v342: 132 pgs: 2 creating+activating, 18 creating+peering, 84 active+clean, 28 unknown; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 767 B/s wr, 4 op/s 2024-03-26T03:44:23.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:23 smithi175 bash[29284]: audit 2024-03-26T03:44:23.301718+0000 mon.a (mon.0) 712 : audit [INF] from='client.? 172.21.15.175:0/1435275536' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-26T03:44:23.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:23 smithi175 bash[29284]: audit 2024-03-26T03:44:23.301936+0000 mon.a (mon.0) 713 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-26T03:44:23.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:23 smithi175 bash[29284]: cluster 2024-03-26T03:44:23.308528+0000 mon.a (mon.0) 714 : cluster [DBG] osdmap e68: 8 total, 8 up, 8 in 2024-03-26T03:44:23.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:23 smithi175 bash[21527]: cluster 2024-03-26T03:44:22.318104+0000 mgr.y (mgr.14152) 353 : cluster [DBG] pgmap v342: 132 pgs: 2 creating+activating, 18 creating+peering, 84 active+clean, 28 unknown; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 767 B/s wr, 4 op/s 2024-03-26T03:44:23.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:23 smithi175 bash[21527]: audit 2024-03-26T03:44:23.301718+0000 mon.a (mon.0) 712 : audit [INF] from='client.? 172.21.15.175:0/1435275536' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-26T03:44:23.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:23 smithi175 bash[21527]: audit 2024-03-26T03:44:23.301936+0000 mon.a (mon.0) 713 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-26T03:44:23.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:23 smithi175 bash[21527]: cluster 2024-03-26T03:44:23.308528+0000 mon.a (mon.0) 714 : cluster [DBG] osdmap e68: 8 total, 8 up, 8 in 2024-03-26T03:44:23.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:23 smithi190 bash[23392]: cluster 2024-03-26T03:44:22.318104+0000 mgr.y (mgr.14152) 353 : cluster [DBG] pgmap v342: 132 pgs: 2 creating+activating, 18 creating+peering, 84 active+clean, 28 unknown; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 767 B/s wr, 4 op/s 2024-03-26T03:44:23.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:23 smithi190 bash[23392]: audit 2024-03-26T03:44:23.301718+0000 mon.a (mon.0) 712 : audit [INF] from='client.? 172.21.15.175:0/1435275536' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-26T03:44:23.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:23 smithi190 bash[23392]: audit 2024-03-26T03:44:23.301936+0000 mon.a (mon.0) 713 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-26T03:44:23.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:23 smithi190 bash[23392]: cluster 2024-03-26T03:44:23.308528+0000 mon.a (mon.0) 714 : cluster [DBG] osdmap e68: 8 total, 8 up, 8 in 2024-03-26T03:44:24.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:24 smithi175 bash[21527]: audit 2024-03-26T03:44:23.605259+0000 mon.a (mon.0) 715 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:24.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:24 smithi175 bash[21527]: audit 2024-03-26T03:44:23.612260+0000 mon.a (mon.0) 716 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:24.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:24 smithi175 bash[21527]: audit 2024-03-26T03:44:23.624777+0000 mon.a (mon.0) 717 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:24.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:24 smithi175 bash[21527]: audit 2024-03-26T03:44:23.701785+0000 mon.a (mon.0) 718 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:44:24.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:24 smithi175 bash[21527]: audit 2024-03-26T03:44:23.704125+0000 mon.a (mon.0) 719 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:44:24.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:24 smithi175 bash[21527]: audit 2024-03-26T03:44:23.704902+0000 mon.a (mon.0) 720 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:44:24.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:24 smithi175 bash[21527]: cephadm 2024-03-26T03:44:23.709657+0000 mgr.y (mgr.14152) 354 : cephadm [INF] Checking dashboard <-> RGW credentials 2024-03-26T03:44:24.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:24 smithi175 bash[21527]: audit 2024-03-26T03:44:23.960463+0000 mon.a (mon.0) 721 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:24.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:24 smithi175 bash[29284]: audit 2024-03-26T03:44:23.605259+0000 mon.a (mon.0) 715 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:24.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:24 smithi175 bash[29284]: audit 2024-03-26T03:44:23.612260+0000 mon.a (mon.0) 716 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:24.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:24 smithi175 bash[29284]: audit 2024-03-26T03:44:23.624777+0000 mon.a (mon.0) 717 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:24.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:24 smithi175 bash[29284]: audit 2024-03-26T03:44:23.701785+0000 mon.a (mon.0) 718 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:44:24.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:24 smithi175 bash[29284]: audit 2024-03-26T03:44:23.704125+0000 mon.a (mon.0) 719 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:44:24.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:24 smithi175 bash[29284]: audit 2024-03-26T03:44:23.704902+0000 mon.a (mon.0) 720 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:44:24.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:24 smithi175 bash[29284]: cephadm 2024-03-26T03:44:23.709657+0000 mgr.y (mgr.14152) 354 : cephadm [INF] Checking dashboard <-> RGW credentials 2024-03-26T03:44:24.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:24 smithi175 bash[29284]: audit 2024-03-26T03:44:23.960463+0000 mon.a (mon.0) 721 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:24.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:24 smithi190 bash[23392]: audit 2024-03-26T03:44:23.605259+0000 mon.a (mon.0) 715 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:24.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:24 smithi190 bash[23392]: audit 2024-03-26T03:44:23.612260+0000 mon.a (mon.0) 716 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:24.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:24 smithi190 bash[23392]: audit 2024-03-26T03:44:23.624777+0000 mon.a (mon.0) 717 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:24.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:24 smithi190 bash[23392]: audit 2024-03-26T03:44:23.701785+0000 mon.a (mon.0) 718 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:44:24.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:24 smithi190 bash[23392]: audit 2024-03-26T03:44:23.704125+0000 mon.a (mon.0) 719 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:44:24.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:24 smithi190 bash[23392]: audit 2024-03-26T03:44:23.704902+0000 mon.a (mon.0) 720 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:44:24.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:24 smithi190 bash[23392]: cephadm 2024-03-26T03:44:23.709657+0000 mgr.y (mgr.14152) 354 : cephadm [INF] Checking dashboard <-> RGW credentials 2024-03-26T03:44:24.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:24 smithi190 bash[23392]: audit 2024-03-26T03:44:23.960463+0000 mon.a (mon.0) 721 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:25.900 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T03:44:26.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:25 smithi175 bash[21527]: cluster 2024-03-26T03:44:24.319452+0000 mgr.y (mgr.14152) 355 : cluster [DBG] pgmap v344: 132 pgs: 13 creating+peering, 108 active+clean, 11 unknown; 579 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 35 KiB/s rd, 2.2 KiB/s wr, 73 op/s 2024-03-26T03:44:26.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:25 smithi175 bash[29284]: cluster 2024-03-26T03:44:24.319452+0000 mgr.y (mgr.14152) 355 : cluster [DBG] pgmap v344: 132 pgs: 13 creating+peering, 108 active+clean, 11 unknown; 579 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 35 KiB/s rd, 2.2 KiB/s wr, 73 op/s 2024-03-26T03:44:26.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:25 smithi190 bash[23392]: cluster 2024-03-26T03:44:24.319452+0000 mgr.y (mgr.14152) 355 : cluster [DBG] pgmap v344: 132 pgs: 13 creating+peering, 108 active+clean, 11 unknown; 579 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 35 KiB/s rd, 2.2 KiB/s wr, 73 op/s 2024-03-26T03:44:28.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:27 smithi175 bash[21527]: cluster 2024-03-26T03:44:26.320606+0000 mgr.y (mgr.14152) 356 : cluster [DBG] pgmap v345: 132 pgs: 12 creating+peering, 120 active+clean; 580 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s rd, 3.5 KiB/s wr, 143 op/s 2024-03-26T03:44:28.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:27 smithi175 bash[29284]: cluster 2024-03-26T03:44:26.320606+0000 mgr.y (mgr.14152) 356 : cluster [DBG] pgmap v345: 132 pgs: 12 creating+peering, 120 active+clean; 580 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s rd, 3.5 KiB/s wr, 143 op/s 2024-03-26T03:44:28.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:27 smithi190 bash[23392]: cluster 2024-03-26T03:44:26.320606+0000 mgr.y (mgr.14152) 356 : cluster [DBG] pgmap v345: 132 pgs: 12 creating+peering, 120 active+clean; 580 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s rd, 3.5 KiB/s wr, 143 op/s 2024-03-26T03:44:30.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:29 smithi175 bash[21527]: cluster 2024-03-26T03:44:28.322057+0000 mgr.y (mgr.14152) 357 : cluster [DBG] pgmap v346: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 105 KiB/s rd, 5.1 KiB/s wr, 231 op/s 2024-03-26T03:44:30.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:29 smithi175 bash[21527]: audit 2024-03-26T03:44:29.173851+0000 mon.a (mon.0) 722 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-03-26T03:44:30.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:29 smithi175 bash[21527]: audit 2024-03-26T03:44:29.177417+0000 mon.b (mon.1) 27 : audit [INF] from='client.? 172.21.15.190:0/2815482186' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-03-26T03:44:30.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:29 smithi175 bash[29284]: cluster 2024-03-26T03:44:28.322057+0000 mgr.y (mgr.14152) 357 : cluster [DBG] pgmap v346: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 105 KiB/s rd, 5.1 KiB/s wr, 231 op/s 2024-03-26T03:44:30.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:29 smithi175 bash[29284]: audit 2024-03-26T03:44:29.173851+0000 mon.a (mon.0) 722 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-03-26T03:44:30.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:29 smithi175 bash[29284]: audit 2024-03-26T03:44:29.177417+0000 mon.b (mon.1) 27 : audit [INF] from='client.? 172.21.15.190:0/2815482186' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-03-26T03:44:30.243 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:29 smithi190 bash[23392]: cluster 2024-03-26T03:44:28.322057+0000 mgr.y (mgr.14152) 357 : cluster [DBG] pgmap v346: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 105 KiB/s rd, 5.1 KiB/s wr, 231 op/s 2024-03-26T03:44:30.243 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:29 smithi190 bash[23392]: audit 2024-03-26T03:44:29.173851+0000 mon.a (mon.0) 722 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-03-26T03:44:30.243 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:29 smithi190 bash[23392]: audit 2024-03-26T03:44:29.177417+0000 mon.b (mon.1) 27 : audit [INF] from='client.? 172.21.15.190:0/2815482186' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-03-26T03:44:31.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:31 smithi175 bash[21527]: audit 2024-03-26T03:44:29.985162+0000 mon.a (mon.0) 723 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-03-26T03:44:31.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:31 smithi175 bash[21527]: cluster 2024-03-26T03:44:29.996111+0000 mon.a (mon.0) 724 : cluster [DBG] osdmap e69: 8 total, 8 up, 8 in 2024-03-26T03:44:31.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:31 smithi175 bash[29284]: audit 2024-03-26T03:44:29.985162+0000 mon.a (mon.0) 723 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-03-26T03:44:31.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:31 smithi175 bash[29284]: cluster 2024-03-26T03:44:29.996111+0000 mon.a (mon.0) 724 : cluster [DBG] osdmap e69: 8 total, 8 up, 8 in 2024-03-26T03:44:31.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:31 smithi190 bash[23392]: audit 2024-03-26T03:44:29.985162+0000 mon.a (mon.0) 723 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-03-26T03:44:31.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:31 smithi190 bash[23392]: cluster 2024-03-26T03:44:29.996111+0000 mon.a (mon.0) 724 : cluster [DBG] osdmap e69: 8 total, 8 up, 8 in 2024-03-26T03:44:32.307 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:32 smithi190 bash[23392]: cluster 2024-03-26T03:44:30.322882+0000 mgr.y (mgr.14152) 358 : cluster [DBG] pgmap v348: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 92 KiB/s rd, 4.5 KiB/s wr, 202 op/s 2024-03-26T03:44:32.307 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:32 smithi190 bash[23392]: cluster 2024-03-26T03:44:30.985957+0000 mon.a (mon.0) 725 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-26T03:44:32.308 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:32 smithi190 bash[23392]: cluster 2024-03-26T03:44:30.986014+0000 mon.a (mon.0) 726 : cluster [INF] Cluster is now healthy 2024-03-26T03:44:32.308 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:32 smithi190 bash[23392]: cluster 2024-03-26T03:44:30.995901+0000 mon.a (mon.0) 727 : cluster [DBG] osdmap e70: 8 total, 8 up, 8 in 2024-03-26T03:44:32.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:32 smithi175 bash[21527]: cluster 2024-03-26T03:44:30.322882+0000 mgr.y (mgr.14152) 358 : cluster [DBG] pgmap v348: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 92 KiB/s rd, 4.5 KiB/s wr, 202 op/s 2024-03-26T03:44:32.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:32 smithi175 bash[21527]: cluster 2024-03-26T03:44:30.985957+0000 mon.a (mon.0) 725 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-26T03:44:32.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:32 smithi175 bash[21527]: cluster 2024-03-26T03:44:30.986014+0000 mon.a (mon.0) 726 : cluster [INF] Cluster is now healthy 2024-03-26T03:44:32.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:32 smithi175 bash[21527]: cluster 2024-03-26T03:44:30.995901+0000 mon.a (mon.0) 727 : cluster [DBG] osdmap e70: 8 total, 8 up, 8 in 2024-03-26T03:44:32.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:32 smithi175 bash[29284]: cluster 2024-03-26T03:44:30.322882+0000 mgr.y (mgr.14152) 358 : cluster [DBG] pgmap v348: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 92 KiB/s rd, 4.5 KiB/s wr, 202 op/s 2024-03-26T03:44:32.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:32 smithi175 bash[29284]: cluster 2024-03-26T03:44:30.985957+0000 mon.a (mon.0) 725 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-26T03:44:32.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:32 smithi175 bash[29284]: cluster 2024-03-26T03:44:30.986014+0000 mon.a (mon.0) 726 : cluster [INF] Cluster is now healthy 2024-03-26T03:44:32.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:32 smithi175 bash[29284]: cluster 2024-03-26T03:44:30.995901+0000 mon.a (mon.0) 727 : cluster [DBG] osdmap e70: 8 total, 8 up, 8 in 2024-03-26T03:44:32.863 DEBUG:teuthology.orchestra.run.smithi190:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.190 --placement '1;smithi190=iscsi.a' 2024-03-26T03:44:33.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:33 smithi190 bash[23392]: cluster 2024-03-26T03:44:32.008226+0000 mon.a (mon.0) 728 : cluster [DBG] osdmap e71: 8 total, 8 up, 8 in 2024-03-26T03:44:33.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:33 smithi175 bash[29284]: cluster 2024-03-26T03:44:32.008226+0000 mon.a (mon.0) 728 : cluster [DBG] osdmap e71: 8 total, 8 up, 8 in 2024-03-26T03:44:33.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:33 smithi175 bash[21527]: cluster 2024-03-26T03:44:32.008226+0000 mon.a (mon.0) 728 : cluster [DBG] osdmap e71: 8 total, 8 up, 8 in 2024-03-26T03:44:34.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:34 smithi175 bash[21527]: cluster 2024-03-26T03:44:32.323827+0000 mgr.y (mgr.14152) 359 : cluster [DBG] pgmap v351: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 2.7 KiB/s wr, 128 op/s 2024-03-26T03:44:34.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:34 smithi175 bash[29284]: cluster 2024-03-26T03:44:32.323827+0000 mgr.y (mgr.14152) 359 : cluster [DBG] pgmap v351: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 2.7 KiB/s wr, 128 op/s 2024-03-26T03:44:34.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:34 smithi190 bash[23392]: cluster 2024-03-26T03:44:32.323827+0000 mgr.y (mgr.14152) 359 : cluster [DBG] pgmap v351: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 2.7 KiB/s wr, 128 op/s 2024-03-26T03:44:36.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:36 smithi175 bash[21527]: cluster 2024-03-26T03:44:34.324800+0000 mgr.y (mgr.14152) 360 : cluster [DBG] pgmap v352: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:36.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:36 smithi175 bash[29284]: cluster 2024-03-26T03:44:34.324800+0000 mgr.y (mgr.14152) 360 : cluster [DBG] pgmap v352: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:36.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:36 smithi190 bash[23392]: cluster 2024-03-26T03:44:34.324800+0000 mgr.y (mgr.14152) 360 : cluster [DBG] pgmap v352: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:37.611 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T03:44:38.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:38 smithi175 bash[21527]: cluster 2024-03-26T03:44:36.325942+0000 mgr.y (mgr.14152) 361 : cluster [DBG] pgmap v353: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 323 B/s wr, 0 op/s 2024-03-26T03:44:38.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:38 smithi175 bash[29284]: cluster 2024-03-26T03:44:36.325942+0000 mgr.y (mgr.14152) 361 : cluster [DBG] pgmap v353: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 323 B/s wr, 0 op/s 2024-03-26T03:44:38.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:38 smithi190 bash[23392]: cluster 2024-03-26T03:44:36.325942+0000 mgr.y (mgr.14152) 361 : cluster [DBG] pgmap v353: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 323 B/s wr, 0 op/s 2024-03-26T03:44:40.474 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:40 smithi190 bash[23392]: cluster 2024-03-26T03:44:38.327035+0000 mgr.y (mgr.14152) 362 : cluster [DBG] pgmap v354: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-26T03:44:40.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:40 smithi175 bash[21527]: cluster 2024-03-26T03:44:38.327035+0000 mgr.y (mgr.14152) 362 : cluster [DBG] pgmap v354: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-26T03:44:40.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:40 smithi175 bash[29284]: cluster 2024-03-26T03:44:38.327035+0000 mgr.y (mgr.14152) 362 : cluster [DBG] pgmap v354: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-26T03:44:40.635 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f393bf72700 1 -- 172.21.15.190:0/2986401194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39341015d0 msgr2=0x7f39341019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:44:40.635 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f393bf72700 1 --2- 172.21.15.190:0/2986401194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39341015d0 0x7f39341019f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f39300099c0 tx=0x7f3930009cd0 comp rx=0 tx=0).stop 2024-03-26T03:44:40.635 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f393bf72700 1 -- 172.21.15.190:0/2986401194 shutdown_connections 2024-03-26T03:44:40.635 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f393bf72700 1 --2- 172.21.15.190:0/2986401194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3934103170 0x7f3934109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:40.635 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f393bf72700 1 --2- 172.21.15.190:0/2986401194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39341015d0 0x7f39341019f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:40.636 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f393bf72700 1 --2- 172.21.15.190:0/2986401194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39341027d0 0x7f3934102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:40.636 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f393bf72700 1 -- 172.21.15.190:0/2986401194 >> 172.21.15.190:0/2986401194 conn(0x7f39340fcb70 msgr2=0x7f39340fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:44:40.636 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f393bf72700 1 -- 172.21.15.190:0/2986401194 shutdown_connections 2024-03-26T03:44:40.636 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f393bf72700 1 -- 172.21.15.190:0/2986401194 wait complete. 2024-03-26T03:44:40.638 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f393bf72700 1 Processor -- start 2024-03-26T03:44:40.638 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f393bf72700 1 -- start start 2024-03-26T03:44:40.638 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f393bf72700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39341015d0 0x7f393419a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:44:40.638 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f393bf72700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39341027d0 0x7f393419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:44:40.638 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f393bf72700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3934103170 0x7f39341a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:44:40.639 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f393bf72700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f393410c700 con 0x7f39341027d0 2024-03-26T03:44:40.639 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f393bf72700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f393410c580 con 0x7f39341015d0 2024-03-26T03:44:40.639 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f393bf72700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f393410c880 con 0x7f3934103170 2024-03-26T03:44:40.639 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f3939d0e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39341015d0 0x7f393419a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:44:40.639 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f3939d0e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39341015d0 0x7f393419a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:60652/0 (socket says 172.21.15.190:60652) 2024-03-26T03:44:40.639 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f3939d0e700 1 -- 172.21.15.190:0/604515079 learned_addr learned my addr 172.21.15.190:0/604515079 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-03-26T03:44:40.639 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f393950d700 1 --2- 172.21.15.190:0/604515079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39341027d0 0x7f393419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:44:40.639 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f393a50f700 1 --2- 172.21.15.190:0/604515079 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3934103170 0x7f39341a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:44:40.639 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f3939d0e700 1 -- 172.21.15.190:0/604515079 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3934103170 msgr2=0x7f39341a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:44:40.639 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f3939d0e700 1 --2- 172.21.15.190:0/604515079 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3934103170 0x7f39341a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:40.639 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f3939d0e700 1 -- 172.21.15.190:0/604515079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39341027d0 msgr2=0x7f393419a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:44:40.639 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f3939d0e700 1 --2- 172.21.15.190:0/604515079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39341027d0 0x7f393419a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:40.640 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f3939d0e700 1 -- 172.21.15.190:0/604515079 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39341a1d00 con 0x7f39341015d0 2024-03-26T03:44:40.640 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f393a50f700 1 --2- 172.21.15.190:0/604515079 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3934103170 0x7f39341a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T03:44:40.640 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f393950d700 1 --2- 172.21.15.190:0/604515079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39341027d0 0x7f393419a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T03:44:40.640 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f3939d0e700 1 --2- 172.21.15.190:0/604515079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39341015d0 0x7f393419a0a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f393000c010 tx=0x7f39300047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:44:40.640 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f392affd700 1 -- 172.21.15.190:0/604515079 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3930027070 con 0x7f39341015d0 2024-03-26T03:44:40.640 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f392affd700 1 -- 172.21.15.190:0/604515079 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f3930004090 con 0x7f39341015d0 2024-03-26T03:44:40.640 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f393bf72700 1 -- 172.21.15.190:0/604515079 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39341a1f90 con 0x7f39341015d0 2024-03-26T03:44:40.640 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f392affd700 1 -- 172.21.15.190:0/604515079 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3930022c50 con 0x7f39341015d0 2024-03-26T03:44:40.640 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f393bf72700 1 -- 172.21.15.190:0/604515079 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39341a2380 con 0x7f39341015d0 2024-03-26T03:44:40.640 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.641+0000 7f393bf72700 1 -- 172.21.15.190:0/604515079 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3934050300 con 0x7f39341015d0 2024-03-26T03:44:40.640 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.645+0000 7f392affd700 1 -- 172.21.15.190:0/604515079 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 15) ==== 92475+0+0 (secure 0 0 0) 0x7f393002a070 con 0x7f39341015d0 2024-03-26T03:44:40.641 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.645+0000 7f392affd700 1 --2- 172.21.15.190:0/604515079 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f392006e6f0 0x7f3920070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:44:40.641 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.645+0000 7f392affd700 1 -- 172.21.15.190:0/604515079 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7176+0+0 (secure 0 0 0) 0x7f393008f970 con 0x7f39341015d0 2024-03-26T03:44:40.641 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.645+0000 7f393950d700 1 --2- 172.21.15.190:0/604515079 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f392006e6f0 0x7f3920070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:44:40.641 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.645+0000 7f393950d700 1 --2- 172.21.15.190:0/604515079 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f392006e6f0 0x7f3920070bb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3924005fd0 tx=0x7f3924005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:44:40.644 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.649+0000 7f392affd700 1 -- 172.21.15.190:0/604515079 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f3930013070 con 0x7f39341015d0 2024-03-26T03:44:40.863 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.869+0000 7f393bf72700 1 -- 172.21.15.190:0/604515079 --> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] -- mgr_command(tid 0: {"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.190", "placement": "1;smithi190=iscsi.a", "target": ["mon-mgr", ""]}) -- 0x7f39340fee90 con 0x7f392006e6f0 2024-03-26T03:44:40.881 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.885+0000 7f392affd700 1 -- 172.21.15.190:0/604515079 <== mgr.14152 v2:172.21.15.175:6800/2826039805 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+35 (secure 0 0 0) 0x7f39340fee90 con 0x7f392006e6f0 2024-03-26T03:44:40.881 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled iscsi.datapool update... 2024-03-26T03:44:40.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.889+0000 7f393bf72700 1 -- 172.21.15.190:0/604515079 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f392006e6f0 msgr2=0x7f3920070bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:44:40.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.889+0000 7f393bf72700 1 --2- 172.21.15.190:0/604515079 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f392006e6f0 0x7f3920070bb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3924005fd0 tx=0x7f3924005ee0 comp rx=0 tx=0).stop 2024-03-26T03:44:40.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.889+0000 7f393bf72700 1 -- 172.21.15.190:0/604515079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39341015d0 msgr2=0x7f393419a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:44:40.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.889+0000 7f393bf72700 1 --2- 172.21.15.190:0/604515079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39341015d0 0x7f393419a0a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f393000c010 tx=0x7f39300047b0 comp rx=0 tx=0).stop 2024-03-26T03:44:40.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.889+0000 7f393bf72700 1 -- 172.21.15.190:0/604515079 shutdown_connections 2024-03-26T03:44:40.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.889+0000 7f393bf72700 1 --2- 172.21.15.190:0/604515079 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f392006e6f0 0x7f3920070bb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:40.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.889+0000 7f393bf72700 1 --2- 172.21.15.190:0/604515079 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3934103170 0x7f39341a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:40.884 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.889+0000 7f393bf72700 1 --2- 172.21.15.190:0/604515079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39341015d0 0x7f393419a0a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:40.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.889+0000 7f393bf72700 1 --2- 172.21.15.190:0/604515079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39341027d0 0x7f393419a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:40.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.889+0000 7f393bf72700 1 -- 172.21.15.190:0/604515079 >> 172.21.15.190:0/604515079 conn(0x7f39340fcb70 msgr2=0x7f39340fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:44:40.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.889+0000 7f393bf72700 1 -- 172.21.15.190:0/604515079 shutdown_connections 2024-03-26T03:44:40.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:40.889+0000 7f393bf72700 1 -- 172.21.15.190:0/604515079 wait complete. 2024-03-26T03:44:41.679 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-03-26T03:44:41.680 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-26T03:44:41.680 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-03-26T03:44:41.693 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-03-26T03:44:41.693 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-03-26T03:44:41.707 DEBUG:teuthology.orchestra.run.smithi190:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@iscsi.iscsi.a.service 2024-03-26T03:44:41.752 INFO:tasks.cephadm:Adding prometheus.a on smithi190 2024-03-26T03:44:41.752 DEBUG:teuthology.orchestra.run.smithi190:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph orch apply prometheus '1;smithi190=a' 2024-03-26T03:44:41.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:41 smithi175 bash[21527]: cluster 2024-03-26T03:44:40.328060+0000 mgr.y (mgr.14152) 363 : cluster [DBG] pgmap v355: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 219 B/s wr, 0 op/s 2024-03-26T03:44:41.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:41 smithi175 bash[21527]: audit 2024-03-26T03:44:40.869253+0000 mgr.y (mgr.14152) 364 : audit [DBG] from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.190", "placement": "1;smithi190=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:44:41.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:41 smithi175 bash[21527]: cephadm 2024-03-26T03:44:40.875847+0000 mgr.y (mgr.14152) 365 : cephadm [INF] Saving service iscsi.datapool spec with placement smithi190=iscsi.a;count:1 2024-03-26T03:44:41.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:41 smithi175 bash[21527]: audit 2024-03-26T03:44:40.883616+0000 mon.a (mon.0) 729 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:41.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:41 smithi175 bash[21527]: audit 2024-03-26T03:44:40.906118+0000 mon.a (mon.0) 730 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:44:41.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:41 smithi175 bash[21527]: cluster 2024-03-26T03:44:41.296715+0000 mon.a (mon.0) 731 : cluster [DBG] osdmap e72: 8 total, 8 up, 8 in 2024-03-26T03:44:41.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:41 smithi175 bash[21527]: audit 2024-03-26T03:44:41.431548+0000 mon.a (mon.0) 732 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:41.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:41 smithi175 bash[21527]: audit 2024-03-26T03:44:41.438326+0000 mon.a (mon.0) 733 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:41 smithi175 bash[29284]: cluster 2024-03-26T03:44:40.328060+0000 mgr.y (mgr.14152) 363 : cluster [DBG] pgmap v355: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 219 B/s wr, 0 op/s 2024-03-26T03:44:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:41 smithi175 bash[29284]: audit 2024-03-26T03:44:40.869253+0000 mgr.y (mgr.14152) 364 : audit [DBG] from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.190", "placement": "1;smithi190=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:44:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:41 smithi175 bash[29284]: cephadm 2024-03-26T03:44:40.875847+0000 mgr.y (mgr.14152) 365 : cephadm [INF] Saving service iscsi.datapool spec with placement smithi190=iscsi.a;count:1 2024-03-26T03:44:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:41 smithi175 bash[29284]: audit 2024-03-26T03:44:40.883616+0000 mon.a (mon.0) 729 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:41 smithi175 bash[29284]: audit 2024-03-26T03:44:40.906118+0000 mon.a (mon.0) 730 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:44:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:41 smithi175 bash[29284]: cluster 2024-03-26T03:44:41.296715+0000 mon.a (mon.0) 731 : cluster [DBG] osdmap e72: 8 total, 8 up, 8 in 2024-03-26T03:44:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:41 smithi175 bash[29284]: audit 2024-03-26T03:44:41.431548+0000 mon.a (mon.0) 732 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:41 smithi175 bash[29284]: audit 2024-03-26T03:44:41.438326+0000 mon.a (mon.0) 733 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:41.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:41 smithi190 bash[23392]: cluster 2024-03-26T03:44:40.328060+0000 mgr.y (mgr.14152) 363 : cluster [DBG] pgmap v355: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 219 B/s wr, 0 op/s 2024-03-26T03:44:41.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:41 smithi190 bash[23392]: audit 2024-03-26T03:44:40.869253+0000 mgr.y (mgr.14152) 364 : audit [DBG] from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.190", "placement": "1;smithi190=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:44:41.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:41 smithi190 bash[23392]: cephadm 2024-03-26T03:44:40.875847+0000 mgr.y (mgr.14152) 365 : cephadm [INF] Saving service iscsi.datapool spec with placement smithi190=iscsi.a;count:1 2024-03-26T03:44:41.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:41 smithi190 bash[23392]: audit 2024-03-26T03:44:40.883616+0000 mon.a (mon.0) 729 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:41.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:41 smithi190 bash[23392]: audit 2024-03-26T03:44:40.906118+0000 mon.a (mon.0) 730 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:44:41.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:41 smithi190 bash[23392]: cluster 2024-03-26T03:44:41.296715+0000 mon.a (mon.0) 731 : cluster [DBG] osdmap e72: 8 total, 8 up, 8 in 2024-03-26T03:44:41.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:41 smithi190 bash[23392]: audit 2024-03-26T03:44:41.431548+0000 mon.a (mon.0) 732 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:41.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:41 smithi190 bash[23392]: audit 2024-03-26T03:44:41.438326+0000 mon.a (mon.0) 733 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:43.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:42 smithi175 bash[21527]: audit 2024-03-26T03:44:42.014017+0000 mon.a (mon.0) 734 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:44:43.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:42 smithi175 bash[21527]: audit 2024-03-26T03:44:42.015611+0000 mon.a (mon.0) 735 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:44:43.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:42 smithi175 bash[21527]: audit 2024-03-26T03:44:42.024908+0000 mon.a (mon.0) 736 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:43.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:42 smithi175 bash[21527]: audit 2024-03-26T03:44:42.030347+0000 mon.a (mon.0) 737 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-26T03:44:43.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:42 smithi175 bash[21527]: audit 2024-03-26T03:44:42.033637+0000 mon.a (mon.0) 738 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-03-26T03:44:43.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:42 smithi175 bash[21527]: audit 2024-03-26T03:44:42.055552+0000 mon.a (mon.0) 739 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:44:43.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:42 smithi175 bash[21527]: cephadm 2024-03-26T03:44:42.057709+0000 mgr.y (mgr.14152) 366 : cephadm [INF] Deploying daemon iscsi.iscsi.a on smithi190 2024-03-26T03:44:43.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:42 smithi175 bash[29284]: audit 2024-03-26T03:44:42.014017+0000 mon.a (mon.0) 734 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:44:43.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:42 smithi175 bash[29284]: audit 2024-03-26T03:44:42.015611+0000 mon.a (mon.0) 735 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:44:43.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:42 smithi175 bash[29284]: audit 2024-03-26T03:44:42.024908+0000 mon.a (mon.0) 736 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:43.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:42 smithi175 bash[29284]: audit 2024-03-26T03:44:42.030347+0000 mon.a (mon.0) 737 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-26T03:44:43.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:42 smithi175 bash[29284]: audit 2024-03-26T03:44:42.033637+0000 mon.a (mon.0) 738 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-03-26T03:44:43.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:42 smithi175 bash[29284]: audit 2024-03-26T03:44:42.055552+0000 mon.a (mon.0) 739 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:44:43.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:42 smithi175 bash[29284]: cephadm 2024-03-26T03:44:42.057709+0000 mgr.y (mgr.14152) 366 : cephadm [INF] Deploying daemon iscsi.iscsi.a on smithi190 2024-03-26T03:44:43.243 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:42 smithi190 bash[23392]: audit 2024-03-26T03:44:42.014017+0000 mon.a (mon.0) 734 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:44:43.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:42 smithi190 bash[23392]: audit 2024-03-26T03:44:42.015611+0000 mon.a (mon.0) 735 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:44:43.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:42 smithi190 bash[23392]: audit 2024-03-26T03:44:42.024908+0000 mon.a (mon.0) 736 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:43.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:42 smithi190 bash[23392]: audit 2024-03-26T03:44:42.030347+0000 mon.a (mon.0) 737 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-26T03:44:43.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:42 smithi190 bash[23392]: audit 2024-03-26T03:44:42.033637+0000 mon.a (mon.0) 738 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-03-26T03:44:43.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:42 smithi190 bash[23392]: audit 2024-03-26T03:44:42.055552+0000 mon.a (mon.0) 739 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:44:43.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:42 smithi190 bash[23392]: cephadm 2024-03-26T03:44:42.057709+0000 mgr.y (mgr.14152) 366 : cephadm [INF] Deploying daemon iscsi.iscsi.a on smithi190 2024-03-26T03:44:44.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:44 smithi190 bash[23392]: cluster 2024-03-26T03:44:42.329018+0000 mgr.y (mgr.14152) 367 : cluster [DBG] pgmap v357: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 204 B/s wr, 0 op/s 2024-03-26T03:44:44.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:44 smithi175 bash[21527]: cluster 2024-03-26T03:44:42.329018+0000 mgr.y (mgr.14152) 367 : cluster [DBG] pgmap v357: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 204 B/s wr, 0 op/s 2024-03-26T03:44:44.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:44 smithi175 bash[29284]: cluster 2024-03-26T03:44:42.329018+0000 mgr.y (mgr.14152) 367 : cluster [DBG] pgmap v357: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 204 B/s wr, 0 op/s 2024-03-26T03:44:44.769 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:44 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:44:44.769 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:44:44 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:44:44.769 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 03:44:44 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:44:44.769 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 03:44:44 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:44:44.770 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 03:44:44 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:44:44.770 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 03:44:44 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:44:45.020 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:44 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:44:45.020 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:44:44 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:44:45.020 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 03:44:44 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:44:45.021 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 03:44:44 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:44:45.021 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 03:44:44 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:44:45.021 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 03:44:44 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:44:46.328 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:46 smithi190 bash[23392]: cluster 2024-03-26T03:44:44.330089+0000 mgr.y (mgr.14152) 368 : cluster [DBG] pgmap v358: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 204 B/s wr, 0 op/s 2024-03-26T03:44:46.328 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:46 smithi190 bash[23392]: audit 2024-03-26T03:44:45.079375+0000 mon.a (mon.0) 740 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:46.328 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:46 smithi190 bash[23392]: audit 2024-03-26T03:44:45.090531+0000 mon.a (mon.0) 741 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:46.328 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:46 smithi190 bash[23392]: audit 2024-03-26T03:44:45.138999+0000 mon.a (mon.0) 742 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:46.328 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:46 smithi190 bash[23392]: cephadm 2024-03-26T03:44:45.140800+0000 mgr.y (mgr.14152) 369 : cephadm [INF] Checking pool "datapool" exists for service iscsi.datapool 2024-03-26T03:44:46.328 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:46 smithi190 bash[23392]: audit 2024-03-26T03:44:45.157244+0000 mon.a (mon.0) 743 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:46.329 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:46 smithi190 bash[23392]: audit 2024-03-26T03:44:45.222489+0000 mon.a (mon.0) 744 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:44:46.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:46 smithi175 bash[21527]: cluster 2024-03-26T03:44:44.330089+0000 mgr.y (mgr.14152) 368 : cluster [DBG] pgmap v358: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 204 B/s wr, 0 op/s 2024-03-26T03:44:46.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:46 smithi175 bash[21527]: audit 2024-03-26T03:44:45.079375+0000 mon.a (mon.0) 740 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:46.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:46 smithi175 bash[21527]: audit 2024-03-26T03:44:45.090531+0000 mon.a (mon.0) 741 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:46.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:46 smithi175 bash[21527]: audit 2024-03-26T03:44:45.138999+0000 mon.a (mon.0) 742 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:46.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:46 smithi175 bash[21527]: cephadm 2024-03-26T03:44:45.140800+0000 mgr.y (mgr.14152) 369 : cephadm [INF] Checking pool "datapool" exists for service iscsi.datapool 2024-03-26T03:44:46.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:46 smithi175 bash[21527]: audit 2024-03-26T03:44:45.157244+0000 mon.a (mon.0) 743 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:46.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:46 smithi175 bash[21527]: audit 2024-03-26T03:44:45.222489+0000 mon.a (mon.0) 744 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:44:46.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:46 smithi175 bash[29284]: cluster 2024-03-26T03:44:44.330089+0000 mgr.y (mgr.14152) 368 : cluster [DBG] pgmap v358: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 204 B/s wr, 0 op/s 2024-03-26T03:44:46.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:46 smithi175 bash[29284]: audit 2024-03-26T03:44:45.079375+0000 mon.a (mon.0) 740 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:46.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:46 smithi175 bash[29284]: audit 2024-03-26T03:44:45.090531+0000 mon.a (mon.0) 741 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:46.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:46 smithi175 bash[29284]: audit 2024-03-26T03:44:45.138999+0000 mon.a (mon.0) 742 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:46.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:46 smithi175 bash[29284]: cephadm 2024-03-26T03:44:45.140800+0000 mgr.y (mgr.14152) 369 : cephadm [INF] Checking pool "datapool" exists for service iscsi.datapool 2024-03-26T03:44:46.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:46 smithi175 bash[29284]: audit 2024-03-26T03:44:45.157244+0000 mon.a (mon.0) 743 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:46.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:46 smithi175 bash[29284]: audit 2024-03-26T03:44:45.222489+0000 mon.a (mon.0) 744 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:44:46.539 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T03:44:46.992 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:44:46 smithi190 bash[47836]: debug Processing osd blocklist entries for this node 2024-03-26T03:44:47.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:47 smithi175 bash[21527]: cluster 2024-03-26T03:44:46.176986+0000 mon.a (mon.0) 745 : cluster [DBG] mgrmap e16: y(active, since 9m), standbys: x 2024-03-26T03:44:47.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:47 smithi175 bash[21527]: audit 2024-03-26T03:44:46.911946+0000 mon.a (mon.0) 746 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:47.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:47 smithi175 bash[21527]: audit 2024-03-26T03:44:47.075254+0000 mon.b (mon.1) 28 : audit [DBG] from='client.? 172.21.15.190:0/3617495162' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T03:44:47.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:47 smithi175 bash[29284]: cluster 2024-03-26T03:44:46.176986+0000 mon.a (mon.0) 745 : cluster [DBG] mgrmap e16: y(active, since 9m), standbys: x 2024-03-26T03:44:47.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:47 smithi175 bash[29284]: audit 2024-03-26T03:44:46.911946+0000 mon.a (mon.0) 746 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:47.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:47 smithi175 bash[29284]: audit 2024-03-26T03:44:47.075254+0000 mon.b (mon.1) 28 : audit [DBG] from='client.? 172.21.15.190:0/3617495162' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T03:44:47.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:47 smithi190 bash[23392]: cluster 2024-03-26T03:44:46.176986+0000 mon.a (mon.0) 745 : cluster [DBG] mgrmap e16: y(active, since 9m), standbys: x 2024-03-26T03:44:47.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:47 smithi190 bash[23392]: audit 2024-03-26T03:44:46.911946+0000 mon.a (mon.0) 746 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:47.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:47 smithi190 bash[23392]: audit 2024-03-26T03:44:47.075254+0000 mon.b (mon.1) 28 : audit [DBG] from='client.? 172.21.15.190:0/3617495162' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T03:44:47.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:44:47 smithi190 bash[47836]: debug Reading the configuration object to update local LIO configuration 2024-03-26T03:44:47.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:44:47 smithi190 bash[47836]: debug Configuration does not have an entry for this host(smithi190.front.sepia.ceph.com) - nothing to define to LIO 2024-03-26T03:44:47.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:44:47 smithi190 bash[47836]: debug * Running on http://[::]:5000/ (Press CTRL+C to quit) 2024-03-26T03:44:48.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:48 smithi175 bash[21527]: cluster 2024-03-26T03:44:46.331309+0000 mgr.y (mgr.14152) 370 : cluster [DBG] pgmap v359: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:48.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:48 smithi175 bash[29284]: cluster 2024-03-26T03:44:46.331309+0000 mgr.y (mgr.14152) 370 : cluster [DBG] pgmap v359: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:48.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:48 smithi190 bash[23392]: cluster 2024-03-26T03:44:46.331309+0000 mgr.y (mgr.14152) 370 : cluster [DBG] pgmap v359: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:44:49.888 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2f6d58700 1 -- 172.21.15.190:0/1578079549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2f01002c0 msgr2=0x7fb2f01006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:44:49.889 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2f6d58700 1 --2- 172.21.15.190:0/1578079549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2f01002c0 0x7fb2f01006e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000b0a0 tx=0x7fb2e000b3b0 comp rx=0 tx=0).stop 2024-03-26T03:44:49.889 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2f6d58700 1 -- 172.21.15.190:0/1578079549 shutdown_connections 2024-03-26T03:44:49.889 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2f6d58700 1 --2- 172.21.15.190:0/1578079549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2f01083d0 0x7fb2f010a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:49.889 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2f6d58700 1 --2- 172.21.15.190:0/1578079549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2f01002c0 0x7fb2f01006e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:49.889 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2f6d58700 1 --2- 172.21.15.190:0/1578079549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2f0100c20 0x7fb2f0107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:49.889 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2f6d58700 1 -- 172.21.15.190:0/1578079549 >> 172.21.15.190:0/1578079549 conn(0x7fb2f00fbea0 msgr2=0x7fb2f00fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:44:49.889 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2f6d58700 1 -- 172.21.15.190:0/1578079549 shutdown_connections 2024-03-26T03:44:49.889 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2f6d58700 1 -- 172.21.15.190:0/1578079549 wait complete. 2024-03-26T03:44:49.889 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2f6d58700 1 Processor -- start 2024-03-26T03:44:49.889 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2f6d58700 1 -- start start 2024-03-26T03:44:49.889 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2f6d58700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2f01002c0 0x7fb2f0197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:44:49.889 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2f6d58700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2f0100c20 0x7fb2f0198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:44:49.891 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2f6d58700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2f01083d0 0x7fb2f019f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:44:49.891 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2f6d58700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f010d780 con 0x7fb2f0100c20 2024-03-26T03:44:49.891 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2f6d58700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f010d600 con 0x7fb2f01083d0 2024-03-26T03:44:49.891 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2f6d58700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb2f010d900 con 0x7fb2f01002c0 2024-03-26T03:44:49.891 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2f52f5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2f01083d0 0x7fb2f019f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:44:49.891 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2f52f5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2f01083d0 0x7fb2f019f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:35280/0 (socket says 172.21.15.190:35280) 2024-03-26T03:44:49.891 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2f52f5700 1 -- 172.21.15.190:0/1058845205 learned_addr learned my addr 172.21.15.190:0/1058845205 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-03-26T03:44:49.891 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2effff700 1 --2- 172.21.15.190:0/1058845205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2f0100c20 0x7fb2f0198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:44:49.891 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2f4af4700 1 --2- 172.21.15.190:0/1058845205 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2f01002c0 0x7fb2f0197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:44:49.891 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2f52f5700 1 -- 172.21.15.190:0/1058845205 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2f01002c0 msgr2=0x7fb2f0197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:44:49.891 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2f52f5700 1 --2- 172.21.15.190:0/1058845205 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2f01002c0 0x7fb2f0197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:49.892 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2f52f5700 1 -- 172.21.15.190:0/1058845205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2f0100c20 msgr2=0x7fb2f0198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:44:49.892 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2f52f5700 1 --2- 172.21.15.190:0/1058845205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2f0100c20 0x7fb2f0198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:49.892 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2f52f5700 1 -- 172.21.15.190:0/1058845205 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f019fa20 con 0x7fb2f01083d0 2024-03-26T03:44:49.892 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2f52f5700 1 --2- 172.21.15.190:0/1058845205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2f01083d0 0x7fb2f019f380 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb2e800c900 tx=0x7fb2e800ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:44:49.892 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2edffb700 1 -- 172.21.15.190:0/1058845205 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8004230 con 0x7fb2f01083d0 2024-03-26T03:44:49.892 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2edffb700 1 -- 172.21.15.190:0/1058845205 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fb2e8007d80 con 0x7fb2f01083d0 2024-03-26T03:44:49.892 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2f6d58700 1 -- 172.21.15.190:0/1058845205 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2f019fd10 con 0x7fb2f01083d0 2024-03-26T03:44:49.892 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2edffb700 1 -- 172.21.15.190:0/1058845205 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8003c90 con 0x7fb2f01083d0 2024-03-26T03:44:49.892 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.893+0000 7fb2f6d58700 1 -- 172.21.15.190:0/1058845205 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2f01a01a0 con 0x7fb2f01083d0 2024-03-26T03:44:49.892 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.897+0000 7fb2f6d58700 1 -- 172.21.15.190:0/1058845205 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2f0050300 con 0x7fb2f01083d0 2024-03-26T03:44:49.893 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.897+0000 7fb2edffb700 1 -- 172.21.15.190:0/1058845205 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) ==== 92526+0+0 (secure 0 0 0) 0x7fb2e8007900 con 0x7fb2f01083d0 2024-03-26T03:44:49.893 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.897+0000 7fb2edffb700 1 --2- 172.21.15.190:0/1058845205 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fb2d806e780 0x7fb2d8070c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:44:49.893 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.897+0000 7fb2edffb700 1 -- 172.21.15.190:0/1058845205 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7148+0+0 (secure 0 0 0) 0x7fb2e808ea40 con 0x7fb2f01083d0 2024-03-26T03:44:49.893 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.897+0000 7fb2f4af4700 1 --2- 172.21.15.190:0/1058845205 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fb2d806e780 0x7fb2d8070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:44:49.893 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.897+0000 7fb2f4af4700 1 --2- 172.21.15.190:0/1058845205 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fb2d806e780 0x7fb2d8070c40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb2e00062a0 tx=0x7fb2e0006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:44:49.896 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:49.901+0000 7fb2edffb700 1 -- 172.21.15.190:0/1058845205 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7fb2e8014070 con 0x7fb2f01083d0 2024-03-26T03:44:50.121 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:50.125+0000 7fb2f6d58700 1 -- 172.21.15.190:0/1058845205 --> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi190=a", "target": ["mon-mgr", ""]}) -- 0x7fb2f0062a80 con 0x7fb2d806e780 2024-03-26T03:44:50.135 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:50.141+0000 7fb2edffb700 1 -- 172.21.15.190:0/1058845205 <== mgr.14152 v2:172.21.15.175:6800/2826039805 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+31 (secure 0 0 0) 0x7fb2f0062a80 con 0x7fb2d806e780 2024-03-26T03:44:50.136 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled prometheus update... 2024-03-26T03:44:50.138 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:50.141+0000 7fb2f6d58700 1 -- 172.21.15.190:0/1058845205 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fb2d806e780 msgr2=0x7fb2d8070c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:44:50.138 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:50.141+0000 7fb2f6d58700 1 --2- 172.21.15.190:0/1058845205 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fb2d806e780 0x7fb2d8070c40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb2e00062a0 tx=0x7fb2e0006210 comp rx=0 tx=0).stop 2024-03-26T03:44:50.138 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:50.141+0000 7fb2f6d58700 1 -- 172.21.15.190:0/1058845205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2f01083d0 msgr2=0x7fb2f019f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:44:50.138 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:50.141+0000 7fb2f6d58700 1 --2- 172.21.15.190:0/1058845205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2f01083d0 0x7fb2f019f380 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb2e800c900 tx=0x7fb2e800ccc0 comp rx=0 tx=0).stop 2024-03-26T03:44:50.138 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:50.145+0000 7fb2f6d58700 1 -- 172.21.15.190:0/1058845205 shutdown_connections 2024-03-26T03:44:50.138 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:50.145+0000 7fb2f6d58700 1 --2- 172.21.15.190:0/1058845205 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fb2d806e780 0x7fb2d8070c40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:50.139 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:50.145+0000 7fb2f6d58700 1 --2- 172.21.15.190:0/1058845205 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2f01002c0 0x7fb2f0197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:50.139 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:50.145+0000 7fb2f6d58700 1 --2- 172.21.15.190:0/1058845205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2f01083d0 0x7fb2f019f380 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:50.139 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:50.145+0000 7fb2f6d58700 1 --2- 172.21.15.190:0/1058845205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2f0100c20 0x7fb2f0198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:44:50.139 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:50.145+0000 7fb2f6d58700 1 -- 172.21.15.190:0/1058845205 >> 172.21.15.190:0/1058845205 conn(0x7fb2f00fbea0 msgr2=0x7fb2f0103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:44:50.139 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:50.145+0000 7fb2f6d58700 1 -- 172.21.15.190:0/1058845205 shutdown_connections 2024-03-26T03:44:50.139 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:44:50.145+0000 7fb2f6d58700 1 -- 172.21.15.190:0/1058845205 wait complete. 2024-03-26T03:44:50.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:50 smithi175 bash[21527]: cluster 2024-03-26T03:44:48.332395+0000 mgr.y (mgr.14152) 371 : cluster [DBG] pgmap v360: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 102 B/s rd, 102 B/s wr, 0 op/s 2024-03-26T03:44:50.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:50 smithi175 bash[21527]: audit 2024-03-26T03:44:50.138410+0000 mon.a (mon.0) 747 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:50.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:50 smithi175 bash[29284]: cluster 2024-03-26T03:44:48.332395+0000 mgr.y (mgr.14152) 371 : cluster [DBG] pgmap v360: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 102 B/s rd, 102 B/s wr, 0 op/s 2024-03-26T03:44:50.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:50 smithi175 bash[29284]: audit 2024-03-26T03:44:50.138410+0000 mon.a (mon.0) 747 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:50.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:50 smithi190 bash[23392]: cluster 2024-03-26T03:44:48.332395+0000 mgr.y (mgr.14152) 371 : cluster [DBG] pgmap v360: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 102 B/s rd, 102 B/s wr, 0 op/s 2024-03-26T03:44:50.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:50 smithi190 bash[23392]: audit 2024-03-26T03:44:50.138410+0000 mon.a (mon.0) 747 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:50.914 DEBUG:teuthology.orchestra.run.smithi190:prometheus.a> sudo journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@prometheus.a.service 2024-03-26T03:44:50.917 INFO:tasks.cephadm:Adding node-exporter.a on smithi175 2024-03-26T03:44:50.917 INFO:tasks.cephadm:Adding node-exporter.b on smithi190 2024-03-26T03:44:50.917 DEBUG:teuthology.orchestra.run.smithi190:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph orch apply node-exporter '2;smithi175=a;smithi190=b' 2024-03-26T03:44:51.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:51 smithi175 bash[21527]: audit 2024-03-26T03:44:50.127226+0000 mgr.y (mgr.14152) 372 : audit [DBG] from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi190=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:44:51.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:51 smithi175 bash[21527]: cephadm 2024-03-26T03:44:50.130893+0000 mgr.y (mgr.14152) 373 : cephadm [INF] Saving service prometheus spec with placement smithi190=a;count:1 2024-03-26T03:44:51.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:51 smithi175 bash[29284]: audit 2024-03-26T03:44:50.127226+0000 mgr.y (mgr.14152) 372 : audit [DBG] from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi190=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:44:51.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:51 smithi175 bash[29284]: cephadm 2024-03-26T03:44:50.130893+0000 mgr.y (mgr.14152) 373 : cephadm [INF] Saving service prometheus spec with placement smithi190=a;count:1 2024-03-26T03:44:51.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:51 smithi190 bash[23392]: audit 2024-03-26T03:44:50.127226+0000 mgr.y (mgr.14152) 372 : audit [DBG] from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi190=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:44:51.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:51 smithi190 bash[23392]: cephadm 2024-03-26T03:44:50.130893+0000 mgr.y (mgr.14152) 373 : cephadm [INF] Saving service prometheus spec with placement smithi190=a;count:1 2024-03-26T03:44:52.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:52 smithi175 bash[29284]: cluster 2024-03-26T03:44:50.333426+0000 mgr.y (mgr.14152) 374 : cluster [DBG] pgmap v361: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 102 B/s rd, 102 B/s wr, 0 op/s 2024-03-26T03:44:52.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:52 smithi175 bash[21527]: cluster 2024-03-26T03:44:50.333426+0000 mgr.y (mgr.14152) 374 : cluster [DBG] pgmap v361: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 102 B/s rd, 102 B/s wr, 0 op/s 2024-03-26T03:44:52.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:52 smithi190 bash[23392]: cluster 2024-03-26T03:44:50.333426+0000 mgr.y (mgr.14152) 374 : cluster [DBG] pgmap v361: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 102 B/s rd, 102 B/s wr, 0 op/s 2024-03-26T03:44:53.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:53 smithi190 bash[23392]: audit 2024-03-26T03:44:52.254906+0000 mon.a (mon.0) 748 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2024-03-26T03:44:53.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:53 smithi190 bash[23392]: audit 2024-03-26T03:44:52.255631+0000 mon.a (mon.0) 749 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-03-26T03:44:53.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:53 smithi190 bash[23392]: audit 2024-03-26T03:44:52.260347+0000 mon.a (mon.0) 750 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-03-26T03:44:53.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:53 smithi190 bash[23392]: audit 2024-03-26T03:44:52.260955+0000 mon.a (mon.0) 751 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-03-26T03:44:53.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:53 smithi190 bash[23392]: audit 2024-03-26T03:44:52.977879+0000 mon.a (mon.0) 752 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:53.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:53 smithi190 bash[23392]: audit 2024-03-26T03:44:52.991205+0000 mon.a (mon.0) 753 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:53.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:53 smithi190 bash[23392]: audit 2024-03-26T03:44:52.993864+0000 mon.a (mon.0) 754 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:44:53.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:53 smithi190 bash[23392]: audit 2024-03-26T03:44:52.995849+0000 mon.a (mon.0) 755 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:44:53.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:53 smithi190 bash[23392]: audit 2024-03-26T03:44:53.008157+0000 mon.a (mon.0) 756 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:53.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:53 smithi175 bash[21527]: audit 2024-03-26T03:44:52.254906+0000 mon.a (mon.0) 748 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2024-03-26T03:44:53.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:53 smithi175 bash[21527]: audit 2024-03-26T03:44:52.255631+0000 mon.a (mon.0) 749 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-03-26T03:44:53.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:53 smithi175 bash[21527]: audit 2024-03-26T03:44:52.260347+0000 mon.a (mon.0) 750 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-03-26T03:44:53.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:53 smithi175 bash[21527]: audit 2024-03-26T03:44:52.260955+0000 mon.a (mon.0) 751 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-03-26T03:44:53.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:53 smithi175 bash[21527]: audit 2024-03-26T03:44:52.977879+0000 mon.a (mon.0) 752 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:53.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:53 smithi175 bash[21527]: audit 2024-03-26T03:44:52.991205+0000 mon.a (mon.0) 753 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:53.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:53 smithi175 bash[21527]: audit 2024-03-26T03:44:52.993864+0000 mon.a (mon.0) 754 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:44:53.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:53 smithi175 bash[21527]: audit 2024-03-26T03:44:52.995849+0000 mon.a (mon.0) 755 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:44:53.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:53 smithi175 bash[21527]: audit 2024-03-26T03:44:53.008157+0000 mon.a (mon.0) 756 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:53.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:53 smithi175 bash[29284]: audit 2024-03-26T03:44:52.254906+0000 mon.a (mon.0) 748 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]: dispatch 2024-03-26T03:44:53.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:53 smithi175 bash[29284]: audit 2024-03-26T03:44:52.255631+0000 mon.a (mon.0) 749 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-03-26T03:44:53.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:53 smithi175 bash[29284]: audit 2024-03-26T03:44:52.260347+0000 mon.a (mon.0) 750 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-03-26T03:44:53.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:53 smithi175 bash[29284]: audit 2024-03-26T03:44:52.260955+0000 mon.a (mon.0) 751 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-03-26T03:44:53.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:53 smithi175 bash[29284]: audit 2024-03-26T03:44:52.977879+0000 mon.a (mon.0) 752 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:53.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:53 smithi175 bash[29284]: audit 2024-03-26T03:44:52.991205+0000 mon.a (mon.0) 753 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:53.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:53 smithi175 bash[29284]: audit 2024-03-26T03:44:52.993864+0000 mon.a (mon.0) 754 : audit [DBG] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:44:53.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:53 smithi175 bash[29284]: audit 2024-03-26T03:44:52.995849+0000 mon.a (mon.0) 755 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:44:53.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:53 smithi175 bash[29284]: audit 2024-03-26T03:44:53.008157+0000 mon.a (mon.0) 756 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:44:53.992 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:44:53 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:44:54.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:54 smithi175 bash[21527]: cluster 2024-03-26T03:44:52.334872+0000 mgr.y (mgr.14152) 375 : cluster [DBG] pgmap v362: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 556 B/s rd, 92 B/s wr, 1 op/s 2024-03-26T03:44:54.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:54 smithi175 bash[21527]: audit 2024-03-26T03:44:53.202676+0000 mon.a (mon.0) 757 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]': finished 2024-03-26T03:44:54.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:54 smithi175 bash[21527]: audit 2024-03-26T03:44:53.202859+0000 mon.a (mon.0) 758 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-03-26T03:44:54.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:54 smithi175 bash[21527]: audit 2024-03-26T03:44:53.203054+0000 mon.a (mon.0) 759 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]': finished 2024-03-26T03:44:54.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:54 smithi175 bash[21527]: audit 2024-03-26T03:44:53.203287+0000 mon.a (mon.0) 760 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2024-03-26T03:44:54.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:54 smithi175 bash[21527]: cluster 2024-03-26T03:44:53.213026+0000 mon.a (mon.0) 761 : cluster [DBG] osdmap e73: 8 total, 8 up, 8 in 2024-03-26T03:44:54.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:54 smithi175 bash[29284]: cluster 2024-03-26T03:44:52.334872+0000 mgr.y (mgr.14152) 375 : cluster [DBG] pgmap v362: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 556 B/s rd, 92 B/s wr, 1 op/s 2024-03-26T03:44:54.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:54 smithi175 bash[29284]: audit 2024-03-26T03:44:53.202676+0000 mon.a (mon.0) 757 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]': finished 2024-03-26T03:44:54.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:54 smithi175 bash[29284]: audit 2024-03-26T03:44:53.202859+0000 mon.a (mon.0) 758 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-03-26T03:44:54.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:54 smithi175 bash[29284]: audit 2024-03-26T03:44:53.203054+0000 mon.a (mon.0) 759 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]': finished 2024-03-26T03:44:54.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:54 smithi175 bash[29284]: audit 2024-03-26T03:44:53.203287+0000 mon.a (mon.0) 760 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2024-03-26T03:44:54.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:54 smithi175 bash[29284]: cluster 2024-03-26T03:44:53.213026+0000 mon.a (mon.0) 761 : cluster [DBG] osdmap e73: 8 total, 8 up, 8 in 2024-03-26T03:44:54.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:54 smithi190 bash[23392]: cluster 2024-03-26T03:44:52.334872+0000 mgr.y (mgr.14152) 375 : cluster [DBG] pgmap v362: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 556 B/s rd, 92 B/s wr, 1 op/s 2024-03-26T03:44:54.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:54 smithi190 bash[23392]: audit 2024-03-26T03:44:53.202676+0000 mon.a (mon.0) 757 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [7, 2]}]': finished 2024-03-26T03:44:54.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:54 smithi190 bash[23392]: audit 2024-03-26T03:44:53.202859+0000 mon.a (mon.0) 758 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-03-26T03:44:54.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:54 smithi190 bash[23392]: audit 2024-03-26T03:44:53.203054+0000 mon.a (mon.0) 759 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]': finished 2024-03-26T03:44:54.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:54 smithi190 bash[23392]: audit 2024-03-26T03:44:53.203287+0000 mon.a (mon.0) 760 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2024-03-26T03:44:54.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:54 smithi190 bash[23392]: cluster 2024-03-26T03:44:53.213026+0000 mon.a (mon.0) 761 : cluster [DBG] osdmap e73: 8 total, 8 up, 8 in 2024-03-26T03:44:55.444 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T03:44:55.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:55 smithi190 bash[23392]: cephadm 2024-03-26T03:44:53.308241+0000 mgr.y (mgr.14152) 376 : cephadm [INF] Deploying daemon prometheus.a on smithi190 2024-03-26T03:44:55.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:55 smithi190 bash[23392]: cluster 2024-03-26T03:44:54.219520+0000 mon.a (mon.0) 762 : cluster [DBG] osdmap e74: 8 total, 8 up, 8 in 2024-03-26T03:44:55.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:55 smithi190 bash[23392]: cluster 2024-03-26T03:44:54.335799+0000 mgr.y (mgr.14152) 377 : cluster [DBG] pgmap v365: 132 pgs: 2 peering, 130 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-03-26T03:44:55.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:55 smithi175 bash[21527]: cephadm 2024-03-26T03:44:53.308241+0000 mgr.y (mgr.14152) 376 : cephadm [INF] Deploying daemon prometheus.a on smithi190 2024-03-26T03:44:55.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:55 smithi175 bash[21527]: cluster 2024-03-26T03:44:54.219520+0000 mon.a (mon.0) 762 : cluster [DBG] osdmap e74: 8 total, 8 up, 8 in 2024-03-26T03:44:55.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:55 smithi175 bash[21527]: cluster 2024-03-26T03:44:54.335799+0000 mgr.y (mgr.14152) 377 : cluster [DBG] pgmap v365: 132 pgs: 2 peering, 130 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-03-26T03:44:55.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:55 smithi175 bash[29284]: cephadm 2024-03-26T03:44:53.308241+0000 mgr.y (mgr.14152) 376 : cephadm [INF] Deploying daemon prometheus.a on smithi190 2024-03-26T03:44:55.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:55 smithi175 bash[29284]: cluster 2024-03-26T03:44:54.219520+0000 mon.a (mon.0) 762 : cluster [DBG] osdmap e74: 8 total, 8 up, 8 in 2024-03-26T03:44:55.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:55 smithi175 bash[29284]: cluster 2024-03-26T03:44:54.335799+0000 mgr.y (mgr.14152) 377 : cluster [DBG] pgmap v365: 132 pgs: 2 peering, 130 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-03-26T03:44:57.242 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:44:56 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:44:57.681 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:57 smithi190 bash[23392]: cluster 2024-03-26T03:44:56.336913+0000 mgr.y (mgr.14152) 378 : cluster [DBG] pgmap v366: 132 pgs: 2 peering, 130 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-26T03:44:57.681 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:57 smithi190 bash[23392]: audit 2024-03-26T03:44:56.757459+0000 mgr.y (mgr.14152) 379 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:44:57.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:57 smithi175 bash[21527]: cluster 2024-03-26T03:44:56.336913+0000 mgr.y (mgr.14152) 378 : cluster [DBG] pgmap v366: 132 pgs: 2 peering, 130 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-26T03:44:57.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:57 smithi175 bash[21527]: audit 2024-03-26T03:44:56.757459+0000 mgr.y (mgr.14152) 379 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:44:57.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:57 smithi175 bash[29284]: cluster 2024-03-26T03:44:56.336913+0000 mgr.y (mgr.14152) 378 : cluster [DBG] pgmap v366: 132 pgs: 2 peering, 130 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-26T03:44:57.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:57 smithi175 bash[29284]: audit 2024-03-26T03:44:56.757459+0000 mgr.y (mgr.14152) 379 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:44:59.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:44:59 smithi175 bash[21527]: cluster 2024-03-26T03:44:58.338005+0000 mgr.y (mgr.14152) 380 : cluster [DBG] pgmap v367: 132 pgs: 2 peering, 130 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:44:59.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:44:59 smithi175 bash[29284]: cluster 2024-03-26T03:44:58.338005+0000 mgr.y (mgr.14152) 380 : cluster [DBG] pgmap v367: 132 pgs: 2 peering, 130 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:44:59.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:44:59 smithi190 bash[23392]: cluster 2024-03-26T03:44:58.338005+0000 mgr.y (mgr.14152) 380 : cluster [DBG] pgmap v367: 132 pgs: 2 peering, 130 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:45:01.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:01 smithi175 bash[21527]: cluster 2024-03-26T03:45:00.338960+0000 mgr.y (mgr.14152) 381 : cluster [DBG] pgmap v368: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-26T03:45:01.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:01 smithi175 bash[29284]: cluster 2024-03-26T03:45:00.338960+0000 mgr.y (mgr.14152) 381 : cluster [DBG] pgmap v368: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-26T03:45:01.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:01 smithi190 bash[23392]: cluster 2024-03-26T03:45:00.338960+0000 mgr.y (mgr.14152) 381 : cluster [DBG] pgmap v368: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-26T03:45:02.026 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.029+0000 7fdec492f700 1 -- 172.21.15.190:0/2735870999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdec01038d0 msgr2=0x7fdec0103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:45:02.026 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.029+0000 7fdec492f700 1 --2- 172.21.15.190:0/2735870999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdec01038d0 0x7fdec0103d30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdeb00099c0 tx=0x7fdeb0009cd0 comp rx=0 tx=0).stop 2024-03-26T03:45:02.026 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.029+0000 7fdec492f700 1 -- 172.21.15.190:0/2735870999 shutdown_connections 2024-03-26T03:45:02.026 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.029+0000 7fdec492f700 1 --2- 172.21.15.190:0/2735870999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdec01026d0 0x7fdec0102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:02.026 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.029+0000 7fdec492f700 1 --2- 172.21.15.190:0/2735870999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdec0104270 0x7fdec010ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:02.026 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.029+0000 7fdec492f700 1 --2- 172.21.15.190:0/2735870999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdec01038d0 0x7fdec0103d30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:02.026 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.029+0000 7fdec492f700 1 -- 172.21.15.190:0/2735870999 >> 172.21.15.190:0/2735870999 conn(0x7fdec00fdc70 msgr2=0x7fdec01000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:45:02.027 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.029+0000 7fdec492f700 1 -- 172.21.15.190:0/2735870999 shutdown_connections 2024-03-26T03:45:02.027 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.029+0000 7fdec492f700 1 -- 172.21.15.190:0/2735870999 wait complete. 2024-03-26T03:45:02.027 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.029+0000 7fdec492f700 1 Processor -- start 2024-03-26T03:45:02.027 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.029+0000 7fdec492f700 1 -- start start 2024-03-26T03:45:02.027 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.029+0000 7fdec492f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdec01026d0 0x7fdec019a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:45:02.027 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.029+0000 7fdec492f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdec01038d0 0x7fdec019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:45:02.027 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.029+0000 7fdec492f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdec0104270 0x7fdec01a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:45:02.027 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.029+0000 7fdec492f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdec010d7a0 con 0x7fdec0104270 2024-03-26T03:45:02.027 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.029+0000 7fdec492f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdec010d620 con 0x7fdec01026d0 2024-03-26T03:45:02.027 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.029+0000 7fdec492f700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdec010d920 con 0x7fdec01038d0 2024-03-26T03:45:02.027 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.029+0000 7fdebe59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdec01026d0 0x7fdec019a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:45:02.027 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.029+0000 7fdebe59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdec01026d0 0x7fdec019a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:33204/0 (socket says 172.21.15.190:33204) 2024-03-26T03:45:02.027 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.029+0000 7fdebe59c700 1 -- 172.21.15.190:0/2897734109 learned_addr learned my addr 172.21.15.190:0/2897734109 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-03-26T03:45:02.027 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.029+0000 7fdebed9d700 1 --2- 172.21.15.190:0/2897734109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdec0104270 0x7fdec01a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:45:02.028 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.029+0000 7fdeb7fff700 1 --2- 172.21.15.190:0/2897734109 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdec01038d0 0x7fdec019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:45:02.028 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.029+0000 7fdebe59c700 1 -- 172.21.15.190:0/2897734109 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdec01038d0 msgr2=0x7fdec019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:45:02.028 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.029+0000 7fdebe59c700 1 --2- 172.21.15.190:0/2897734109 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdec01038d0 0x7fdec019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:02.028 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.029+0000 7fdebe59c700 1 -- 172.21.15.190:0/2897734109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdec0104270 msgr2=0x7fdec01a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:45:02.028 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.029+0000 7fdebe59c700 1 --2- 172.21.15.190:0/2897734109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdec0104270 0x7fdec01a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:02.028 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.029+0000 7fdebe59c700 1 -- 172.21.15.190:0/2897734109 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec01a1d20 con 0x7fdec01026d0 2024-03-26T03:45:02.028 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.029+0000 7fdebe59c700 1 --2- 172.21.15.190:0/2897734109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdec01026d0 0x7fdec019a110 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdea800b500 tx=0x7fdea800b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:45:02.028 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.029+0000 7fdeb77fe700 1 -- 172.21.15.190:0/2897734109 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea8015020 con 0x7fdec01026d0 2024-03-26T03:45:02.028 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.029+0000 7fdeb77fe700 1 -- 172.21.15.190:0/2897734109 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fdea8011cb0 con 0x7fdec01026d0 2024-03-26T03:45:02.028 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.033+0000 7fdec492f700 1 -- 172.21.15.190:0/2897734109 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdec01a1fb0 con 0x7fdec01026d0 2024-03-26T03:45:02.028 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.033+0000 7fdeb77fe700 1 -- 172.21.15.190:0/2897734109 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea8010c60 con 0x7fdec01026d0 2024-03-26T03:45:02.029 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.033+0000 7fdec492f700 1 -- 172.21.15.190:0/2897734109 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdec01a24e0 con 0x7fdec01026d0 2024-03-26T03:45:02.029 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.033+0000 7fdec492f700 1 -- 172.21.15.190:0/2897734109 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdec0050300 con 0x7fdec01026d0 2024-03-26T03:45:02.029 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.033+0000 7fdeb77fe700 1 -- 172.21.15.190:0/2897734109 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) ==== 92526+0+0 (secure 0 0 0) 0x7fdea8018020 con 0x7fdec01026d0 2024-03-26T03:45:02.029 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.033+0000 7fdeb77fe700 1 --2- 172.21.15.190:0/2897734109 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fdea006e780 0x7fdea0070c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:45:02.030 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.033+0000 7fdeb77fe700 1 -- 172.21.15.190:0/2897734109 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7236+0+0 (secure 0 0 0) 0x7fdea808e440 con 0x7fdec01026d0 2024-03-26T03:45:02.030 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.033+0000 7fdeb7fff700 1 --2- 172.21.15.190:0/2897734109 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fdea006e780 0x7fdea0070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:45:02.030 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.033+0000 7fdeb7fff700 1 --2- 172.21.15.190:0/2897734109 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fdea006e780 0x7fdea0070c40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdeb00037f0 tx=0x7fdeb0003780 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:45:02.033 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.037+0000 7fdeb77fe700 1 -- 172.21.15.190:0/2897734109 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7fdea801f020 con 0x7fdec01026d0 2024-03-26T03:45:02.255 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.261+0000 7fdec492f700 1 -- 172.21.15.190:0/2897734109 --> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi175=a;smithi190=b", "target": ["mon-mgr", ""]}) -- 0x7fdec0075880 con 0x7fdea006e780 2024-03-26T03:45:02.269 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.273+0000 7fdeb77fe700 1 -- 172.21.15.190:0/2897734109 <== mgr.14152 v2:172.21.15.175:6800/2826039805 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+34 (secure 0 0 0) 0x7fdec0075880 con 0x7fdea006e780 2024-03-26T03:45:02.269 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled node-exporter update... 2024-03-26T03:45:02.271 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.277+0000 7fdec492f700 1 -- 172.21.15.190:0/2897734109 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fdea006e780 msgr2=0x7fdea0070c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:45:02.272 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.277+0000 7fdec492f700 1 --2- 172.21.15.190:0/2897734109 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fdea006e780 0x7fdea0070c40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdeb00037f0 tx=0x7fdeb0003780 comp rx=0 tx=0).stop 2024-03-26T03:45:02.272 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.277+0000 7fdec492f700 1 -- 172.21.15.190:0/2897734109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdec01026d0 msgr2=0x7fdec019a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:45:02.272 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.277+0000 7fdec492f700 1 --2- 172.21.15.190:0/2897734109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdec01026d0 0x7fdec019a110 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdea800b500 tx=0x7fdea800b8c0 comp rx=0 tx=0).stop 2024-03-26T03:45:02.272 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.277+0000 7fdec492f700 1 -- 172.21.15.190:0/2897734109 shutdown_connections 2024-03-26T03:45:02.272 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.277+0000 7fdec492f700 1 --2- 172.21.15.190:0/2897734109 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7fdea006e780 0x7fdea0070c40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:02.272 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.277+0000 7fdec492f700 1 --2- 172.21.15.190:0/2897734109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdec0104270 0x7fdec01a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:02.272 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.277+0000 7fdec492f700 1 --2- 172.21.15.190:0/2897734109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdec01026d0 0x7fdec019a110 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:02.272 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.277+0000 7fdec492f700 1 --2- 172.21.15.190:0/2897734109 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdec01038d0 0x7fdec019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:02.272 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.277+0000 7fdec492f700 1 -- 172.21.15.190:0/2897734109 >> 172.21.15.190:0/2897734109 conn(0x7fdec00fdc70 msgr2=0x7fdec00ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:45:02.272 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.277+0000 7fdec492f700 1 -- 172.21.15.190:0/2897734109 shutdown_connections 2024-03-26T03:45:02.272 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:02.277+0000 7fdec492f700 1 -- 172.21.15.190:0/2897734109 wait complete. 2024-03-26T03:45:03.642 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:03 smithi190 bash[23392]: audit 2024-03-26T03:45:02.260849+0000 mgr.y (mgr.14152) 382 : audit [DBG] from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi175=a;smithi190=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:45:03.643 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:03 smithi190 bash[23392]: cephadm 2024-03-26T03:45:02.264571+0000 mgr.y (mgr.14152) 383 : cephadm [INF] Saving service node-exporter spec with placement smithi175=a;smithi190=b;count:2 2024-03-26T03:45:03.643 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:03 smithi190 bash[23392]: audit 2024-03-26T03:45:02.272015+0000 mon.a (mon.0) 763 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:45:03.643 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:03 smithi190 bash[23392]: cluster 2024-03-26T03:45:02.340279+0000 mgr.y (mgr.14152) 384 : cluster [DBG] pgmap v369: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-26T03:45:03.644 DEBUG:teuthology.orchestra.run.smithi175:node-exporter.a> sudo journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@node-exporter.a.service 2024-03-26T03:45:03.647 DEBUG:teuthology.orchestra.run.smithi190:node-exporter.b> sudo journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@node-exporter.b.service 2024-03-26T03:45:03.649 INFO:tasks.cephadm:Adding alertmanager.a on smithi175 2024-03-26T03:45:03.649 DEBUG:teuthology.orchestra.run.smithi190:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph orch apply alertmanager '1;smithi175=a' 2024-03-26T03:45:03.655 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:03 smithi175 bash[29284]: audit 2024-03-26T03:45:02.260849+0000 mgr.y (mgr.14152) 382 : audit [DBG] from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi175=a;smithi190=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:45:03.656 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:03 smithi175 bash[29284]: cephadm 2024-03-26T03:45:02.264571+0000 mgr.y (mgr.14152) 383 : cephadm [INF] Saving service node-exporter spec with placement smithi175=a;smithi190=b;count:2 2024-03-26T03:45:03.656 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:03 smithi175 bash[29284]: audit 2024-03-26T03:45:02.272015+0000 mon.a (mon.0) 763 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:45:03.656 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:03 smithi175 bash[29284]: cluster 2024-03-26T03:45:02.340279+0000 mgr.y (mgr.14152) 384 : cluster [DBG] pgmap v369: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-26T03:45:03.656 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:03 smithi175 bash[21527]: audit 2024-03-26T03:45:02.260849+0000 mgr.y (mgr.14152) 382 : audit [DBG] from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi175=a;smithi190=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:45:03.656 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:03 smithi175 bash[21527]: cephadm 2024-03-26T03:45:02.264571+0000 mgr.y (mgr.14152) 383 : cephadm [INF] Saving service node-exporter spec with placement smithi175=a;smithi190=b;count:2 2024-03-26T03:45:03.656 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:03 smithi175 bash[21527]: audit 2024-03-26T03:45:02.272015+0000 mon.a (mon.0) 763 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:45:03.656 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:03 smithi175 bash[21527]: cluster 2024-03-26T03:45:02.340279+0000 mgr.y (mgr.14152) 384 : cluster [DBG] pgmap v369: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-26T03:45:05.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:05 smithi175 bash[29284]: cluster 2024-03-26T03:45:04.341269+0000 mgr.y (mgr.14152) 385 : cluster [DBG] pgmap v370: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1011 B/s rd, 0 op/s 2024-03-26T03:45:05.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:05 smithi175 bash[21527]: cluster 2024-03-26T03:45:04.341269+0000 mgr.y (mgr.14152) 385 : cluster [DBG] pgmap v370: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1011 B/s rd, 0 op/s 2024-03-26T03:45:05.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:05 smithi190 bash[23392]: cluster 2024-03-26T03:45:04.341269+0000 mgr.y (mgr.14152) 385 : cluster [DBG] pgmap v370: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1011 B/s rd, 0 op/s 2024-03-26T03:45:06.989 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:45:06 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:45:07.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:07 smithi190 bash[23392]: cluster 2024-03-26T03:45:06.342327+0000 mgr.y (mgr.14152) 386 : cluster [DBG] pgmap v371: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:45:07.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:07 smithi190 bash[23392]: audit 2024-03-26T03:45:06.766675+0000 mgr.y (mgr.14152) 387 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:45:07.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:07 smithi190 bash[23392]: audit 2024-03-26T03:45:06.933181+0000 mon.a (mon.0) 764 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:45:08.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:07 smithi175 bash[21527]: cluster 2024-03-26T03:45:06.342327+0000 mgr.y (mgr.14152) 386 : cluster [DBG] pgmap v371: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:45:08.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:07 smithi175 bash[21527]: audit 2024-03-26T03:45:06.766675+0000 mgr.y (mgr.14152) 387 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:45:08.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:07 smithi175 bash[21527]: audit 2024-03-26T03:45:06.933181+0000 mon.a (mon.0) 764 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:45:08.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:07 smithi175 bash[29284]: cluster 2024-03-26T03:45:06.342327+0000 mgr.y (mgr.14152) 386 : cluster [DBG] pgmap v371: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:45:08.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:07 smithi175 bash[29284]: audit 2024-03-26T03:45:06.766675+0000 mgr.y (mgr.14152) 387 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:45:08.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:07 smithi175 bash[29284]: audit 2024-03-26T03:45:06.933181+0000 mon.a (mon.0) 764 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:45:08.920 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:45:08 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:08.920 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:08 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:08.921 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 03:45:08 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:08.921 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 03:45:08 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:08.921 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 03:45:08 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:08.922 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 03:45:08 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:08.922 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:08 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:09.006 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T03:45:09.173 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:09 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:09.174 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 03:45:09 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:09.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:45:09 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:09.174 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:09 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:09.174 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 03:45:09 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:09.175 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:45:08 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:09.175 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:45:08 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:09.175 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:45:08 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:09.175 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:45:09 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:09.175 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:45:09 smithi190 systemd[1]: Started Ceph prometheus.a for 762e930c-eb21-11ee-95d0-87774f69a715. 2024-03-26T03:45:09.175 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 03:45:09 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:09.176 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 03:45:09 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:10.491 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:10 smithi175 bash[21783]: ignoring --setuser ceph since I am not root 2024-03-26T03:45:10.491 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:10 smithi175 bash[21783]: ignoring --setgroup ceph since I am not root 2024-03-26T03:45:10.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:10 smithi175 bash[21527]: cluster 2024-03-26T03:45:08.343501+0000 mgr.y (mgr.14152) 388 : cluster [DBG] pgmap v372: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:45:10.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:10 smithi175 bash[21527]: audit 2024-03-26T03:45:09.224946+0000 mon.a (mon.0) 765 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:45:10.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:10 smithi175 bash[21527]: audit 2024-03-26T03:45:09.235635+0000 mon.a (mon.0) 766 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:45:10.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:10 smithi175 bash[21527]: audit 2024-03-26T03:45:09.277345+0000 mon.a (mon.0) 767 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:45:10.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:10 smithi175 bash[21527]: audit 2024-03-26T03:45:09.286679+0000 mon.a (mon.0) 768 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-26T03:45:10.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:10 smithi175 bash[29284]: cluster 2024-03-26T03:45:08.343501+0000 mgr.y (mgr.14152) 388 : cluster [DBG] pgmap v372: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:45:10.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:10 smithi175 bash[29284]: audit 2024-03-26T03:45:09.224946+0000 mon.a (mon.0) 765 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:45:10.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:10 smithi175 bash[29284]: audit 2024-03-26T03:45:09.235635+0000 mon.a (mon.0) 766 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:45:10.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:10 smithi175 bash[29284]: audit 2024-03-26T03:45:09.277345+0000 mon.a (mon.0) 767 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:45:10.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:10 smithi175 bash[29284]: audit 2024-03-26T03:45:09.286679+0000 mon.a (mon.0) 768 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-26T03:45:10.494 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:10 smithi190 bash[24341]: ignoring --setuser ceph since I am not root 2024-03-26T03:45:10.494 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:10 smithi190 bash[24341]: ignoring --setgroup ceph since I am not root 2024-03-26T03:45:10.494 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:10 smithi190 bash[24341]: debug 2024-03-26T03:45:10.357+0000 7fae97ac5700 1 -- 172.21.15.190:0/1944445387 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558a8b1025a0 con 0x558a8be80400 2024-03-26T03:45:10.495 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:10 smithi190 bash[23392]: cluster 2024-03-26T03:45:08.343501+0000 mgr.y (mgr.14152) 388 : cluster [DBG] pgmap v372: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:45:10.495 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:10 smithi190 bash[23392]: audit 2024-03-26T03:45:09.224946+0000 mon.a (mon.0) 765 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:45:10.495 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:10 smithi190 bash[23392]: audit 2024-03-26T03:45:09.235635+0000 mon.a (mon.0) 766 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:45:10.495 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:10 smithi190 bash[23392]: audit 2024-03-26T03:45:09.277345+0000 mon.a (mon.0) 767 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' 2024-03-26T03:45:10.495 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:10 smithi190 bash[23392]: audit 2024-03-26T03:45:09.286679+0000 mon.a (mon.0) 768 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-26T03:45:10.753 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:10 smithi190 bash[24341]: debug 2024-03-26T03:45:10.505+0000 7faea18b6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T03:45:10.991 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:10 smithi175 bash[21783]: debug 2024-03-26T03:45:10.490+0000 7f6f58acf200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T03:45:11.169 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:45:10 smithi190 bash[48954]: ts=2024-03-26T03:45:10.847Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-03-26T03:45:11.170 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:45:10 smithi190 bash[48954]: ts=2024-03-26T03:45:10.848Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-03-26T03:45:11.170 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:45:10 smithi190 bash[48954]: ts=2024-03-26T03:45:10.848Z caller=main.go:570 level=info host_details="(Linux 5.15.0-101-generic #111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024 x86_64 smithi190 (none))" 2024-03-26T03:45:11.170 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:45:10 smithi190 bash[48954]: ts=2024-03-26T03:45:10.848Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-03-26T03:45:11.170 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:45:10 smithi190 bash[48954]: ts=2024-03-26T03:45:10.848Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-03-26T03:45:11.170 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:45:10 smithi190 bash[48954]: ts=2024-03-26T03:45:10.849Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-03-26T03:45:11.170 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:45:10 smithi190 bash[48954]: ts=2024-03-26T03:45:10.849Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-03-26T03:45:11.170 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:45:10 smithi190 bash[48954]: ts=2024-03-26T03:45:10.851Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-03-26T03:45:11.170 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:45:10 smithi190 bash[48954]: ts=2024-03-26T03:45:10.851Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-03-26T03:45:11.170 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:45:10 smithi190 bash[48954]: ts=2024-03-26T03:45:10.853Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-03-26T03:45:11.170 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:45:10 smithi190 bash[48954]: ts=2024-03-26T03:45:10.853Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.822µs 2024-03-26T03:45:11.170 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:45:10 smithi190 bash[48954]: ts=2024-03-26T03:45:10.853Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-03-26T03:45:11.171 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:45:10 smithi190 bash[48954]: ts=2024-03-26T03:45:10.853Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-03-26T03:45:11.171 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:45:10 smithi190 bash[48954]: ts=2024-03-26T03:45:10.853Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=19.684µs wal_replay_duration=347.117µs wbl_replay_duration=91ns total_replay_duration=386.201µs 2024-03-26T03:45:11.171 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:45:10 smithi190 bash[48954]: ts=2024-03-26T03:45:10.854Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-03-26T03:45:11.171 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:45:10 smithi190 bash[48954]: ts=2024-03-26T03:45:10.854Z caller=main.go:1029 level=info msg="TSDB started" 2024-03-26T03:45:11.171 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:45:10 smithi190 bash[48954]: ts=2024-03-26T03:45:10.854Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-03-26T03:45:11.171 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:45:10 smithi190 bash[48954]: ts=2024-03-26T03:45:10.856Z caller=refresh.go:80 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.175:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.175:8765: connect: connection refused" 2024-03-26T03:45:11.171 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:45:10 smithi190 bash[48954]: ts=2024-03-26T03:45:10.856Z caller=refresh.go:80 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.175:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.175:8765: connect: connection refused" 2024-03-26T03:45:11.171 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:45:10 smithi190 bash[48954]: ts=2024-03-26T03:45:10.856Z caller=refresh.go:80 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.175:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.175:8765: connect: connection refused" 2024-03-26T03:45:11.171 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:45:10 smithi190 bash[48954]: ts=2024-03-26T03:45:10.884Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=30.010479ms db_storage=1.099µs remote_storage=1.357µs web_handler=600ns query_engine=1.05µs scrape=399.668µs scrape_sd=150.514µs notify=1.122µs notify_sd=3.087µs rules=28.982388ms tracing=6.591µs 2024-03-26T03:45:11.171 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:45:10 smithi190 bash[48954]: ts=2024-03-26T03:45:10.885Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-03-26T03:45:11.171 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:45:10 smithi190 bash[48954]: ts=2024-03-26T03:45:10.885Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-03-26T03:45:11.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:11 smithi190 bash[23392]: audit 2024-03-26T03:45:10.278124+0000 mon.a (mon.0) 769 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-26T03:45:11.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:11 smithi190 bash[23392]: cluster 2024-03-26T03:45:10.292207+0000 mon.a (mon.0) 770 : cluster [DBG] mgrmap e17: y(active, since 10m), standbys: x 2024-03-26T03:45:11.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:11 smithi175 bash[21527]: audit 2024-03-26T03:45:10.278124+0000 mon.a (mon.0) 769 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-26T03:45:11.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:11 smithi175 bash[21527]: cluster 2024-03-26T03:45:10.292207+0000 mon.a (mon.0) 770 : cluster [DBG] mgrmap e17: y(active, since 10m), standbys: x 2024-03-26T03:45:11.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:11 smithi175 bash[29284]: audit 2024-03-26T03:45:10.278124+0000 mon.a (mon.0) 769 : audit [INF] from='mgr.14152 172.21.15.175:0/1842356026' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-26T03:45:11.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:11 smithi175 bash[29284]: cluster 2024-03-26T03:45:10.292207+0000 mon.a (mon.0) 770 : cluster [DBG] mgrmap e17: y(active, since 10m), standbys: x 2024-03-26T03:45:12.403 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:12 smithi175 bash[21783]: debug 2024-03-26T03:45:12.054+0000 7f6f58acf200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T03:45:12.403 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:12 smithi175 bash[21783]: debug 2024-03-26T03:45:12.150+0000 7f6f58acf200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T03:45:12.482 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:12 smithi190 bash[24341]: debug 2024-03-26T03:45:12.137+0000 7faea18b6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T03:45:12.483 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:12 smithi190 bash[24341]: debug 2024-03-26T03:45:12.233+0000 7faea18b6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T03:45:12.743 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:12 smithi190 bash[24341]: debug 2024-03-26T03:45:12.484+0000 7faea18b6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T03:45:12.914 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e216a6700 1 -- 172.21.15.190:0/204612778 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e1c1083c0 msgr2=0x7f9e1c10a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:45:12.915 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e216a6700 1 --2- 172.21.15.190:0/204612778 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e1c1083c0 0x7f9e1c10a830 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9e1400b0a0 tx=0x7f9e1400b3b0 comp rx=0 tx=0).stop 2024-03-26T03:45:12.915 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e216a6700 1 -- 172.21.15.190:0/204612778 shutdown_connections 2024-03-26T03:45:12.915 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e216a6700 1 --2- 172.21.15.190:0/204612778 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e1c103160 0x7f9e1c105550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:12.915 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e216a6700 1 --2- 172.21.15.190:0/204612778 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e1c1083c0 0x7f9e1c10a830 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:12.915 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e216a6700 1 --2- 172.21.15.190:0/204612778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e1c105a90 0x7f9e1c107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:12.915 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e216a6700 1 -- 172.21.15.190:0/204612778 >> 172.21.15.190:0/204612778 conn(0x7f9e1c0fcb90 msgr2=0x7f9e1c0fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:45:12.915 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e216a6700 1 -- 172.21.15.190:0/204612778 shutdown_connections 2024-03-26T03:45:12.915 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e216a6700 1 -- 172.21.15.190:0/204612778 wait complete. 2024-03-26T03:45:12.915 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e216a6700 1 Processor -- start 2024-03-26T03:45:12.915 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e216a6700 1 -- start start 2024-03-26T03:45:12.917 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e216a6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e1c103160 0x7f9e1c197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:45:12.917 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e216a6700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e1c105a90 0x7f9e1c1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:45:12.917 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e216a6700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e1c1083c0 0x7f9e1c19f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:45:12.917 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e216a6700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c10d7b0 con 0x7f9e1c1083c0 2024-03-26T03:45:12.917 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e216a6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c10d630 con 0x7f9e1c103160 2024-03-26T03:45:12.917 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e216a6700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9e1c10d930 con 0x7f9e1c105a90 2024-03-26T03:45:12.917 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e1affd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e1c103160 0x7f9e1c197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:45:12.918 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e1affd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e1c103160 0x7f9e1c197d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:56170/0 (socket says 172.21.15.190:56170) 2024-03-26T03:45:12.918 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e1affd700 1 -- 172.21.15.190:0/3606450410 learned_addr learned my addr 172.21.15.190:0/3606450410 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-03-26T03:45:12.918 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e1a7fc700 1 --2- 172.21.15.190:0/3606450410 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e1c105a90 0x7f9e1c1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:45:12.918 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e1b7fe700 1 --2- 172.21.15.190:0/3606450410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e1c1083c0 0x7f9e1c19f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:45:12.918 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e1affd700 1 -- 172.21.15.190:0/3606450410 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e1c105a90 msgr2=0x7f9e1c1982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:45:12.918 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e1affd700 1 --2- 172.21.15.190:0/3606450410 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e1c105a90 0x7f9e1c1982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:12.918 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e1affd700 1 -- 172.21.15.190:0/3606450410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e1c1083c0 msgr2=0x7f9e1c19f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:45:12.918 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e1affd700 1 --2- 172.21.15.190:0/3606450410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e1c1083c0 0x7f9e1c19f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:12.918 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e1affd700 1 -- 172.21.15.190:0/3606450410 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1c19f9c0 con 0x7f9e1c103160 2024-03-26T03:45:12.918 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e1b7fe700 1 --2- 172.21.15.190:0/3606450410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e1c1083c0 0x7f9e1c19f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T03:45:12.918 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e1affd700 1 --2- 172.21.15.190:0/3606450410 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e1c103160 0x7f9e1c197d60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00d980 tx=0x7f9e0c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:45:12.918 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e03fff700 1 -- 172.21.15.190:0/3606450410 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c017070 con 0x7f9e1c103160 2024-03-26T03:45:12.918 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e216a6700 1 -- 172.21.15.190:0/3606450410 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e1c19fcb0 con 0x7f9e1c103160 2024-03-26T03:45:12.919 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e03fff700 1 -- 172.21.15.190:0/3606450410 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f9e0c005020 con 0x7f9e1c103160 2024-03-26T03:45:12.919 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e216a6700 1 -- 172.21.15.190:0/3606450410 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e1c1a01a0 con 0x7f9e1c103160 2024-03-26T03:45:12.919 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e03fff700 1 -- 172.21.15.190:0/3606450410 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c009ad0 con 0x7f9e1c103160 2024-03-26T03:45:12.919 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e216a6700 1 -- 172.21.15.190:0/3606450410 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e1c050300 con 0x7f9e1c103160 2024-03-26T03:45:12.919 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.920+0000 7f9e03fff700 1 -- 172.21.15.190:0/3606450410 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 17) ==== 92540+0+0 (secure 0 0 0) 0x7f9e0c009cb0 con 0x7f9e1c103160 2024-03-26T03:45:12.919 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.924+0000 7f9e03fff700 1 --2- 172.21.15.190:0/3606450410 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f9e0406e6b0 0x7f9e04070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:45:12.919 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.924+0000 7f9e03fff700 1 -- 172.21.15.190:0/3606450410 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7236+0+0 (secure 0 0 0) 0x7f9e0c08ecc0 con 0x7f9e1c103160 2024-03-26T03:45:12.920 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.924+0000 7f9e1a7fc700 1 -- 172.21.15.190:0/3606450410 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f9e0406e6b0 msgr2=0x7f9e04070b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2826039805 2024-03-26T03:45:12.920 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.924+0000 7f9e1a7fc700 1 --2- 172.21.15.190:0/3606450410 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f9e0406e6b0 0x7f9e04070b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T03:45:12.923 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:12.928+0000 7f9e03fff700 1 -- 172.21.15.190:0/3606450410 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f9e0c020020 con 0x7f9e1c103160 2024-03-26T03:45:13.122 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:13.128+0000 7f9e1a7fc700 1 -- 172.21.15.190:0/3606450410 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f9e0406e6b0 msgr2=0x7f9e04070b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2826039805 2024-03-26T03:45:13.122 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:13.128+0000 7f9e1a7fc700 1 --2- 172.21.15.190:0/3606450410 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f9e0406e6b0 0x7f9e04070b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T03:45:13.142 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:13.144+0000 7f9e216a6700 1 -- 172.21.15.190:0/3606450410 --> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi175=a", "target": ["mon-mgr", ""]}) -- 0x7f9e1c062a80 con 0x7f9e0406e6b0 2024-03-26T03:45:13.491 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:12 smithi175 bash[21783]: debug 2024-03-26T03:45:12.402+0000 7f6f58acf200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T03:45:13.492 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:13 smithi175 bash[21783]: debug 2024-03-26T03:45:13.198+0000 7f6f58acf200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T03:45:13.492 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:13 smithi175 bash[21783]: debug 2024-03-26T03:45:13.310+0000 7f6f58acf200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T03:45:13.492 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:13 smithi175 bash[21783]: debug 2024-03-26T03:45:13.414+0000 7f6f58acf200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T03:45:13.526 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:13.532+0000 7f9e1a7fc700 1 -- 172.21.15.190:0/3606450410 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f9e0406e6b0 msgr2=0x7f9e04070b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2826039805 2024-03-26T03:45:13.526 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:13.532+0000 7f9e1a7fc700 1 --2- 172.21.15.190:0/3606450410 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f9e0406e6b0 0x7f9e04070b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-26T03:45:13.741 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:13 smithi190 bash[24341]: debug 2024-03-26T03:45:13.296+0000 7faea18b6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T03:45:13.741 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:13 smithi190 bash[24341]: debug 2024-03-26T03:45:13.408+0000 7faea18b6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T03:45:13.742 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:13 smithi190 bash[24341]: debug 2024-03-26T03:45:13.512+0000 7faea18b6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T03:45:14.242 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:13 smithi190 bash[24341]: debug 2024-03-26T03:45:13.868+0000 7faea18b6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T03:45:14.242 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:13 smithi190 bash[24341]: debug 2024-03-26T03:45:13.980+0000 7faea18b6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T03:45:14.242 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:14 smithi190 bash[24341]: debug 2024-03-26T03:45:14.084+0000 7faea18b6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T03:45:14.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:13 smithi175 bash[21783]: debug 2024-03-26T03:45:13.774+0000 7f6f58acf200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T03:45:14.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:13 smithi175 bash[21783]: debug 2024-03-26T03:45:13.890+0000 7f6f58acf200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T03:45:14.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:13 smithi175 bash[21783]: debug 2024-03-26T03:45:13.994+0000 7f6f58acf200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T03:45:14.330 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:14.336+0000 7f9e1a7fc700 1 -- 172.21.15.190:0/3606450410 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f9e0406e6b0 msgr2=0x7f9e04070b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2826039805 2024-03-26T03:45:14.330 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:14.336+0000 7f9e1a7fc700 1 --2- 172.21.15.190:0/3606450410 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f9e0406e6b0 0x7f9e04070b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-26T03:45:14.741 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:14 smithi175 bash[21783]: debug 2024-03-26T03:45:14.358+0000 7f6f58acf200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T03:45:14.741 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:14 smithi175 bash[21783]: debug 2024-03-26T03:45:14.602+0000 7f6f58acf200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T03:45:14.742 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:14 smithi190 bash[24341]: debug 2024-03-26T03:45:14.444+0000 7faea18b6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T03:45:14.742 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:14 smithi190 bash[24341]: debug 2024-03-26T03:45:14.688+0000 7faea18b6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T03:45:15.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:14 smithi175 bash[21783]: debug 2024-03-26T03:45:14.862+0000 7f6f58acf200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T03:45:15.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:14 smithi175 bash[21783]: debug 2024-03-26T03:45:14.978+0000 7f6f58acf200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T03:45:15.242 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:14 smithi190 bash[24341]: debug 2024-03-26T03:45:14.948+0000 7faea18b6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T03:45:15.242 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:15 smithi190 bash[24341]: debug 2024-03-26T03:45:15.064+0000 7faea18b6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T03:45:15.673 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:15 smithi175 bash[21783]: debug 2024-03-26T03:45:15.254+0000 7f6f58acf200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T03:45:15.741 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:15 smithi190 bash[24341]: debug 2024-03-26T03:45:15.336+0000 7faea18b6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T03:45:15.934 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:15.940+0000 7f9e1a7fc700 1 -- 172.21.15.190:0/3606450410 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f9e0406e6b0 msgr2=0x7f9e04070b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.175:6800/2826039805 2024-03-26T03:45:15.938 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:15.940+0000 7f9e1a7fc700 1 --2- 172.21.15.190:0/3606450410 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f9e0406e6b0 0x7f9e04070b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-26T03:45:15.991 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:15 smithi175 bash[21783]: debug 2024-03-26T03:45:15.674+0000 7f6f58acf200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T03:45:15.991 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:15 smithi175 bash[21783]: debug 2024-03-26T03:45:15.914+0000 7f6f58acf200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T03:45:16.101 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:15 smithi190 bash[24341]: debug 2024-03-26T03:45:15.752+0000 7faea18b6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T03:45:16.101 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:15 smithi190 bash[24341]: debug 2024-03-26T03:45:15.988+0000 7faea18b6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T03:45:16.286 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:16 smithi175 bash[21783]: debug 2024-03-26T03:45:16.034+0000 7f6f58acf200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T03:45:16.351 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:16 smithi190 bash[24341]: debug 2024-03-26T03:45:16.104+0000 7faea18b6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T03:45:16.741 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:16 smithi175 bash[21783]: debug 2024-03-26T03:45:16.286+0000 7f6f58acf200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T03:45:16.741 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:16 smithi190 bash[24341]: debug 2024-03-26T03:45:16.356+0000 7faea18b6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T03:45:17.720 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:17 smithi175 bash[21783]: debug 2024-03-26T03:45:17.310+0000 7f6f58acf200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T03:45:17.721 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:17 smithi175 bash[21783]: debug 2024-03-26T03:45:17.426+0000 7f6f58acf200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T03:45:17.741 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:17 smithi190 bash[24341]: debug 2024-03-26T03:45:17.368+0000 7faea18b6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T03:45:17.741 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:17 smithi190 bash[24341]: debug 2024-03-26T03:45:17.484+0000 7faea18b6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T03:45:17.991 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:17 smithi175 bash[21783]: debug 2024-03-26T03:45:17.722+0000 7f6f58acf200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T03:45:17.991 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:17 smithi175 bash[21783]: debug 2024-03-26T03:45:17.834+0000 7f6f58acf200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T03:45:18.241 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:17 smithi190 bash[24341]: debug 2024-03-26T03:45:17.776+0000 7faea18b6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T03:45:18.241 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:17 smithi190 bash[24341]: debug 2024-03-26T03:45:17.888+0000 7faea18b6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T03:45:18.452 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:18.456+0000 7f9e03fff700 1 -- 172.21.15.190:0/3606450410 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mgrmap(e 18) ==== 92156+0+0 (secure 0 0 0) 0x7f9e0c095080 con 0x7f9e1c103160 2024-03-26T03:45:18.452 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:18.456+0000 7f9e03fff700 1 -- 172.21.15.190:0/3606450410 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f9e0406e6b0 msgr2=0x7f9e04070b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T03:45:18.453 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:18.456+0000 7f9e03fff700 1 --2- 172.21.15.190:0/3606450410 >> [v2:172.21.15.175:6800/2826039805,v1:172.21.15.175:6801/2826039805] conn(0x7f9e0406e6b0 0x7f9e04070b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:18.572 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21783]: debug 2024-03-26T03:45:18.290+0000 7f6f58acf200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T03:45:18.572 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21783]: debug 2024-03-26T03:45:18.418+0000 7f6f58acf200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T03:45:18.573 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21527]: cluster 2024-03-26T03:45:18.442361+0000 mon.a (mon.0) 771 : cluster [INF] Active manager daemon y restarted 2024-03-26T03:45:18.573 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21527]: cluster 2024-03-26T03:45:18.443113+0000 mon.a (mon.0) 772 : cluster [INF] Activating manager daemon y 2024-03-26T03:45:18.573 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21527]: cluster 2024-03-26T03:45:18.454261+0000 mon.a (mon.0) 773 : cluster [DBG] osdmap e75: 8 total, 8 up, 8 in 2024-03-26T03:45:18.573 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21527]: cluster 2024-03-26T03:45:18.454825+0000 mon.a (mon.0) 774 : cluster [DBG] mgrmap e18: y(active, starting, since 0.0119113s), standbys: x 2024-03-26T03:45:18.573 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21527]: audit 2024-03-26T03:45:18.465282+0000 mon.c (mon.2) 18 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T03:45:18.573 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21527]: audit 2024-03-26T03:45:18.465763+0000 mon.c (mon.2) 19 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:45:18.573 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21527]: audit 2024-03-26T03:45:18.466205+0000 mon.c (mon.2) 20 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:45:18.573 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21527]: audit 2024-03-26T03:45:18.467261+0000 mon.c (mon.2) 21 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T03:45:18.573 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21527]: audit 2024-03-26T03:45:18.467740+0000 mon.c (mon.2) 22 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T03:45:18.573 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21527]: audit 2024-03-26T03:45:18.468299+0000 mon.c (mon.2) 23 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T03:45:18.573 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21527]: audit 2024-03-26T03:45:18.469296+0000 mon.c (mon.2) 24 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T03:45:18.574 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21527]: audit 2024-03-26T03:45:18.469997+0000 mon.c (mon.2) 25 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T03:45:18.574 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21527]: audit 2024-03-26T03:45:18.470640+0000 mon.c (mon.2) 26 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T03:45:18.574 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21527]: audit 2024-03-26T03:45:18.471283+0000 mon.c (mon.2) 27 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T03:45:18.574 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21527]: audit 2024-03-26T03:45:18.471939+0000 mon.c (mon.2) 28 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T03:45:18.574 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21527]: audit 2024-03-26T03:45:18.472573+0000 mon.c (mon.2) 29 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T03:45:18.574 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21527]: audit 2024-03-26T03:45:18.473168+0000 mon.c (mon.2) 30 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T03:45:18.574 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21527]: audit 2024-03-26T03:45:18.473876+0000 mon.c (mon.2) 31 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T03:45:18.574 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21527]: audit 2024-03-26T03:45:18.474244+0000 mon.c (mon.2) 32 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T03:45:18.574 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21527]: audit 2024-03-26T03:45:18.475490+0000 mon.c (mon.2) 33 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T03:45:18.574 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21527]: cluster 2024-03-26T03:45:18.483966+0000 mon.a (mon.0) 775 : cluster [DBG] Standby manager daemon x restarted 2024-03-26T03:45:18.574 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21527]: cluster 2024-03-26T03:45:18.484065+0000 mon.a (mon.0) 776 : cluster [DBG] Standby manager daemon x started 2024-03-26T03:45:18.574 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21527]: audit 2024-03-26T03:45:18.493178+0000 mon.b (mon.1) 29 : audit [DBG] from='mgr.? 172.21.15.190:0/1290957953' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-26T03:45:18.574 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21527]: audit 2024-03-26T03:45:18.496436+0000 mon.b (mon.1) 30 : audit [DBG] from='mgr.? 172.21.15.190:0/1290957953' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T03:45:18.574 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21527]: audit 2024-03-26T03:45:18.497752+0000 mon.b (mon.1) 31 : audit [DBG] from='mgr.? 172.21.15.190:0/1290957953' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-26T03:45:18.575 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21527]: audit 2024-03-26T03:45:18.498075+0000 mon.b (mon.1) 32 : audit [DBG] from='mgr.? 172.21.15.190:0/1290957953' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T03:45:18.575 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[29284]: cluster 2024-03-26T03:45:18.442361+0000 mon.a (mon.0) 771 : cluster [INF] Active manager daemon y restarted 2024-03-26T03:45:18.575 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[29284]: cluster 2024-03-26T03:45:18.443113+0000 mon.a (mon.0) 772 : cluster [INF] Activating manager daemon y 2024-03-26T03:45:18.575 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[29284]: cluster 2024-03-26T03:45:18.454261+0000 mon.a (mon.0) 773 : cluster [DBG] osdmap e75: 8 total, 8 up, 8 in 2024-03-26T03:45:18.575 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[29284]: cluster 2024-03-26T03:45:18.454825+0000 mon.a (mon.0) 774 : cluster [DBG] mgrmap e18: y(active, starting, since 0.0119113s), standbys: x 2024-03-26T03:45:18.575 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[29284]: audit 2024-03-26T03:45:18.465282+0000 mon.c (mon.2) 18 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T03:45:18.575 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[29284]: audit 2024-03-26T03:45:18.465763+0000 mon.c (mon.2) 19 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:45:18.575 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[29284]: audit 2024-03-26T03:45:18.466205+0000 mon.c (mon.2) 20 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:45:18.575 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[29284]: audit 2024-03-26T03:45:18.467261+0000 mon.c (mon.2) 21 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T03:45:18.576 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[29284]: audit 2024-03-26T03:45:18.467740+0000 mon.c (mon.2) 22 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T03:45:18.576 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[29284]: audit 2024-03-26T03:45:18.468299+0000 mon.c (mon.2) 23 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T03:45:18.576 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[29284]: audit 2024-03-26T03:45:18.469296+0000 mon.c (mon.2) 24 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T03:45:18.576 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[29284]: audit 2024-03-26T03:45:18.469997+0000 mon.c (mon.2) 25 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T03:45:18.576 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[29284]: audit 2024-03-26T03:45:18.470640+0000 mon.c (mon.2) 26 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T03:45:18.576 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[29284]: audit 2024-03-26T03:45:18.471283+0000 mon.c (mon.2) 27 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T03:45:18.576 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[29284]: audit 2024-03-26T03:45:18.471939+0000 mon.c (mon.2) 28 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T03:45:18.576 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[29284]: audit 2024-03-26T03:45:18.472573+0000 mon.c (mon.2) 29 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T03:45:18.576 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[29284]: audit 2024-03-26T03:45:18.473168+0000 mon.c (mon.2) 30 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T03:45:18.576 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[29284]: audit 2024-03-26T03:45:18.473876+0000 mon.c (mon.2) 31 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T03:45:18.576 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[29284]: audit 2024-03-26T03:45:18.474244+0000 mon.c (mon.2) 32 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T03:45:18.576 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[29284]: audit 2024-03-26T03:45:18.475490+0000 mon.c (mon.2) 33 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T03:45:18.576 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[29284]: cluster 2024-03-26T03:45:18.483966+0000 mon.a (mon.0) 775 : cluster [DBG] Standby manager daemon x restarted 2024-03-26T03:45:18.576 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[29284]: cluster 2024-03-26T03:45:18.484065+0000 mon.a (mon.0) 776 : cluster [DBG] Standby manager daemon x started 2024-03-26T03:45:18.576 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[29284]: audit 2024-03-26T03:45:18.493178+0000 mon.b (mon.1) 29 : audit [DBG] from='mgr.? 172.21.15.190:0/1290957953' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-26T03:45:18.577 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[29284]: audit 2024-03-26T03:45:18.496436+0000 mon.b (mon.1) 30 : audit [DBG] from='mgr.? 172.21.15.190:0/1290957953' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T03:45:18.577 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[29284]: audit 2024-03-26T03:45:18.497752+0000 mon.b (mon.1) 31 : audit [DBG] from='mgr.? 172.21.15.190:0/1290957953' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-26T03:45:18.577 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[29284]: audit 2024-03-26T03:45:18.498075+0000 mon.b (mon.1) 32 : audit [DBG] from='mgr.? 172.21.15.190:0/1290957953' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T03:45:18.587 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[24341]: debug 2024-03-26T03:45:18.340+0000 7faea18b6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T03:45:18.587 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[24341]: debug 2024-03-26T03:45:18.464+0000 7faea18b6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T03:45:18.587 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[24341]: [26/Mar/2024:03:45:18] ENGINE Bus STARTING 2024-03-26T03:45:18.587 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[24341]: CherryPy Checker: 2024-03-26T03:45:18.588 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[23392]: cluster 2024-03-26T03:45:18.442361+0000 mon.a (mon.0) 771 : cluster [INF] Active manager daemon y restarted 2024-03-26T03:45:18.588 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[23392]: cluster 2024-03-26T03:45:18.443113+0000 mon.a (mon.0) 772 : cluster [INF] Activating manager daemon y 2024-03-26T03:45:18.588 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[23392]: cluster 2024-03-26T03:45:18.454261+0000 mon.a (mon.0) 773 : cluster [DBG] osdmap e75: 8 total, 8 up, 8 in 2024-03-26T03:45:18.588 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[23392]: cluster 2024-03-26T03:45:18.454825+0000 mon.a (mon.0) 774 : cluster [DBG] mgrmap e18: y(active, starting, since 0.0119113s), standbys: x 2024-03-26T03:45:18.588 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[23392]: audit 2024-03-26T03:45:18.465282+0000 mon.c (mon.2) 18 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T03:45:18.588 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[23392]: audit 2024-03-26T03:45:18.465763+0000 mon.c (mon.2) 19 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T03:45:18.588 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[23392]: audit 2024-03-26T03:45:18.466205+0000 mon.c (mon.2) 20 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T03:45:18.588 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[23392]: audit 2024-03-26T03:45:18.467261+0000 mon.c (mon.2) 21 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T03:45:18.588 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[23392]: audit 2024-03-26T03:45:18.467740+0000 mon.c (mon.2) 22 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T03:45:18.589 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[23392]: audit 2024-03-26T03:45:18.468299+0000 mon.c (mon.2) 23 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T03:45:18.589 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[23392]: audit 2024-03-26T03:45:18.469296+0000 mon.c (mon.2) 24 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T03:45:18.589 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[23392]: audit 2024-03-26T03:45:18.469997+0000 mon.c (mon.2) 25 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T03:45:18.589 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[23392]: audit 2024-03-26T03:45:18.470640+0000 mon.c (mon.2) 26 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T03:45:18.589 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[23392]: audit 2024-03-26T03:45:18.471283+0000 mon.c (mon.2) 27 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T03:45:18.589 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[23392]: audit 2024-03-26T03:45:18.471939+0000 mon.c (mon.2) 28 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T03:45:18.589 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[23392]: audit 2024-03-26T03:45:18.472573+0000 mon.c (mon.2) 29 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T03:45:18.589 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[23392]: audit 2024-03-26T03:45:18.473168+0000 mon.c (mon.2) 30 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T03:45:18.589 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[23392]: audit 2024-03-26T03:45:18.473876+0000 mon.c (mon.2) 31 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T03:45:18.589 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[23392]: audit 2024-03-26T03:45:18.474244+0000 mon.c (mon.2) 32 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T03:45:18.589 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[23392]: audit 2024-03-26T03:45:18.475490+0000 mon.c (mon.2) 33 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T03:45:18.589 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[23392]: cluster 2024-03-26T03:45:18.483966+0000 mon.a (mon.0) 775 : cluster [DBG] Standby manager daemon x restarted 2024-03-26T03:45:18.589 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[23392]: cluster 2024-03-26T03:45:18.484065+0000 mon.a (mon.0) 776 : cluster [DBG] Standby manager daemon x started 2024-03-26T03:45:18.589 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[23392]: audit 2024-03-26T03:45:18.493178+0000 mon.b (mon.1) 29 : audit [DBG] from='mgr.? 172.21.15.190:0/1290957953' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-26T03:45:18.589 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[23392]: audit 2024-03-26T03:45:18.496436+0000 mon.b (mon.1) 30 : audit [DBG] from='mgr.? 172.21.15.190:0/1290957953' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T03:45:18.590 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[23392]: audit 2024-03-26T03:45:18.497752+0000 mon.b (mon.1) 31 : audit [DBG] from='mgr.? 172.21.15.190:0/1290957953' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-26T03:45:18.590 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[23392]: audit 2024-03-26T03:45:18.498075+0000 mon.b (mon.1) 32 : audit [DBG] from='mgr.? 172.21.15.190:0/1290957953' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T03:45:18.991 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21783]: [26/Mar/2024:03:45:18] ENGINE Bus STARTING 2024-03-26T03:45:18.991 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21783]: CherryPy Checker: 2024-03-26T03:45:18.991 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21783]: The Application mounted at '' has an empty config. 2024-03-26T03:45:18.991 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21783]: [26/Mar/2024:03:45:18] ENGINE Serving on http://:::9283 2024-03-26T03:45:18.991 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:18 smithi175 bash[21783]: [26/Mar/2024:03:45:18] ENGINE Bus STARTED 2024-03-26T03:45:18.992 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[24341]: The Application mounted at '' has an empty config. 2024-03-26T03:45:18.992 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[24341]: [26/Mar/2024:03:45:18] ENGINE Serving on http://:::9283 2024-03-26T03:45:18.992 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:18 smithi190 bash[24341]: [26/Mar/2024:03:45:18] ENGINE Bus STARTED 2024-03-26T03:45:19.462 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:19.464+0000 7f9e03fff700 1 -- 172.21.15.190:0/3606450410 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mgrmap(e 19) ==== 92283+0+0 (secure 0 0 0) 0x7f9e0c057c50 con 0x7f9e1c103160 2024-03-26T03:45:19.462 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:19.468+0000 7f9e03fff700 1 --2- 172.21.15.190:0/3606450410 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f9e04077eb0 0x7f9e0407a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:45:19.462 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:19.468+0000 7f9e03fff700 1 -- 172.21.15.190:0/3606450410 --> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi175=a", "target": ["mon-mgr", ""]}) -- 0x7f9e1c062a80 con 0x7f9e04077eb0 2024-03-26T03:45:19.464 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:19.468+0000 7f9e1a7fc700 1 --2- 172.21.15.190:0/3606450410 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f9e04077eb0 0x7f9e0407a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:45:19.467 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:19.472+0000 7f9e1a7fc700 1 --2- 172.21.15.190:0/3606450410 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f9e04077eb0 0x7f9e0407a2a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9e1000a6a0 tx=0x7f9e1000a380 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:45:19.479 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:19.484+0000 7f9e03fff700 1 -- 172.21.15.190:0/3606450410 <== mgr.24431 v2:172.21.15.175:6800/1140903233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+33 (secure 0 0 0) 0x7f9e1c062a80 con 0x7f9e04077eb0 2024-03-26T03:45:19.479 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled alertmanager update... 2024-03-26T03:45:19.482 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:19.488+0000 7f9e216a6700 1 -- 172.21.15.190:0/3606450410 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f9e04077eb0 msgr2=0x7f9e0407a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:45:19.482 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:19.488+0000 7f9e216a6700 1 --2- 172.21.15.190:0/3606450410 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f9e04077eb0 0x7f9e0407a2a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9e1000a6a0 tx=0x7f9e1000a380 comp rx=0 tx=0).stop 2024-03-26T03:45:19.482 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:19.488+0000 7f9e216a6700 1 -- 172.21.15.190:0/3606450410 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e1c103160 msgr2=0x7f9e1c197d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:45:19.482 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:19.488+0000 7f9e216a6700 1 --2- 172.21.15.190:0/3606450410 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e1c103160 0x7f9e1c197d60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00d980 tx=0x7f9e0c00dc90 comp rx=0 tx=0).stop 2024-03-26T03:45:19.482 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:19.488+0000 7f9e1b7fe700 1 -- 172.21.15.190:0/3606450410 reap_dead start 2024-03-26T03:45:19.482 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:19.488+0000 7f9e216a6700 1 -- 172.21.15.190:0/3606450410 shutdown_connections 2024-03-26T03:45:19.482 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:19.488+0000 7f9e216a6700 1 -- 172.21.15.190:0/3606450410 >> 172.21.15.190:0/3606450410 conn(0x7f9e1c0fcb90 msgr2=0x7f9e1c103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:45:19.483 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:19.488+0000 7f9e216a6700 1 -- 172.21.15.190:0/3606450410 shutdown_connections 2024-03-26T03:45:19.483 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:19.488+0000 7f9e216a6700 1 -- 172.21.15.190:0/3606450410 wait complete. 2024-03-26T03:45:19.741 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:45:19 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:45:19.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:19 smithi190 bash[23392]: cluster 2024-03-26T03:45:18.502601+0000 mon.a (mon.0) 777 : cluster [INF] Manager daemon y is now available 2024-03-26T03:45:19.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:19 smithi190 bash[23392]: audit 2024-03-26T03:45:18.547284+0000 mon.c (mon.2) 34 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:45:19.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:19 smithi190 bash[23392]: audit 2024-03-26T03:45:18.571113+0000 mon.a (mon.0) 778 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:19.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:19 smithi190 bash[23392]: audit 2024-03-26T03:45:18.575520+0000 mon.c (mon.2) 35 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:45:19.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:19 smithi190 bash[23392]: audit 2024-03-26T03:45:18.586907+0000 mon.c (mon.2) 36 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-26T03:45:19.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:19 smithi190 bash[23392]: audit 2024-03-26T03:45:18.587238+0000 mon.a (mon.0) 779 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-26T03:45:19.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:19 smithi190 bash[23392]: audit 2024-03-26T03:45:18.604302+0000 mon.c (mon.2) 37 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-26T03:45:19.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:19 smithi190 bash[23392]: audit 2024-03-26T03:45:18.604722+0000 mon.a (mon.0) 780 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-26T03:45:19.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:19 smithi190 bash[23392]: cephadm 2024-03-26T03:45:19.014611+0000 mgr.y (mgr.24431) 1 : cephadm [INF] [26/Mar/2024:03:45:19] ENGINE Bus STARTING 2024-03-26T03:45:19.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:19 smithi190 bash[23392]: cephadm 2024-03-26T03:45:19.116359+0000 mgr.y (mgr.24431) 2 : cephadm [INF] [26/Mar/2024:03:45:19] ENGINE Serving on http://172.21.15.175:8765 2024-03-26T03:45:19.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:19 smithi190 bash[23392]: cephadm 2024-03-26T03:45:19.233785+0000 mgr.y (mgr.24431) 3 : cephadm [INF] [26/Mar/2024:03:45:19] ENGINE Serving on https://172.21.15.175:7150 2024-03-26T03:45:19.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:19 smithi190 bash[23392]: cephadm 2024-03-26T03:45:19.233996+0000 mgr.y (mgr.24431) 4 : cephadm [INF] [26/Mar/2024:03:45:19] ENGINE Bus STARTED 2024-03-26T03:45:19.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:19 smithi190 bash[23392]: cluster 2024-03-26T03:45:19.459347+0000 mon.a (mon.0) 781 : cluster [DBG] mgrmap e19: y(active, since 1.01644s), standbys: x 2024-03-26T03:45:19.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:19 smithi190 bash[23392]: audit 2024-03-26T03:45:19.479542+0000 mon.a (mon.0) 782 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:19.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:19 smithi175 bash[29284]: cluster 2024-03-26T03:45:18.502601+0000 mon.a (mon.0) 777 : cluster [INF] Manager daemon y is now available 2024-03-26T03:45:19.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:19 smithi175 bash[29284]: audit 2024-03-26T03:45:18.547284+0000 mon.c (mon.2) 34 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:45:19.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:19 smithi175 bash[29284]: audit 2024-03-26T03:45:18.571113+0000 mon.a (mon.0) 778 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:19.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:19 smithi175 bash[29284]: audit 2024-03-26T03:45:18.575520+0000 mon.c (mon.2) 35 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:45:19.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:19 smithi175 bash[29284]: audit 2024-03-26T03:45:18.586907+0000 mon.c (mon.2) 36 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-26T03:45:19.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:19 smithi175 bash[29284]: audit 2024-03-26T03:45:18.587238+0000 mon.a (mon.0) 779 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-26T03:45:19.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:19 smithi175 bash[29284]: audit 2024-03-26T03:45:18.604302+0000 mon.c (mon.2) 37 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-26T03:45:19.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:19 smithi175 bash[29284]: audit 2024-03-26T03:45:18.604722+0000 mon.a (mon.0) 780 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-26T03:45:19.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:19 smithi175 bash[29284]: cephadm 2024-03-26T03:45:19.014611+0000 mgr.y (mgr.24431) 1 : cephadm [INF] [26/Mar/2024:03:45:19] ENGINE Bus STARTING 2024-03-26T03:45:19.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:19 smithi175 bash[29284]: cephadm 2024-03-26T03:45:19.116359+0000 mgr.y (mgr.24431) 2 : cephadm [INF] [26/Mar/2024:03:45:19] ENGINE Serving on http://172.21.15.175:8765 2024-03-26T03:45:19.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:19 smithi175 bash[29284]: cephadm 2024-03-26T03:45:19.233785+0000 mgr.y (mgr.24431) 3 : cephadm [INF] [26/Mar/2024:03:45:19] ENGINE Serving on https://172.21.15.175:7150 2024-03-26T03:45:19.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:19 smithi175 bash[29284]: cephadm 2024-03-26T03:45:19.233996+0000 mgr.y (mgr.24431) 4 : cephadm [INF] [26/Mar/2024:03:45:19] ENGINE Bus STARTED 2024-03-26T03:45:19.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:19 smithi175 bash[29284]: cluster 2024-03-26T03:45:19.459347+0000 mon.a (mon.0) 781 : cluster [DBG] mgrmap e19: y(active, since 1.01644s), standbys: x 2024-03-26T03:45:19.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:19 smithi175 bash[29284]: audit 2024-03-26T03:45:19.479542+0000 mon.a (mon.0) 782 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:19 smithi175 bash[21527]: cluster 2024-03-26T03:45:18.502601+0000 mon.a (mon.0) 777 : cluster [INF] Manager daemon y is now available 2024-03-26T03:45:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:19 smithi175 bash[21527]: audit 2024-03-26T03:45:18.547284+0000 mon.c (mon.2) 34 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:45:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:19 smithi175 bash[21527]: audit 2024-03-26T03:45:18.571113+0000 mon.a (mon.0) 778 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:19 smithi175 bash[21527]: audit 2024-03-26T03:45:18.575520+0000 mon.c (mon.2) 35 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:45:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:19 smithi175 bash[21527]: audit 2024-03-26T03:45:18.586907+0000 mon.c (mon.2) 36 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-26T03:45:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:19 smithi175 bash[21527]: audit 2024-03-26T03:45:18.587238+0000 mon.a (mon.0) 779 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-26T03:45:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:19 smithi175 bash[21527]: audit 2024-03-26T03:45:18.604302+0000 mon.c (mon.2) 37 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-26T03:45:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:19 smithi175 bash[21527]: audit 2024-03-26T03:45:18.604722+0000 mon.a (mon.0) 780 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-26T03:45:19.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:19 smithi175 bash[21527]: cephadm 2024-03-26T03:45:19.014611+0000 mgr.y (mgr.24431) 1 : cephadm [INF] [26/Mar/2024:03:45:19] ENGINE Bus STARTING 2024-03-26T03:45:19.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:19 smithi175 bash[21527]: cephadm 2024-03-26T03:45:19.116359+0000 mgr.y (mgr.24431) 2 : cephadm [INF] [26/Mar/2024:03:45:19] ENGINE Serving on http://172.21.15.175:8765 2024-03-26T03:45:19.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:19 smithi175 bash[21527]: cephadm 2024-03-26T03:45:19.233785+0000 mgr.y (mgr.24431) 3 : cephadm [INF] [26/Mar/2024:03:45:19] ENGINE Serving on https://172.21.15.175:7150 2024-03-26T03:45:19.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:19 smithi175 bash[21527]: cephadm 2024-03-26T03:45:19.233996+0000 mgr.y (mgr.24431) 4 : cephadm [INF] [26/Mar/2024:03:45:19] ENGINE Bus STARTED 2024-03-26T03:45:19.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:19 smithi175 bash[21527]: cluster 2024-03-26T03:45:19.459347+0000 mon.a (mon.0) 781 : cluster [DBG] mgrmap e19: y(active, since 1.01644s), standbys: x 2024-03-26T03:45:19.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:19 smithi175 bash[21527]: audit 2024-03-26T03:45:19.479542+0000 mon.a (mon.0) 782 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:20.463 DEBUG:teuthology.orchestra.run.smithi175:alertmanager.a> sudo journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@alertmanager.a.service 2024-03-26T03:45:20.466 INFO:tasks.cephadm:Adding grafana.a on smithi190 2024-03-26T03:45:20.466 DEBUG:teuthology.orchestra.run.smithi190:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph orch apply grafana '1;smithi190=a' 2024-03-26T03:45:20.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:20 smithi190 bash[23392]: audit 2024-03-26T03:45:19.472233+0000 mgr.y (mgr.24431) 5 : audit [DBG] from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi175=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:45:20.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:20 smithi190 bash[23392]: audit 2024-03-26T03:45:19.473410+0000 mgr.y (mgr.24431) 6 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:45:20.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:20 smithi190 bash[23392]: cephadm 2024-03-26T03:45:19.475076+0000 mgr.y (mgr.24431) 7 : cephadm [INF] Saving service alertmanager spec with placement smithi175=a;count:1 2024-03-26T03:45:20.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:20 smithi190 bash[23392]: cluster 2024-03-26T03:45:19.478080+0000 mgr.y (mgr.24431) 8 : cluster [DBG] pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:45:20.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:20 smithi175 bash[21527]: audit 2024-03-26T03:45:19.472233+0000 mgr.y (mgr.24431) 5 : audit [DBG] from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi175=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:45:20.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:20 smithi175 bash[21527]: audit 2024-03-26T03:45:19.473410+0000 mgr.y (mgr.24431) 6 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:45:20.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:20 smithi175 bash[21527]: cephadm 2024-03-26T03:45:19.475076+0000 mgr.y (mgr.24431) 7 : cephadm [INF] Saving service alertmanager spec with placement smithi175=a;count:1 2024-03-26T03:45:20.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:20 smithi175 bash[21527]: cluster 2024-03-26T03:45:19.478080+0000 mgr.y (mgr.24431) 8 : cluster [DBG] pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:45:20.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:20 smithi175 bash[29284]: audit 2024-03-26T03:45:19.472233+0000 mgr.y (mgr.24431) 5 : audit [DBG] from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi175=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:45:20.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:20 smithi175 bash[29284]: audit 2024-03-26T03:45:19.473410+0000 mgr.y (mgr.24431) 6 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:45:20.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:20 smithi175 bash[29284]: cephadm 2024-03-26T03:45:19.475076+0000 mgr.y (mgr.24431) 7 : cephadm [INF] Saving service alertmanager spec with placement smithi175=a;count:1 2024-03-26T03:45:20.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:20 smithi175 bash[29284]: cluster 2024-03-26T03:45:19.478080+0000 mgr.y (mgr.24431) 8 : cluster [DBG] pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:45:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:21 smithi190 bash[23392]: cluster 2024-03-26T03:45:20.469397+0000 mgr.y (mgr.24431) 9 : cluster [DBG] pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:45:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:21 smithi190 bash[23392]: cluster 2024-03-26T03:45:20.525580+0000 mon.a (mon.0) 783 : cluster [DBG] mgrmap e20: y(active, since 2s), standbys: x 2024-03-26T03:45:21.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:21 smithi175 bash[21527]: cluster 2024-03-26T03:45:20.469397+0000 mgr.y (mgr.24431) 9 : cluster [DBG] pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:45:21.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:21 smithi175 bash[21527]: cluster 2024-03-26T03:45:20.525580+0000 mon.a (mon.0) 783 : cluster [DBG] mgrmap e20: y(active, since 2s), standbys: x 2024-03-26T03:45:21.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:21 smithi175 bash[29284]: cluster 2024-03-26T03:45:20.469397+0000 mgr.y (mgr.24431) 9 : cluster [DBG] pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:45:21.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:21 smithi175 bash[29284]: cluster 2024-03-26T03:45:20.525580+0000 mon.a (mon.0) 783 : cluster [DBG] mgrmap e20: y(active, since 2s), standbys: x 2024-03-26T03:45:23.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:23 smithi175 bash[21527]: cluster 2024-03-26T03:45:22.470471+0000 mgr.y (mgr.24431) 10 : cluster [DBG] pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:45:23.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:23 smithi175 bash[21527]: cluster 2024-03-26T03:45:22.538062+0000 mon.a (mon.0) 784 : cluster [DBG] mgrmap e21: y(active, since 4s), standbys: x 2024-03-26T03:45:23.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:23 smithi190 bash[23392]: cluster 2024-03-26T03:45:22.470471+0000 mgr.y (mgr.24431) 10 : cluster [DBG] pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:45:23.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:23 smithi190 bash[23392]: cluster 2024-03-26T03:45:22.538062+0000 mon.a (mon.0) 784 : cluster [DBG] mgrmap e21: y(active, since 4s), standbys: x 2024-03-26T03:45:23.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:23 smithi175 bash[29284]: cluster 2024-03-26T03:45:22.470471+0000 mgr.y (mgr.24431) 10 : cluster [DBG] pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:45:23.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:23 smithi175 bash[29284]: cluster 2024-03-26T03:45:22.538062+0000 mon.a (mon.0) 784 : cluster [DBG] mgrmap e21: y(active, since 4s), standbys: x 2024-03-26T03:45:24.968 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T03:45:25.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:25 smithi190 bash[23392]: audit 2024-03-26T03:45:24.355707+0000 mon.a (mon.0) 785 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:25.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:25 smithi190 bash[23392]: audit 2024-03-26T03:45:24.361921+0000 mon.a (mon.0) 786 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:25.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:25 smithi190 bash[23392]: audit 2024-03-26T03:45:24.668129+0000 mon.a (mon.0) 787 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:25.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:25 smithi190 bash[23392]: audit 2024-03-26T03:45:24.682968+0000 mon.a (mon.0) 788 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:25.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:25 smithi175 bash[29284]: audit 2024-03-26T03:45:24.355707+0000 mon.a (mon.0) 785 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:25.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:25 smithi175 bash[29284]: audit 2024-03-26T03:45:24.361921+0000 mon.a (mon.0) 786 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:25.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:25 smithi175 bash[29284]: audit 2024-03-26T03:45:24.668129+0000 mon.a (mon.0) 787 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:25.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:25 smithi175 bash[29284]: audit 2024-03-26T03:45:24.682968+0000 mon.a (mon.0) 788 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:25.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:25 smithi175 bash[21527]: audit 2024-03-26T03:45:24.355707+0000 mon.a (mon.0) 785 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:25.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:25 smithi175 bash[21527]: audit 2024-03-26T03:45:24.361921+0000 mon.a (mon.0) 786 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:25.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:25 smithi175 bash[21527]: audit 2024-03-26T03:45:24.668129+0000 mon.a (mon.0) 787 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:25.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:25 smithi175 bash[21527]: audit 2024-03-26T03:45:24.682968+0000 mon.a (mon.0) 788 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:26.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[21527]: cluster 2024-03-26T03:45:24.471023+0000 mgr.y (mgr.24431) 11 : cluster [DBG] pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:45:26.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[21527]: audit 2024-03-26T03:45:25.371736+0000 mon.a (mon.0) 789 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:26.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[21527]: audit 2024-03-26T03:45:25.669656+0000 mon.a (mon.0) 790 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:26.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[21527]: audit 2024-03-26T03:45:25.673451+0000 mon.c (mon.2) 38 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[21527]: audit 2024-03-26T03:45:25.674190+0000 mon.a (mon.0) 791 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[21527]: audit 2024-03-26T03:45:25.676467+0000 mon.c (mon.2) 39 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[21527]: audit 2024-03-26T03:45:25.677145+0000 mon.a (mon.0) 792 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[21527]: audit 2024-03-26T03:45:25.678872+0000 mon.c (mon.2) 40 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[21527]: audit 2024-03-26T03:45:25.679622+0000 mon.a (mon.0) 793 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[21527]: audit 2024-03-26T03:45:25.681237+0000 mon.c (mon.2) 41 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[21527]: audit 2024-03-26T03:45:25.681837+0000 mon.a (mon.0) 794 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[21527]: audit 2024-03-26T03:45:25.690241+0000 mon.a (mon.0) 795 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:26.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[21527]: audit 2024-03-26T03:45:26.116105+0000 mon.a (mon.0) 796 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:26.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[21527]: audit 2024-03-26T03:45:26.131066+0000 mon.a (mon.0) 797 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:26.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[21527]: audit 2024-03-26T03:45:26.135588+0000 mon.c (mon.2) 42 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[21527]: audit 2024-03-26T03:45:26.136267+0000 mon.a (mon.0) 798 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[21527]: audit 2024-03-26T03:45:26.138598+0000 mon.c (mon.2) 43 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[21527]: audit 2024-03-26T03:45:26.139295+0000 mon.a (mon.0) 799 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[21527]: audit 2024-03-26T03:45:26.141439+0000 mon.c (mon.2) 44 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[21527]: audit 2024-03-26T03:45:26.142055+0000 mon.a (mon.0) 800 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[21527]: audit 2024-03-26T03:45:26.144029+0000 mon.c (mon.2) 45 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[21527]: audit 2024-03-26T03:45:26.144664+0000 mon.a (mon.0) 801 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[21527]: audit 2024-03-26T03:45:26.153486+0000 mon.a (mon.0) 802 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:26.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[21527]: audit 2024-03-26T03:45:26.158533+0000 mon.c (mon.2) 46 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:45:26.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[21527]: audit 2024-03-26T03:45:26.160354+0000 mon.c (mon.2) 47 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:45:26.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[29284]: cluster 2024-03-26T03:45:24.471023+0000 mgr.y (mgr.24431) 11 : cluster [DBG] pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:45:26.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[29284]: audit 2024-03-26T03:45:25.371736+0000 mon.a (mon.0) 789 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:26.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[29284]: audit 2024-03-26T03:45:25.669656+0000 mon.a (mon.0) 790 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:26.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[29284]: audit 2024-03-26T03:45:25.673451+0000 mon.c (mon.2) 38 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[29284]: audit 2024-03-26T03:45:25.674190+0000 mon.a (mon.0) 791 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[29284]: audit 2024-03-26T03:45:25.676467+0000 mon.c (mon.2) 39 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[29284]: audit 2024-03-26T03:45:25.677145+0000 mon.a (mon.0) 792 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[29284]: audit 2024-03-26T03:45:25.678872+0000 mon.c (mon.2) 40 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[29284]: audit 2024-03-26T03:45:25.679622+0000 mon.a (mon.0) 793 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[29284]: audit 2024-03-26T03:45:25.681237+0000 mon.c (mon.2) 41 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[29284]: audit 2024-03-26T03:45:25.681837+0000 mon.a (mon.0) 794 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[29284]: audit 2024-03-26T03:45:25.690241+0000 mon.a (mon.0) 795 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:26.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[29284]: audit 2024-03-26T03:45:26.116105+0000 mon.a (mon.0) 796 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:26.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[29284]: audit 2024-03-26T03:45:26.131066+0000 mon.a (mon.0) 797 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:26.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[29284]: audit 2024-03-26T03:45:26.135588+0000 mon.c (mon.2) 42 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[29284]: audit 2024-03-26T03:45:26.136267+0000 mon.a (mon.0) 798 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[29284]: audit 2024-03-26T03:45:26.138598+0000 mon.c (mon.2) 43 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[29284]: audit 2024-03-26T03:45:26.139295+0000 mon.a (mon.0) 799 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[29284]: audit 2024-03-26T03:45:26.141439+0000 mon.c (mon.2) 44 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[29284]: audit 2024-03-26T03:45:26.142055+0000 mon.a (mon.0) 800 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[29284]: audit 2024-03-26T03:45:26.144029+0000 mon.c (mon.2) 45 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[29284]: audit 2024-03-26T03:45:26.144664+0000 mon.a (mon.0) 801 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[29284]: audit 2024-03-26T03:45:26.153486+0000 mon.a (mon.0) 802 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:26.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[29284]: audit 2024-03-26T03:45:26.158533+0000 mon.c (mon.2) 46 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:45:26.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:26 smithi175 bash[29284]: audit 2024-03-26T03:45:26.160354+0000 mon.c (mon.2) 47 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:45:26.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:26 smithi190 bash[23392]: cluster 2024-03-26T03:45:24.471023+0000 mgr.y (mgr.24431) 11 : cluster [DBG] pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:45:26.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:26 smithi190 bash[23392]: audit 2024-03-26T03:45:25.371736+0000 mon.a (mon.0) 789 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:26.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:26 smithi190 bash[23392]: audit 2024-03-26T03:45:25.669656+0000 mon.a (mon.0) 790 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:26.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:26 smithi190 bash[23392]: audit 2024-03-26T03:45:25.673451+0000 mon.c (mon.2) 38 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:26 smithi190 bash[23392]: audit 2024-03-26T03:45:25.674190+0000 mon.a (mon.0) 791 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:26 smithi190 bash[23392]: audit 2024-03-26T03:45:25.676467+0000 mon.c (mon.2) 39 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:26 smithi190 bash[23392]: audit 2024-03-26T03:45:25.677145+0000 mon.a (mon.0) 792 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:26 smithi190 bash[23392]: audit 2024-03-26T03:45:25.678872+0000 mon.c (mon.2) 40 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:26 smithi190 bash[23392]: audit 2024-03-26T03:45:25.679622+0000 mon.a (mon.0) 793 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:26 smithi190 bash[23392]: audit 2024-03-26T03:45:25.681237+0000 mon.c (mon.2) 41 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:26 smithi190 bash[23392]: audit 2024-03-26T03:45:25.681837+0000 mon.a (mon.0) 794 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:26 smithi190 bash[23392]: audit 2024-03-26T03:45:25.690241+0000 mon.a (mon.0) 795 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:26.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:26 smithi190 bash[23392]: audit 2024-03-26T03:45:26.116105+0000 mon.a (mon.0) 796 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:26.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:26 smithi190 bash[23392]: audit 2024-03-26T03:45:26.131066+0000 mon.a (mon.0) 797 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:26.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:26 smithi190 bash[23392]: audit 2024-03-26T03:45:26.135588+0000 mon.c (mon.2) 42 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:26 smithi190 bash[23392]: audit 2024-03-26T03:45:26.136267+0000 mon.a (mon.0) 798 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:26 smithi190 bash[23392]: audit 2024-03-26T03:45:26.138598+0000 mon.c (mon.2) 43 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:26 smithi190 bash[23392]: audit 2024-03-26T03:45:26.139295+0000 mon.a (mon.0) 799 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:26 smithi190 bash[23392]: audit 2024-03-26T03:45:26.141439+0000 mon.c (mon.2) 44 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:26 smithi190 bash[23392]: audit 2024-03-26T03:45:26.142055+0000 mon.a (mon.0) 800 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:26 smithi190 bash[23392]: audit 2024-03-26T03:45:26.144029+0000 mon.c (mon.2) 45 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:26 smithi190 bash[23392]: audit 2024-03-26T03:45:26.144664+0000 mon.a (mon.0) 801 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:45:26.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:26 smithi190 bash[23392]: audit 2024-03-26T03:45:26.153486+0000 mon.a (mon.0) 802 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:26.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:26 smithi190 bash[23392]: audit 2024-03-26T03:45:26.158533+0000 mon.c (mon.2) 46 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:45:26.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:26 smithi190 bash[23392]: audit 2024-03-26T03:45:26.160354+0000 mon.c (mon.2) 47 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:45:27.601 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[21527]: cephadm 2024-03-26T03:45:25.683061+0000 mgr.y (mgr.24431) 12 : cephadm [INF] Adjusting osd_memory_target on smithi175 to 3804M 2024-03-26T03:45:27.601 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[21527]: cephadm 2024-03-26T03:45:26.146357+0000 mgr.y (mgr.24431) 13 : cephadm [INF] Adjusting osd_memory_target on smithi190 to 3804M 2024-03-26T03:45:27.602 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[21527]: cephadm 2024-03-26T03:45:26.163031+0000 mgr.y (mgr.24431) 14 : cephadm [INF] Updating smithi175:/etc/ceph/ceph.conf 2024-03-26T03:45:27.602 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[21527]: cephadm 2024-03-26T03:45:26.163565+0000 mgr.y (mgr.24431) 15 : cephadm [INF] Updating smithi190:/etc/ceph/ceph.conf 2024-03-26T03:45:27.602 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[21527]: cephadm 2024-03-26T03:45:26.379673+0000 mgr.y (mgr.24431) 16 : cephadm [INF] Updating smithi175:/var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T03:45:27.602 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[21527]: cephadm 2024-03-26T03:45:26.381242+0000 mgr.y (mgr.24431) 17 : cephadm [INF] Updating smithi190:/var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T03:45:27.602 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[21527]: cluster 2024-03-26T03:45:26.471692+0000 mgr.y (mgr.24431) 18 : cluster [DBG] pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-03-26T03:45:27.602 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[21527]: cephadm 2024-03-26T03:45:26.545118+0000 mgr.y (mgr.24431) 19 : cephadm [INF] Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-03-26T03:45:27.602 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[21527]: cephadm 2024-03-26T03:45:26.546274+0000 mgr.y (mgr.24431) 20 : cephadm [INF] Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-03-26T03:45:27.602 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[21527]: cephadm 2024-03-26T03:45:26.709802+0000 mgr.y (mgr.24431) 21 : cephadm [INF] Updating smithi175:/var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T03:45:27.602 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[21527]: cephadm 2024-03-26T03:45:26.710694+0000 mgr.y (mgr.24431) 22 : cephadm [INF] Updating smithi190:/var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T03:45:27.602 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[21527]: audit 2024-03-26T03:45:26.882726+0000 mon.a (mon.0) 803 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:27.602 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[21527]: audit 2024-03-26T03:45:26.888286+0000 mon.a (mon.0) 804 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:27.602 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[21527]: audit 2024-03-26T03:45:26.899771+0000 mon.a (mon.0) 805 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:27.603 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[21527]: audit 2024-03-26T03:45:26.905785+0000 mon.a (mon.0) 806 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:27.603 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[21527]: audit 2024-03-26T03:45:26.915965+0000 mon.a (mon.0) 807 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:27.603 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[21527]: cephadm 2024-03-26T03:45:26.922529+0000 mgr.y (mgr.24431) 23 : cephadm [INF] Deploying daemon node-exporter.a on smithi175 2024-03-26T03:45:27.603 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[29284]: cephadm 2024-03-26T03:45:25.683061+0000 mgr.y (mgr.24431) 12 : cephadm [INF] Adjusting osd_memory_target on smithi175 to 3804M 2024-03-26T03:45:27.603 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[29284]: cephadm 2024-03-26T03:45:26.146357+0000 mgr.y (mgr.24431) 13 : cephadm [INF] Adjusting osd_memory_target on smithi190 to 3804M 2024-03-26T03:45:27.603 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[29284]: cephadm 2024-03-26T03:45:26.163031+0000 mgr.y (mgr.24431) 14 : cephadm [INF] Updating smithi175:/etc/ceph/ceph.conf 2024-03-26T03:45:27.603 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[29284]: cephadm 2024-03-26T03:45:26.163565+0000 mgr.y (mgr.24431) 15 : cephadm [INF] Updating smithi190:/etc/ceph/ceph.conf 2024-03-26T03:45:27.603 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[29284]: cephadm 2024-03-26T03:45:26.379673+0000 mgr.y (mgr.24431) 16 : cephadm [INF] Updating smithi175:/var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T03:45:27.603 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[29284]: cephadm 2024-03-26T03:45:26.381242+0000 mgr.y (mgr.24431) 17 : cephadm [INF] Updating smithi190:/var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T03:45:27.603 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[29284]: cluster 2024-03-26T03:45:26.471692+0000 mgr.y (mgr.24431) 18 : cluster [DBG] pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-03-26T03:45:27.604 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[29284]: cephadm 2024-03-26T03:45:26.545118+0000 mgr.y (mgr.24431) 19 : cephadm [INF] Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-03-26T03:45:27.604 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[29284]: cephadm 2024-03-26T03:45:26.546274+0000 mgr.y (mgr.24431) 20 : cephadm [INF] Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-03-26T03:45:27.604 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[29284]: cephadm 2024-03-26T03:45:26.709802+0000 mgr.y (mgr.24431) 21 : cephadm [INF] Updating smithi175:/var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T03:45:27.604 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[29284]: cephadm 2024-03-26T03:45:26.710694+0000 mgr.y (mgr.24431) 22 : cephadm [INF] Updating smithi190:/var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T03:45:27.604 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[29284]: audit 2024-03-26T03:45:26.882726+0000 mon.a (mon.0) 803 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:27.604 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[29284]: audit 2024-03-26T03:45:26.888286+0000 mon.a (mon.0) 804 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:27.604 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[29284]: audit 2024-03-26T03:45:26.899771+0000 mon.a (mon.0) 805 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:27.604 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[29284]: audit 2024-03-26T03:45:26.905785+0000 mon.a (mon.0) 806 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:27.604 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[29284]: audit 2024-03-26T03:45:26.915965+0000 mon.a (mon.0) 807 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:27.604 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:27 smithi175 bash[29284]: cephadm 2024-03-26T03:45:26.922529+0000 mgr.y (mgr.24431) 23 : cephadm [INF] Deploying daemon node-exporter.a on smithi175 2024-03-26T03:45:27.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:27 smithi190 bash[23392]: cephadm 2024-03-26T03:45:25.683061+0000 mgr.y (mgr.24431) 12 : cephadm [INF] Adjusting osd_memory_target on smithi175 to 3804M 2024-03-26T03:45:27.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:27 smithi190 bash[23392]: cephadm 2024-03-26T03:45:26.146357+0000 mgr.y (mgr.24431) 13 : cephadm [INF] Adjusting osd_memory_target on smithi190 to 3804M 2024-03-26T03:45:27.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:27 smithi190 bash[23392]: cephadm 2024-03-26T03:45:26.163031+0000 mgr.y (mgr.24431) 14 : cephadm [INF] Updating smithi175:/etc/ceph/ceph.conf 2024-03-26T03:45:27.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:27 smithi190 bash[23392]: cephadm 2024-03-26T03:45:26.163565+0000 mgr.y (mgr.24431) 15 : cephadm [INF] Updating smithi190:/etc/ceph/ceph.conf 2024-03-26T03:45:27.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:27 smithi190 bash[23392]: cephadm 2024-03-26T03:45:26.379673+0000 mgr.y (mgr.24431) 16 : cephadm [INF] Updating smithi175:/var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T03:45:27.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:27 smithi190 bash[23392]: cephadm 2024-03-26T03:45:26.381242+0000 mgr.y (mgr.24431) 17 : cephadm [INF] Updating smithi190:/var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T03:45:27.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:27 smithi190 bash[23392]: cluster 2024-03-26T03:45:26.471692+0000 mgr.y (mgr.24431) 18 : cluster [DBG] pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-03-26T03:45:27.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:27 smithi190 bash[23392]: cephadm 2024-03-26T03:45:26.545118+0000 mgr.y (mgr.24431) 19 : cephadm [INF] Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-03-26T03:45:27.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:27 smithi190 bash[23392]: cephadm 2024-03-26T03:45:26.546274+0000 mgr.y (mgr.24431) 20 : cephadm [INF] Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2024-03-26T03:45:27.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:27 smithi190 bash[23392]: cephadm 2024-03-26T03:45:26.709802+0000 mgr.y (mgr.24431) 21 : cephadm [INF] Updating smithi175:/var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T03:45:27.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:27 smithi190 bash[23392]: cephadm 2024-03-26T03:45:26.710694+0000 mgr.y (mgr.24431) 22 : cephadm [INF] Updating smithi190:/var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T03:45:27.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:27 smithi190 bash[23392]: audit 2024-03-26T03:45:26.882726+0000 mon.a (mon.0) 803 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:27.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:27 smithi190 bash[23392]: audit 2024-03-26T03:45:26.888286+0000 mon.a (mon.0) 804 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:27.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:27 smithi190 bash[23392]: audit 2024-03-26T03:45:26.899771+0000 mon.a (mon.0) 805 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:27.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:27 smithi190 bash[23392]: audit 2024-03-26T03:45:26.905785+0000 mon.a (mon.0) 806 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:27.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:27 smithi190 bash[23392]: audit 2024-03-26T03:45:26.915965+0000 mon.a (mon.0) 807 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:27.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:27 smithi190 bash[23392]: cephadm 2024-03-26T03:45:26.922529+0000 mgr.y (mgr.24431) 23 : cephadm [INF] Deploying daemon node-exporter.a on smithi175 2024-03-26T03:45:27.866 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:27 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:27.866 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:27 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:27.867 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 26 03:45:27 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:27.867 INFO:journalctl@ceph.osd.3.smithi175.stdout:Mar 26 03:45:27 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:27.867 INFO:journalctl@ceph.rgw.foo.a.smithi175.stdout:Mar 26 03:45:27 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:27.867 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:27 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:27.868 INFO:journalctl@ceph.osd.0.smithi175.stdout:Mar 26 03:45:27 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:27.868 INFO:journalctl@ceph.osd.1.smithi175.stdout:Mar 26 03:45:27 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:28.150 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.152+0000 7f1f30be7700 1 -- 172.21.15.190:0/601107092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f2c103080 msgr2=0x7f1f2c10f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:45:28.150 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.152+0000 7f1f30be7700 1 --2- 172.21.15.190:0/601107092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f2c103080 0x7f1f2c10f510 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1f2400b0a0 tx=0x7f1f2400b3b0 comp rx=0 tx=0).stop 2024-03-26T03:45:28.150 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.152+0000 7f1f30be7700 1 -- 172.21.15.190:0/601107092 shutdown_connections 2024-03-26T03:45:28.150 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.152+0000 7f1f30be7700 1 --2- 172.21.15.190:0/601107092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f2c103080 0x7f1f2c10f510 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:28.150 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.152+0000 7f1f30be7700 1 --2- 172.21.15.190:0/601107092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f2c1086b0 0x7f1f2c108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:28.150 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.152+0000 7f1f30be7700 1 --2- 172.21.15.190:0/601107092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f2c1026c0 0x7f1f2c102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:28.151 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.156+0000 7f1f30be7700 1 -- 172.21.15.190:0/601107092 >> 172.21.15.190:0/601107092 conn(0x7f1f2c0797f0 msgr2=0x7f1f2c079c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:45:28.151 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.156+0000 7f1f30be7700 1 -- 172.21.15.190:0/601107092 shutdown_connections 2024-03-26T03:45:28.151 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.156+0000 7f1f30be7700 1 -- 172.21.15.190:0/601107092 wait complete. 2024-03-26T03:45:28.151 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.156+0000 7f1f30be7700 1 Processor -- start 2024-03-26T03:45:28.151 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.156+0000 7f1f30be7700 1 -- start start 2024-03-26T03:45:28.152 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.156+0000 7f1f30be7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f2c1026c0 0x7f1f2c198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:45:28.152 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.156+0000 7f1f30be7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f2c103080 0x7f1f2c1987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:45:28.152 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.156+0000 7f1f30be7700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f2c1086b0 0x7f1f2c19cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:45:28.152 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.156+0000 7f1f30be7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2c111ec0 con 0x7f1f2c1026c0 2024-03-26T03:45:28.152 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.156+0000 7f1f30be7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2c111d40 con 0x7f1f2c103080 2024-03-26T03:45:28.152 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.156+0000 7f1f30be7700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1f2c112040 con 0x7f1f2c1086b0 2024-03-26T03:45:28.152 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.156+0000 7f1f29d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f2c103080 0x7f1f2c1987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:45:28.153 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.156+0000 7f1f29d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f2c103080 0x7f1f2c1987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:36758/0 (socket says 172.21.15.190:36758) 2024-03-26T03:45:28.153 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.156+0000 7f1f29d9b700 1 -- 172.21.15.190:0/1315808201 learned_addr learned my addr 172.21.15.190:0/1315808201 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-03-26T03:45:28.153 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.156+0000 7f1f2a59c700 1 --2- 172.21.15.190:0/1315808201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f2c1026c0 0x7f1f2c198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:45:28.153 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.156+0000 7f1f2ad9d700 1 --2- 172.21.15.190:0/1315808201 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f2c1086b0 0x7f1f2c19cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:45:28.153 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.156+0000 7f1f29d9b700 1 -- 172.21.15.190:0/1315808201 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f2c1086b0 msgr2=0x7f1f2c19cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:45:28.153 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.156+0000 7f1f29d9b700 1 --2- 172.21.15.190:0/1315808201 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f2c1086b0 0x7f1f2c19cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:28.153 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.156+0000 7f1f29d9b700 1 -- 172.21.15.190:0/1315808201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f2c1026c0 msgr2=0x7f1f2c198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:45:28.153 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.156+0000 7f1f29d9b700 1 --2- 172.21.15.190:0/1315808201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f2c1026c0 0x7f1f2c198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:28.153 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.156+0000 7f1f29d9b700 1 -- 172.21.15.190:0/1315808201 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f2c19d2a0 con 0x7f1f2c103080 2024-03-26T03:45:28.153 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.156+0000 7f1f2ad9d700 1 --2- 172.21.15.190:0/1315808201 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f2c1086b0 0x7f1f2c19cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T03:45:28.153 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.156+0000 7f1f29d9b700 1 --2- 172.21.15.190:0/1315808201 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f2c103080 0x7f1f2c1987d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c00d980 tx=0x7f1f1c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:45:28.153 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.156+0000 7f1f1b7fe700 1 -- 172.21.15.190:0/1315808201 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c017070 con 0x7f1f2c103080 2024-03-26T03:45:28.153 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.156+0000 7f1f1b7fe700 1 -- 172.21.15.190:0/1315808201 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f1f1c005020 con 0x7f1f2c103080 2024-03-26T03:45:28.153 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.156+0000 7f1f30be7700 1 -- 172.21.15.190:0/1315808201 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f2c19d590 con 0x7f1f2c103080 2024-03-26T03:45:28.153 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.156+0000 7f1f1b7fe700 1 -- 172.21.15.190:0/1315808201 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c009a30 con 0x7f1f2c103080 2024-03-26T03:45:28.154 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.156+0000 7f1f30be7700 1 -- 172.21.15.190:0/1315808201 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f2c1a4e10 con 0x7f1f2c103080 2024-03-26T03:45:28.154 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.156+0000 7f1f30be7700 1 -- 172.21.15.190:0/1315808201 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f2c050300 con 0x7f1f2c103080 2024-03-26T03:45:28.155 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.156+0000 7f1f1b7fe700 1 -- 172.21.15.190:0/1315808201 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7f1f1c009bd0 con 0x7f1f2c103080 2024-03-26T03:45:28.155 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.160+0000 7f1f1b7fe700 1 --2- 172.21.15.190:0/1315808201 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f1f1406e7e0 0x7f1f14070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:45:28.155 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.160+0000 7f1f1b7fe700 1 -- 172.21.15.190:0/1315808201 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7537+0+0 (secure 0 0 0) 0x7f1f1c08edb0 con 0x7f1f2c103080 2024-03-26T03:45:28.155 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.160+0000 7f1f2a59c700 1 --2- 172.21.15.190:0/1315808201 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f1f1406e7e0 0x7f1f14070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:45:28.155 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.160+0000 7f1f2a59c700 1 --2- 172.21.15.190:0/1315808201 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f1f1406e7e0 0x7f1f14070ca0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1f2000abd0 tx=0x7f1f2000a380 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:45:28.159 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.164+0000 7f1f1b7fe700 1 -- 172.21.15.190:0/1315808201 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f1f1c020020 con 0x7f1f2c103080 2024-03-26T03:45:28.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:27 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:28.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:27 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:28.242 INFO:journalctl@ceph.osd.0.smithi175.stdout:Mar 26 03:45:27 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:28.242 INFO:journalctl@ceph.osd.1.smithi175.stdout:Mar 26 03:45:27 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:28.242 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 26 03:45:27 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:28.243 INFO:journalctl@ceph.rgw.foo.a.smithi175.stdout:Mar 26 03:45:27 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:28.243 INFO:journalctl@ceph.osd.3.smithi175.stdout:Mar 26 03:45:27 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:28.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:27 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:28.244 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:27 smithi175 systemd[1]: Started Ceph node-exporter.a for 762e930c-eb21-11ee-95d0-87774f69a715. 2024-03-26T03:45:28.244 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:28 smithi175 bash[57876]: Unable to find image 'quay.io/prometheus/node-exporter:v1.5.0' locally 2024-03-26T03:45:28.386 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.392+0000 7f1f30be7700 1 -- 172.21.15.190:0/1315808201 --> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi190=a", "target": ["mon-mgr", ""]}) -- 0x7f1f2c1995f0 con 0x7f1f1406e7e0 2024-03-26T03:45:28.402 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.408+0000 7f1f1b7fe700 1 -- 172.21.15.190:0/1315808201 <== mgr.24431 v2:172.21.15.175:6800/1140903233 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f1f2c1995f0 con 0x7f1f1406e7e0 2024-03-26T03:45:28.402 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled grafana update... 2024-03-26T03:45:28.406 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.412+0000 7f1f30be7700 1 -- 172.21.15.190:0/1315808201 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f1f1406e7e0 msgr2=0x7f1f14070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:45:28.406 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.412+0000 7f1f30be7700 1 --2- 172.21.15.190:0/1315808201 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f1f1406e7e0 0x7f1f14070ca0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1f2000abd0 tx=0x7f1f2000a380 comp rx=0 tx=0).stop 2024-03-26T03:45:28.406 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.412+0000 7f1f30be7700 1 -- 172.21.15.190:0/1315808201 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f2c103080 msgr2=0x7f1f2c1987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:45:28.406 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.412+0000 7f1f30be7700 1 --2- 172.21.15.190:0/1315808201 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f2c103080 0x7f1f2c1987d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c00d980 tx=0x7f1f1c00dc90 comp rx=0 tx=0).stop 2024-03-26T03:45:28.406 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.412+0000 7f1f30be7700 1 -- 172.21.15.190:0/1315808201 shutdown_connections 2024-03-26T03:45:28.406 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.412+0000 7f1f30be7700 1 --2- 172.21.15.190:0/1315808201 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f1f1406e7e0 0x7f1f14070ca0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:28.406 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.412+0000 7f1f30be7700 1 --2- 172.21.15.190:0/1315808201 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f2c1086b0 0x7f1f2c19cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:28.406 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.412+0000 7f1f30be7700 1 --2- 172.21.15.190:0/1315808201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f2c1026c0 0x7f1f2c198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:28.406 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.412+0000 7f1f30be7700 1 --2- 172.21.15.190:0/1315808201 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f2c103080 0x7f1f2c1987d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:28.406 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.412+0000 7f1f30be7700 1 -- 172.21.15.190:0/1315808201 >> 172.21.15.190:0/1315808201 conn(0x7f1f2c0797f0 msgr2=0x7f1f2c1012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:45:28.406 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.412+0000 7f1f30be7700 1 -- 172.21.15.190:0/1315808201 shutdown_connections 2024-03-26T03:45:28.407 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:28.412+0000 7f1f30be7700 1 -- 172.21.15.190:0/1315808201 wait complete. 2024-03-26T03:45:28.804 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:28 smithi175 bash[57876]: v1.5.0: Pulling from prometheus/node-exporter 2024-03-26T03:45:29.065 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:28 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:29.066 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 03:45:28 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:29.066 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:45:28 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:29.066 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 03:45:28 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:29.067 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 03:45:28 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:29.067 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 03:45:28 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:29.067 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:45:28 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:29.067 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:28 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:29.068 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:29 smithi190 bash[23392]: audit 2024-03-26T03:45:28.029580+0000 mon.a (mon.0) 808 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:29.068 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:29 smithi190 bash[23392]: audit 2024-03-26T03:45:28.035843+0000 mon.a (mon.0) 809 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:29.068 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:29 smithi190 bash[23392]: audit 2024-03-26T03:45:28.041598+0000 mon.a (mon.0) 810 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:29.068 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:29 smithi190 bash[23392]: cephadm 2024-03-26T03:45:28.042599+0000 mgr.y (mgr.24431) 24 : cephadm [INF] Deploying daemon node-exporter.b on smithi190 2024-03-26T03:45:29.068 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:29 smithi190 bash[23392]: audit 2024-03-26T03:45:28.404336+0000 mon.a (mon.0) 811 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:29.072 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:29 smithi175 bash[21527]: audit 2024-03-26T03:45:28.029580+0000 mon.a (mon.0) 808 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:29.072 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:29 smithi175 bash[21527]: audit 2024-03-26T03:45:28.035843+0000 mon.a (mon.0) 809 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:29.072 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:29 smithi175 bash[21527]: audit 2024-03-26T03:45:28.041598+0000 mon.a (mon.0) 810 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:29.073 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:29 smithi175 bash[21527]: cephadm 2024-03-26T03:45:28.042599+0000 mgr.y (mgr.24431) 24 : cephadm [INF] Deploying daemon node-exporter.b on smithi190 2024-03-26T03:45:29.073 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:29 smithi175 bash[21527]: audit 2024-03-26T03:45:28.404336+0000 mon.a (mon.0) 811 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:29.073 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:29 smithi175 bash[29284]: audit 2024-03-26T03:45:28.029580+0000 mon.a (mon.0) 808 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:29.073 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:29 smithi175 bash[29284]: audit 2024-03-26T03:45:28.035843+0000 mon.a (mon.0) 809 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:29.073 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:29 smithi175 bash[29284]: audit 2024-03-26T03:45:28.041598+0000 mon.a (mon.0) 810 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:29.073 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:29 smithi175 bash[29284]: cephadm 2024-03-26T03:45:28.042599+0000 mgr.y (mgr.24431) 24 : cephadm [INF] Deploying daemon node-exporter.b on smithi190 2024-03-26T03:45:29.073 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:29 smithi175 bash[29284]: audit 2024-03-26T03:45:28.404336+0000 mon.a (mon.0) 811 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:29.074 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:28 smithi175 bash[57876]: 22b70bddd3ac: Pulling fs layer 2024-03-26T03:45:29.074 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:28 smithi175 bash[57876]: 5c12815fee55: Pulling fs layer 2024-03-26T03:45:29.074 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:28 smithi175 bash[57876]: c0e87333d380: Pulling fs layer 2024-03-26T03:45:29.074 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:28 smithi175 bash[57876]: 5c12815fee55: Verifying Checksum 2024-03-26T03:45:29.074 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:28 smithi175 bash[57876]: 5c12815fee55: Download complete 2024-03-26T03:45:29.074 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:28 smithi175 bash[57876]: 22b70bddd3ac: Verifying Checksum 2024-03-26T03:45:29.074 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:28 smithi175 bash[57876]: 22b70bddd3ac: Download complete 2024-03-26T03:45:29.387 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:29 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:29.387 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 03:45:29 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:29.387 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 03:45:29 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:29.388 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 03:45:29 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:29.388 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:45:29 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:29.388 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:45:29 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:29.389 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:29 smithi190 systemd[1]: Started Ceph node-exporter.b for 762e930c-eb21-11ee-95d0-87774f69a715. 2024-03-26T03:45:29.389 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:29 smithi190 bash[49947]: Unable to find image 'quay.io/prometheus/node-exporter:v1.5.0' locally 2024-03-26T03:45:29.389 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 03:45:29 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:29.389 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:45:29 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:29.390 DEBUG:teuthology.orchestra.run.smithi190:grafana.a> sudo journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@grafana.a.service 2024-03-26T03:45:29.392 INFO:tasks.cephadm:Setting up client nodes... 2024-03-26T03:45:29.392 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-26T03:45:29.402 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:29 smithi175 bash[57876]: c0e87333d380: Verifying Checksum 2024-03-26T03:45:29.402 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:29 smithi175 bash[57876]: c0e87333d380: Download complete 2024-03-26T03:45:29.402 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:29 smithi175 bash[57876]: 22b70bddd3ac: Pull complete 2024-03-26T03:45:29.741 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:29 smithi175 bash[57876]: 5c12815fee55: Pull complete 2024-03-26T03:45:29.741 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:45:29 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:45:29.935 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:45:29 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:30.211 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:29 smithi175 bash[57876]: c0e87333d380: Pull complete 2024-03-26T03:45:30.211 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:30 smithi175 bash[57876]: Digest: sha256:39c642b2b337e38c18e80266fb14383754178202f40103646337722a594d984c 2024-03-26T03:45:30.232 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:29 smithi190 bash[49947]: v1.5.0: Pulling from prometheus/node-exporter 2024-03-26T03:45:30.232 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:30 smithi190 bash[49947]: 22b70bddd3ac: Pulling fs layer 2024-03-26T03:45:30.232 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:30 smithi190 bash[49947]: 5c12815fee55: Pulling fs layer 2024-03-26T03:45:30.232 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:30 smithi190 bash[49947]: c0e87333d380: Pulling fs layer 2024-03-26T03:45:30.491 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:30 smithi190 bash[49947]: 5c12815fee55: Download complete 2024-03-26T03:45:30.491 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:30 smithi190 bash[49947]: 22b70bddd3ac: Verifying Checksum 2024-03-26T03:45:30.492 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:30 smithi190 bash[49947]: 22b70bddd3ac: Download complete 2024-03-26T03:45:30.492 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:30 smithi190 bash[49947]: c0e87333d380: Download complete 2024-03-26T03:45:30.492 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:30 smithi190 bash[49947]: 22b70bddd3ac: Pull complete 2024-03-26T03:45:30.492 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:30 smithi175 bash[57876]: Status: Downloaded newer image for quay.io/prometheus/node-exporter:v1.5.0 2024-03-26T03:45:30.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:30 smithi190 bash[23392]: audit 2024-03-26T03:45:28.391745+0000 mgr.y (mgr.24431) 25 : audit [DBG] from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi190=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:45:30.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:30 smithi190 bash[23392]: cephadm 2024-03-26T03:45:28.396008+0000 mgr.y (mgr.24431) 26 : cephadm [INF] Saving service grafana spec with placement smithi190=a;count:1 2024-03-26T03:45:30.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:30 smithi190 bash[23392]: cluster 2024-03-26T03:45:28.472311+0000 mgr.y (mgr.24431) 27 : cluster [DBG] pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-03-26T03:45:30.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:30 smithi190 bash[23392]: audit 2024-03-26T03:45:29.234609+0000 mon.a (mon.0) 812 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:30.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:30 smithi190 bash[23392]: audit 2024-03-26T03:45:29.247120+0000 mon.a (mon.0) 813 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:30.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:30 smithi190 bash[23392]: audit 2024-03-26T03:45:29.268084+0000 mon.a (mon.0) 814 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:30.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:30 smithi190 bash[23392]: audit 2024-03-26T03:45:29.279817+0000 mon.a (mon.0) 815 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:30.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:30 smithi190 bash[23392]: audit 2024-03-26T03:45:29.290863+0000 mon.a (mon.0) 816 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:30.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:30 smithi190 bash[23392]: cephadm 2024-03-26T03:45:29.307276+0000 mgr.y (mgr.24431) 28 : cephadm [INF] Deploying daemon alertmanager.a on smithi175 2024-03-26T03:45:30.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:30 smithi175 bash[21527]: audit 2024-03-26T03:45:28.391745+0000 mgr.y (mgr.24431) 25 : audit [DBG] from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi190=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:45:30.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:30 smithi175 bash[21527]: cephadm 2024-03-26T03:45:28.396008+0000 mgr.y (mgr.24431) 26 : cephadm [INF] Saving service grafana spec with placement smithi190=a;count:1 2024-03-26T03:45:30.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:30 smithi175 bash[21527]: cluster 2024-03-26T03:45:28.472311+0000 mgr.y (mgr.24431) 27 : cluster [DBG] pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-03-26T03:45:30.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:30 smithi175 bash[21527]: audit 2024-03-26T03:45:29.234609+0000 mon.a (mon.0) 812 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:30.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:30 smithi175 bash[21527]: audit 2024-03-26T03:45:29.247120+0000 mon.a (mon.0) 813 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:30.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:30 smithi175 bash[21527]: audit 2024-03-26T03:45:29.268084+0000 mon.a (mon.0) 814 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:30.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:30 smithi175 bash[21527]: audit 2024-03-26T03:45:29.279817+0000 mon.a (mon.0) 815 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:30.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:30 smithi175 bash[21527]: audit 2024-03-26T03:45:29.290863+0000 mon.a (mon.0) 816 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:30.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:30 smithi175 bash[21527]: cephadm 2024-03-26T03:45:29.307276+0000 mgr.y (mgr.24431) 28 : cephadm [INF] Deploying daemon alertmanager.a on smithi175 2024-03-26T03:45:30.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:30 smithi175 bash[29284]: audit 2024-03-26T03:45:28.391745+0000 mgr.y (mgr.24431) 25 : audit [DBG] from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi190=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T03:45:30.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:30 smithi175 bash[29284]: cephadm 2024-03-26T03:45:28.396008+0000 mgr.y (mgr.24431) 26 : cephadm [INF] Saving service grafana spec with placement smithi190=a;count:1 2024-03-26T03:45:30.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:30 smithi175 bash[29284]: cluster 2024-03-26T03:45:28.472311+0000 mgr.y (mgr.24431) 27 : cluster [DBG] pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-03-26T03:45:30.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:30 smithi175 bash[29284]: audit 2024-03-26T03:45:29.234609+0000 mon.a (mon.0) 812 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:30.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:30 smithi175 bash[29284]: audit 2024-03-26T03:45:29.247120+0000 mon.a (mon.0) 813 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:30.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:30 smithi175 bash[29284]: audit 2024-03-26T03:45:29.268084+0000 mon.a (mon.0) 814 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:30.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:30 smithi175 bash[29284]: audit 2024-03-26T03:45:29.279817+0000 mon.a (mon.0) 815 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:30.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:30 smithi175 bash[29284]: audit 2024-03-26T03:45:29.290863+0000 mon.a (mon.0) 816 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:30.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:30 smithi175 bash[29284]: cephadm 2024-03-26T03:45:29.307276+0000 mgr.y (mgr.24431) 28 : cephadm [INF] Deploying daemon alertmanager.a on smithi175 2024-03-26T03:45:31.383 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:31 smithi190 bash[23392]: audit 2024-03-26T03:45:29.482218+0000 mgr.y (mgr.24431) 29 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:45:31.384 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:31 smithi190 bash[49947]: 5c12815fee55: Pull complete 2024-03-26T03:45:31.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:31 smithi175 bash[29284]: audit 2024-03-26T03:45:29.482218+0000 mgr.y (mgr.24431) 29 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:45:31.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:31 smithi175 bash[21527]: audit 2024-03-26T03:45:29.482218+0000 mgr.y (mgr.24431) 29 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:45:31.651 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:31 smithi190 bash[49947]: c0e87333d380: Pull complete 2024-03-26T03:45:31.652 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:31 smithi190 bash[49947]: Digest: sha256:39c642b2b337e38c18e80266fb14383754178202f40103646337722a594d984c 2024-03-26T03:45:31.991 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:31 smithi190 bash[49947]: Status: Downloaded newer image for quay.io/prometheus/node-exporter:v1.5.0 2024-03-26T03:45:32.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:32 smithi190 bash[23392]: cluster 2024-03-26T03:45:30.473804+0000 mgr.y (mgr.24431) 30 : cluster [DBG] pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-26T03:45:32.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[21527]: cluster 2024-03-26T03:45:30.473804+0000 mgr.y (mgr.24431) 30 : cluster [DBG] pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-26T03:45:32.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[29284]: cluster 2024-03-26T03:45:30.473804+0000 mgr.y (mgr.24431) 30 : cluster [DBG] pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-26T03:45:32.742 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.382Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-03-26T03:45:32.742 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.383Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-03-26T03:45:32.742 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-03-26T03:45:32.742 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-03-26T03:45:32.742 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-03-26T03:45:32.743 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-03-26T03:45:32.743 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-03-26T03:45:32.743 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=arp 2024-03-26T03:45:32.743 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=bcache 2024-03-26T03:45:32.743 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=bonding 2024-03-26T03:45:32.743 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=btrfs 2024-03-26T03:45:32.743 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=conntrack 2024-03-26T03:45:32.743 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=cpu 2024-03-26T03:45:32.743 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-03-26T03:45:32.743 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=diskstats 2024-03-26T03:45:32.743 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=dmi 2024-03-26T03:45:32.743 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=edac 2024-03-26T03:45:32.743 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=entropy 2024-03-26T03:45:32.743 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-03-26T03:45:32.743 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=filefd 2024-03-26T03:45:32.744 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=filesystem 2024-03-26T03:45:32.744 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=hwmon 2024-03-26T03:45:32.744 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=infiniband 2024-03-26T03:45:32.744 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=ipvs 2024-03-26T03:45:32.744 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=loadavg 2024-03-26T03:45:32.744 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=mdadm 2024-03-26T03:45:32.744 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=meminfo 2024-03-26T03:45:32.744 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=netclass 2024-03-26T03:45:32.744 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=netdev 2024-03-26T03:45:32.744 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=netstat 2024-03-26T03:45:32.744 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=nfs 2024-03-26T03:45:32.744 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=nfsd 2024-03-26T03:45:32.744 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=nvme 2024-03-26T03:45:32.744 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=os 2024-03-26T03:45:32.744 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-03-26T03:45:32.744 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=pressure 2024-03-26T03:45:32.744 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=rapl 2024-03-26T03:45:32.745 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=schedstat 2024-03-26T03:45:32.745 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=selinux 2024-03-26T03:45:32.745 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=sockstat 2024-03-26T03:45:32.745 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=softnet 2024-03-26T03:45:32.745 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=stat 2024-03-26T03:45:32.745 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=tapestats 2024-03-26T03:45:32.745 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=textfile 2024-03-26T03:45:32.745 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-03-26T03:45:32.745 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=time 2024-03-26T03:45:32.745 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-03-26T03:45:32.745 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=uname 2024-03-26T03:45:32.745 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=vmstat 2024-03-26T03:45:32.745 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=xfs 2024-03-26T03:45:32.745 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.384Z caller=node_exporter.go:117 level=info collector=zfs 2024-03-26T03:45:32.745 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.385Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-03-26T03:45:32.745 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:32 smithi175 bash[57876]: ts=2024-03-26T03:45:32.385Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-03-26T03:45:33.492 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.197Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-03-26T03:45:33.492 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.198Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-03-26T03:45:33.492 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.198Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-03-26T03:45:33.492 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.198Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-03-26T03:45:33.492 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-03-26T03:45:33.492 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-03-26T03:45:33.492 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-03-26T03:45:33.492 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=arp 2024-03-26T03:45:33.492 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=bcache 2024-03-26T03:45:33.492 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=bonding 2024-03-26T03:45:33.493 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=btrfs 2024-03-26T03:45:33.493 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=conntrack 2024-03-26T03:45:33.493 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=cpu 2024-03-26T03:45:33.493 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-03-26T03:45:33.493 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=diskstats 2024-03-26T03:45:33.493 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=dmi 2024-03-26T03:45:33.493 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=edac 2024-03-26T03:45:33.493 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=entropy 2024-03-26T03:45:33.493 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-03-26T03:45:33.493 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=filefd 2024-03-26T03:45:33.493 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=filesystem 2024-03-26T03:45:33.493 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=hwmon 2024-03-26T03:45:33.493 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=infiniband 2024-03-26T03:45:33.493 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=ipvs 2024-03-26T03:45:33.494 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=loadavg 2024-03-26T03:45:33.494 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=mdadm 2024-03-26T03:45:33.494 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=meminfo 2024-03-26T03:45:33.494 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=netclass 2024-03-26T03:45:33.494 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=netdev 2024-03-26T03:45:33.494 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=netstat 2024-03-26T03:45:33.494 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=nfs 2024-03-26T03:45:33.494 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=nfsd 2024-03-26T03:45:33.494 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=nvme 2024-03-26T03:45:33.494 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=os 2024-03-26T03:45:33.495 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-03-26T03:45:33.495 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=pressure 2024-03-26T03:45:33.495 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=rapl 2024-03-26T03:45:33.495 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=schedstat 2024-03-26T03:45:33.495 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=selinux 2024-03-26T03:45:33.495 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=sockstat 2024-03-26T03:45:33.495 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=softnet 2024-03-26T03:45:33.495 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=stat 2024-03-26T03:45:33.495 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=tapestats 2024-03-26T03:45:33.495 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=textfile 2024-03-26T03:45:33.495 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-03-26T03:45:33.495 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=time 2024-03-26T03:45:33.495 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-03-26T03:45:33.495 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=uname 2024-03-26T03:45:33.495 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=vmstat 2024-03-26T03:45:33.495 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=xfs 2024-03-26T03:45:33.496 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.199Z caller=node_exporter.go:117 level=info collector=zfs 2024-03-26T03:45:33.496 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.200Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-03-26T03:45:33.496 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:45:33 smithi190 bash[49947]: ts=2024-03-26T03:45:33.200Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-03-26T03:45:34.148 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:45:34.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:34 smithi175 bash[21527]: cluster 2024-03-26T03:45:32.474257+0000 mgr.y (mgr.24431) 31 : cluster [DBG] pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-26T03:45:34.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:34 smithi175 bash[21527]: audit 2024-03-26T03:45:33.579151+0000 mon.a (mon.0) 817 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:34.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:34 smithi175 bash[21527]: audit 2024-03-26T03:45:33.604986+0000 mon.c (mon.2) 48 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:45:34.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:34 smithi175 bash[29284]: cluster 2024-03-26T03:45:32.474257+0000 mgr.y (mgr.24431) 31 : cluster [DBG] pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-26T03:45:34.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:34 smithi175 bash[29284]: audit 2024-03-26T03:45:33.579151+0000 mon.a (mon.0) 817 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:34.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:34 smithi175 bash[29284]: audit 2024-03-26T03:45:33.604986+0000 mon.c (mon.2) 48 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:45:34.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:34 smithi190 bash[23392]: cluster 2024-03-26T03:45:32.474257+0000 mgr.y (mgr.24431) 31 : cluster [DBG] pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-26T03:45:34.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:34 smithi190 bash[23392]: audit 2024-03-26T03:45:33.579151+0000 mon.a (mon.0) 817 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:34.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:34 smithi190 bash[23392]: audit 2024-03-26T03:45:33.604986+0000 mon.c (mon.2) 48 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:45:36.735 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:36 smithi175 bash[21527]: cluster 2024-03-26T03:45:34.475481+0000 mgr.y (mgr.24431) 32 : cluster [DBG] pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-26T03:45:36.735 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:36 smithi175 bash[29284]: cluster 2024-03-26T03:45:34.475481+0000 mgr.y (mgr.24431) 32 : cluster [DBG] pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-26T03:45:36.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:36 smithi190 bash[23392]: cluster 2024-03-26T03:45:34.475481+0000 mgr.y (mgr.24431) 32 : cluster [DBG] pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-26T03:45:38.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:38 smithi175 bash[29284]: cluster 2024-03-26T03:45:36.476550+0000 mgr.y (mgr.24431) 33 : cluster [DBG] pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-26T03:45:38.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:38 smithi175 bash[21527]: cluster 2024-03-26T03:45:36.476550+0000 mgr.y (mgr.24431) 33 : cluster [DBG] pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-26T03:45:38.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:38 smithi190 bash[23392]: cluster 2024-03-26T03:45:36.476550+0000 mgr.y (mgr.24431) 33 : cluster [DBG] pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-26T03:45:39.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.537+0000 7f79ada91700 1 -- 172.21.15.175:0/455754121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79a80749e0 msgr2=0x7f79a8074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:45:39.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.537+0000 7f79ada91700 1 --2- 172.21.15.175:0/455754121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79a80749e0 0x7f79a8074dc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f799c009990 tx=0x7f799c009ca0 comp rx=0 tx=0).stop 2024-03-26T03:45:39.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.537+0000 7f79ada91700 1 -- 172.21.15.175:0/455754121 shutdown_connections 2024-03-26T03:45:39.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.537+0000 7f79ada91700 1 --2- 172.21.15.175:0/455754121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79a80749e0 0x7f79a8074dc0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:39.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.537+0000 7f79ada91700 1 --2- 172.21.15.175:0/455754121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79a8113680 0x7f79a8115a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:39.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.537+0000 7f79ada91700 1 --2- 172.21.15.175:0/455754121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79a8075300 0x7f79a8113140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:39.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.537+0000 7f79ada91700 1 -- 172.21.15.175:0/455754121 >> 172.21.15.175:0/455754121 conn(0x7f79a80fe140 msgr2=0x7f79a8100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:45:39.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.537+0000 7f79ada91700 1 -- 172.21.15.175:0/455754121 shutdown_connections 2024-03-26T03:45:39.538 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.537+0000 7f79ada91700 1 -- 172.21.15.175:0/455754121 wait complete. 2024-03-26T03:45:39.540 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.537+0000 7f79ada91700 1 Processor -- start 2024-03-26T03:45:39.540 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.537+0000 7f79ada91700 1 -- start start 2024-03-26T03:45:39.540 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.537+0000 7f79ada91700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79a80749e0 0x7f79a81a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:45:39.540 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.537+0000 7f79ada91700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79a8075300 0x7f79a81a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:45:39.540 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.537+0000 7f79ada91700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79a8113680 0x7f79a819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:45:39.540 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.537+0000 7f79ada91700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f79a8118530 con 0x7f79a80749e0 2024-03-26T03:45:39.540 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.537+0000 7f79ada91700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f79a81183b0 con 0x7f79a8113680 2024-03-26T03:45:39.540 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.537+0000 7f79ada91700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f79a81186b0 con 0x7f79a8075300 2024-03-26T03:45:39.540 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.537+0000 7f79a77fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79a80749e0 0x7f79a81a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:45:39.540 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.537+0000 7f79a77fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79a80749e0 0x7f79a81a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55228/0 (socket says 172.21.15.175:55228) 2024-03-26T03:45:39.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.537+0000 7f79a77fe700 1 -- 172.21.15.175:0/2634654928 learned_addr learned my addr 172.21.15.175:0/2634654928 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:45:39.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.537+0000 7f79a6ffd700 1 --2- 172.21.15.175:0/2634654928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79a8075300 0x7f79a81a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:45:39.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.537+0000 7f79a7fff700 1 --2- 172.21.15.175:0/2634654928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79a8113680 0x7f79a819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:45:39.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.537+0000 7f79a77fe700 1 -- 172.21.15.175:0/2634654928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79a8075300 msgr2=0x7f79a81a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:45:39.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.537+0000 7f79a77fe700 1 --2- 172.21.15.175:0/2634654928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79a8075300 0x7f79a81a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:39.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.537+0000 7f79a77fe700 1 -- 172.21.15.175:0/2634654928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79a8113680 msgr2=0x7f79a819e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:45:39.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.537+0000 7f79a77fe700 1 --2- 172.21.15.175:0/2634654928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79a8113680 0x7f79a819e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:39.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.537+0000 7f79a77fe700 1 -- 172.21.15.175:0/2634654928 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a819efd0 con 0x7f79a80749e0 2024-03-26T03:45:39.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.541+0000 7f79a77fe700 1 --2- 172.21.15.175:0/2634654928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79a80749e0 0x7f79a81a46b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f799c005910 tx=0x7f799c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:45:39.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.541+0000 7f79a4ff9700 1 -- 172.21.15.175:0/2634654928 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c027070 con 0x7f79a80749e0 2024-03-26T03:45:39.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.541+0000 7f79ada91700 1 -- 172.21.15.175:0/2634654928 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a819f260 con 0x7f79a80749e0 2024-03-26T03:45:39.541 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.541+0000 7f79a4ff9700 1 -- 172.21.15.175:0/2634654928 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f799c022470 con 0x7f79a80749e0 2024-03-26T03:45:39.542 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.541+0000 7f79a4ff9700 1 -- 172.21.15.175:0/2634654928 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c017650 con 0x7f79a80749e0 2024-03-26T03:45:39.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.541+0000 7f79ada91700 1 -- 172.21.15.175:0/2634654928 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a81ab580 con 0x7f79a80749e0 2024-03-26T03:45:39.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.541+0000 7f79ada91700 1 -- 172.21.15.175:0/2634654928 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79a8050300 con 0x7f79a80749e0 2024-03-26T03:45:39.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.541+0000 7f79a4ff9700 1 -- 172.21.15.175:0/2634654928 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7f799c02a070 con 0x7f79a80749e0 2024-03-26T03:45:39.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.541+0000 7f79a4ff9700 1 --2- 172.21.15.175:0/2634654928 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f799006e7e0 0x7f7990070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:45:39.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.541+0000 7f79a4ff9700 1 -- 172.21.15.175:0/2634654928 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7537+0+0 (secure 0 0 0) 0x7f799c08ffc0 con 0x7f79a80749e0 2024-03-26T03:45:39.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.541+0000 7f79a6ffd700 1 --2- 172.21.15.175:0/2634654928 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f799006e7e0 0x7f7990070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:45:39.543 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.541+0000 7f79a6ffd700 1 --2- 172.21.15.175:0/2634654928 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f799006e7e0 0x7f7990070ca0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f79a81008b0 tx=0x7f799800a300 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:45:39.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.549+0000 7f79a4ff9700 1 -- 172.21.15.175:0/2634654928 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f799c013070 con 0x7f79a80749e0 2024-03-26T03:45:39.741 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:45:39 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:45:39.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.845+0000 7f79ada91700 1 -- 172.21.15.175:0/2634654928 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f79a802dd30 con 0x7f79a80749e0 2024-03-26T03:45:39.847 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.849+0000 7f79a4ff9700 1 -- 172.21.15.175:0/2634654928 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) ==== 170+0+59 (secure 0 0 0) 0x7f799c05d0d0 con 0x7f79a80749e0 2024-03-26T03:45:39.848 INFO:teuthology.orchestra.run.smithi175.stdout:[client.0] 2024-03-26T03:45:39.848 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQDjRAJmdLeTMhAAtsjgv2fDJZiUHFBTAcDYMQ== 2024-03-26T03:45:39.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.849+0000 7f79ada91700 1 -- 172.21.15.175:0/2634654928 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f799006e7e0 msgr2=0x7f7990070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:45:39.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.849+0000 7f79ada91700 1 --2- 172.21.15.175:0/2634654928 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f799006e7e0 0x7f7990070ca0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f79a81008b0 tx=0x7f799800a300 comp rx=0 tx=0).stop 2024-03-26T03:45:39.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.849+0000 7f79ada91700 1 -- 172.21.15.175:0/2634654928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79a80749e0 msgr2=0x7f79a81a46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:45:39.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.849+0000 7f79ada91700 1 --2- 172.21.15.175:0/2634654928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79a80749e0 0x7f79a81a46b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f799c005910 tx=0x7f799c0047b0 comp rx=0 tx=0).stop 2024-03-26T03:45:39.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.849+0000 7f79ada91700 1 -- 172.21.15.175:0/2634654928 shutdown_connections 2024-03-26T03:45:39.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.849+0000 7f79ada91700 1 --2- 172.21.15.175:0/2634654928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79a8113680 0x7f79a819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:39.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.849+0000 7f79ada91700 1 --2- 172.21.15.175:0/2634654928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79a80749e0 0x7f79a81a46b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:39.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.849+0000 7f79ada91700 1 --2- 172.21.15.175:0/2634654928 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f799006e7e0 0x7f7990070ca0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:39.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.849+0000 7f79ada91700 1 --2- 172.21.15.175:0/2634654928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f79a8075300 0x7f79a81a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:39.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.849+0000 7f79ada91700 1 -- 172.21.15.175:0/2634654928 >> 172.21.15.175:0/2634654928 conn(0x7f79a80fe140 msgr2=0x7f79a80ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:45:39.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.849+0000 7f79ada91700 1 -- 172.21.15.175:0/2634654928 shutdown_connections 2024-03-26T03:45:39.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:39.849+0000 7f79ada91700 1 -- 172.21.15.175:0/2634654928 wait complete. 2024-03-26T03:45:40.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:40 smithi175 bash[29284]: cluster 2024-03-26T03:45:38.477247+0000 mgr.y (mgr.24431) 34 : cluster [DBG] pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:45:40.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:40 smithi175 bash[29284]: audit 2024-03-26T03:45:39.848227+0000 mon.a (mon.0) 818 : audit [INF] from='client.? 172.21.15.175:0/2634654928' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T03:45:40.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:40 smithi175 bash[29284]: audit 2024-03-26T03:45:39.851496+0000 mon.a (mon.0) 819 : audit [INF] from='client.? 172.21.15.175:0/2634654928' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-26T03:45:40.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:40 smithi190 bash[23392]: cluster 2024-03-26T03:45:38.477247+0000 mgr.y (mgr.24431) 34 : cluster [DBG] pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:45:40.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:40 smithi190 bash[23392]: audit 2024-03-26T03:45:39.848227+0000 mon.a (mon.0) 818 : audit [INF] from='client.? 172.21.15.175:0/2634654928' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T03:45:40.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:40 smithi190 bash[23392]: audit 2024-03-26T03:45:39.851496+0000 mon.a (mon.0) 819 : audit [INF] from='client.? 172.21.15.175:0/2634654928' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-26T03:45:40.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:40 smithi175 bash[21527]: cluster 2024-03-26T03:45:38.477247+0000 mgr.y (mgr.24431) 34 : cluster [DBG] pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:45:40.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:40 smithi175 bash[21527]: audit 2024-03-26T03:45:39.848227+0000 mon.a (mon.0) 818 : audit [INF] from='client.? 172.21.15.175:0/2634654928' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T03:45:40.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:40 smithi175 bash[21527]: audit 2024-03-26T03:45:39.851496+0000 mon.a (mon.0) 819 : audit [INF] from='client.? 172.21.15.175:0/2634654928' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-26T03:45:41.041 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-26T03:45:41.041 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-26T03:45:41.041 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-26T03:45:41.099 DEBUG:teuthology.orchestra.run.smithi190:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-26T03:45:41.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:41 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:41.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:41 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:41.242 INFO:journalctl@ceph.osd.3.smithi175.stdout:Mar 26 03:45:41 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:41.242 INFO:journalctl@ceph.rgw.foo.a.smithi175.stdout:Mar 26 03:45:41 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:41.242 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:41 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:41.243 INFO:journalctl@ceph.osd.0.smithi175.stdout:Mar 26 03:45:41 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:41.243 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 26 03:45:41 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:41.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:41 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:41.244 INFO:journalctl@ceph.osd.1.smithi175.stdout:Mar 26 03:45:41 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:41.378 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:41 smithi175 bash[21527]: audit 2024-03-26T03:45:39.490083+0000 mgr.y (mgr.24431) 35 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:45:41.378 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:41 smithi175 bash[29284]: audit 2024-03-26T03:45:39.490083+0000 mgr.y (mgr.24431) 35 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:45:41.379 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:45:41 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:41.379 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:45:41 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:41.379 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:45:41 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:41.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:41 smithi190 bash[23392]: audit 2024-03-26T03:45:39.490083+0000 mgr.y (mgr.24431) 35 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:45:41.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:41 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:41.494 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:41 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:41.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:41 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:41.495 INFO:journalctl@ceph.osd.0.smithi175.stdout:Mar 26 03:45:41 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:41.495 INFO:journalctl@ceph.osd.1.smithi175.stdout:Mar 26 03:45:41 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:41.495 INFO:journalctl@ceph.rgw.foo.a.smithi175.stdout:Mar 26 03:45:41 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:41.496 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 03:45:41 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:41.496 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 26 03:45:41 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:41.496 INFO:journalctl@ceph.osd.3.smithi175.stdout:Mar 26 03:45:41 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:41.496 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:45:41 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:45:41.496 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:45:41 smithi175 systemd[1]: Started Ceph alertmanager.a for 762e930c-eb21-11ee-95d0-87774f69a715. 2024-03-26T03:45:42.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:42 smithi175 bash[29284]: cluster 2024-03-26T03:45:40.478747+0000 mgr.y (mgr.24431) 36 : cluster [DBG] pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:45:42.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:42 smithi175 bash[29284]: audit 2024-03-26T03:45:41.545534+0000 mon.a (mon.0) 820 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:42.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:42 smithi175 bash[29284]: audit 2024-03-26T03:45:41.555092+0000 mon.a (mon.0) 821 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:42.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:42 smithi175 bash[29284]: audit 2024-03-26T03:45:41.600442+0000 mon.a (mon.0) 822 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:42.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:42 smithi175 bash[29284]: audit 2024-03-26T03:45:41.608815+0000 mon.a (mon.0) 823 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:42.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:42 smithi175 bash[29284]: audit 2024-03-26T03:45:41.677011+0000 mon.c (mon.2) 49 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:45:42.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:42 smithi175 bash[29284]: audit 2024-03-26T03:45:42.203710+0000 mon.a (mon.0) 824 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:42.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:42 smithi175 bash[29284]: audit 2024-03-26T03:45:42.218626+0000 mon.a (mon.0) 825 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:42.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:42 smithi175 bash[21527]: cluster 2024-03-26T03:45:40.478747+0000 mgr.y (mgr.24431) 36 : cluster [DBG] pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:45:42.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:42 smithi175 bash[21527]: audit 2024-03-26T03:45:41.545534+0000 mon.a (mon.0) 820 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:42.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:42 smithi175 bash[21527]: audit 2024-03-26T03:45:41.555092+0000 mon.a (mon.0) 821 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:42.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:42 smithi175 bash[21527]: audit 2024-03-26T03:45:41.600442+0000 mon.a (mon.0) 822 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:42.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:42 smithi175 bash[21527]: audit 2024-03-26T03:45:41.608815+0000 mon.a (mon.0) 823 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:42.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:42 smithi175 bash[21527]: audit 2024-03-26T03:45:41.677011+0000 mon.c (mon.2) 49 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:45:42.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:42 smithi175 bash[21527]: audit 2024-03-26T03:45:42.203710+0000 mon.a (mon.0) 824 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:42.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:42 smithi175 bash[21527]: audit 2024-03-26T03:45:42.218626+0000 mon.a (mon.0) 825 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:42.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:42 smithi190 bash[23392]: cluster 2024-03-26T03:45:40.478747+0000 mgr.y (mgr.24431) 36 : cluster [DBG] pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:45:42.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:42 smithi190 bash[23392]: audit 2024-03-26T03:45:41.545534+0000 mon.a (mon.0) 820 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:42.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:42 smithi190 bash[23392]: audit 2024-03-26T03:45:41.555092+0000 mon.a (mon.0) 821 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:42.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:42 smithi190 bash[23392]: audit 2024-03-26T03:45:41.600442+0000 mon.a (mon.0) 822 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:42.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:42 smithi190 bash[23392]: audit 2024-03-26T03:45:41.608815+0000 mon.a (mon.0) 823 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:42.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:42 smithi190 bash[23392]: audit 2024-03-26T03:45:41.677011+0000 mon.c (mon.2) 49 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:45:42.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:42 smithi190 bash[23392]: audit 2024-03-26T03:45:42.203710+0000 mon.a (mon.0) 824 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:42.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:42 smithi190 bash[23392]: audit 2024-03-26T03:45:42.218626+0000 mon.a (mon.0) 825 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:42.991 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:45:42 smithi175 bash[58426]: ts=2024-03-26T03:45:42.752Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-03-26T03:45:42.991 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:45:42 smithi175 bash[58426]: ts=2024-03-26T03:45:42.752Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-03-26T03:45:42.992 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:45:42 smithi175 bash[58426]: ts=2024-03-26T03:45:42.754Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.175 port=9094 2024-03-26T03:45:42.992 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:45:42 smithi175 bash[58426]: ts=2024-03-26T03:45:42.754Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-03-26T03:45:42.992 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:45:42 smithi175 bash[58426]: ts=2024-03-26T03:45:42.785Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-03-26T03:45:42.992 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:45:42 smithi175 bash[58426]: ts=2024-03-26T03:45:42.786Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-03-26T03:45:42.992 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:45:42 smithi175 bash[58426]: ts=2024-03-26T03:45:42.789Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-03-26T03:45:42.992 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:45:42 smithi175 bash[58426]: ts=2024-03-26T03:45:42.789Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-03-26T03:45:44.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:44 smithi190 bash[23392]: cluster 2024-03-26T03:45:42.479483+0000 mgr.y (mgr.24431) 37 : cluster [DBG] pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:45:44.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:44 smithi190 bash[23392]: audit 2024-03-26T03:45:43.593759+0000 mon.a (mon.0) 826 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:44.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:44 smithi175 bash[21527]: cluster 2024-03-26T03:45:42.479483+0000 mgr.y (mgr.24431) 37 : cluster [DBG] pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:45:44.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:44 smithi175 bash[21527]: audit 2024-03-26T03:45:43.593759+0000 mon.a (mon.0) 826 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:44.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:44 smithi175 bash[29284]: cluster 2024-03-26T03:45:42.479483+0000 mgr.y (mgr.24431) 37 : cluster [DBG] pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:45:44.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:44 smithi175 bash[29284]: audit 2024-03-26T03:45:43.593759+0000 mon.a (mon.0) 826 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:45.241 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:45:44 smithi175 bash[58426]: ts=2024-03-26T03:45:44.755Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000707107s 2024-03-26T03:45:45.878 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T03:45:46.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:46 smithi190 bash[23392]: cluster 2024-03-26T03:45:44.480864+0000 mgr.y (mgr.24431) 38 : cluster [DBG] pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:45:46.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:46 smithi175 bash[21527]: cluster 2024-03-26T03:45:44.480864+0000 mgr.y (mgr.24431) 38 : cluster [DBG] pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:45:46.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:46 smithi175 bash[29284]: cluster 2024-03-26T03:45:44.480864+0000 mgr.y (mgr.24431) 38 : cluster [DBG] pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:45:48.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:47 smithi175 bash[21783]: [26/Mar/2024:03:45:47] ENGINE Bus STOPPING 2024-03-26T03:45:48.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:47 smithi175 bash[21783]: [26/Mar/2024:03:45:47] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-26T03:45:48.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:47 smithi175 bash[21783]: [26/Mar/2024:03:45:47] ENGINE Bus STOPPED 2024-03-26T03:45:48.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:47 smithi175 bash[21783]: [26/Mar/2024:03:45:47] ENGINE Bus STARTING 2024-03-26T03:45:48.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:48 smithi175 bash[21783]: [26/Mar/2024:03:45:48] ENGINE Serving on http://:::9283 2024-03-26T03:45:48.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:45:48 smithi175 bash[21783]: [26/Mar/2024:03:45:48] ENGINE Bus STARTED 2024-03-26T03:45:48.602 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:48 smithi190 bash[23392]: cluster 2024-03-26T03:45:46.481573+0000 mgr.y (mgr.24431) 39 : cluster [DBG] pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:45:48.602 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:48 smithi190 bash[23392]: audit 2024-03-26T03:45:47.319804+0000 mon.a (mon.0) 827 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:48.602 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:48 smithi190 bash[23392]: audit 2024-03-26T03:45:47.330822+0000 mon.a (mon.0) 828 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:48.602 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:48 smithi190 bash[23392]: audit 2024-03-26T03:45:47.662935+0000 mon.a (mon.0) 829 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:48.602 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:48 smithi190 bash[23392]: audit 2024-03-26T03:45:47.678846+0000 mon.a (mon.0) 830 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:48.602 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:48 smithi190 bash[23392]: audit 2024-03-26T03:45:47.683775+0000 mon.c (mon.2) 50 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:45:48.602 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:48 smithi190 bash[23392]: audit 2024-03-26T03:45:47.685789+0000 mon.c (mon.2) 51 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:45:48.602 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:48 smithi190 bash[23392]: audit 2024-03-26T03:45:47.697289+0000 mon.a (mon.0) 831 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:48.602 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:48 smithi190 bash[23392]: audit 2024-03-26T03:45:47.818814+0000 mon.a (mon.0) 832 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:48.603 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:48 smithi190 bash[23392]: audit 2024-03-26T03:45:47.827713+0000 mon.a (mon.0) 833 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:48.603 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:48 smithi190 bash[23392]: audit 2024-03-26T03:45:47.832284+0000 mon.c (mon.2) 52 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-26T03:45:48.603 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:48 smithi190 bash[23392]: audit 2024-03-26T03:45:47.840699+0000 mon.a (mon.0) 834 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:48.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:48 smithi175 bash[21527]: cluster 2024-03-26T03:45:46.481573+0000 mgr.y (mgr.24431) 39 : cluster [DBG] pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:45:48.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:48 smithi175 bash[21527]: audit 2024-03-26T03:45:47.319804+0000 mon.a (mon.0) 827 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:48.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:48 smithi175 bash[21527]: audit 2024-03-26T03:45:47.330822+0000 mon.a (mon.0) 828 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:48.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:48 smithi175 bash[21527]: audit 2024-03-26T03:45:47.662935+0000 mon.a (mon.0) 829 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:48.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:48 smithi175 bash[21527]: audit 2024-03-26T03:45:47.678846+0000 mon.a (mon.0) 830 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:48.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:48 smithi175 bash[21527]: audit 2024-03-26T03:45:47.683775+0000 mon.c (mon.2) 50 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:45:48.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:48 smithi175 bash[21527]: audit 2024-03-26T03:45:47.685789+0000 mon.c (mon.2) 51 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:45:48.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:48 smithi175 bash[21527]: audit 2024-03-26T03:45:47.697289+0000 mon.a (mon.0) 831 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:48.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:48 smithi175 bash[21527]: audit 2024-03-26T03:45:47.818814+0000 mon.a (mon.0) 832 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:48.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:48 smithi175 bash[21527]: audit 2024-03-26T03:45:47.827713+0000 mon.a (mon.0) 833 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:48.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:48 smithi175 bash[21527]: audit 2024-03-26T03:45:47.832284+0000 mon.c (mon.2) 52 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-26T03:45:48.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:48 smithi175 bash[21527]: audit 2024-03-26T03:45:47.840699+0000 mon.a (mon.0) 834 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:48.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:48 smithi175 bash[29284]: cluster 2024-03-26T03:45:46.481573+0000 mgr.y (mgr.24431) 39 : cluster [DBG] pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:45:48.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:48 smithi175 bash[29284]: audit 2024-03-26T03:45:47.319804+0000 mon.a (mon.0) 827 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:48.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:48 smithi175 bash[29284]: audit 2024-03-26T03:45:47.330822+0000 mon.a (mon.0) 828 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:48.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:48 smithi175 bash[29284]: audit 2024-03-26T03:45:47.662935+0000 mon.a (mon.0) 829 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:48.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:48 smithi175 bash[29284]: audit 2024-03-26T03:45:47.678846+0000 mon.a (mon.0) 830 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:48.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:48 smithi175 bash[29284]: audit 2024-03-26T03:45:47.683775+0000 mon.c (mon.2) 50 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:45:48.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:48 smithi175 bash[29284]: audit 2024-03-26T03:45:47.685789+0000 mon.c (mon.2) 51 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:45:48.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:48 smithi175 bash[29284]: audit 2024-03-26T03:45:47.697289+0000 mon.a (mon.0) 831 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:48.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:48 smithi175 bash[29284]: audit 2024-03-26T03:45:47.818814+0000 mon.a (mon.0) 832 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:48.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:48 smithi175 bash[29284]: audit 2024-03-26T03:45:47.827713+0000 mon.a (mon.0) 833 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:48.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:48 smithi175 bash[29284]: audit 2024-03-26T03:45:47.832284+0000 mon.c (mon.2) 52 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-26T03:45:48.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:48 smithi175 bash[29284]: audit 2024-03-26T03:45:47.840699+0000 mon.a (mon.0) 834 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:45:48.795 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe537350700 1 -- 172.21.15.190:0/3449112423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe53010cca0 msgr2=0x7fe53010f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:45:48.795 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe537350700 1 --2- 172.21.15.190:0/3449112423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe53010cca0 0x7fe53010f140 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe52800b0a0 tx=0x7fe52800b3b0 comp rx=0 tx=0).stop 2024-03-26T03:45:48.795 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe537350700 1 -- 172.21.15.190:0/3449112423 shutdown_connections 2024-03-26T03:45:48.795 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe537350700 1 --2- 172.21.15.190:0/3449112423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe530103950 0x7fe530103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:48.795 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe537350700 1 --2- 172.21.15.190:0/3449112423 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe53010cca0 0x7fe53010f140 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:48.795 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe537350700 1 --2- 172.21.15.190:0/3449112423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe530104270 0x7fe53010c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:48.795 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe537350700 1 -- 172.21.15.190:0/3449112423 >> 172.21.15.190:0/3449112423 conn(0x7fe5300786f0 msgr2=0x7fe530078b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:45:48.795 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe537350700 1 -- 172.21.15.190:0/3449112423 shutdown_connections 2024-03-26T03:45:48.795 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe537350700 1 -- 172.21.15.190:0/3449112423 wait complete. 2024-03-26T03:45:48.797 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe537350700 1 Processor -- start 2024-03-26T03:45:48.797 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe537350700 1 -- start start 2024-03-26T03:45:48.797 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe537350700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe530103950 0x7fe530198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:45:48.797 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe537350700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe530104270 0x7fe5301987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:45:48.797 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe537350700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe53010cca0 0x7fe53019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:45:48.797 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe537350700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe530111eb0 con 0x7fe53010cca0 2024-03-26T03:45:48.797 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe537350700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe530111d30 con 0x7fe530104270 2024-03-26T03:45:48.797 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe537350700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe530112030 con 0x7fe530103950 2024-03-26T03:45:48.797 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe5348eb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe530104270 0x7fe5301987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:45:48.797 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe5348eb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe530104270 0x7fe5301987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:48494/0 (socket says 172.21.15.190:48494) 2024-03-26T03:45:48.798 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe5348eb700 1 -- 172.21.15.190:0/812971764 learned_addr learned my addr 172.21.15.190:0/812971764 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-03-26T03:45:48.798 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe5358ed700 1 --2- 172.21.15.190:0/812971764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe53010cca0 0x7fe53019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:45:48.798 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe5350ec700 1 --2- 172.21.15.190:0/812971764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe530103950 0x7fe530198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:45:48.798 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe5348eb700 1 -- 172.21.15.190:0/812971764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe530103950 msgr2=0x7fe530198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:45:48.798 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe5348eb700 1 --2- 172.21.15.190:0/812971764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe530103950 0x7fe530198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:48.798 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe5348eb700 1 -- 172.21.15.190:0/812971764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe53010cca0 msgr2=0x7fe53019cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:45:48.798 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe5348eb700 1 --2- 172.21.15.190:0/812971764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe53010cca0 0x7fe53019cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:48.798 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe5348eb700 1 -- 172.21.15.190:0/812971764 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe53019d2a0 con 0x7fe530104270 2024-03-26T03:45:48.798 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe5348eb700 1 --2- 172.21.15.190:0/812971764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe530104270 0x7fe5301987d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe52000c9e0 tx=0x7fe52000ccf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:45:48.798 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe5267fc700 1 -- 172.21.15.190:0/812971764 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520007aa0 con 0x7fe530104270 2024-03-26T03:45:48.798 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe537350700 1 -- 172.21.15.190:0/812971764 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe53019d590 con 0x7fe530104270 2024-03-26T03:45:48.798 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe5267fc700 1 -- 172.21.15.190:0/812971764 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fe520007c40 con 0x7fe530104270 2024-03-26T03:45:48.798 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe5267fc700 1 -- 172.21.15.190:0/812971764 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520018660 con 0x7fe530104270 2024-03-26T03:45:48.803 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.800+0000 7fe537350700 1 -- 172.21.15.190:0/812971764 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5301014e0 con 0x7fe530104270 2024-03-26T03:45:48.803 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.804+0000 7fe537350700 1 -- 172.21.15.190:0/812971764 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe530050300 con 0x7fe530104270 2024-03-26T03:45:48.803 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.804+0000 7fe5267fc700 1 -- 172.21.15.190:0/812971764 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7fe52000f450 con 0x7fe530104270 2024-03-26T03:45:48.803 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.804+0000 7fe5267fc700 1 --2- 172.21.15.190:0/812971764 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fe51c06e7e0 0x7fe51c070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:45:48.804 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.804+0000 7fe5267fc700 1 -- 172.21.15.190:0/812971764 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7537+0+0 (secure 0 0 0) 0x7fe52008ed80 con 0x7fe530104270 2024-03-26T03:45:48.804 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.804+0000 7fe5350ec700 1 --2- 172.21.15.190:0/812971764 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fe51c06e7e0 0x7fe51c070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:45:48.804 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.804+0000 7fe5350ec700 1 --2- 172.21.15.190:0/812971764 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fe51c06e7e0 0x7fe51c070ca0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe52c005fd0 tx=0x7fe52c00a560 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:45:48.804 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:48.808+0000 7fe5267fc700 1 -- 172.21.15.190:0/812971764 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7fe520014030 con 0x7fe530104270 2024-03-26T03:45:49.082 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:49.088+0000 7fe537350700 1 -- 172.21.15.190:0/812971764 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fe5301995f0 con 0x7fe530104270 2024-03-26T03:45:49.089 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:49.092+0000 7fe5267fc700 1 -- 172.21.15.190:0/812971764 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v19) ==== 170+0+59 (secure 0 0 0) 0x7fe52005bdf0 con 0x7fe530104270 2024-03-26T03:45:49.089 INFO:teuthology.orchestra.run.smithi190.stdout:[client.1] 2024-03-26T03:45:49.089 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQDtRAJm7kdCBRAAj1Ms4PdzawkQOzgNeXZGKA== 2024-03-26T03:45:49.091 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:49.096+0000 7fe537350700 1 -- 172.21.15.190:0/812971764 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fe51c06e7e0 msgr2=0x7fe51c070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:45:49.091 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:49.096+0000 7fe537350700 1 --2- 172.21.15.190:0/812971764 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fe51c06e7e0 0x7fe51c070ca0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe52c005fd0 tx=0x7fe52c00a560 comp rx=0 tx=0).stop 2024-03-26T03:45:49.092 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:49.096+0000 7fe537350700 1 -- 172.21.15.190:0/812971764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe530104270 msgr2=0x7fe5301987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:45:49.092 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:49.096+0000 7fe537350700 1 --2- 172.21.15.190:0/812971764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe530104270 0x7fe5301987d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe52000c9e0 tx=0x7fe52000ccf0 comp rx=0 tx=0).stop 2024-03-26T03:45:49.092 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:49.096+0000 7fe537350700 1 -- 172.21.15.190:0/812971764 shutdown_connections 2024-03-26T03:45:49.092 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:49.096+0000 7fe537350700 1 --2- 172.21.15.190:0/812971764 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fe51c06e7e0 0x7fe51c070ca0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:49.092 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:49.096+0000 7fe537350700 1 --2- 172.21.15.190:0/812971764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe530103950 0x7fe530198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:49.092 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:49.096+0000 7fe537350700 1 --2- 172.21.15.190:0/812971764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe53010cca0 0x7fe53019cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:49.092 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:49.096+0000 7fe537350700 1 --2- 172.21.15.190:0/812971764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe530104270 0x7fe5301987d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:49.092 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:49.096+0000 7fe537350700 1 -- 172.21.15.190:0/812971764 >> 172.21.15.190:0/812971764 conn(0x7fe5300786f0 msgr2=0x7fe530104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:45:49.092 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:49.096+0000 7fe537350700 1 -- 172.21.15.190:0/812971764 shutdown_connections 2024-03-26T03:45:49.092 INFO:teuthology.orchestra.run.smithi190.stderr:2024-03-26T03:45:49.096+0000 7fe537350700 1 -- 172.21.15.190:0/812971764 wait complete. 2024-03-26T03:45:49.386 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:49 smithi190 bash[23392]: cephadm 2024-03-26T03:45:47.739946+0000 mgr.y (mgr.24431) 40 : cephadm [INF] Regenerating cephadm self-signed grafana TLS certificates 2024-03-26T03:45:49.387 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:49 smithi190 bash[23392]: audit 2024-03-26T03:45:47.833443+0000 mgr.y (mgr.24431) 41 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-26T03:45:49.387 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:49 smithi190 bash[23392]: cephadm 2024-03-26T03:45:47.867687+0000 mgr.y (mgr.24431) 42 : cephadm [INF] Deploying daemon grafana.a on smithi190 2024-03-26T03:45:49.387 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:49 smithi190 bash[23392]: audit 2024-03-26T03:45:48.635588+0000 mon.c (mon.2) 53 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:45:49.387 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:49 smithi190 bash[23392]: audit 2024-03-26T03:45:49.087876+0000 mon.a (mon.0) 835 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T03:45:49.387 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:49 smithi190 bash[23392]: audit 2024-03-26T03:45:49.091310+0000 mon.a (mon.0) 836 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-26T03:45:49.387 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:49 smithi190 bash[23392]: audit 2024-03-26T03:45:49.092569+0000 mon.b (mon.1) 33 : audit [INF] from='client.? 172.21.15.190:0/812971764' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T03:45:49.741 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:45:49 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:45:49.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:49 smithi175 bash[21527]: cephadm 2024-03-26T03:45:47.739946+0000 mgr.y (mgr.24431) 40 : cephadm [INF] Regenerating cephadm self-signed grafana TLS certificates 2024-03-26T03:45:49.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:49 smithi175 bash[21527]: audit 2024-03-26T03:45:47.833443+0000 mgr.y (mgr.24431) 41 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-26T03:45:49.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:49 smithi175 bash[21527]: cephadm 2024-03-26T03:45:47.867687+0000 mgr.y (mgr.24431) 42 : cephadm [INF] Deploying daemon grafana.a on smithi190 2024-03-26T03:45:49.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:49 smithi175 bash[21527]: audit 2024-03-26T03:45:48.635588+0000 mon.c (mon.2) 53 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:45:49.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:49 smithi175 bash[21527]: audit 2024-03-26T03:45:49.087876+0000 mon.a (mon.0) 835 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T03:45:49.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:49 smithi175 bash[21527]: audit 2024-03-26T03:45:49.091310+0000 mon.a (mon.0) 836 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-26T03:45:49.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:49 smithi175 bash[21527]: audit 2024-03-26T03:45:49.092569+0000 mon.b (mon.1) 33 : audit [INF] from='client.? 172.21.15.190:0/812971764' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T03:45:49.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:49 smithi175 bash[29284]: cephadm 2024-03-26T03:45:47.739946+0000 mgr.y (mgr.24431) 40 : cephadm [INF] Regenerating cephadm self-signed grafana TLS certificates 2024-03-26T03:45:49.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:49 smithi175 bash[29284]: audit 2024-03-26T03:45:47.833443+0000 mgr.y (mgr.24431) 41 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-26T03:45:49.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:49 smithi175 bash[29284]: cephadm 2024-03-26T03:45:47.867687+0000 mgr.y (mgr.24431) 42 : cephadm [INF] Deploying daemon grafana.a on smithi190 2024-03-26T03:45:49.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:49 smithi175 bash[29284]: audit 2024-03-26T03:45:48.635588+0000 mon.c (mon.2) 53 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:45:49.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:49 smithi175 bash[29284]: audit 2024-03-26T03:45:49.087876+0000 mon.a (mon.0) 835 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T03:45:49.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:49 smithi175 bash[29284]: audit 2024-03-26T03:45:49.091310+0000 mon.a (mon.0) 836 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-26T03:45:49.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:49 smithi175 bash[29284]: audit 2024-03-26T03:45:49.092569+0000 mon.b (mon.1) 33 : audit [INF] from='client.? 172.21.15.190:0/812971764' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T03:45:50.182 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-03-26T03:45:50.182 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-03-26T03:45:50.182 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-26T03:45:50.202 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-26T03:45:50.203 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-26T03:45:50.203 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph mgr dump --format=json 2024-03-26T03:45:50.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:50 smithi190 bash[23392]: cluster 2024-03-26T03:45:48.482228+0000 mgr.y (mgr.24431) 43 : cluster [DBG] pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:45:50.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:50 smithi175 bash[21527]: cluster 2024-03-26T03:45:48.482228+0000 mgr.y (mgr.24431) 43 : cluster [DBG] pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:45:50.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:50 smithi175 bash[29284]: cluster 2024-03-26T03:45:48.482228+0000 mgr.y (mgr.24431) 43 : cluster [DBG] pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:45:51.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:51 smithi190 bash[23392]: audit 2024-03-26T03:45:49.498153+0000 mgr.y (mgr.24431) 44 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:45:51.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:51 smithi175 bash[21527]: audit 2024-03-26T03:45:49.498153+0000 mgr.y (mgr.24431) 44 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:45:51.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:51 smithi175 bash[29284]: audit 2024-03-26T03:45:49.498153+0000 mgr.y (mgr.24431) 44 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:45:52.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:52 smithi190 bash[23392]: cluster 2024-03-26T03:45:50.483835+0000 mgr.y (mgr.24431) 45 : cluster [DBG] pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:45:52.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:52 smithi175 bash[21527]: cluster 2024-03-26T03:45:50.483835+0000 mgr.y (mgr.24431) 45 : cluster [DBG] pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:45:52.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:52 smithi175 bash[29284]: cluster 2024-03-26T03:45:50.483835+0000 mgr.y (mgr.24431) 45 : cluster [DBG] pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:45:53.241 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:45:52 smithi175 bash[58426]: ts=2024-03-26T03:45:52.759Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.004560574s 2024-03-26T03:45:54.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:54 smithi190 bash[23392]: cluster 2024-03-26T03:45:52.484665+0000 mgr.y (mgr.24431) 46 : cluster [DBG] pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:45:54.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:54 smithi175 bash[21527]: cluster 2024-03-26T03:45:52.484665+0000 mgr.y (mgr.24431) 46 : cluster [DBG] pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:45:54.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:54 smithi175 bash[29284]: cluster 2024-03-26T03:45:52.484665+0000 mgr.y (mgr.24431) 46 : cluster [DBG] pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:45:54.991 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:45:56.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:56 smithi190 bash[23392]: cluster 2024-03-26T03:45:54.486085+0000 mgr.y (mgr.24431) 47 : cluster [DBG] pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:45:56.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:56 smithi175 bash[21527]: cluster 2024-03-26T03:45:54.486085+0000 mgr.y (mgr.24431) 47 : cluster [DBG] pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:45:56.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:56 smithi175 bash[29284]: cluster 2024-03-26T03:45:54.486085+0000 mgr.y (mgr.24431) 47 : cluster [DBG] pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:45:57.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a4203a700 1 -- 172.21.15.175:0/2590235093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a3c06adb0 msgr2=0x7f6a3c06b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:45:57.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a4203a700 1 --2- 172.21.15.175:0/2590235093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a3c06adb0 0x7f6a3c06b230 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6a30009990 tx=0x7f6a30009ca0 comp rx=0 tx=0).stop 2024-03-26T03:45:57.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a4203a700 1 -- 172.21.15.175:0/2590235093 shutdown_connections 2024-03-26T03:45:57.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a4203a700 1 --2- 172.21.15.175:0/2590235093 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a3c104570 0x7f6a3c104950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:57.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a4203a700 1 --2- 172.21.15.175:0/2590235093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a3c06b770 0x7f6a3c10f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:57.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a4203a700 1 --2- 172.21.15.175:0/2590235093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a3c06adb0 0x7f6a3c06b230 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:57.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a4203a700 1 -- 172.21.15.175:0/2590235093 >> 172.21.15.175:0/2590235093 conn(0x7f6a3c0775e0 msgr2=0x7f6a3c0779f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:45:57.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a4203a700 1 -- 172.21.15.175:0/2590235093 shutdown_connections 2024-03-26T03:45:57.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a4203a700 1 -- 172.21.15.175:0/2590235093 wait complete. 2024-03-26T03:45:57.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a4203a700 1 Processor -- start 2024-03-26T03:45:57.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a4203a700 1 -- start start 2024-03-26T03:45:57.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a4203a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a3c06adb0 0x7f6a3c101710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:45:57.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a4203a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a3c06b770 0x7f6a3c101c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:45:57.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a4203a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a3c104570 0x7f6a3c102190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:45:57.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a4203a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c112060 con 0x7f6a3c104570 2024-03-26T03:45:57.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a4203a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c111ee0 con 0x7f6a3c06b770 2024-03-26T03:45:57.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a4203a700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6a3c1121e0 con 0x7f6a3c06adb0 2024-03-26T03:45:57.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a3b7fe700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a3c06adb0 0x7f6a3c101710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:45:57.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a3b7fe700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a3c06adb0 0x7f6a3c101710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:57068/0 (socket says 172.21.15.175:57068) 2024-03-26T03:45:57.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a3b7fe700 1 -- 172.21.15.175:0/3689182781 learned_addr learned my addr 172.21.15.175:0/3689182781 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:45:57.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a3bfff700 1 --2- 172.21.15.175:0/3689182781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a3c104570 0x7f6a3c102190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:45:57.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a3affd700 1 --2- 172.21.15.175:0/3689182781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a3c06b770 0x7f6a3c101c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:45:57.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a3b7fe700 1 -- 172.21.15.175:0/3689182781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a3c06b770 msgr2=0x7f6a3c101c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:45:57.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a3b7fe700 1 --2- 172.21.15.175:0/3689182781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a3c06b770 0x7f6a3c101c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:57.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a3b7fe700 1 -- 172.21.15.175:0/3689182781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a3c104570 msgr2=0x7f6a3c102190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:45:57.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a3b7fe700 1 --2- 172.21.15.175:0/3689182781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a3c104570 0x7f6a3c102190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:57.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a3b7fe700 1 -- 172.21.15.175:0/3689182781 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a3c1a69f0 con 0x7f6a3c06adb0 2024-03-26T03:45:57.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a3b7fe700 1 --2- 172.21.15.175:0/3689182781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a3c06adb0 0x7f6a3c101710 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00eb30 tx=0x7f6a2c00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:45:57.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a38ff9700 1 -- 172.21.15.175:0/3689182781 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c00cc90 con 0x7f6a3c06adb0 2024-03-26T03:45:57.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a38ff9700 1 -- 172.21.15.175:0/3689182781 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f6a2c00ce30 con 0x7f6a3c06adb0 2024-03-26T03:45:57.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a4203a700 1 -- 172.21.15.175:0/3689182781 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a3c1a6ce0 con 0x7f6a3c06adb0 2024-03-26T03:45:57.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a4203a700 1 -- 172.21.15.175:0/3689182781 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a3c1a7210 con 0x7f6a3c06adb0 2024-03-26T03:45:57.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.965+0000 7f6a38ff9700 1 -- 172.21.15.175:0/3689182781 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c003d80 con 0x7f6a3c06adb0 2024-03-26T03:45:57.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.969+0000 7f6a4203a700 1 -- 172.21.15.175:0/3689182781 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a28005120 con 0x7f6a3c06adb0 2024-03-26T03:45:57.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.969+0000 7f6a38ff9700 1 -- 172.21.15.175:0/3689182781 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7f6a2c003fc0 con 0x7f6a3c06adb0 2024-03-26T03:45:57.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.969+0000 7f6a38ff9700 1 --2- 172.21.15.175:0/3689182781 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f6a2406e7e0 0x7f6a24070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:45:57.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.969+0000 7f6a38ff9700 1 -- 172.21.15.175:0/3689182781 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(75..75 src has 1..75) ==== 7537+0+0 (secure 0 0 0) 0x7f6a2c08f070 con 0x7f6a3c06adb0 2024-03-26T03:45:57.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.969+0000 7f6a3affd700 1 --2- 172.21.15.175:0/3689182781 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f6a2406e7e0 0x7f6a24070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:45:57.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.969+0000 7f6a3affd700 1 --2- 172.21.15.175:0/3689182781 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f6a2406e7e0 0x7f6a24070ca0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6a3000b2e0 tx=0x7f6a30005e80 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:45:57.973 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:57.973+0000 7f6a38ff9700 1 -- 172.21.15.175:0/3689182781 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f6a2c05c0e0 con 0x7f6a3c06adb0 2024-03-26T03:45:58.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:58.253+0000 7f6a4203a700 1 -- 172.21.15.175:0/3689182781 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f6a280056e0 con 0x7f6a3c06adb0 2024-03-26T03:45:58.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:58.257+0000 7f6a38ff9700 1 -- 172.21.15.175:0/3689182781 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v21) ==== 74+0+176775 (secure 0 0 0) 0x7f6a2c05fd90 con 0x7f6a3c06adb0 2024-03-26T03:45:58.261 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-26T03:45:58.262 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:58.261+0000 7f6a4203a700 1 -- 172.21.15.175:0/3689182781 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f6a2406e7e0 msgr2=0x7f6a24070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:45:58.262 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:58.261+0000 7f6a4203a700 1 --2- 172.21.15.175:0/3689182781 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f6a2406e7e0 0x7f6a24070ca0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6a3000b2e0 tx=0x7f6a30005e80 comp rx=0 tx=0).stop 2024-03-26T03:45:58.262 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:58.261+0000 7f6a4203a700 1 -- 172.21.15.175:0/3689182781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a3c06adb0 msgr2=0x7f6a3c101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:45:58.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:58.261+0000 7f6a4203a700 1 --2- 172.21.15.175:0/3689182781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a3c06adb0 0x7f6a3c101710 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00eb30 tx=0x7f6a2c00ee40 comp rx=0 tx=0).stop 2024-03-26T03:45:58.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:58.261+0000 7f6a4203a700 1 -- 172.21.15.175:0/3689182781 shutdown_connections 2024-03-26T03:45:58.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:58.261+0000 7f6a4203a700 1 --2- 172.21.15.175:0/3689182781 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f6a2406e7e0 0x7f6a24070ca0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:58.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:58.261+0000 7f6a4203a700 1 --2- 172.21.15.175:0/3689182781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a3c06adb0 0x7f6a3c101710 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:58.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:58.261+0000 7f6a4203a700 1 --2- 172.21.15.175:0/3689182781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a3c104570 0x7f6a3c102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:58.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:58.261+0000 7f6a4203a700 1 --2- 172.21.15.175:0/3689182781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a3c06b770 0x7f6a3c101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:45:58.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:58.261+0000 7f6a4203a700 1 -- 172.21.15.175:0/3689182781 >> 172.21.15.175:0/3689182781 conn(0x7f6a3c0775e0 msgr2=0x7f6a3c100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:45:58.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:58.261+0000 7f6a4203a700 1 -- 172.21.15.175:0/3689182781 shutdown_connections 2024-03-26T03:45:58.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:45:58.261+0000 7f6a4203a700 1 -- 172.21.15.175:0/3689182781 wait complete. 2024-03-26T03:45:58.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:58 smithi190 bash[23392]: cluster 2024-03-26T03:45:56.486774+0000 mgr.y (mgr.24431) 48 : cluster [DBG] pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:45:58.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:45:58 smithi190 bash[23392]: audit 2024-03-26T03:45:58.258103+0000 mon.c (mon.2) 54 : audit [DBG] from='client.? 172.21.15.175:0/3689182781' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-26T03:45:58.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:58 smithi175 bash[29284]: cluster 2024-03-26T03:45:56.486774+0000 mgr.y (mgr.24431) 48 : cluster [DBG] pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:45:58.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:45:58 smithi175 bash[29284]: audit 2024-03-26T03:45:58.258103+0000 mon.c (mon.2) 54 : audit [DBG] from='client.? 172.21.15.175:0/3689182781' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-26T03:45:58.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:58 smithi175 bash[21527]: cluster 2024-03-26T03:45:56.486774+0000 mgr.y (mgr.24431) 48 : cluster [DBG] pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:45:58.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:45:58 smithi175 bash[21527]: audit 2024-03-26T03:45:58.258103+0000 mon.c (mon.2) 54 : audit [DBG] from='client.? 172.21.15.175:0/3689182781' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-26T03:45:59.053 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":21,"flags":0,"active_gid":24431,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1140903233},{"type":"v1","addr":"172.21.15.175:6801","nonce":1140903233}]},"active_addr":"172.21.15.175:6801/1140903233","active_change":"2024-03-26T03:45:18.442898+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24421,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.175:8443/","prometheus":"http://172.21.15.175:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":75,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":3981582793}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":2252628800}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":3200491542}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":759223296}]}]} 2024-03-26T03:45:59.057 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-26T03:45:59.057 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-26T03:45:59.057 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-26T03:45:59.990 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:45:59 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:46:00.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:00 smithi190 bash[23392]: cluster 2024-03-26T03:45:58.487520+0000 mgr.y (mgr.24431) 49 : cluster [DBG] pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:00.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:00 smithi175 bash[29284]: cluster 2024-03-26T03:45:58.487520+0000 mgr.y (mgr.24431) 49 : cluster [DBG] pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:00.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:00 smithi175 bash[21527]: cluster 2024-03-26T03:45:58.487520+0000 mgr.y (mgr.24431) 49 : cluster [DBG] pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:01.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:01 smithi175 bash[21527]: audit 2024-03-26T03:45:59.509260+0000 mgr.y (mgr.24431) 50 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:46:01.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:01 smithi190 bash[23392]: audit 2024-03-26T03:45:59.509260+0000 mgr.y (mgr.24431) 50 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:46:01.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:01 smithi175 bash[29284]: audit 2024-03-26T03:45:59.509260+0000 mgr.y (mgr.24431) 50 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:46:02.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:02 smithi190 bash[23392]: cluster 2024-03-26T03:46:00.488933+0000 mgr.y (mgr.24431) 51 : cluster [DBG] pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:02.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:02 smithi175 bash[29284]: cluster 2024-03-26T03:46:00.488933+0000 mgr.y (mgr.24431) 51 : cluster [DBG] pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:02.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:02 smithi175 bash[21527]: cluster 2024-03-26T03:46:00.488933+0000 mgr.y (mgr.24431) 51 : cluster [DBG] pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:03.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:03 smithi190 bash[23392]: cluster 2024-03-26T03:46:02.489648+0000 mgr.y (mgr.24431) 52 : cluster [DBG] pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:03.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:03 smithi175 bash[21527]: cluster 2024-03-26T03:46:02.489648+0000 mgr.y (mgr.24431) 52 : cluster [DBG] pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:03.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:03 smithi175 bash[29284]: cluster 2024-03-26T03:46:02.489648+0000 mgr.y (mgr.24431) 52 : cluster [DBG] pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:03.846 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:46:04.693 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 03:46:04 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:46:04.694 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 03:46:04 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:46:04.694 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 03:46:04 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:46:04.694 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:46:04 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:46:04.695 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:04 smithi190 bash[23392]: audit 2024-03-26T03:46:03.665473+0000 mon.c (mon.2) 55 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:46:04.695 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:04 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:46:04.695 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:46:04 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:46:04.695 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:46:04 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:46:04.696 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 03:46:04 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:46:04.696 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:04 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:46:04.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:04 smithi175 bash[21527]: audit 2024-03-26T03:46:03.665473+0000 mon.c (mon.2) 55 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:46:04.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:04 smithi175 bash[29284]: audit 2024-03-26T03:46:03.665473+0000 mon.c (mon.2) 55 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:46:04.988 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:04 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:46:04.988 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:46:04 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:46:04.989 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 03:46:04 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:46:04.989 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:04 smithi190 systemd[1]: Started Ceph grafana.a for 762e930c-eb21-11ee-95d0-87774f69a715. 2024-03-26T03:46:04.989 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 03:46:04 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:46:04.990 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 03:46:04 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:46:04.990 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 03:46:04 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:46:04.990 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:04 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:46:04.993 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 03:46:04 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:46:04.994 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 03:46:04 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T03:46:06.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:05 smithi190 bash[23392]: cluster 2024-03-26T03:46:04.491216+0000 mgr.y (mgr.24431) 53 : cluster [DBG] pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:06.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:05 smithi190 bash[23392]: audit 2024-03-26T03:46:04.991738+0000 mon.a (mon.0) 837 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:06.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:05 smithi190 bash[23392]: audit 2024-03-26T03:46:04.999312+0000 mon.a (mon.0) 838 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:06.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:05 smithi190 bash[23392]: audit 2024-03-26T03:46:05.007893+0000 mon.a (mon.0) 839 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:06.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:05 smithi190 bash[23392]: audit 2024-03-26T03:46:05.017223+0000 mon.a (mon.0) 840 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:06.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:05 smithi190 bash[23392]: audit 2024-03-26T03:46:05.052655+0000 mon.c (mon.2) 56 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:46:06.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:05 smithi175 bash[21527]: cluster 2024-03-26T03:46:04.491216+0000 mgr.y (mgr.24431) 53 : cluster [DBG] pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:06.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:05 smithi175 bash[21527]: audit 2024-03-26T03:46:04.991738+0000 mon.a (mon.0) 837 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:06.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:05 smithi175 bash[21527]: audit 2024-03-26T03:46:04.999312+0000 mon.a (mon.0) 838 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:06.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:05 smithi175 bash[21527]: audit 2024-03-26T03:46:05.007893+0000 mon.a (mon.0) 839 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:06.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:05 smithi175 bash[21527]: audit 2024-03-26T03:46:05.017223+0000 mon.a (mon.0) 840 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:06.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:05 smithi175 bash[21527]: audit 2024-03-26T03:46:05.052655+0000 mon.c (mon.2) 56 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:46:06.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:05 smithi175 bash[29284]: cluster 2024-03-26T03:46:04.491216+0000 mgr.y (mgr.24431) 53 : cluster [DBG] pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:06.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:05 smithi175 bash[29284]: audit 2024-03-26T03:46:04.991738+0000 mon.a (mon.0) 837 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:06.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:05 smithi175 bash[29284]: audit 2024-03-26T03:46:04.999312+0000 mon.a (mon.0) 838 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:06.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:05 smithi175 bash[29284]: audit 2024-03-26T03:46:05.007893+0000 mon.a (mon.0) 839 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:06.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:05 smithi175 bash[29284]: audit 2024-03-26T03:46:05.017223+0000 mon.a (mon.0) 840 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:06.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:05 smithi175 bash[29284]: audit 2024-03-26T03:46:05.052655+0000 mon.c (mon.2) 56 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:46:06.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf5f84a700 1 -- 172.21.15.175:0/1440015961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf581087e0 msgr2=0x7fbf58108bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:06.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf5f84a700 1 --2- 172.21.15.175:0/1440015961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf581087e0 0x7fbf58108bc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009990 tx=0x7fbf48009ca0 comp rx=0 tx=0).stop 2024-03-26T03:46:06.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf5f84a700 1 -- 172.21.15.175:0/1440015961 shutdown_connections 2024-03-26T03:46:06.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf5f84a700 1 --2- 172.21.15.175:0/1440015961 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf5806b690 0x7fbf5810f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:06.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf5f84a700 1 --2- 172.21.15.175:0/1440015961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf581087e0 0x7fbf58108bc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:06.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf5f84a700 1 --2- 172.21.15.175:0/1440015961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf5806acd0 0x7fbf5806b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:06.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf5f84a700 1 -- 172.21.15.175:0/1440015961 >> 172.21.15.175:0/1440015961 conn(0x7fbf58078430 msgr2=0x7fbf58078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:46:06.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf5f84a700 1 -- 172.21.15.175:0/1440015961 shutdown_connections 2024-03-26T03:46:06.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf5f84a700 1 -- 172.21.15.175:0/1440015961 wait complete. 2024-03-26T03:46:06.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf5f84a700 1 Processor -- start 2024-03-26T03:46:06.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf5f84a700 1 -- start start 2024-03-26T03:46:06.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf5f84a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf5806acd0 0x7fbf581982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:06.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf5f84a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf5806b690 0x7fbf581987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:06.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf5f84a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf581087e0 0x7fbf5819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:06.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf5f84a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbf58111f10 con 0x7fbf581087e0 2024-03-26T03:46:06.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf5f84a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbf58111d90 con 0x7fbf5806b690 2024-03-26T03:46:06.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf5f84a700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbf58112090 con 0x7fbf5806acd0 2024-03-26T03:46:06.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf5d5e6700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf5806acd0 0x7fbf581982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:06.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf5d5e6700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf5806acd0 0x7fbf581982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:60056/0 (socket says 172.21.15.175:60056) 2024-03-26T03:46:06.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf5d5e6700 1 -- 172.21.15.175:0/3005121607 learned_addr learned my addr 172.21.15.175:0/3005121607 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:46:06.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf5dde7700 1 --2- 172.21.15.175:0/3005121607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf581087e0 0x7fbf5819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:06.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf5cde5700 1 --2- 172.21.15.175:0/3005121607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf5806b690 0x7fbf581987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:06.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf5d5e6700 1 -- 172.21.15.175:0/3005121607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf5806b690 msgr2=0x7fbf581987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:06.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf5d5e6700 1 --2- 172.21.15.175:0/3005121607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf5806b690 0x7fbf581987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:06.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf5d5e6700 1 -- 172.21.15.175:0/3005121607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf581087e0 msgr2=0x7fbf5819cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:06.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf5d5e6700 1 --2- 172.21.15.175:0/3005121607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf581087e0 0x7fbf5819cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:06.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf5d5e6700 1 -- 172.21.15.175:0/3005121607 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf5819d2b0 con 0x7fbf5806acd0 2024-03-26T03:46:06.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf5d5e6700 1 --2- 172.21.15.175:0/3005121607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf5806acd0 0x7fbf581982a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbf48000c00 tx=0x7fbf480047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:46:06.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf567fc700 1 -- 172.21.15.175:0/3005121607 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf48027070 con 0x7fbf5806acd0 2024-03-26T03:46:06.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf567fc700 1 -- 172.21.15.175:0/3005121607 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fbf48022470 con 0x7fbf5806acd0 2024-03-26T03:46:06.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf5f84a700 1 -- 172.21.15.175:0/3005121607 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf5819d540 con 0x7fbf5806acd0 2024-03-26T03:46:06.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf567fc700 1 -- 172.21.15.175:0/3005121607 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf48017650 con 0x7fbf5806acd0 2024-03-26T03:46:06.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf5f84a700 1 -- 172.21.15.175:0/3005121607 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf581a4e20 con 0x7fbf5806acd0 2024-03-26T03:46:06.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.613+0000 7fbf5f84a700 1 -- 172.21.15.175:0/3005121607 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf58050300 con 0x7fbf5806acd0 2024-03-26T03:46:06.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.617+0000 7fbf567fc700 1 -- 172.21.15.175:0/3005121607 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7fbf4802a070 con 0x7fbf5806acd0 2024-03-26T03:46:06.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.617+0000 7fbf567fc700 1 --2- 172.21.15.175:0/3005121607 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fbf4006e7e0 0x7fbf40070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:06.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.617+0000 7fbf567fc700 1 -- 172.21.15.175:0/3005121607 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(75..75 src has 1..75) ==== 7537+0+0 (secure 0 0 0) 0x7fbf4808fe90 con 0x7fbf5806acd0 2024-03-26T03:46:06.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.617+0000 7fbf5cde5700 1 --2- 172.21.15.175:0/3005121607 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fbf4006e7e0 0x7fbf40070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:06.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.617+0000 7fbf5cde5700 1 --2- 172.21.15.175:0/3005121607 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fbf4006e7e0 0x7fbf40070ca0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbf580789a0 tx=0x7fbf4c009040 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:46:06.621 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.621+0000 7fbf567fc700 1 -- 172.21.15.175:0/3005121607 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7fbf48013070 con 0x7fbf5806acd0 2024-03-26T03:46:06.661 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=settings t=2024-03-26T03:46:06.377272452Z level=info msg="Starting Grafana" version=9.4.12 commit=37ee25ba46 branch=HEAD compiled=2023-05-22T15:58:05Z 2024-03-26T03:46:06.661 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=settings t=2024-03-26T03:46:06.377469323Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-03-26T03:46:06.661 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=settings t=2024-03-26T03:46:06.377483313Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-03-26T03:46:06.661 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=settings t=2024-03-26T03:46:06.37748743Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-03-26T03:46:06.661 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=settings t=2024-03-26T03:46:06.37749075Z level=info msg="Config overridden from command line" arg="default.paths.data=/var/lib/grafana" 2024-03-26T03:46:06.661 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=settings t=2024-03-26T03:46:06.377493853Z level=info msg="Config overridden from command line" arg="default.paths.logs=/var/log/grafana" 2024-03-26T03:46:06.661 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=settings t=2024-03-26T03:46:06.377496925Z level=info msg="Config overridden from command line" arg="default.paths.plugins=/var/lib/grafana/plugins" 2024-03-26T03:46:06.661 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=settings t=2024-03-26T03:46:06.377499831Z level=info msg="Config overridden from command line" arg="default.paths.provisioning=/etc/grafana/provisioning" 2024-03-26T03:46:06.661 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=settings t=2024-03-26T03:46:06.377502818Z level=info msg="Config overridden from command line" arg="default.log.mode=console" 2024-03-26T03:46:06.661 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=settings t=2024-03-26T03:46:06.377505993Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-03-26T03:46:06.661 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=settings t=2024-03-26T03:46:06.377509528Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-03-26T03:46:06.662 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=settings t=2024-03-26T03:46:06.377512369Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-03-26T03:46:06.662 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=settings t=2024-03-26T03:46:06.37751535Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-03-26T03:46:06.662 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=settings t=2024-03-26T03:46:06.37752203Z level=info msg="Path Home" path=/usr/share/grafana 2024-03-26T03:46:06.662 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=settings t=2024-03-26T03:46:06.377525339Z level=info msg="Path Data" path=/var/lib/grafana 2024-03-26T03:46:06.662 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=settings t=2024-03-26T03:46:06.377528253Z level=info msg="Path Logs" path=/var/log/grafana 2024-03-26T03:46:06.662 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=settings t=2024-03-26T03:46:06.377531263Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-03-26T03:46:06.662 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=settings t=2024-03-26T03:46:06.377534617Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-03-26T03:46:06.662 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=settings t=2024-03-26T03:46:06.377537799Z level=info msg="App mode production" 2024-03-26T03:46:06.662 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=sqlstore t=2024-03-26T03:46:06.377606108Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-03-26T03:46:06.662 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=sqlstore t=2024-03-26T03:46:06.37762694Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-03-26T03:46:06.663 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:06.378645498Z level=info msg="Starting DB migrations" 2024-03-26T03:46:06.663 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:06.378740429Z level=info msg="Executing migration" id="create migration_log table" 2024-03-26T03:46:06.663 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:06.47621274Z level=info msg="Executing migration" id="create user table" 2024-03-26T03:46:06.663 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:06.518271792Z level=info msg="Executing migration" id="add unique index user.login" 2024-03-26T03:46:06.663 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:06.568397334Z level=info msg="Executing migration" id="add unique index user.email" 2024-03-26T03:46:06.663 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:06.602034398Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-03-26T03:46:06.663 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:06.669135889Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-03-26T03:46:06.830 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.829+0000 7fbf5f84a700 1 -- 172.21.15.175:0/3005121607 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf58199600 con 0x7fbf5806acd0 2024-03-26T03:46:06.833 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.833+0000 7fbf567fc700 1 -- 172.21.15.175:0/3005121607 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+23247 (secure 0 0 0) 0x7fbf4805cf00 con 0x7fbf5806acd0 2024-03-26T03:46:06.833 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-26T03:46:06.833 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":75,"fsid":"762e930c-eb21-11ee-95d0-87774f69a715","created":"2024-03-26T03:33:45.388461+0000","modified":"2024-03-26T03:45:18.442567+0000","last_up_change":"2024-03-26T03:43:33.976035+0000","last_in_change":"2024-03-26T03:43:07.107635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T03:39:08.367712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-03-26T03:44:15.186218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.log","create_time":"2024-03-26T03:44:16.391387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.control","create_time":"2024-03-26T03:44:18.319611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"datapool","create_time":"2024-03-26T03:44:19.627074+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":71,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-03-26T03:44:20.435692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41a2182b-5eeb-47fa-be41-1d7639df2f35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":241042466},{"type":"v1","addr":"172.21.15.175:6803","nonce":241042466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":241042466},{"type":"v1","addr":"172.21.15.175:6805","nonce":241042466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":241042466},{"type":"v1","addr":"172.21.15.175:6809","nonce":241042466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":241042466},{"type":"v1","addr":"172.21.15.175:6807","nonce":241042466}]},"public_addr":"172.21.15.175:6803/241042466","cluster_addr":"172.21.15.175:6805/241042466","heartbeat_back_addr":"172.21.15.175:6809/241042466","heartbeat_front_addr":"172.21.15.175:6807/241042466","state":["exists","up"]},{"osd":1,"uuid":"8fbe50bf-5723-4174-9c85-6010a32bd55b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":931168929},{"type":"v1","addr":"172.21.15.175:6811","nonce":931168929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":931168929},{"type":"v1","addr":"172.21.15.175:6813","nonce":931168929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":931168929},{"type":"v1","addr":"172.21.15.175:6817","nonce":931168929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":931168929},{"type":"v1","addr":"172.21.15.175:6815","nonce":931168929}]},"public_addr":"172.21.15.175:6811/931168929","cluster_addr":"172.21.15.175:6813/931168929","heartbeat_back_addr":"172.21.15.175:6817/931168929","heartbeat_front_addr":"172.21.15.175:6815/931168929","state":["exists","up"]},{"osd":2,"uuid":"02333e36-f498-4ac2-ab6e-5ae1e1148e97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3775031674},{"type":"v1","addr":"172.21.15.175:6819","nonce":3775031674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":3775031674},{"type":"v1","addr":"172.21.15.175:6821","nonce":3775031674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3775031674},{"type":"v1","addr":"172.21.15.175:6825","nonce":3775031674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":3775031674},{"type":"v1","addr":"172.21.15.175:6823","nonce":3775031674}]},"public_addr":"172.21.15.175:6819/3775031674","cluster_addr":"172.21.15.175:6821/3775031674","heartbeat_back_addr":"172.21.15.175:6825/3775031674","heartbeat_front_addr":"172.21.15.175:6823/3775031674","state":["exists","up"]},{"osd":3,"uuid":"8add17f6-b408-4dc0-af05-b3b64845d8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":672326154},{"type":"v1","addr":"172.21.15.175:6827","nonce":672326154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":672326154},{"type":"v1","addr":"172.21.15.175:6829","nonce":672326154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":672326154},{"type":"v1","addr":"172.21.15.175:6833","nonce":672326154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":672326154},{"type":"v1","addr":"172.21.15.175:6831","nonce":672326154}]},"public_addr":"172.21.15.175:6827/672326154","cluster_addr":"172.21.15.175:6829/672326154","heartbeat_back_addr":"172.21.15.175:6833/672326154","heartbeat_front_addr":"172.21.15.175:6831/672326154","state":["exists","up"]},{"osd":4,"uuid":"54aac0c2-d77e-4bf4-b1e5-856b4e997426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":874343325},{"type":"v1","addr":"172.21.15.190:6801","nonce":874343325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":874343325},{"type":"v1","addr":"172.21.15.190:6803","nonce":874343325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":874343325},{"type":"v1","addr":"172.21.15.190:6807","nonce":874343325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":874343325},{"type":"v1","addr":"172.21.15.190:6805","nonce":874343325}]},"public_addr":"172.21.15.190:6801/874343325","cluster_addr":"172.21.15.190:6803/874343325","heartbeat_back_addr":"172.21.15.190:6807/874343325","heartbeat_front_addr":"172.21.15.190:6805/874343325","state":["exists","up"]},{"osd":5,"uuid":"d1eb5f92-1729-48c7-a89c-3265609fe40d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":4089468657},{"type":"v1","addr":"172.21.15.190:6809","nonce":4089468657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":4089468657},{"type":"v1","addr":"172.21.15.190:6811","nonce":4089468657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":4089468657},{"type":"v1","addr":"172.21.15.190:6815","nonce":4089468657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":4089468657},{"type":"v1","addr":"172.21.15.190:6813","nonce":4089468657}]},"public_addr":"172.21.15.190:6809/4089468657","cluster_addr":"172.21.15.190:6811/4089468657","heartbeat_back_addr":"172.21.15.190:6815/4089468657","heartbeat_front_addr":"172.21.15.190:6813/4089468657","state":["exists","up"]},{"osd":6,"uuid":"01cebff3-5246-49f3-893d-706cbfd9d829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4077005791},{"type":"v1","addr":"172.21.15.190:6817","nonce":4077005791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4077005791},{"type":"v1","addr":"172.21.15.190:6819","nonce":4077005791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4077005791},{"type":"v1","addr":"172.21.15.190:6823","nonce":4077005791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":4077005791},{"type":"v1","addr":"172.21.15.190:6821","nonce":4077005791}]},"public_addr":"172.21.15.190:6817/4077005791","cluster_addr":"172.21.15.190:6819/4077005791","heartbeat_back_addr":"172.21.15.190:6823/4077005791","heartbeat_front_addr":"172.21.15.190:6821/4077005791","state":["exists","up"]},{"osd":7,"uuid":"7ee736b1-45ad-4daf-aea6-ae92e1f4c170","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3444881554},{"type":"v1","addr":"172.21.15.190:6825","nonce":3444881554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3444881554},{"type":"v1","addr":"172.21.15.190:6827","nonce":3444881554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3444881554},{"type":"v1","addr":"172.21.15.190:6831","nonce":3444881554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3444881554},{"type":"v1","addr":"172.21.15.190:6829","nonce":3444881554}]},"public_addr":"172.21.15.190:6825/3444881554","cluster_addr":"172.21.15.190:6827/3444881554","heartbeat_back_addr":"172.21.15.190:6831/3444881554","heartbeat_front_addr":"172.21.15.190:6829/3444881554","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T03:37:17.820423+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T03:38:11.242278+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T03:39:06.127491+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T03:39:59.306276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T03:40:50.284254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T03:41:45.327530+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T03:42:38.089070+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T03:43:32.633741+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.9","mappings":[{"from":7,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/4127351834":"2024-03-27T03:34:52.164937+0000","172.21.15.175:0/1112608733":"2024-03-27T03:34:16.885473+0000","172.21.15.175:0/202905760":"2024-03-27T03:45:18.442397+0000","172.21.15.175:0/4269788583":"2024-03-27T03:34:16.885473+0000","172.21.15.175:6801/1363563398":"2024-03-27T03:34:16.885473+0000","172.21.15.175:6800/1363563398":"2024-03-27T03:34:16.885473+0000","172.21.15.175:6801/2826039805":"2024-03-27T03:45:18.442397+0000","172.21.15.175:0/544124769":"2024-03-27T03:34:52.164937+0000","172.21.15.175:0/2856312475":"2024-03-27T03:45:18.442397+0000","172.21.15.175:0/2156166628":"2024-03-27T03:34:52.164937+0000","172.21.15.175:6800/712246762":"2024-03-27T03:34:52.164937+0000","172.21.15.175:6801/712246762":"2024-03-27T03:34:52.164937+0000","172.21.15.175:0/94496267":"2024-03-27T03:45:18.442397+0000","172.21.15.175:0/3518793794":"2024-03-27T03:45:18.442397+0000","172.21.15.175:0/460197073":"2024-03-27T03:45:18.442397+0000","172.21.15.175:0/4149587290":"2024-03-27T03:34:16.885473+0000","172.21.15.175:6800/2826039805":"2024-03-27T03:45:18.442397+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T03:46:06.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.837+0000 7fbf5f84a700 1 -- 172.21.15.175:0/3005121607 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fbf4006e7e0 msgr2=0x7fbf40070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:06.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.837+0000 7fbf5f84a700 1 --2- 172.21.15.175:0/3005121607 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fbf4006e7e0 0x7fbf40070ca0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbf580789a0 tx=0x7fbf4c009040 comp rx=0 tx=0).stop 2024-03-26T03:46:06.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.837+0000 7fbf5f84a700 1 -- 172.21.15.175:0/3005121607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf5806acd0 msgr2=0x7fbf581982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:06.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.837+0000 7fbf5f84a700 1 --2- 172.21.15.175:0/3005121607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf5806acd0 0x7fbf581982a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbf48000c00 tx=0x7fbf480047b0 comp rx=0 tx=0).stop 2024-03-26T03:46:06.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.837+0000 7fbf5f84a700 1 -- 172.21.15.175:0/3005121607 shutdown_connections 2024-03-26T03:46:06.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.837+0000 7fbf5f84a700 1 --2- 172.21.15.175:0/3005121607 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fbf4006e7e0 0x7fbf40070ca0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:06.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.837+0000 7fbf5f84a700 1 --2- 172.21.15.175:0/3005121607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf5806acd0 0x7fbf581982a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:06.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.837+0000 7fbf5f84a700 1 --2- 172.21.15.175:0/3005121607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf581087e0 0x7fbf5819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:06.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.837+0000 7fbf5f84a700 1 --2- 172.21.15.175:0/3005121607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf5806b690 0x7fbf581987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:06.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.837+0000 7fbf5f84a700 1 -- 172.21.15.175:0/3005121607 >> 172.21.15.175:0/3005121607 conn(0x7fbf58078430 msgr2=0x7fbf580ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:46:06.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.837+0000 7fbf5f84a700 1 -- 172.21.15.175:0/3005121607 shutdown_connections 2024-03-26T03:46:06.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:06.837+0000 7fbf5f84a700 1 -- 172.21.15.175:0/3005121607 wait complete. 2024-03-26T03:46:06.928 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:06.719479559Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-03-26T03:46:06.929 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:06.753267024Z level=info msg="Executing migration" id="create user table v2" 2024-03-26T03:46:06.929 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:06.786770231Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-03-26T03:46:06.929 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:06.820360405Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-03-26T03:46:06.929 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:06.854028028Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-03-26T03:46:06.929 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:06.887523892Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-03-26T03:46:06.929 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:06.937767986Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-03-26T03:46:07.180 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:07 smithi190 bash[23392]: audit 2024-03-26T03:46:06.835234+0000 mon.c (mon.2) 57 : audit [DBG] from='client.? 172.21.15.175:0/3005121607' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T03:46:07.180 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:06 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:06.98812941Z level=info msg="Executing migration" id="Update user table charset" 2024-03-26T03:46:07.180 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:07 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:07.038543907Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-03-26T03:46:07.180 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:07 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:07.088838781Z level=info msg="Executing migration" id="Add missing user data" 2024-03-26T03:46:07.180 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:07 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:07.139128529Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-03-26T03:46:07.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:06 smithi175 bash[21527]: audit 2024-03-26T03:46:06.835234+0000 mon.c (mon.2) 57 : audit [DBG] from='client.? 172.21.15.175:0/3005121607' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T03:46:07.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:07 smithi175 bash[29284]: audit 2024-03-26T03:46:06.835234+0000 mon.c (mon.2) 57 : audit [DBG] from='client.? 172.21.15.175:0/3005121607' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T03:46:07.448 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:07 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:07.189442176Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-03-26T03:46:07.449 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:07 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:07.222883416Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-03-26T03:46:07.449 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:07 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:07.256526145Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-03-26T03:46:07.449 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:07 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:07.290176509Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-03-26T03:46:07.449 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:07 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:07.323703153Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-03-26T03:46:07.449 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:07 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:07.357408242Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-03-26T03:46:07.449 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:07 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:07.390867475Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-03-26T03:46:07.449 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:07 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:07.424448445Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-03-26T03:46:07.449 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:07 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:07.45813928Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-03-26T03:46:07.632 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-26T03:46:07.633 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-26T03:46:07.701 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:07 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:07.508919864Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-03-26T03:46:07.701 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:07 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:07.559140231Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-03-26T03:46:07.701 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:07 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:07.592701675Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-03-26T03:46:07.701 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:07 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:07.626354631Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-03-26T03:46:07.701 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:07 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:07.659925148Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-03-26T03:46:07.701 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:07 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:07.710238292Z level=info msg="Executing migration" id="create temp_user v2" 2024-03-26T03:46:07.987 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:07 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:07.743798503Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-03-26T03:46:07.987 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:07 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:07.777424759Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-03-26T03:46:07.987 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:07 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:07.811001699Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-03-26T03:46:07.987 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:07 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:07.836205664Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-03-26T03:46:07.987 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:07 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:07.861615694Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-03-26T03:46:07.987 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:07 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:07.886686713Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-03-26T03:46:07.987 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:07 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:07.911957048Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-03-26T03:46:07.987 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:07 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:07.953980066Z level=info msg="Executing migration" id="create star table" 2024-03-26T03:46:07.987 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:07 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:07.995956262Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-03-26T03:46:08.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:08 smithi190 bash[23392]: cluster 2024-03-26T03:46:06.491724+0000 mgr.y (mgr.24431) 54 : cluster [DBG] pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:08.240 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:08 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:08.037942451Z level=info msg="Executing migration" id="create org table v1" 2024-03-26T03:46:08.241 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:08 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:08.063230963Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-03-26T03:46:08.241 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:08 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:08.088554666Z level=info msg="Executing migration" id="create org_user table v1" 2024-03-26T03:46:08.241 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:08 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:08.138761754Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-03-26T03:46:08.241 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:08 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:08.172437112Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-03-26T03:46:08.241 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:08 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:08.222840883Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-03-26T03:46:08.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:08 smithi175 bash[21527]: cluster 2024-03-26T03:46:06.491724+0000 mgr.y (mgr.24431) 54 : cluster [DBG] pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:08.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:08 smithi175 bash[29284]: cluster 2024-03-26T03:46:06.491724+0000 mgr.y (mgr.24431) 54 : cluster [DBG] pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:08.525 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:08 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:08.273233345Z level=info msg="Executing migration" id="Update org table charset" 2024-03-26T03:46:08.526 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:08 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:08.31528482Z level=info msg="Executing migration" id="Update org_user table charset" 2024-03-26T03:46:08.526 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:08 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:08.348923427Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-03-26T03:46:08.526 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:08 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:08.382557506Z level=info msg="Executing migration" id="create dashboard table" 2024-03-26T03:46:08.526 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:08 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:08.41612861Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-03-26T03:46:08.526 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:08 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:08.441384348Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-03-26T03:46:08.526 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:08 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:08.491649689Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-03-26T03:46:08.526 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:08 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:08.53410364Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-03-26T03:46:08.776 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:08 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:08.584422164Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-03-26T03:46:08.777 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:08 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:08.634761487Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-03-26T03:46:08.777 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:08 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:08.668445737Z level=info msg="Executing migration" id="create dashboard v2" 2024-03-26T03:46:08.777 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:08 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:08.718692455Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-03-26T03:46:08.777 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:08 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:08.752346598Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-03-26T03:46:08.777 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:08 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:08.785935438Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-03-26T03:46:09.045 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:08 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:08.819410769Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-03-26T03:46:09.045 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:08 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:08.869876968Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-03-26T03:46:09.045 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:08 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:08.903505529Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-03-26T03:46:09.045 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:08 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:08.953866009Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-03-26T03:46:09.046 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:09 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:09.004221409Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-03-26T03:46:09.314 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:09 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:09.054588273Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-03-26T03:46:09.314 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:09 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:09.10488731Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-03-26T03:46:09.314 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:09 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:09.138574543Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-03-26T03:46:09.315 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:09 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:09.17214509Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-03-26T03:46:09.315 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:09 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:09.222435037Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-03-26T03:46:09.315 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:09 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:09.255977801Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-03-26T03:46:09.315 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:09 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:09.289600108Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-03-26T03:46:09.591 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:46:09 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:46:09.591 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:09 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:09.323291313Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-03-26T03:46:09.591 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:09 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:09.373530262Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-03-26T03:46:09.592 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:09 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:09.407128748Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-03-26T03:46:09.592 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:09 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:09.440712135Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-03-26T03:46:09.592 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:09 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:09.491042466Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-03-26T03:46:09.592 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:09 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:09.533246842Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-03-26T03:46:09.592 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:09 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:09.558471756Z level=info msg="Executing migration" id="Update dashboard title length" 2024-03-26T03:46:09.592 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:09 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:09.600600586Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-03-26T03:46:09.852 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:09 smithi190 bash[23392]: cluster 2024-03-26T03:46:08.492451+0000 mgr.y (mgr.24431) 55 : cluster [DBG] pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:09.852 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:09 smithi190 bash[23392]: audit 2024-03-26T03:46:08.611981+0000 mon.a (mon.0) 841 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:09.852 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:09 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:09.642761512Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-03-26T03:46:09.853 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:09 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:09.66806904Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-03-26T03:46:09.853 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:09 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:09.693322575Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-03-26T03:46:09.853 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:09 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:09.735270949Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-03-26T03:46:09.853 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:09 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:09.760667914Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-03-26T03:46:09.853 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:09 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:09.785813484Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-03-26T03:46:09.853 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:09 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:09.811020979Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-03-26T03:46:09.853 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:09 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:09.836594854Z level=info msg="Executing migration" id="Add check_sum column" 2024-03-26T03:46:09.853 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:09 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:09.861786194Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-03-26T03:46:09.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:09 smithi175 bash[29284]: cluster 2024-03-26T03:46:08.492451+0000 mgr.y (mgr.24431) 55 : cluster [DBG] pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:09.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:09 smithi175 bash[29284]: audit 2024-03-26T03:46:08.611981+0000 mon.a (mon.0) 841 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:09.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:09 smithi175 bash[21527]: cluster 2024-03-26T03:46:08.492451+0000 mgr.y (mgr.24431) 55 : cluster [DBG] pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:09.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:09 smithi175 bash[21527]: audit 2024-03-26T03:46:08.611981+0000 mon.a (mon.0) 841 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:10.129 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:09 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:09.887004748Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-03-26T03:46:10.129 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:09 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:09.929001528Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-03-26T03:46:10.129 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:09 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:09.979475629Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-03-26T03:46:10.129 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:10 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:10.013108735Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-03-26T03:46:10.129 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:10 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:10.046690576Z level=info msg="Executing migration" id="create data_source table" 2024-03-26T03:46:10.129 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:10 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:10.09687087Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-03-26T03:46:10.398 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:10 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:10.138771262Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-03-26T03:46:10.398 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:10 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:10.172573467Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-03-26T03:46:10.398 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:10 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:10.206240989Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-03-26T03:46:10.398 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:10 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:10.239859563Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-03-26T03:46:10.398 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:10 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:10.273404911Z level=info msg="Executing migration" id="create data_source table v2" 2024-03-26T03:46:10.398 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:10 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:10.340421936Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-03-26T03:46:10.398 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:10 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:10.374126722Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-03-26T03:46:10.399 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:10 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:10.407713415Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-03-26T03:46:10.683 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:10 smithi190 bash[23392]: audit 2024-03-26T03:46:09.517903+0000 mgr.y (mgr.24431) 56 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:46:10.684 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:10 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:10.458100306Z level=info msg="Executing migration" id="Add column with_credentials" 2024-03-26T03:46:10.684 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:10 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:10.508378127Z level=info msg="Executing migration" id="Add secure json data column" 2024-03-26T03:46:10.684 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:10 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:10.558979029Z level=info msg="Executing migration" id="Update data_source table charset" 2024-03-26T03:46:10.684 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:10 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:10.642954634Z level=info msg="Executing migration" id="Update initial version to 1" 2024-03-26T03:46:10.969 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:10 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:10.693393335Z level=info msg="Executing migration" id="Add read_only data column" 2024-03-26T03:46:10.970 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:10 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:10.718842438Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-03-26T03:46:10.970 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:10 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:10.769358153Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-03-26T03:46:10.970 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:10 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:10.811206981Z level=info msg="Executing migration" id="Add uid column" 2024-03-26T03:46:10.970 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:10 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:10.878491181Z level=info msg="Executing migration" id="Update uid value" 2024-03-26T03:46:10.970 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:10 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:10.928941981Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-03-26T03:46:10.970 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:10 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:10.979255785Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-03-26T03:46:10.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:10 smithi175 bash[21527]: audit 2024-03-26T03:46:09.517903+0000 mgr.y (mgr.24431) 56 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:46:10.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:10 smithi175 bash[29284]: audit 2024-03-26T03:46:09.517903+0000 mgr.y (mgr.24431) 56 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:46:11.240 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:10 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:10.996293723Z level=info msg="Executing migration" id="create api_key table" 2024-03-26T03:46:11.240 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:11.013210098Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-03-26T03:46:11.240 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:11.046844628Z level=info msg="Executing migration" id="add index api_key.key" 2024-03-26T03:46:11.240 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:11.113230319Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-03-26T03:46:11.241 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:11.223580704Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-03-26T03:46:11.616 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:11.373283916Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-03-26T03:46:11.616 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:11.43188293Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-03-26T03:46:11.616 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:11.473903508Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-03-26T03:46:11.616 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:11.515849977Z level=info msg="Executing migration" id="create api_key table v2" 2024-03-26T03:46:11.616 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:11.557875324Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-03-26T03:46:11.616 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:11.59986074Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-03-26T03:46:11.616 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:11.625127551Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-03-26T03:46:11.885 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[23392]: cluster 2024-03-26T03:46:10.493216+0000 mgr.y (mgr.24431) 57 : cluster [DBG] pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:11.886 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[23392]: audit 2024-03-26T03:46:10.737099+0000 mon.a (mon.0) 842 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:11.886 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[23392]: audit 2024-03-26T03:46:10.743337+0000 mon.a (mon.0) 843 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:11.886 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[23392]: audit 2024-03-26T03:46:11.145737+0000 mon.a (mon.0) 844 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:11.886 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[23392]: audit 2024-03-26T03:46:11.165834+0000 mon.a (mon.0) 845 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:11.886 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[23392]: audit 2024-03-26T03:46:11.171533+0000 mon.c (mon.2) 58 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:46:11.886 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[23392]: audit 2024-03-26T03:46:11.173523+0000 mon.c (mon.2) 59 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:46:11.886 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[23392]: audit 2024-03-26T03:46:11.184266+0000 mon.a (mon.0) 846 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:11.886 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[23392]: cephadm 2024-03-26T03:46:11.266259+0000 mgr.y (mgr.24431) 58 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2024-03-26T03:46:11.886 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[23392]: cephadm 2024-03-26T03:46:11.274912+0000 mgr.y (mgr.24431) 59 : cephadm [INF] Reconfiguring daemon alertmanager.a on smithi175 2024-03-26T03:46:11.886 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:11.650436785Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-03-26T03:46:11.887 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:11.675922818Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-03-26T03:46:11.887 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:11.701171342Z level=info msg="Executing migration" id="Update api_key table charset" 2024-03-26T03:46:11.887 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:11.726392994Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-03-26T03:46:11.887 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:11.7683975Z level=info msg="Executing migration" id="Add service account foreign key" 2024-03-26T03:46:11.887 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:11.793724076Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-03-26T03:46:11.887 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:11.819089173Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-03-26T03:46:11.887 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:11.844300465Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-03-26T03:46:11.887 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:11.869613716Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-03-26T03:46:11.887 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:11.894762841Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-03-26T03:46:11.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:11 smithi175 bash[21527]: cluster 2024-03-26T03:46:10.493216+0000 mgr.y (mgr.24431) 57 : cluster [DBG] pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:11.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:11 smithi175 bash[21527]: audit 2024-03-26T03:46:10.737099+0000 mon.a (mon.0) 842 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:11.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:11 smithi175 bash[21527]: audit 2024-03-26T03:46:10.743337+0000 mon.a (mon.0) 843 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:11.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:11 smithi175 bash[21527]: audit 2024-03-26T03:46:11.145737+0000 mon.a (mon.0) 844 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:11.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:11 smithi175 bash[21527]: audit 2024-03-26T03:46:11.165834+0000 mon.a (mon.0) 845 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:11.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:11 smithi175 bash[21527]: audit 2024-03-26T03:46:11.171533+0000 mon.c (mon.2) 58 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:46:11.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:11 smithi175 bash[21527]: audit 2024-03-26T03:46:11.173523+0000 mon.c (mon.2) 59 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:46:11.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:11 smithi175 bash[21527]: audit 2024-03-26T03:46:11.184266+0000 mon.a (mon.0) 846 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:11.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:11 smithi175 bash[21527]: cephadm 2024-03-26T03:46:11.266259+0000 mgr.y (mgr.24431) 58 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2024-03-26T03:46:11.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:11 smithi175 bash[21527]: cephadm 2024-03-26T03:46:11.274912+0000 mgr.y (mgr.24431) 59 : cephadm [INF] Reconfiguring daemon alertmanager.a on smithi175 2024-03-26T03:46:11.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:11 smithi175 bash[29284]: cluster 2024-03-26T03:46:10.493216+0000 mgr.y (mgr.24431) 57 : cluster [DBG] pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:11.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:11 smithi175 bash[29284]: audit 2024-03-26T03:46:10.737099+0000 mon.a (mon.0) 842 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:11.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:11 smithi175 bash[29284]: audit 2024-03-26T03:46:10.743337+0000 mon.a (mon.0) 843 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:11.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:11 smithi175 bash[29284]: audit 2024-03-26T03:46:11.145737+0000 mon.a (mon.0) 844 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:11.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:11 smithi175 bash[29284]: audit 2024-03-26T03:46:11.165834+0000 mon.a (mon.0) 845 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:11.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:11 smithi175 bash[29284]: audit 2024-03-26T03:46:11.171533+0000 mon.c (mon.2) 58 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:46:11.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:11 smithi175 bash[29284]: audit 2024-03-26T03:46:11.173523+0000 mon.c (mon.2) 59 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:46:11.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:11 smithi175 bash[29284]: audit 2024-03-26T03:46:11.184266+0000 mon.a (mon.0) 846 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:11.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:11 smithi175 bash[29284]: cephadm 2024-03-26T03:46:11.266259+0000 mgr.y (mgr.24431) 58 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2024-03-26T03:46:11.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:11 smithi175 bash[29284]: cephadm 2024-03-26T03:46:11.274912+0000 mgr.y (mgr.24431) 59 : cephadm [INF] Reconfiguring daemon alertmanager.a on smithi175 2024-03-26T03:46:12.145 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:11.920096815Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-03-26T03:46:12.146 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:11 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:11.970389219Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-03-26T03:46:12.146 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:12 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:12.004075125Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-03-26T03:46:12.146 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:12 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:12.054424248Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-03-26T03:46:12.146 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:12 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:12.104697586Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-03-26T03:46:12.146 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:12 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:12.155064977Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-03-26T03:46:12.174 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:46:12.431 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:12 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:12.205399867Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-03-26T03:46:12.431 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:12 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:12.238985314Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-03-26T03:46:12.431 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:12 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:12.27264231Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-03-26T03:46:12.432 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:12 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:12.306177957Z level=info msg="Executing migration" id="create quota table v1" 2024-03-26T03:46:12.432 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:12 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:12.356549024Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-03-26T03:46:12.432 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:12 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:12.390300673Z level=info msg="Executing migration" id="Update quota table charset" 2024-03-26T03:46:12.432 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:12 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:12.440596497Z level=info msg="Executing migration" id="create plugin_setting table" 2024-03-26T03:46:12.700 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:12 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:12.47423378Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-03-26T03:46:12.700 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:12 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:12.524543997Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-03-26T03:46:12.700 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:12 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:12.574851674Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-03-26T03:46:12.700 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:12 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:12.608450348Z level=info msg="Executing migration" id="create session table" 2024-03-26T03:46:12.700 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:12 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:12.642174804Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-03-26T03:46:12.700 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:12 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:12.667485978Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-03-26T03:46:12.701 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:12 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:12.709595285Z level=info msg="Executing migration" id="create playlist table v2" 2024-03-26T03:46:12.952 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:12 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:12.734952366Z level=info msg="Executing migration" id="create playlist item table v2" 2024-03-26T03:46:12.952 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:12 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:12.776825445Z level=info msg="Executing migration" id="Update playlist table charset" 2024-03-26T03:46:12.953 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:12 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:12.80211205Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-03-26T03:46:12.953 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:12 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:12.827417356Z level=info msg="Executing migration" id="drop preferences table v2" 2024-03-26T03:46:12.953 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:12 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:12.852656201Z level=info msg="Executing migration" id="drop preferences table v3" 2024-03-26T03:46:12.953 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:12 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:12.877923529Z level=info msg="Executing migration" id="create preferences table v3" 2024-03-26T03:46:12.953 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:12 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:12.911600745Z level=info msg="Executing migration" id="Update preferences table charset" 2024-03-26T03:46:13.204 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:12 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:12.961944015Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-03-26T03:46:13.204 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:13 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:13.012316246Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-03-26T03:46:13.204 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:13 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:13.045879762Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-03-26T03:46:13.205 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:13 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:13.096146684Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-03-26T03:46:13.205 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:13 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:13.129799373Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-03-26T03:46:13.205 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:13 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:13.180101322Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-03-26T03:46:13.205 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:13 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:13.213760164Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-03-26T03:46:13.490 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:13 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:13.24741405Z level=info msg="Executing migration" id="create alert table v1" 2024-03-26T03:46:13.490 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:13 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:13.280966937Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-03-26T03:46:13.490 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:13 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:13.314658496Z level=info msg="Executing migration" id="add index alert state" 2024-03-26T03:46:13.490 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:13 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:13.364921351Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-03-26T03:46:13.490 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:13 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:13.398537958Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-03-26T03:46:13.490 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:13 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:13.43219923Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-03-26T03:46:13.490 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:13 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:13.457485904Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-03-26T03:46:13.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:13 smithi190 bash[23392]: cluster 2024-03-26T03:46:12.493965+0000 mgr.y (mgr.24431) 60 : cluster [DBG] pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:13.740 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:13 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:13.49936877Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-03-26T03:46:13.740 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:13 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:13.549748871Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-03-26T03:46:13.741 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:13 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:13.575031158Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-03-26T03:46:13.741 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:13 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:13.617018994Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-03-26T03:46:13.741 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:13 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:13.642239302Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-03-26T03:46:13.741 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:13 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:13.667744207Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-03-26T03:46:13.741 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:13 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:13.693003292Z level=info msg="Executing migration" id="Add column is_default" 2024-03-26T03:46:13.741 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:13 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:13.718290676Z level=info msg="Executing migration" id="Add column frequency" 2024-03-26T03:46:14.020 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:13 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:13.760261028Z level=info msg="Executing migration" id="Add column send_reminder" 2024-03-26T03:46:14.020 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:13 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:13.802333786Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-03-26T03:46:14.020 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:13 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:13.827671557Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-03-26T03:46:14.020 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:13 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:13.852929211Z level=info msg="Executing migration" id="Update alert table charset" 2024-03-26T03:46:14.020 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:13 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:13.894843683Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-03-26T03:46:14.020 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:13 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:13.920102985Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-03-26T03:46:14.020 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:13 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:13.962116023Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-03-26T03:46:14.020 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:14 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:14.004135428Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-03-26T03:46:14.021 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:14 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:14.02935018Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-03-26T03:46:14.045 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:13 smithi175 bash[21527]: cluster 2024-03-26T03:46:12.493965+0000 mgr.y (mgr.24431) 60 : cluster [DBG] pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:14.045 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:13 smithi175 bash[29284]: cluster 2024-03-26T03:46:12.493965+0000 mgr.y (mgr.24431) 60 : cluster [DBG] pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:14.297 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:14 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:14.054601787Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-03-26T03:46:14.297 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:14 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:14.104864288Z level=info msg="Executing migration" id="Add for to alert table" 2024-03-26T03:46:14.297 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:14 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:14.155227169Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-03-26T03:46:14.297 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:14 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:14.188872795Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-03-26T03:46:14.297 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:14 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:14.239338218Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-03-26T03:46:14.297 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:14 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:14.272962771Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-03-26T03:46:14.549 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:14 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:14.306526275Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-03-26T03:46:14.549 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:14 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:14.356742249Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-03-26T03:46:14.549 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:14 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:14.407364834Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-03-26T03:46:14.549 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:14 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:14.457631385Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-03-26T03:46:14.549 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:14 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:14.508032948Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-03-26T03:46:14.549 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:14 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:14.558352546Z level=info msg="Executing migration" id="create annotation table v5" 2024-03-26T03:46:14.826 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:14 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:14.592063998Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-03-26T03:46:14.826 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:14 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:14.625625809Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-03-26T03:46:14.826 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:14 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:14.701146801Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-03-26T03:46:14.826 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:14 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:14.73478417Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-03-26T03:46:14.826 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:14 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:14.801819235Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-03-26T03:46:14.826 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:14 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:14.835403271Z level=info msg="Executing migration" id="Update annotation table charset" 2024-03-26T03:46:15.120 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:14 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:14.868884406Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-03-26T03:46:15.120 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:14 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:14.902505012Z level=info msg="Executing migration" id="Drop category_id index" 2024-03-26T03:46:15.120 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:14 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:14.927751003Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-03-26T03:46:15.120 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:14 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:14.969805732Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-03-26T03:46:15.120 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:15 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:15.011891868Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-03-26T03:46:15.120 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:15 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:15.037191892Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-03-26T03:46:15.120 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:15 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:15.079137325Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-03-26T03:46:15.120 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:15 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:15.129397546Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-03-26T03:46:15.381 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:15 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:15.171468007Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-03-26T03:46:15.381 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:15 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:15.213558862Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-03-26T03:46:15.381 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:15 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:15.238723496Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-03-26T03:46:15.381 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:15 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:15.263905582Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-03-26T03:46:15.381 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:15 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:15.289208862Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-03-26T03:46:15.381 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:15 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:15.314476044Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-03-26T03:46:15.381 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:15 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:15.339678876Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-03-26T03:46:15.381 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:15 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:15.364933126Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-03-26T03:46:15.382 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:15 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:15.390267707Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-03-26T03:46:15.649 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:15 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:15.432235595Z level=info msg="Executing migration" id="Add epoch_end column" 2024-03-26T03:46:15.650 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:15 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:15.457512603Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-03-26T03:46:15.650 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:15 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:15.507941346Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-03-26T03:46:15.650 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:15 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:15.549870112Z level=info msg="Executing migration" id="Move region to single row" 2024-03-26T03:46:15.650 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:15 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:15.575123516Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-03-26T03:46:15.650 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:15 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:15.608689537Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-03-26T03:46:15.650 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:15 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:15.659008658Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-03-26T03:46:15.902 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:15 smithi190 bash[23392]: cluster 2024-03-26T03:46:14.495619+0000 mgr.y (mgr.24431) 61 : cluster [DBG] pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:15.902 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:15 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:15.751409451Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-03-26T03:46:15.902 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:15 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:15.793471989Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-03-26T03:46:15.902 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:15 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:15.835501521Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-03-26T03:46:15.902 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:15 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:15.860840228Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-03-26T03:46:15.902 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:15 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:15.886207022Z level=info msg="Executing migration" id="create test_data table" 2024-03-26T03:46:15.903 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:15 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:15.911539679Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-03-26T03:46:16.171 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:15 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:15.970152509Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-03-26T03:46:16.171 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:15 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:15.995480123Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-03-26T03:46:16.171 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:16 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:16.020694218Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-03-26T03:46:16.171 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:16 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:16.045757357Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-03-26T03:46:16.171 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:16 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:16.087807945Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-03-26T03:46:16.171 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:16 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:16.129846363Z level=info msg="Executing migration" id="create team table" 2024-03-26T03:46:16.171 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:16 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:16.155175657Z level=info msg="Executing migration" id="add index team.org_id" 2024-03-26T03:46:16.172 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:16 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:16.180491984Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-03-26T03:46:16.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:15 smithi175 bash[21527]: cluster 2024-03-26T03:46:14.495619+0000 mgr.y (mgr.24431) 61 : cluster [DBG] pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:16.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:15 smithi175 bash[29284]: cluster 2024-03-26T03:46:14.495619+0000 mgr.y (mgr.24431) 61 : cluster [DBG] pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:16.423 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:16 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:16.230751399Z level=info msg="Executing migration" id="create team member table" 2024-03-26T03:46:16.423 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:16 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:16.25604837Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-03-26T03:46:16.423 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:16 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:16.298020182Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-03-26T03:46:16.423 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:16 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:16.348545758Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-03-26T03:46:16.423 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:16 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:16.398862724Z level=info msg="Executing migration" id="Add column email to team table" 2024-03-26T03:46:16.423 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:16 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:16.432503888Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-03-26T03:46:16.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.461+0000 7f731db3f700 1 -- 172.21.15.175:0/2343787927 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7318108ed0 msgr2=0x7f73181092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:16.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.461+0000 7f731db3f700 1 --2- 172.21.15.175:0/2343787927 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7318108ed0 0x7f73181092b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f73080099c0 tx=0x7f7308009cd0 comp rx=0 tx=0).stop 2024-03-26T03:46:16.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.461+0000 7f731db3f700 1 -- 172.21.15.175:0/2343787927 shutdown_connections 2024-03-26T03:46:16.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.461+0000 7f731db3f700 1 --2- 172.21.15.175:0/2343787927 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7318108ed0 0x7f73181092b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:16.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.461+0000 7f731db3f700 1 --2- 172.21.15.175:0/2343787927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f731810dfb0 0x7f7318115b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:16.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.461+0000 7f731db3f700 1 --2- 172.21.15.175:0/2343787927 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7318109880 0x7f731810d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:16.462 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.461+0000 7f731db3f700 1 -- 172.21.15.175:0/2343787927 >> 172.21.15.175:0/2343787927 conn(0x7f7318075e10 msgr2=0x7f7318078230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:46:16.463 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.461+0000 7f731db3f700 1 -- 172.21.15.175:0/2343787927 shutdown_connections 2024-03-26T03:46:16.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.461+0000 7f731db3f700 1 -- 172.21.15.175:0/2343787927 wait complete. 2024-03-26T03:46:16.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.461+0000 7f731db3f700 1 Processor -- start 2024-03-26T03:46:16.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.461+0000 7f731db3f700 1 -- start start 2024-03-26T03:46:16.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.461+0000 7f731db3f700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7318108ed0 0x7f73181a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:16.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.461+0000 7f731db3f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7318109880 0x7f73181a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:16.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.461+0000 7f731db3f700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f731810dfb0 0x7f731819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:16.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.461+0000 7f731db3f700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f73181184c0 con 0x7f731810dfb0 2024-03-26T03:46:16.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.461+0000 7f731db3f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7318118340 con 0x7f7318109880 2024-03-26T03:46:16.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.461+0000 7f731db3f700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7318118640 con 0x7f7318108ed0 2024-03-26T03:46:16.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.461+0000 7f73177fe700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7318108ed0 0x7f73181a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:16.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.461+0000 7f73177fe700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7318108ed0 0x7f73181a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:36212/0 (socket says 172.21.15.175:36212) 2024-03-26T03:46:16.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.461+0000 7f73177fe700 1 -- 172.21.15.175:0/2489340523 learned_addr learned my addr 172.21.15.175:0/2489340523 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:46:16.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.461+0000 7f7317fff700 1 --2- 172.21.15.175:0/2489340523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f731810dfb0 0x7f731819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:16.464 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.461+0000 7f7316ffd700 1 --2- 172.21.15.175:0/2489340523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7318109880 0x7f73181a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:16.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.461+0000 7f73177fe700 1 -- 172.21.15.175:0/2489340523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7318109880 msgr2=0x7f73181a4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:16.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.461+0000 7f73177fe700 1 --2- 172.21.15.175:0/2489340523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7318109880 0x7f73181a4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:16.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.461+0000 7f73177fe700 1 -- 172.21.15.175:0/2489340523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f731810dfb0 msgr2=0x7f731819e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:16.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.461+0000 7f73177fe700 1 --2- 172.21.15.175:0/2489340523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f731810dfb0 0x7f731819e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:16.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.461+0000 7f73177fe700 1 -- 172.21.15.175:0/2489340523 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f731819efa0 con 0x7f7318108ed0 2024-03-26T03:46:16.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.461+0000 7f7317fff700 1 --2- 172.21.15.175:0/2489340523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f731810dfb0 0x7f731819e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T03:46:16.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.465+0000 7f73177fe700 1 --2- 172.21.15.175:0/2489340523 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7318108ed0 0x7f73181a4640 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7308006010 tx=0x7f73080056a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:46:16.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.465+0000 7f7314ff9700 1 -- 172.21.15.175:0/2489340523 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308026070 con 0x7f7318108ed0 2024-03-26T03:46:16.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.465+0000 7f7314ff9700 1 -- 172.21.15.175:0/2489340523 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f7308003e50 con 0x7f7318108ed0 2024-03-26T03:46:16.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.465+0000 7f731db3f700 1 -- 172.21.15.175:0/2489340523 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f731819f230 con 0x7f7318108ed0 2024-03-26T03:46:16.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.465+0000 7f7314ff9700 1 -- 172.21.15.175:0/2489340523 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730800fba0 con 0x7f7318108ed0 2024-03-26T03:46:16.465 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.465+0000 7f731db3f700 1 -- 172.21.15.175:0/2489340523 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f73181ab4a0 con 0x7f7318108ed0 2024-03-26T03:46:16.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.465+0000 7f731db3f700 1 -- 172.21.15.175:0/2489340523 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f731810a3b0 con 0x7f7318108ed0 2024-03-26T03:46:16.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.465+0000 7f7314ff9700 1 -- 172.21.15.175:0/2489340523 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7f7308029070 con 0x7f7318108ed0 2024-03-26T03:46:16.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.465+0000 7f7314ff9700 1 --2- 172.21.15.175:0/2489340523 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f730006e7e0 0x7f7300070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:16.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.465+0000 7f7316ffd700 1 --2- 172.21.15.175:0/2489340523 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f730006e7e0 0x7f7300070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:16.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.465+0000 7f7314ff9700 1 -- 172.21.15.175:0/2489340523 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(75..75 src has 1..75) ==== 7537+0+0 (secure 0 0 0) 0x7f730808f900 con 0x7f7318108ed0 2024-03-26T03:46:16.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.465+0000 7f7316ffd700 1 --2- 172.21.15.175:0/2489340523 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f730006e7e0 0x7f7300070ca0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7318078580 tx=0x7f730c009260 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:46:16.470 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.469+0000 7f7314ff9700 1 -- 172.21.15.175:0/2489340523 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f7308013070 con 0x7f7318108ed0 2024-03-26T03:46:16.674 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:16 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:16.482823831Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-03-26T03:46:16.675 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:16 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:16.533169795Z level=info msg="Executing migration" id="create dashboard acl table" 2024-03-26T03:46:16.675 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:16 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:16.583436357Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-03-26T03:46:16.675 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:16 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:16.617144495Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-03-26T03:46:16.675 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:16 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:16.650696486Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-03-26T03:46:16.675 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:16 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:16.684310199Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-03-26T03:46:16.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.681+0000 7f731db3f700 1 -- 172.21.15.175:0/2489340523 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7318050300 con 0x7f7318108ed0 2024-03-26T03:46:16.683 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.685+0000 7f7314ff9700 1 -- 172.21.15.175:0/2489340523 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+23247 (secure 0 0 0) 0x7f730805c8f0 con 0x7f7318108ed0 2024-03-26T03:46:16.684 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-26T03:46:16.684 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":75,"fsid":"762e930c-eb21-11ee-95d0-87774f69a715","created":"2024-03-26T03:33:45.388461+0000","modified":"2024-03-26T03:45:18.442567+0000","last_up_change":"2024-03-26T03:43:33.976035+0000","last_in_change":"2024-03-26T03:43:07.107635+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T03:39:08.367712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-03-26T03:44:15.186218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.log","create_time":"2024-03-26T03:44:16.391387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.control","create_time":"2024-03-26T03:44:18.319611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"datapool","create_time":"2024-03-26T03:44:19.627074+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":71,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-03-26T03:44:20.435692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"41a2182b-5eeb-47fa-be41-1d7639df2f35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":241042466},{"type":"v1","addr":"172.21.15.175:6803","nonce":241042466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":241042466},{"type":"v1","addr":"172.21.15.175:6805","nonce":241042466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":241042466},{"type":"v1","addr":"172.21.15.175:6809","nonce":241042466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":241042466},{"type":"v1","addr":"172.21.15.175:6807","nonce":241042466}]},"public_addr":"172.21.15.175:6803/241042466","cluster_addr":"172.21.15.175:6805/241042466","heartbeat_back_addr":"172.21.15.175:6809/241042466","heartbeat_front_addr":"172.21.15.175:6807/241042466","state":["exists","up"]},{"osd":1,"uuid":"8fbe50bf-5723-4174-9c85-6010a32bd55b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":931168929},{"type":"v1","addr":"172.21.15.175:6811","nonce":931168929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":931168929},{"type":"v1","addr":"172.21.15.175:6813","nonce":931168929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":931168929},{"type":"v1","addr":"172.21.15.175:6817","nonce":931168929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":931168929},{"type":"v1","addr":"172.21.15.175:6815","nonce":931168929}]},"public_addr":"172.21.15.175:6811/931168929","cluster_addr":"172.21.15.175:6813/931168929","heartbeat_back_addr":"172.21.15.175:6817/931168929","heartbeat_front_addr":"172.21.15.175:6815/931168929","state":["exists","up"]},{"osd":2,"uuid":"02333e36-f498-4ac2-ab6e-5ae1e1148e97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":3775031674},{"type":"v1","addr":"172.21.15.175:6819","nonce":3775031674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":3775031674},{"type":"v1","addr":"172.21.15.175:6821","nonce":3775031674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":3775031674},{"type":"v1","addr":"172.21.15.175:6825","nonce":3775031674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":3775031674},{"type":"v1","addr":"172.21.15.175:6823","nonce":3775031674}]},"public_addr":"172.21.15.175:6819/3775031674","cluster_addr":"172.21.15.175:6821/3775031674","heartbeat_back_addr":"172.21.15.175:6825/3775031674","heartbeat_front_addr":"172.21.15.175:6823/3775031674","state":["exists","up"]},{"osd":3,"uuid":"8add17f6-b408-4dc0-af05-b3b64845d8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":672326154},{"type":"v1","addr":"172.21.15.175:6827","nonce":672326154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":672326154},{"type":"v1","addr":"172.21.15.175:6829","nonce":672326154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":672326154},{"type":"v1","addr":"172.21.15.175:6833","nonce":672326154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":672326154},{"type":"v1","addr":"172.21.15.175:6831","nonce":672326154}]},"public_addr":"172.21.15.175:6827/672326154","cluster_addr":"172.21.15.175:6829/672326154","heartbeat_back_addr":"172.21.15.175:6833/672326154","heartbeat_front_addr":"172.21.15.175:6831/672326154","state":["exists","up"]},{"osd":4,"uuid":"54aac0c2-d77e-4bf4-b1e5-856b4e997426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":874343325},{"type":"v1","addr":"172.21.15.190:6801","nonce":874343325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":874343325},{"type":"v1","addr":"172.21.15.190:6803","nonce":874343325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":874343325},{"type":"v1","addr":"172.21.15.190:6807","nonce":874343325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":874343325},{"type":"v1","addr":"172.21.15.190:6805","nonce":874343325}]},"public_addr":"172.21.15.190:6801/874343325","cluster_addr":"172.21.15.190:6803/874343325","heartbeat_back_addr":"172.21.15.190:6807/874343325","heartbeat_front_addr":"172.21.15.190:6805/874343325","state":["exists","up"]},{"osd":5,"uuid":"d1eb5f92-1729-48c7-a89c-3265609fe40d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":4089468657},{"type":"v1","addr":"172.21.15.190:6809","nonce":4089468657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":4089468657},{"type":"v1","addr":"172.21.15.190:6811","nonce":4089468657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":4089468657},{"type":"v1","addr":"172.21.15.190:6815","nonce":4089468657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":4089468657},{"type":"v1","addr":"172.21.15.190:6813","nonce":4089468657}]},"public_addr":"172.21.15.190:6809/4089468657","cluster_addr":"172.21.15.190:6811/4089468657","heartbeat_back_addr":"172.21.15.190:6815/4089468657","heartbeat_front_addr":"172.21.15.190:6813/4089468657","state":["exists","up"]},{"osd":6,"uuid":"01cebff3-5246-49f3-893d-706cbfd9d829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":4077005791},{"type":"v1","addr":"172.21.15.190:6817","nonce":4077005791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":4077005791},{"type":"v1","addr":"172.21.15.190:6819","nonce":4077005791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":4077005791},{"type":"v1","addr":"172.21.15.190:6823","nonce":4077005791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":4077005791},{"type":"v1","addr":"172.21.15.190:6821","nonce":4077005791}]},"public_addr":"172.21.15.190:6817/4077005791","cluster_addr":"172.21.15.190:6819/4077005791","heartbeat_back_addr":"172.21.15.190:6823/4077005791","heartbeat_front_addr":"172.21.15.190:6821/4077005791","state":["exists","up"]},{"osd":7,"uuid":"7ee736b1-45ad-4daf-aea6-ae92e1f4c170","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3444881554},{"type":"v1","addr":"172.21.15.190:6825","nonce":3444881554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3444881554},{"type":"v1","addr":"172.21.15.190:6827","nonce":3444881554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3444881554},{"type":"v1","addr":"172.21.15.190:6831","nonce":3444881554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3444881554},{"type":"v1","addr":"172.21.15.190:6829","nonce":3444881554}]},"public_addr":"172.21.15.190:6825/3444881554","cluster_addr":"172.21.15.190:6827/3444881554","heartbeat_back_addr":"172.21.15.190:6831/3444881554","heartbeat_front_addr":"172.21.15.190:6829/3444881554","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T03:37:17.820423+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T03:38:11.242278+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T03:39:06.127491+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T03:39:59.306276+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T03:40:50.284254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T03:41:45.327530+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T03:42:38.089070+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T03:43:32.633741+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.9","mappings":[{"from":7,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:0/4127351834":"2024-03-27T03:34:52.164937+0000","172.21.15.175:0/1112608733":"2024-03-27T03:34:16.885473+0000","172.21.15.175:0/202905760":"2024-03-27T03:45:18.442397+0000","172.21.15.175:0/4269788583":"2024-03-27T03:34:16.885473+0000","172.21.15.175:6801/1363563398":"2024-03-27T03:34:16.885473+0000","172.21.15.175:6800/1363563398":"2024-03-27T03:34:16.885473+0000","172.21.15.175:6801/2826039805":"2024-03-27T03:45:18.442397+0000","172.21.15.175:0/544124769":"2024-03-27T03:34:52.164937+0000","172.21.15.175:0/2856312475":"2024-03-27T03:45:18.442397+0000","172.21.15.175:0/2156166628":"2024-03-27T03:34:52.164937+0000","172.21.15.175:6800/712246762":"2024-03-27T03:34:52.164937+0000","172.21.15.175:6801/712246762":"2024-03-27T03:34:52.164937+0000","172.21.15.175:0/94496267":"2024-03-27T03:45:18.442397+0000","172.21.15.175:0/3518793794":"2024-03-27T03:45:18.442397+0000","172.21.15.175:0/460197073":"2024-03-27T03:45:18.442397+0000","172.21.15.175:0/4149587290":"2024-03-27T03:34:16.885473+0000","172.21.15.175:6800/2826039805":"2024-03-27T03:45:18.442397+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T03:46:16.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.685+0000 7f731db3f700 1 -- 172.21.15.175:0/2489340523 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f730006e7e0 msgr2=0x7f7300070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:16.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.685+0000 7f731db3f700 1 --2- 172.21.15.175:0/2489340523 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f730006e7e0 0x7f7300070ca0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7318078580 tx=0x7f730c009260 comp rx=0 tx=0).stop 2024-03-26T03:46:16.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.685+0000 7f731db3f700 1 -- 172.21.15.175:0/2489340523 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7318108ed0 msgr2=0x7f73181a4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:16.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.685+0000 7f731db3f700 1 --2- 172.21.15.175:0/2489340523 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7318108ed0 0x7f73181a4640 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7308006010 tx=0x7f73080056a0 comp rx=0 tx=0).stop 2024-03-26T03:46:16.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.685+0000 7f731db3f700 1 -- 172.21.15.175:0/2489340523 shutdown_connections 2024-03-26T03:46:16.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.685+0000 7f731db3f700 1 --2- 172.21.15.175:0/2489340523 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f730006e7e0 0x7f7300070ca0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:16.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.685+0000 7f731db3f700 1 --2- 172.21.15.175:0/2489340523 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7318108ed0 0x7f73181a4640 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:16.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.685+0000 7f731db3f700 1 --2- 172.21.15.175:0/2489340523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f731810dfb0 0x7f731819e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:16.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.685+0000 7f731db3f700 1 --2- 172.21.15.175:0/2489340523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7318109880 0x7f73181a4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:16.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.685+0000 7f731db3f700 1 -- 172.21.15.175:0/2489340523 >> 172.21.15.175:0/2489340523 conn(0x7f7318075e10 msgr2=0x7f731810d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:46:16.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.685+0000 7f731db3f700 1 -- 172.21.15.175:0/2489340523 shutdown_connections 2024-03-26T03:46:16.687 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:16.685+0000 7f731db3f700 1 -- 172.21.15.175:0/2489340523 wait complete. 2024-03-26T03:46:16.960 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:16 smithi190 bash[23392]: audit 2024-03-26T03:46:16.684411+0000 mon.c (mon.2) 60 : audit [DBG] from='client.? 172.21.15.175:0/2489340523' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T03:46:16.960 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:16 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:16.735003269Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-03-26T03:46:16.961 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:16 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:16.785414444Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-03-26T03:46:16.961 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:16 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:16.818964674Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-03-26T03:46:16.961 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:16 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:16.852511657Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-03-26T03:46:16.961 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:16 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:16.886111528Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-03-26T03:46:16.961 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:16 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:16.919526095Z level=info msg="Executing migration" id="create tag table" 2024-03-26T03:46:16.961 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:16 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:16.969992635Z level=info msg="Executing migration" id="add index tag.key_value" 2024-03-26T03:46:16.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:16 smithi175 bash[21527]: audit 2024-03-26T03:46:16.684411+0000 mon.c (mon.2) 60 : audit [DBG] from='client.? 172.21.15.175:0/2489340523' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T03:46:16.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:16 smithi175 bash[29284]: audit 2024-03-26T03:46:16.684411+0000 mon.c (mon.2) 60 : audit [DBG] from='client.? 172.21.15.175:0/2489340523' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T03:46:17.237 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:17 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:17.020303676Z level=info msg="Executing migration" id="create login attempt table" 2024-03-26T03:46:17.238 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:17 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:17.070761982Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-03-26T03:46:17.238 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:17 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:17.104372912Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-03-26T03:46:17.238 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:17 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:17.137921239Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-03-26T03:46:17.238 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:17 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:17.179838288Z level=info msg="Executing migration" id="create login_attempt v2" 2024-03-26T03:46:17.238 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:17 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:17.213470938Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-03-26T03:46:17.238 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:17 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:17.247081884Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-03-26T03:46:17.489 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:17 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:17.297285839Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-03-26T03:46:17.489 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:17 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:17.34760071Z level=info msg="Executing migration" id="create user auth table" 2024-03-26T03:46:17.489 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:17 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:17.381274966Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-03-26T03:46:17.489 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:17 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:17.41474525Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-03-26T03:46:17.489 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:17 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:17.448406416Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-03-26T03:46:17.741 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:17 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:17.498732027Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-03-26T03:46:17.741 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:17 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:17.532394727Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-03-26T03:46:17.741 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:17 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:17.565981743Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-03-26T03:46:17.741 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:17 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:17.616325741Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-03-26T03:46:17.741 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:17 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:17.649914514Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-03-26T03:46:17.741 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:17 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:17.700067579Z level=info msg="Executing migration" id="create server_lock table" 2024-03-26T03:46:17.741 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:17 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:17.733838684Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-03-26T03:46:17.741 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:46:17 smithi175 systemd[1]: Stopping Ceph alertmanager.a for 762e930c-eb21-11ee-95d0-87774f69a715... 2024-03-26T03:46:17.906 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-26T03:46:17.906 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-26T03:46:17.907 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-26T03:46:17.907 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-26T03:46:17.907 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-26T03:46:17.908 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-26T03:46:17.908 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-26T03:46:17.909 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-26T03:46:18.021 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:17 smithi175 bash[21527]: cluster 2024-03-26T03:46:16.496130+0000 mgr.y (mgr.24431) 62 : cluster [DBG] pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:18.021 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:17 smithi175 bash[29284]: cluster 2024-03-26T03:46:16.496130+0000 mgr.y (mgr.24431) 62 : cluster [DBG] pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:18.022 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:46:17 smithi175 bash[58426]: ts=2024-03-26T03:46:17.785Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-03-26T03:46:18.043 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:17 smithi190 bash[23392]: cluster 2024-03-26T03:46:16.496130+0000 mgr.y (mgr.24431) 62 : cluster [DBG] pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:18.043 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:17 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:17.784281264Z level=info msg="Executing migration" id="create user auth token table" 2024-03-26T03:46:18.043 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:17 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:17.817928427Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-03-26T03:46:18.044 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:17 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:17.868396397Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-03-26T03:46:18.044 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:17 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:17.901926263Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-03-26T03:46:18.044 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:17 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:17.952329794Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-03-26T03:46:18.044 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:18 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:18.002599398Z level=info msg="Executing migration" id="create cache_data table" 2024-03-26T03:46:18.044 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:18 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:18.052965572Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-03-26T03:46:18.312 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:18 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:18.078409621Z level=info msg="Executing migration" id="create short_url table v1" 2024-03-26T03:46:18.312 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:18 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:18.120365537Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-03-26T03:46:18.312 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:18 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:18.162315805Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-03-26T03:46:18.313 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:18 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:18.187476891Z level=info msg="Executing migration" id="delete alert_definition table" 2024-03-26T03:46:18.313 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:18 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:18.212834245Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-03-26T03:46:18.313 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:18 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:18.238132092Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-03-26T03:46:18.313 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:18 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:18.271630529Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-03-26T03:46:18.313 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:18 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:18.32200046Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-03-26T03:46:18.573 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:18 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:18.347440577Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-03-26T03:46:18.573 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:18 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:18.381005633Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-03-26T03:46:18.573 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:18 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:18.414609896Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-03-26T03:46:18.573 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:18 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:18.448167617Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-03-26T03:46:18.573 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:18 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:18.498869795Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-03-26T03:46:18.573 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:18 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:18.532079577Z level=info msg="Executing migration" id="drop alert_definition table" 2024-03-26T03:46:18.573 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:18 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:18.582377741Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-03-26T03:46:18.621 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:46:18 smithi175 bash[59920]: ceph-762e930c-eb21-11ee-95d0-87774f69a715-alertmanager-a 2024-03-26T03:46:18.833 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:18 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:18.632689496Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-03-26T03:46:18.833 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:18 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:18.683656994Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-03-26T03:46:18.833 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:18 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:18.724984918Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-03-26T03:46:18.833 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:18 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:18.75023443Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-03-26T03:46:18.833 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:18 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:18.792158689Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-03-26T03:46:18.833 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:18 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:18.817514289Z level=info msg="Executing migration" id="create alert_instance table" 2024-03-26T03:46:18.834 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:18 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:18.842747653Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-03-26T03:46:18.991 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:46:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:46:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:46:18.991 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:46:18 smithi175 bash[60114]: Error response from daemon: No such container: ceph-762e930c-eb21-11ee-95d0-87774f69a715-alertmanager-a 2024-03-26T03:46:18.991 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:46:18 smithi175 systemd[1]: ceph-762e930c-eb21-11ee-95d0-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-03-26T03:46:18.991 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:46:18 smithi175 systemd[1]: Stopped Ceph alertmanager.a for 762e930c-eb21-11ee-95d0-87774f69a715. 2024-03-26T03:46:18.991 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:46:18 smithi175 systemd[1]: Started Ceph alertmanager.a for 762e930c-eb21-11ee-95d0-87774f69a715. 2024-03-26T03:46:19.102 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:18 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:18.876602801Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-03-26T03:46:19.102 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:18 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:18.918648566Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-03-26T03:46:19.102 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:18 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:18.960585884Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-03-26T03:46:19.102 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:18 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:18.985803643Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-03-26T03:46:19.102 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:19 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:19.061415414Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-03-26T03:46:19.370 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:19 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:19.111611633Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-03-26T03:46:19.370 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:19 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:19.161890507Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-03-26T03:46:19.370 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:19 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:19.195466874Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-03-26T03:46:19.370 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:19 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:19.229096637Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-03-26T03:46:19.370 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:19 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:19.279413307Z level=info msg="Executing migration" id="create alert_rule table" 2024-03-26T03:46:19.370 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:19 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:19.329693096Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-03-26T03:46:19.370 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:19 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:19.379831522Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-03-26T03:46:19.666 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:46:19 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:46:19.667 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:19 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:19.619417305Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-03-26T03:46:19.990 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:19 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:19.721365999Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-03-26T03:46:19.990 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:19 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:19.77732234Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-03-26T03:46:19.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:19 smithi190 bash[23392]: cluster 2024-03-26T03:46:18.496810+0000 mgr.y (mgr.24431) 63 : cluster [DBG] pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:19.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:19 smithi190 bash[23392]: audit 2024-03-26T03:46:18.670165+0000 mon.a (mon.0) 847 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:19.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:19 smithi190 bash[23392]: audit 2024-03-26T03:46:18.678229+0000 mon.a (mon.0) 848 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:19.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:19 smithi190 bash[23392]: audit 2024-03-26T03:46:18.695194+0000 mon.c (mon.2) 61 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:46:19.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:19 smithi190 bash[23392]: cephadm 2024-03-26T03:46:18.708675+0000 mgr.y (mgr.24431) 64 : cephadm [INF] Reconfiguring prometheus.a (dependencies changed)... 2024-03-26T03:46:19.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:19 smithi190 bash[23392]: cephadm 2024-03-26T03:46:18.936946+0000 mgr.y (mgr.24431) 65 : cephadm [INF] Reconfiguring daemon prometheus.a on smithi190 2024-03-26T03:46:19.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:19 smithi175 bash[29284]: cluster 2024-03-26T03:46:18.496810+0000 mgr.y (mgr.24431) 63 : cluster [DBG] pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:19.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:19 smithi175 bash[29284]: audit 2024-03-26T03:46:18.670165+0000 mon.a (mon.0) 847 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:19.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:19 smithi175 bash[29284]: audit 2024-03-26T03:46:18.678229+0000 mon.a (mon.0) 848 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:19.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:19 smithi175 bash[29284]: audit 2024-03-26T03:46:18.695194+0000 mon.c (mon.2) 61 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:46:19.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:19 smithi175 bash[29284]: cephadm 2024-03-26T03:46:18.708675+0000 mgr.y (mgr.24431) 64 : cephadm [INF] Reconfiguring prometheus.a (dependencies changed)... 2024-03-26T03:46:19.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:19 smithi175 bash[29284]: cephadm 2024-03-26T03:46:18.936946+0000 mgr.y (mgr.24431) 65 : cephadm [INF] Reconfiguring daemon prometheus.a on smithi190 2024-03-26T03:46:19.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:19 smithi175 bash[21527]: cluster 2024-03-26T03:46:18.496810+0000 mgr.y (mgr.24431) 63 : cluster [DBG] pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:19.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:19 smithi175 bash[21527]: audit 2024-03-26T03:46:18.670165+0000 mon.a (mon.0) 847 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:19.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:19 smithi175 bash[21527]: audit 2024-03-26T03:46:18.678229+0000 mon.a (mon.0) 848 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:19.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:19 smithi175 bash[21527]: audit 2024-03-26T03:46:18.695194+0000 mon.c (mon.2) 61 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:46:19.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:19 smithi175 bash[21527]: cephadm 2024-03-26T03:46:18.708675+0000 mgr.y (mgr.24431) 64 : cephadm [INF] Reconfiguring prometheus.a (dependencies changed)... 2024-03-26T03:46:19.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:19 smithi175 bash[21527]: cephadm 2024-03-26T03:46:18.936946+0000 mgr.y (mgr.24431) 65 : cephadm [INF] Reconfiguring daemon prometheus.a on smithi190 2024-03-26T03:46:19.992 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:46:19 smithi175 bash[60142]: ts=2024-03-26T03:46:19.781Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-03-26T03:46:19.992 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:46:19 smithi175 bash[60142]: ts=2024-03-26T03:46:19.781Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-03-26T03:46:19.992 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:46:19 smithi175 bash[60142]: ts=2024-03-26T03:46:19.782Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.175 port=9094 2024-03-26T03:46:19.993 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:46:19 smithi175 bash[60142]: ts=2024-03-26T03:46:19.783Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-03-26T03:46:19.993 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:46:19 smithi175 bash[60142]: ts=2024-03-26T03:46:19.813Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-03-26T03:46:19.993 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:46:19 smithi175 bash[60142]: ts=2024-03-26T03:46:19.814Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-03-26T03:46:19.993 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:46:19 smithi175 bash[60142]: ts=2024-03-26T03:46:19.818Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-03-26T03:46:19.993 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:46:19 smithi175 bash[60142]: ts=2024-03-26T03:46:19.818Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-03-26T03:46:20.429 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:20 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:20.183114484Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-03-26T03:46:20.429 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:20 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:20.426764095Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-03-26T03:46:20.705 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:20 smithi190 bash[23392]: audit 2024-03-26T03:46:19.525826+0000 mgr.y (mgr.24431) 66 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:46:20.990 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:20 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:20.714982233Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-03-26T03:46:20.990 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:20 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:20.902800426Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-03-26T03:46:20.990 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:20 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:20.964514174Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-03-26T03:46:20.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:20 smithi175 bash[29284]: audit 2024-03-26T03:46:19.525826+0000 mgr.y (mgr.24431) 66 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:46:20.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:20 smithi175 bash[21527]: audit 2024-03-26T03:46:19.525826+0000 mgr.y (mgr.24431) 66 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:46:21.376 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:21 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:21.056081623Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-03-26T03:46:21.376 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:21 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:21.164355282Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-03-26T03:46:21.376 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:21 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:21.221536437Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-03-26T03:46:21.376 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:21 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:21.305287662Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-03-26T03:46:21.634 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:21 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:21.385397274Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-03-26T03:46:21.635 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:21 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:21.447965152Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-03-26T03:46:21.635 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:21 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:21.507328238Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-03-26T03:46:21.635 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:21 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:21.582691896Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-03-26T03:46:21.635 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:21 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:21.616151561Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-03-26T03:46:21.936 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:21 smithi190 bash[23392]: cluster 2024-03-26T03:46:20.498441+0000 mgr.y (mgr.24431) 67 : cluster [DBG] pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:21.936 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:21 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:21.683398339Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-03-26T03:46:21.937 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:21 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:21.77596023Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-03-26T03:46:21.937 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:21 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:21.875815304Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-03-26T03:46:21.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:21 smithi175 bash[21527]: cluster 2024-03-26T03:46:20.498441+0000 mgr.y (mgr.24431) 67 : cluster [DBG] pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:21.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:21 smithi175 bash[29284]: cluster 2024-03-26T03:46:20.498441+0000 mgr.y (mgr.24431) 67 : cluster [DBG] pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:21.991 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:46:21 smithi175 bash[60142]: ts=2024-03-26T03:46:21.784Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000812239s 2024-03-26T03:46:22.240 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:22 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:22.028461909Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-03-26T03:46:22.521 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:22 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:22.280177131Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-03-26T03:46:22.838 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:22 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:22.53106308Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-03-26T03:46:22.838 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:22 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:22.670415644Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-03-26T03:46:22.838 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:22 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:22.754621149Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-03-26T03:46:23.240 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:22 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:22.847500667Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-03-26T03:46:23.240 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:23 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:23.060740123Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-03-26T03:46:23.683 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:23 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:23.294588722Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-03-26T03:46:23.683 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:23 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:23.517269052Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-03-26T03:46:23.804 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:46:23.805 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:46:23.806 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:46:23.806 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:46:23.811 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:46:23.811 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:46:23.811 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:46:23.812 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:46:23.959 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:23 smithi190 bash[23392]: cluster 2024-03-26T03:46:22.499165+0000 mgr.y (mgr.24431) 68 : cluster [DBG] pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:23.959 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:23 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:23.83005089Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-03-26T03:46:23.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:23 smithi175 bash[21527]: cluster 2024-03-26T03:46:22.499165+0000 mgr.y (mgr.24431) 68 : cluster [DBG] pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:23.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:23 smithi175 bash[29284]: cluster 2024-03-26T03:46:22.499165+0000 mgr.y (mgr.24431) 68 : cluster [DBG] pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:24.240 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:23 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:23.968898293Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-03-26T03:46:24.240 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:24 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:24.080679581Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-03-26T03:46:24.240 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:24 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:24.205606438Z level=info msg="Executing migration" id="create provenance_type table" 2024-03-26T03:46:24.566 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:24 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:24.321228292Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-03-26T03:46:24.567 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:24 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:24.379380168Z level=info msg="Executing migration" id="create alert_image table" 2024-03-26T03:46:24.567 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:24 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:24.468388711Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-03-26T03:46:24.567 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:24 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:24.538743266Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-03-26T03:46:24.841 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:24 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:24.588993631Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-03-26T03:46:24.841 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:24 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:24.631061147Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-03-26T03:46:24.841 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:24 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:24.707176719Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-03-26T03:46:24.841 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:24 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:24.707484043Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-03-26T03:46:24.841 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:24 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:24.732422436Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-03-26T03:46:24.841 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:24 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:24.784987242Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-03-26T03:46:24.841 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:24 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:24.850010196Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-03-26T03:46:25.169 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:24 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:24.850567209Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-03-26T03:46:25.169 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:24 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:24.851113017Z level=info msg="alerts found to migrate" alerts=0 2024-03-26T03:46:25.169 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:24 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:24.942017061Z level=info msg="Executing migration" id="create library_element table v1" 2024-03-26T03:46:25.425 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:25 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:25.178538328Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-03-26T03:46:25.686 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:25 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:25.435456961Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-03-26T03:46:25.686 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:25 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:25.598030852Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-03-26T03:46:25.939 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:25 smithi190 bash[23392]: cluster 2024-03-26T03:46:24.500871+0000 mgr.y (mgr.24431) 69 : cluster [DBG] pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:25.939 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:25 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:25.745222389Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-03-26T03:46:25.939 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:25 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:25.854121729Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-03-26T03:46:25.939 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:25 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:25.930185901Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-03-26T03:46:25.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:25 smithi175 bash[29284]: cluster 2024-03-26T03:46:24.500871+0000 mgr.y (mgr.24431) 69 : cluster [DBG] pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:25.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:25 smithi175 bash[21527]: cluster 2024-03-26T03:46:24.500871+0000 mgr.y (mgr.24431) 69 : cluster [DBG] pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:26.199 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:25 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:25.96379954Z level=info msg="Executing migration" id="create data_keys table" 2024-03-26T03:46:26.199 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:25 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:25.997427286Z level=info msg="Executing migration" id="create secrets table" 2024-03-26T03:46:26.199 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:26 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:26.09011268Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-03-26T03:46:26.200 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:26 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:26.148825306Z level=info msg="Executing migration" id="add name column into data_keys" 2024-03-26T03:46:26.200 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:25 smithi190 systemd[1]: Stopping Ceph prometheus.a for 762e930c-eb21-11ee-95d0-87774f69a715... 2024-03-26T03:46:26.200 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:26 smithi190 bash[48954]: ts=2024-03-26T03:46:26.117Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-03-26T03:46:26.200 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:26 smithi190 bash[48954]: ts=2024-03-26T03:46:26.117Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-03-26T03:46:26.200 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:26 smithi190 bash[48954]: ts=2024-03-26T03:46:26.117Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-03-26T03:46:26.200 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:26 smithi190 bash[48954]: ts=2024-03-26T03:46:26.117Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-03-26T03:46:26.200 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:26 smithi190 bash[48954]: ts=2024-03-26T03:46:26.117Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-03-26T03:46:26.200 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:26 smithi190 bash[48954]: ts=2024-03-26T03:46:26.117Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-03-26T03:46:26.200 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:26 smithi190 bash[48954]: ts=2024-03-26T03:46:26.117Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-03-26T03:46:26.200 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:26 smithi190 bash[48954]: ts=2024-03-26T03:46:26.117Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-03-26T03:46:26.201 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:26 smithi190 bash[48954]: ts=2024-03-26T03:46:26.117Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-03-26T03:46:26.201 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:26 smithi190 bash[48954]: ts=2024-03-26T03:46:26.118Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-03-26T03:46:26.201 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:26 smithi190 bash[48954]: ts=2024-03-26T03:46:26.118Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-03-26T03:46:26.201 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:26 smithi190 bash[48954]: ts=2024-03-26T03:46:26.118Z caller=main.go:1147 level=info msg="See you next time!" 2024-03-26T03:46:26.490 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:26 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:26.209339826Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-03-26T03:46:26.490 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:26 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:26.266298904Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-03-26T03:46:26.490 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:26 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:26.375128448Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-03-26T03:46:26.944 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:26 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:26.600832954Z level=info msg="Executing migration" id="create kv_store table v1" 2024-03-26T03:46:27.218 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:27 smithi190 bash[51488]: ceph-762e930c-eb21-11ee-95d0-87774f69a715-prometheus-a 2024-03-26T03:46:27.219 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:26 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:26.954279714Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-03-26T03:46:27.490 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:27 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:27.228255722Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-03-26T03:46:27.490 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:27 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:27.312291736Z level=info msg="Executing migration" id="create permission table" 2024-03-26T03:46:27.490 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:27 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:27.379992609Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-03-26T03:46:27.490 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:27 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:27.405206775Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-03-26T03:46:27.490 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:27 smithi190 bash[51548]: Error response from daemon: No such container: ceph-762e930c-eb21-11ee-95d0-87774f69a715-prometheus-a 2024-03-26T03:46:27.490 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:27 smithi190 systemd[1]: ceph-762e930c-eb21-11ee-95d0-87774f69a715@prometheus.a.service: Deactivated successfully. 2024-03-26T03:46:27.490 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:27 smithi190 systemd[1]: Stopped Ceph prometheus.a for 762e930c-eb21-11ee-95d0-87774f69a715. 2024-03-26T03:46:27.491 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:27 smithi190 systemd[1]: Started Ceph prometheus.a for 762e930c-eb21-11ee-95d0-87774f69a715. 2024-03-26T03:46:27.828 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[21783]: [26/Mar/2024:03:46:27] ENGINE Bus STOPPING 2024-03-26T03:46:27.828 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[21783]: [26/Mar/2024:03:46:27] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-26T03:46:27.828 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[21783]: [26/Mar/2024:03:46:27] ENGINE Bus STOPPED 2024-03-26T03:46:27.828 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[21783]: [26/Mar/2024:03:46:27] ENGINE Bus STARTING 2024-03-26T03:46:27.964 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:27 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:27.606814847Z level=info msg="Executing migration" id="create role table" 2024-03-26T03:46:27.964 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:27 smithi190 bash[23392]: cluster 2024-03-26T03:46:26.501645+0000 mgr.y (mgr.24431) 70 : cluster [DBG] pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:27.964 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:27 smithi190 bash[23392]: audit 2024-03-26T03:46:27.417099+0000 mon.a (mon.0) 849 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:27.964 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:27 smithi190 bash[23392]: audit 2024-03-26T03:46:27.433995+0000 mon.a (mon.0) 850 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:27.964 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:27 smithi190 bash[23392]: audit 2024-03-26T03:46:27.490256+0000 mon.c (mon.2) 62 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-26T03:46:27.965 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:27 smithi190 bash[23392]: audit 2024-03-26T03:46:27.493306+0000 mon.c (mon.2) 63 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi175.front.sepia.ceph.com:9093"}]: dispatch 2024-03-26T03:46:27.965 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:27 smithi190 bash[23392]: audit 2024-03-26T03:46:27.500989+0000 mon.a (mon.0) 851 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:27.965 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:27 smithi190 bash[23392]: audit 2024-03-26T03:46:27.518837+0000 mon.c (mon.2) 64 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-26T03:46:27.965 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:27 smithi190 bash[23392]: audit 2024-03-26T03:46:27.525075+0000 mon.c (mon.2) 65 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi190.front.sepia.ceph.com:9095"}]: dispatch 2024-03-26T03:46:27.965 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:27 smithi190 bash[23392]: audit 2024-03-26T03:46:27.532810+0000 mon.a (mon.0) 852 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:27.965 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:27 smithi190 bash[23392]: audit 2024-03-26T03:46:27.548576+0000 mon.c (mon.2) 66 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-26T03:46:27.965 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:27 smithi190 bash[23392]: audit 2024-03-26T03:46:27.551878+0000 mon.c (mon.2) 67 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi190.front.sepia.ceph.com:3000"}]: dispatch 2024-03-26T03:46:27.965 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:27 smithi190 bash[23392]: audit 2024-03-26T03:46:27.557139+0000 mon.a (mon.0) 853 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:27.965 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:27 smithi190 bash[23392]: audit 2024-03-26T03:46:27.574013+0000 mon.c (mon.2) 68 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:46:28.240 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:27 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:27.974106262Z level=info msg="Executing migration" id="add column display_name" 2024-03-26T03:46:28.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[21783]: [26/Mar/2024:03:46:27] ENGINE Serving on http://:::9283 2024-03-26T03:46:28.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[21783]: [26/Mar/2024:03:46:27] ENGINE Bus STARTED 2024-03-26T03:46:28.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[21783]: [26/Mar/2024:03:46:27] ENGINE Bus STOPPING 2024-03-26T03:46:28.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[21783]: [26/Mar/2024:03:46:27] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-26T03:46:28.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[21783]: [26/Mar/2024:03:46:27] ENGINE Bus STOPPED 2024-03-26T03:46:28.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[21783]: [26/Mar/2024:03:46:27] ENGINE Bus STARTING 2024-03-26T03:46:28.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[21783]: [26/Mar/2024:03:46:27] ENGINE Serving on http://:::9283 2024-03-26T03:46:28.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[21783]: [26/Mar/2024:03:46:27] ENGINE Bus STARTED 2024-03-26T03:46:28.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[21783]: [26/Mar/2024:03:46:27] ENGINE Bus STOPPING 2024-03-26T03:46:28.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[21783]: [26/Mar/2024:03:46:27] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-26T03:46:28.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[21783]: [26/Mar/2024:03:46:27] ENGINE Bus STOPPED 2024-03-26T03:46:28.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[21783]: [26/Mar/2024:03:46:27] ENGINE Bus STARTING 2024-03-26T03:46:28.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[21783]: [26/Mar/2024:03:46:27] ENGINE Serving on http://:::9283 2024-03-26T03:46:28.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[21783]: [26/Mar/2024:03:46:27] ENGINE Bus STARTED 2024-03-26T03:46:28.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[21527]: cluster 2024-03-26T03:46:26.501645+0000 mgr.y (mgr.24431) 70 : cluster [DBG] pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:28.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[21527]: audit 2024-03-26T03:46:27.417099+0000 mon.a (mon.0) 849 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:28.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[21527]: audit 2024-03-26T03:46:27.433995+0000 mon.a (mon.0) 850 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:28.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[21527]: audit 2024-03-26T03:46:27.490256+0000 mon.c (mon.2) 62 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-26T03:46:28.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[21527]: audit 2024-03-26T03:46:27.493306+0000 mon.c (mon.2) 63 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi175.front.sepia.ceph.com:9093"}]: dispatch 2024-03-26T03:46:28.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[21527]: audit 2024-03-26T03:46:27.500989+0000 mon.a (mon.0) 851 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:28.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[21527]: audit 2024-03-26T03:46:27.518837+0000 mon.c (mon.2) 64 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-26T03:46:28.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[21527]: audit 2024-03-26T03:46:27.525075+0000 mon.c (mon.2) 65 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi190.front.sepia.ceph.com:9095"}]: dispatch 2024-03-26T03:46:28.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[21527]: audit 2024-03-26T03:46:27.532810+0000 mon.a (mon.0) 852 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:28.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[21527]: audit 2024-03-26T03:46:27.548576+0000 mon.c (mon.2) 66 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-26T03:46:28.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[21527]: audit 2024-03-26T03:46:27.551878+0000 mon.c (mon.2) 67 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi190.front.sepia.ceph.com:3000"}]: dispatch 2024-03-26T03:46:28.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[21527]: audit 2024-03-26T03:46:27.557139+0000 mon.a (mon.0) 853 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:28.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[21527]: audit 2024-03-26T03:46:27.574013+0000 mon.c (mon.2) 68 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:46:28.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[29284]: cluster 2024-03-26T03:46:26.501645+0000 mgr.y (mgr.24431) 70 : cluster [DBG] pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:28.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[29284]: audit 2024-03-26T03:46:27.417099+0000 mon.a (mon.0) 849 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:28.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[29284]: audit 2024-03-26T03:46:27.433995+0000 mon.a (mon.0) 850 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:28.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[29284]: audit 2024-03-26T03:46:27.490256+0000 mon.c (mon.2) 62 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-26T03:46:28.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[29284]: audit 2024-03-26T03:46:27.493306+0000 mon.c (mon.2) 63 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi175.front.sepia.ceph.com:9093"}]: dispatch 2024-03-26T03:46:28.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[29284]: audit 2024-03-26T03:46:27.500989+0000 mon.a (mon.0) 851 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:28.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[29284]: audit 2024-03-26T03:46:27.518837+0000 mon.c (mon.2) 64 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-26T03:46:28.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[29284]: audit 2024-03-26T03:46:27.525075+0000 mon.c (mon.2) 65 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi190.front.sepia.ceph.com:9095"}]: dispatch 2024-03-26T03:46:28.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[29284]: audit 2024-03-26T03:46:27.532810+0000 mon.a (mon.0) 852 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:28.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[29284]: audit 2024-03-26T03:46:27.548576+0000 mon.c (mon.2) 66 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-26T03:46:28.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[29284]: audit 2024-03-26T03:46:27.551878+0000 mon.c (mon.2) 67 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi190.front.sepia.ceph.com:3000"}]: dispatch 2024-03-26T03:46:28.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[29284]: audit 2024-03-26T03:46:27.557139+0000 mon.a (mon.0) 853 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:28.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:27 smithi175 bash[29284]: audit 2024-03-26T03:46:27.574013+0000 mon.c (mon.2) 68 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:46:28.579 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:28 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:28.289279238Z level=info msg="Executing migration" id="add column group_name" 2024-03-26T03:46:28.845 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:28 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:28.588779948Z level=info msg="Executing migration" id="add index role.org_id" 2024-03-26T03:46:28.845 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:28 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:28.744394273Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-03-26T03:46:28.846 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:28 smithi190 bash[23392]: audit 2024-03-26T03:46:27.491171+0000 mgr.y (mgr.24431) 71 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-26T03:46:28.846 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:28 smithi190 bash[23392]: audit 2024-03-26T03:46:27.493944+0000 mgr.y (mgr.24431) 72 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi175.front.sepia.ceph.com:9093"}]: dispatch 2024-03-26T03:46:28.846 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:28 smithi190 bash[23392]: audit 2024-03-26T03:46:27.519670+0000 mgr.y (mgr.24431) 73 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-26T03:46:28.846 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:28 smithi190 bash[23392]: audit 2024-03-26T03:46:27.525591+0000 mgr.y (mgr.24431) 74 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi190.front.sepia.ceph.com:9095"}]: dispatch 2024-03-26T03:46:28.846 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:28 smithi190 bash[23392]: audit 2024-03-26T03:46:27.549288+0000 mgr.y (mgr.24431) 75 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-26T03:46:28.846 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:28 smithi190 bash[23392]: audit 2024-03-26T03:46:27.552290+0000 mgr.y (mgr.24431) 76 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi190.front.sepia.ceph.com:3000"}]: dispatch 2024-03-26T03:46:28.987 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:28 smithi175 bash[29284]: audit 2024-03-26T03:46:27.491171+0000 mgr.y (mgr.24431) 71 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-26T03:46:28.987 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:28 smithi175 bash[29284]: audit 2024-03-26T03:46:27.493944+0000 mgr.y (mgr.24431) 72 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi175.front.sepia.ceph.com:9093"}]: dispatch 2024-03-26T03:46:28.987 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:28 smithi175 bash[29284]: audit 2024-03-26T03:46:27.519670+0000 mgr.y (mgr.24431) 73 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-26T03:46:28.987 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:28 smithi175 bash[29284]: audit 2024-03-26T03:46:27.525591+0000 mgr.y (mgr.24431) 74 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi190.front.sepia.ceph.com:9095"}]: dispatch 2024-03-26T03:46:28.988 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:28 smithi175 bash[29284]: audit 2024-03-26T03:46:27.549288+0000 mgr.y (mgr.24431) 75 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-26T03:46:28.988 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:28 smithi175 bash[29284]: audit 2024-03-26T03:46:27.552290+0000 mgr.y (mgr.24431) 76 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi190.front.sepia.ceph.com:3000"}]: dispatch 2024-03-26T03:46:28.988 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:28 smithi175 bash[21527]: audit 2024-03-26T03:46:27.491171+0000 mgr.y (mgr.24431) 71 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-26T03:46:28.988 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:28 smithi175 bash[21527]: audit 2024-03-26T03:46:27.493944+0000 mgr.y (mgr.24431) 72 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi175.front.sepia.ceph.com:9093"}]: dispatch 2024-03-26T03:46:28.989 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:28 smithi175 bash[21527]: audit 2024-03-26T03:46:27.519670+0000 mgr.y (mgr.24431) 73 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-26T03:46:28.989 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:28 smithi175 bash[21527]: audit 2024-03-26T03:46:27.525591+0000 mgr.y (mgr.24431) 74 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi190.front.sepia.ceph.com:9095"}]: dispatch 2024-03-26T03:46:28.989 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:28 smithi175 bash[21527]: audit 2024-03-26T03:46:27.549288+0000 mgr.y (mgr.24431) 75 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-26T03:46:28.989 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:28 smithi175 bash[21527]: audit 2024-03-26T03:46:27.552290+0000 mgr.y (mgr.24431) 76 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi190.front.sepia.ceph.com:3000"}]: dispatch 2024-03-26T03:46:29.153 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:28 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:28.855198352Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-03-26T03:46:29.153 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:28 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:28.98005795Z level=info msg="Executing migration" id="create team role table" 2024-03-26T03:46:29.153 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:29.093880543Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-03-26T03:46:29.405 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:29.163216882Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-03-26T03:46:29.405 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:29.242616796Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-03-26T03:46:29.405 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:29.330698776Z level=info msg="Executing migration" id="create user role table" 2024-03-26T03:46:29.405 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:29.373153491Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-03-26T03:46:29.675 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:46:29.675 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:29.415143048Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-03-26T03:46:29.675 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:29.473871147Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-03-26T03:46:29.675 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:29.550118581Z level=info msg="Executing migration" id="create builtin role table" 2024-03-26T03:46:29.675 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:29.575380059Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-03-26T03:46:29.675 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:29.600658423Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-03-26T03:46:29.676 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:29.625953757Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-03-26T03:46:29.676 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:29.651258541Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-03-26T03:46:29.676 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:29.684908928Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-03-26T03:46:29.676 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[51574]: ts=2024-03-26T03:46:29.451Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-03-26T03:46:29.676 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[51574]: ts=2024-03-26T03:46:29.451Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-03-26T03:46:29.676 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[51574]: ts=2024-03-26T03:46:29.451Z caller=main.go:570 level=info host_details="(Linux 5.15.0-101-generic #111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024 x86_64 smithi190 (none))" 2024-03-26T03:46:29.676 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[51574]: ts=2024-03-26T03:46:29.451Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-03-26T03:46:29.676 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[51574]: ts=2024-03-26T03:46:29.451Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-03-26T03:46:29.676 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[51574]: ts=2024-03-26T03:46:29.453Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-03-26T03:46:29.677 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[51574]: ts=2024-03-26T03:46:29.454Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-03-26T03:46:29.677 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[51574]: ts=2024-03-26T03:46:29.455Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-03-26T03:46:29.677 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[51574]: ts=2024-03-26T03:46:29.455Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-03-26T03:46:29.677 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[51574]: ts=2024-03-26T03:46:29.456Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-03-26T03:46:29.677 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[51574]: ts=2024-03-26T03:46:29.456Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.673µs 2024-03-26T03:46:29.677 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[51574]: ts=2024-03-26T03:46:29.456Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-03-26T03:46:29.677 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[51574]: ts=2024-03-26T03:46:29.457Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-03-26T03:46:29.677 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[51574]: ts=2024-03-26T03:46:29.458Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-03-26T03:46:29.677 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[51574]: ts=2024-03-26T03:46:29.458Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=36.465µs wal_replay_duration=1.310274ms wbl_replay_duration=141ns total_replay_duration=1.371143ms 2024-03-26T03:46:29.677 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[51574]: ts=2024-03-26T03:46:29.459Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-03-26T03:46:29.677 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[51574]: ts=2024-03-26T03:46:29.459Z caller=main.go:1029 level=info msg="TSDB started" 2024-03-26T03:46:29.677 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[51574]: ts=2024-03-26T03:46:29.459Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-03-26T03:46:29.677 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[51574]: ts=2024-03-26T03:46:29.484Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=24.573885ms db_storage=1.002µs remote_storage=1.293µs web_handler=584ns query_engine=740ns scrape=338.066µs scrape_sd=116.452µs notify=10.634µs notify_sd=12.139µs rules=23.697262ms tracing=7.313µs 2024-03-26T03:46:29.677 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[51574]: ts=2024-03-26T03:46:29.484Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-03-26T03:46:29.678 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[51574]: ts=2024-03-26T03:46:29.484Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-03-26T03:46:29.953 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[23392]: cluster 2024-03-26T03:46:28.502104+0000 mgr.y (mgr.24431) 77 : cluster [DBG] pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:29.953 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:29.727107115Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-03-26T03:46:29.953 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:29.769254378Z level=info msg="Executing migration" id="add unique index role.uid" 2024-03-26T03:46:29.953 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:29.794541266Z level=info msg="Executing migration" id="create seed assignment table" 2024-03-26T03:46:29.953 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:29.819688478Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-03-26T03:46:29.953 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:29.845070124Z level=info msg="Executing migration" id="add column hidden to role table" 2024-03-26T03:46:29.954 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:29.870314226Z level=info msg="Executing migration" id="create query_history table v1" 2024-03-26T03:46:29.954 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:29.895523174Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-03-26T03:46:29.954 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:29.920892135Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-03-26T03:46:29.954 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:29 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:29.96286444Z level=info msg="Executing migration" id="teams permissions migration" 2024-03-26T03:46:29.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:29 smithi175 bash[29284]: cluster 2024-03-26T03:46:28.502104+0000 mgr.y (mgr.24431) 77 : cluster [DBG] pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:29.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:29 smithi175 bash[21527]: cluster 2024-03-26T03:46:28.502104+0000 mgr.y (mgr.24431) 77 : cluster [DBG] pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:29.991 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 03:46:29 smithi175 bash[60142]: ts=2024-03-26T03:46:29.785Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002655566s 2024-03-26T03:46:30.204 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.013227047Z level=info msg="Executing migration" id="dashboard permissions" 2024-03-26T03:46:30.205 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.063448632Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-03-26T03:46:30.205 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.097249677Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-03-26T03:46:30.205 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.130891689Z level=info msg="Executing migration" id="alerting notification permissions" 2024-03-26T03:46:30.205 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.181195363Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-03-26T03:46:30.205 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.214574083Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-03-26T03:46:30.490 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.256784015Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-03-26T03:46:30.490 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.282065753Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-03-26T03:46:30.490 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.307360098Z level=info msg="Executing migration" id="create correlation table v1" 2024-03-26T03:46:30.490 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.349362436Z level=info msg="Executing migration" id="add index correlations.uid" 2024-03-26T03:46:30.490 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.399836621Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-03-26T03:46:30.491 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.450297501Z level=info msg="Executing migration" id="add correlation config column" 2024-03-26T03:46:30.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[23392]: audit 2024-03-26T03:46:29.537141+0000 mgr.y (mgr.24431) 78 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:46:30.745 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.50082526Z level=info msg="Executing migration" id="create entity_events table" 2024-03-26T03:46:30.746 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.526012085Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-03-26T03:46:30.746 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.551165354Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-03-26T03:46:30.746 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.551505074Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-03-26T03:46:30.746 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.576462768Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-03-26T03:46:30.746 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.576773766Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-03-26T03:46:30.746 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.601677553Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-03-26T03:46:30.746 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.670831444Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-03-26T03:46:30.746 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.687820145Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-03-26T03:46:30.746 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.729784934Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-03-26T03:46:30.746 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.755172597Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-03-26T03:46:30.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:30 smithi175 bash[21527]: audit 2024-03-26T03:46:29.537141+0000 mgr.y (mgr.24431) 78 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:46:30.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:30 smithi175 bash[29284]: audit 2024-03-26T03:46:29.537141+0000 mgr.y (mgr.24431) 78 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:46:30.999 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.797090299Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-03-26T03:46:30.999 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.814046532Z level=info msg="Executing migration" id="Drop public config table" 2024-03-26T03:46:30.999 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.831049614Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-03-26T03:46:30.999 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.847872093Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-03-26T03:46:30.999 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.864837148Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-03-26T03:46:30.999 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.881730491Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-03-26T03:46:30.999 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.898623221Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-03-26T03:46:30.999 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.923902882Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-03-26T03:46:30.999 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.957601908Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-03-26T03:46:30.999 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.974505283Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-03-26T03:46:30.999 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:30 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:30.991416312Z level=info msg="Executing migration" id="add share column" 2024-03-26T03:46:30.999 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:31 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:31.008357558Z level=info msg="Executing migration" id="create default alerting folders" 2024-03-26T03:46:31.268 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:31 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:31.025320843Z level=info msg="Executing migration" id="create file table" 2024-03-26T03:46:31.268 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:31 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:31.05887511Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-03-26T03:46:31.268 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:31 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:31.092671099Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-03-26T03:46:31.268 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:31 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:31.117946196Z level=info msg="Executing migration" id="create file_meta table" 2024-03-26T03:46:31.268 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:31 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:31.160082294Z level=info msg="Executing migration" id="file table idx: path key" 2024-03-26T03:46:31.268 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:31 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:31.210274882Z level=info msg="Executing migration" id="set path collation in file table" 2024-03-26T03:46:31.268 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:31 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:31.24380445Z level=info msg="Executing migration" id="managed permissions migration" 2024-03-26T03:46:31.268 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:31 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:31.277434729Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-03-26T03:46:31.520 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:31 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:31.311064856Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-03-26T03:46:31.520 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:31 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:31.361394006Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-03-26T03:46:31.520 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:31 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:31.411711874Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-03-26T03:46:31.520 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:31 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:31.453598598Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-03-26T03:46:31.520 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:31 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:31.504515849Z level=info msg="Executing migration" id="update group index for alert rules" 2024-03-26T03:46:31.520 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:31 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:31.529796263Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-03-26T03:46:31.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:31 smithi175 bash[21527]: cluster 2024-03-26T03:46:30.503533+0000 mgr.y (mgr.24431) 79 : cluster [DBG] pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:31.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:31 smithi175 bash[29284]: cluster 2024-03-26T03:46:30.503533+0000 mgr.y (mgr.24431) 79 : cluster [DBG] pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:31.813 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:31 smithi190 bash[23392]: cluster 2024-03-26T03:46:30.503533+0000 mgr.y (mgr.24431) 79 : cluster [DBG] pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:31.814 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:31 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:31.580126313Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-03-26T03:46:31.814 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:31 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:31.613840332Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-03-26T03:46:31.814 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:31 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:31.647429327Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-03-26T03:46:31.814 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:31 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:31.697767383Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-03-26T03:46:31.814 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:31 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:31.748281962Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-03-26T03:46:32.099 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:31 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:31.823590869Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-03-26T03:46:32.099 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:31 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:31.857253595Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-03-26T03:46:32.099 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:31 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:31.890864289Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-03-26T03:46:32.099 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:31 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:31.95790613Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-03-26T03:46:32.099 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:32 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:32.016537277Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-03-26T03:46:32.099 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:32 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:32.066808367Z level=info msg="Executing migration" id="create folder table" 2024-03-26T03:46:32.100 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:32 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:32.108921702Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-03-26T03:46:32.490 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:32 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:32.134203826Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-03-26T03:46:32.490 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:32 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:32.15948496Z level=info msg="Executing migration" id="Update folder title length" 2024-03-26T03:46:32.490 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:32 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:32.201433049Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-03-26T03:46:32.490 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:32 smithi190 bash[50860]: logger=migrator t=2024-03-26T03:46:32.226640424Z level=info msg="migrations completed" performed=485 skipped=0 duration=25.847928554s 2024-03-26T03:46:32.490 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:32 smithi190 bash[50860]: logger=sqlstore t=2024-03-26T03:46:32.227998953Z level=info msg="Created default organization" 2024-03-26T03:46:32.490 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:32 smithi190 bash[50860]: logger=plugin.finder t=2024-03-26T03:46:32.341428756Z level=warn msg="Skipping finding plugins as directory does not exist" path=/usr/share/grafana/plugins-bundled 2024-03-26T03:46:32.490 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:32 smithi190 bash[50860]: logger=secrets t=2024-03-26T03:46:32.341676788Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-03-26T03:46:32.491 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:32 smithi190 bash[50860]: logger=query_data t=2024-03-26T03:46:32.347944641Z level=info msg="Query Service initialization" 2024-03-26T03:46:32.491 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:32 smithi190 bash[50860]: logger=live.push_http t=2024-03-26T03:46:32.355060682Z level=info msg="Live Push Gateway initialization" 2024-03-26T03:46:33.714 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:33 smithi190 bash[50860]: logger=infra.usagestats.collector t=2024-03-26T03:46:33.38787692Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-03-26T03:46:33.714 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:33 smithi190 bash[50860]: logger=provisioning.datasources t=2024-03-26T03:46:33.389974596Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-03-26T03:46:33.714 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:33 smithi190 bash[50860]: logger=provisioning.datasources t=2024-03-26T03:46:33.436416338Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-03-26T03:46:33.714 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:33 smithi190 bash[50860]: logger=provisioning.alerting t=2024-03-26T03:46:33.503481977Z level=info msg="starting to provision alerting" 2024-03-26T03:46:33.714 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:33 smithi190 bash[50860]: logger=provisioning.alerting t=2024-03-26T03:46:33.503516627Z level=info msg="finished to provision alerting" 2024-03-26T03:46:33.714 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:33 smithi190 bash[50860]: logger=grafanaStorageLogger t=2024-03-26T03:46:33.503982499Z level=info msg="storage starting" 2024-03-26T03:46:33.714 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:33 smithi190 bash[50860]: logger=ngalert.state.manager t=2024-03-26T03:46:33.505040854Z level=info msg="Warming state cache for startup" 2024-03-26T03:46:33.714 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:33 smithi190 bash[50860]: logger=http.server t=2024-03-26T03:46:33.508035228Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-03-26T03:46:33.714 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:33 smithi190 bash[50860]: logger=sqlstore.transactions t=2024-03-26T03:46:33.515239031Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-03-26T03:46:33.964 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:33 smithi175 bash[21527]: cluster 2024-03-26T03:46:32.504209+0000 mgr.y (mgr.24431) 80 : cluster [DBG] pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:33.964 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:33 smithi175 bash[21527]: audit 2024-03-26T03:46:33.707227+0000 mon.c (mon.2) 69 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:46:33.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:33 smithi175 bash[29284]: cluster 2024-03-26T03:46:32.504209+0000 mgr.y (mgr.24431) 80 : cluster [DBG] pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:33.964 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:33 smithi175 bash[29284]: audit 2024-03-26T03:46:33.707227+0000 mon.c (mon.2) 69 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:46:33.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:33 smithi190 bash[23392]: cluster 2024-03-26T03:46:32.504209+0000 mgr.y (mgr.24431) 80 : cluster [DBG] pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:33.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:33 smithi190 bash[23392]: audit 2024-03-26T03:46:33.707227+0000 mon.c (mon.2) 69 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:46:33.990 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:33 smithi190 bash[50860]: logger=ngalert.state.manager t=2024-03-26T03:46:33.834914432Z level=info msg="State cache has been initialized" states=0 duration=329.856877ms 2024-03-26T03:46:33.990 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:33 smithi190 bash[50860]: logger=ticker t=2024-03-26T03:46:33.835007847Z level=info msg=starting first_tick=2024-03-26T03:46:40Z 2024-03-26T03:46:33.990 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:46:33 smithi190 bash[50860]: logger=ngalert.multiorg.alertmanager t=2024-03-26T03:46:33.835003788Z level=info msg="starting MultiOrg Alertmanager" 2024-03-26T03:46:34.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.133+0000 7f20ccc88700 1 -- 172.21.15.175:0/245127335 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20c8103010 msgr2=0x7f20c810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:34.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.133+0000 7f20ccc88700 1 --2- 172.21.15.175:0/245127335 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20c8103010 0x7f20c810f520 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f20b8009990 tx=0x7f20b8009ca0 comp rx=0 tx=0).stop 2024-03-26T03:46:34.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.133+0000 7f20ccc88700 1 -- 172.21.15.175:0/245127335 shutdown_connections 2024-03-26T03:46:34.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.133+0000 7f20ccc88700 1 --2- 172.21.15.175:0/245127335 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20c8103010 0x7f20c810f520 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:34.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.133+0000 7f20ccc88700 1 --2- 172.21.15.175:0/245127335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20c8108650 0x7f20c8108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:34.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.133+0000 7f20ccc88700 1 --2- 172.21.15.175:0/245127335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20c8102650 0x7f20c8102ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:34.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.133+0000 7f20ccc88700 1 -- 172.21.15.175:0/245127335 >> 172.21.15.175:0/245127335 conn(0x7f20c8079a10 msgr2=0x7f20c8079e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:46:34.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.133+0000 7f20ccc88700 1 -- 172.21.15.175:0/245127335 shutdown_connections 2024-03-26T03:46:34.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.133+0000 7f20ccc88700 1 -- 172.21.15.175:0/245127335 wait complete. 2024-03-26T03:46:34.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.133+0000 7f20ccc88700 1 Processor -- start 2024-03-26T03:46:34.135 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.133+0000 7f20ccc88700 1 -- start start 2024-03-26T03:46:34.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.133+0000 7f20ccc88700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20c8102650 0x7f20c8198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:34.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.133+0000 7f20ccc88700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20c8103010 0x7f20c8198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:34.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.133+0000 7f20ccc88700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20c8108650 0x7f20c819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:34.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.133+0000 7f20ccc88700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f20c8111e70 con 0x7f20c8102650 2024-03-26T03:46:34.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.133+0000 7f20ccc88700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f20c8111cf0 con 0x7f20c8108650 2024-03-26T03:46:34.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.133+0000 7f20ccc88700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f20c8111ff0 con 0x7f20c8103010 2024-03-26T03:46:34.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.133+0000 7f20c659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20c8102650 0x7f20c8198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:34.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.137+0000 7f20c5d9b700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20c8103010 0x7f20c8198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:34.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.137+0000 7f20c5d9b700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20c8103010 0x7f20c8198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:52598/0 (socket says 172.21.15.175:52598) 2024-03-26T03:46:34.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.137+0000 7f20c659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20c8102650 0x7f20c8198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:53148/0 (socket says 172.21.15.175:53148) 2024-03-26T03:46:34.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.137+0000 7f20c5d9b700 1 -- 172.21.15.175:0/2975191765 learned_addr learned my addr 172.21.15.175:0/2975191765 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:46:34.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.137+0000 7f20c6d9d700 1 --2- 172.21.15.175:0/2975191765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20c8108650 0x7f20c819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:34.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.137+0000 7f20c659c700 1 -- 172.21.15.175:0/2975191765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20c8103010 msgr2=0x7f20c8198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:34.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.137+0000 7f20c659c700 1 --2- 172.21.15.175:0/2975191765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20c8103010 0x7f20c8198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:34.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.137+0000 7f20c659c700 1 -- 172.21.15.175:0/2975191765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20c8108650 msgr2=0x7f20c819cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:34.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.137+0000 7f20c659c700 1 --2- 172.21.15.175:0/2975191765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20c8108650 0x7f20c819cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:34.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.137+0000 7f20c659c700 1 -- 172.21.15.175:0/2975191765 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c819d310 con 0x7f20c8102650 2024-03-26T03:46:34.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.137+0000 7f20c5d9b700 1 --2- 172.21.15.175:0/2975191765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20c8103010 0x7f20c8198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T03:46:34.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.137+0000 7f20c659c700 1 --2- 172.21.15.175:0/2975191765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20c8102650 0x7f20c8198300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f20c000efc0 tx=0x7f20c000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:46:34.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.137+0000 7f20bf7fe700 1 -- 172.21.15.175:0/2975191765 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c00092e0 con 0x7f20c8102650 2024-03-26T03:46:34.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.137+0000 7f20bf7fe700 1 -- 172.21.15.175:0/2975191765 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f20c0003a00 con 0x7f20c8102650 2024-03-26T03:46:34.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.137+0000 7f20bf7fe700 1 -- 172.21.15.175:0/2975191765 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c00044b0 con 0x7f20c8102650 2024-03-26T03:46:34.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.137+0000 7f20ccc88700 1 -- 172.21.15.175:0/2975191765 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20c819d5a0 con 0x7f20c8102650 2024-03-26T03:46:34.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.137+0000 7f20ccc88700 1 -- 172.21.15.175:0/2975191765 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20c81a4e80 con 0x7f20c8102650 2024-03-26T03:46:34.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.137+0000 7f20ccc88700 1 -- 172.21.15.175:0/2975191765 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f20c8062cd0 con 0x7f20c8102650 2024-03-26T03:46:34.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.137+0000 7f20bf7fe700 1 -- 172.21.15.175:0/2975191765 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7f20c001b030 con 0x7f20c8102650 2024-03-26T03:46:34.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.137+0000 7f20bf7fe700 1 --2- 172.21.15.175:0/2975191765 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f20b406e7e0 0x7f20b4070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:34.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.137+0000 7f20bf7fe700 1 -- 172.21.15.175:0/2975191765 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7537+0+0 (secure 0 0 0) 0x7f20c008e8d0 con 0x7f20c8102650 2024-03-26T03:46:34.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.137+0000 7f20c5d9b700 1 --2- 172.21.15.175:0/2975191765 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f20b406e7e0 0x7f20b4070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:34.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.137+0000 7f20bf7fe700 1 --2- 172.21.15.175:0/2975191765 >> [v2:172.21.15.175:6818/3775031674,v1:172.21.15.175:6819/3775031674] conn(0x7f20b40786f0 0x7f20b407ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:34.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.137+0000 7f20bf7fe700 1 -- 172.21.15.175:0/2975191765 --> [v2:172.21.15.175:6818/3775031674,v1:172.21.15.175:6819/3775031674] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f20c000cd60 con 0x7f20b40786f0 2024-03-26T03:46:34.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.137+0000 7f20bf7fe700 1 -- 172.21.15.175:0/2975191765 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f20c00932c0 con 0x7f20c8102650 2024-03-26T03:46:34.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.137+0000 7f20c6d9d700 1 --2- 172.21.15.175:0/2975191765 >> [v2:172.21.15.175:6818/3775031674,v1:172.21.15.175:6819/3775031674] conn(0x7f20b40786f0 0x7f20b407ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:34.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.137+0000 7f20c5d9b700 1 --2- 172.21.15.175:0/2975191765 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f20b406e7e0 0x7f20b4070ca0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f20c807a010 tx=0x7f20b000b480 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:46:34.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.137+0000 7f20c6d9d700 1 --2- 172.21.15.175:0/2975191765 >> [v2:172.21.15.175:6818/3775031674,v1:172.21.15.175:6819/3775031674] conn(0x7f20b40786f0 0x7f20b407ab90 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:46:34.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.141+0000 7f20bf7fe700 1 -- 172.21.15.175:0/2975191765 <== osd.2 v2:172.21.15.175:6818/3775031674 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f20c000cd60 con 0x7f20b40786f0 2024-03-26T03:46:34.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.157+0000 7f20ccc88700 1 -- 172.21.15.175:0/2975191765 --> [v2:172.21.15.175:6818/3775031674,v1:172.21.15.175:6819/3775031674] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f20c80630d0 con 0x7f20b40786f0 2024-03-26T03:46:34.156 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.157+0000 7f20bf7fe700 1 -- 172.21.15.175:0/2975191765 <== osd.2 v2:172.21.15.175:6818/3775031674 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f20c80630d0 con 0x7f20b40786f0 2024-03-26T03:46:34.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.157+0000 7f20ccc88700 1 -- 172.21.15.175:0/2975191765 >> [v2:172.21.15.175:6818/3775031674,v1:172.21.15.175:6819/3775031674] conn(0x7f20b40786f0 msgr2=0x7f20b407ab90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:34.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.157+0000 7f20ccc88700 1 --2- 172.21.15.175:0/2975191765 >> [v2:172.21.15.175:6818/3775031674,v1:172.21.15.175:6819/3775031674] conn(0x7f20b40786f0 0x7f20b407ab90 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:34.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.157+0000 7f20ccc88700 1 -- 172.21.15.175:0/2975191765 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f20b406e7e0 msgr2=0x7f20b4070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:34.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.157+0000 7f20ccc88700 1 --2- 172.21.15.175:0/2975191765 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f20b406e7e0 0x7f20b4070ca0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f20c807a010 tx=0x7f20b000b480 comp rx=0 tx=0).stop 2024-03-26T03:46:34.157 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.157+0000 7f20ccc88700 1 -- 172.21.15.175:0/2975191765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20c8102650 msgr2=0x7f20c8198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:34.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.157+0000 7f20ccc88700 1 --2- 172.21.15.175:0/2975191765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20c8102650 0x7f20c8198300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f20c000efc0 tx=0x7f20c000c3a0 comp rx=0 tx=0).stop 2024-03-26T03:46:34.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.157+0000 7f20c6d9d700 1 -- 172.21.15.175:0/2975191765 reap_dead start 2024-03-26T03:46:34.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.157+0000 7f20ccc88700 1 -- 172.21.15.175:0/2975191765 shutdown_connections 2024-03-26T03:46:34.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.157+0000 7f20ccc88700 1 -- 172.21.15.175:0/2975191765 >> 172.21.15.175:0/2975191765 conn(0x7f20c8079a10 msgr2=0x7f20c8100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:46:34.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.157+0000 7f20ccc88700 1 -- 172.21.15.175:0/2975191765 shutdown_connections 2024-03-26T03:46:34.158 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:34.157+0000 7f20ccc88700 1 -- 172.21.15.175:0/2975191765 wait complete. 2024-03-26T03:46:35.783 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:35 smithi175 bash[21527]: cluster 2024-03-26T03:46:34.505636+0000 mgr.y (mgr.24431) 81 : cluster [DBG] pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:35.783 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:35 smithi175 bash[21527]: audit 2024-03-26T03:46:34.668264+0000 mon.a (mon.0) 854 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:35.783 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:35 smithi175 bash[21527]: audit 2024-03-26T03:46:34.682904+0000 mon.a (mon.0) 855 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:35.784 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:35 smithi175 bash[29284]: cluster 2024-03-26T03:46:34.505636+0000 mgr.y (mgr.24431) 81 : cluster [DBG] pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:35.784 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:35 smithi175 bash[29284]: audit 2024-03-26T03:46:34.668264+0000 mon.a (mon.0) 854 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:35.784 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:35 smithi175 bash[29284]: audit 2024-03-26T03:46:34.682904+0000 mon.a (mon.0) 855 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:35.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:35 smithi190 bash[23392]: cluster 2024-03-26T03:46:34.505636+0000 mgr.y (mgr.24431) 81 : cluster [DBG] pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:35.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:35 smithi190 bash[23392]: audit 2024-03-26T03:46:34.668264+0000 mon.a (mon.0) 854 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:35.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:35 smithi190 bash[23392]: audit 2024-03-26T03:46:34.682904+0000 mon.a (mon.0) 855 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:37.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f564d258700 1 -- 172.21.15.175:0/1049148988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5648106530 msgr2=0x7f5648106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:37.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f564d258700 1 --2- 172.21.15.175:0/1049148988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5648106530 0x7f5648106910 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f56380098f0 tx=0x7f5638009c00 comp rx=0 tx=0).stop 2024-03-26T03:46:37.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f564d258700 1 -- 172.21.15.175:0/1049148988 shutdown_connections 2024-03-26T03:46:37.872 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f564d258700 1 --2- 172.21.15.175:0/1049148988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5648106530 0x7f5648106910 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:37.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f564d258700 1 --2- 172.21.15.175:0/1049148988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5648101f40 0x7f564810e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:37.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f564d258700 1 --2- 172.21.15.175:0/1049148988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5648101580 0x7f5648101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:37.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f564d258700 1 -- 172.21.15.175:0/1049148988 >> 172.21.15.175:0/1049148988 conn(0x7f56480784e0 msgr2=0x7f56480788f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:46:37.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f564d258700 1 -- 172.21.15.175:0/1049148988 shutdown_connections 2024-03-26T03:46:37.873 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f564d258700 1 -- 172.21.15.175:0/1049148988 wait complete. 2024-03-26T03:46:37.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f564d258700 1 Processor -- start 2024-03-26T03:46:37.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f564d258700 1 -- start start 2024-03-26T03:46:37.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f564d258700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5648101580 0x7f5648198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:37.874 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f564d258700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5648101f40 0x7f56481989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:37.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f564d258700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5648106530 0x7f564819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:37.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f564d258700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5648110df0 con 0x7f5648101f40 2024-03-26T03:46:37.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f564d258700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5648110c70 con 0x7f5648101580 2024-03-26T03:46:37.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f564d258700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5648110f70 con 0x7f5648106530 2024-03-26T03:46:37.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f564659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5648101f40 0x7f56481989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:37.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f564759e700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5648106530 0x7f564819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:37.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f564759e700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5648106530 0x7f564819cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:52614/0 (socket says 172.21.15.175:52614) 2024-03-26T03:46:37.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f564759e700 1 -- 172.21.15.175:0/3381644899 learned_addr learned my addr 172.21.15.175:0/3381644899 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:46:37.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f564659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5648101f40 0x7f56481989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:53158/0 (socket says 172.21.15.175:53158) 2024-03-26T03:46:37.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f5646d9d700 1 --2- 172.21.15.175:0/3381644899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5648101580 0x7f5648198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:37.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f564759e700 1 -- 172.21.15.175:0/3381644899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5648101580 msgr2=0x7f5648198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:37.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f564759e700 1 --2- 172.21.15.175:0/3381644899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5648101580 0x7f5648198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:37.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f564759e700 1 -- 172.21.15.175:0/3381644899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5648101f40 msgr2=0x7f56481989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:37.875 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f564759e700 1 --2- 172.21.15.175:0/3381644899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5648101f40 0x7f56481989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:37.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f564759e700 1 -- 172.21.15.175:0/3381644899 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f564819d480 con 0x7f5648106530 2024-03-26T03:46:37.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f564659c700 1 --2- 172.21.15.175:0/3381644899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5648101f40 0x7f56481989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T03:46:37.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f564759e700 1 --2- 172.21.15.175:0/3381644899 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5648106530 0x7f564819cd40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f564000b970 tx=0x7f564000bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:46:37.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f562ffff700 1 -- 172.21.15.175:0/3381644899 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5640019070 con 0x7f5648106530 2024-03-26T03:46:37.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f564d258700 1 -- 172.21.15.175:0/3381644899 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f564819d770 con 0x7f5648106530 2024-03-26T03:46:37.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f562ffff700 1 -- 172.21.15.175:0/3381644899 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f564000cd70 con 0x7f5648106530 2024-03-26T03:46:37.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f562ffff700 1 -- 172.21.15.175:0/3381644899 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5640014da0 con 0x7f5648106530 2024-03-26T03:46:37.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.873+0000 7f564d258700 1 -- 172.21.15.175:0/3381644899 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f56481a4ff0 con 0x7f5648106530 2024-03-26T03:46:37.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.877+0000 7f564d258700 1 -- 172.21.15.175:0/3381644899 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f564806b660 con 0x7f5648106530 2024-03-26T03:46:37.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.877+0000 7f562ffff700 1 -- 172.21.15.175:0/3381644899 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7f564001c070 con 0x7f5648106530 2024-03-26T03:46:37.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.877+0000 7f562ffff700 1 --2- 172.21.15.175:0/3381644899 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f563006e7e0 0x7f5630070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:37.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.877+0000 7f562ffff700 1 -- 172.21.15.175:0/3381644899 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(75..75 src has 1..75) ==== 7537+0+0 (secure 0 0 0) 0x7f564008e350 con 0x7f5648106530 2024-03-26T03:46:37.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.877+0000 7f5646d9d700 1 --2- 172.21.15.175:0/3381644899 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f563006e7e0 0x7f5630070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:37.877 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.877+0000 7f562ffff700 1 --2- 172.21.15.175:0/3381644899 >> [v2:172.21.15.190:6824/3444881554,v1:172.21.15.190:6825/3444881554] conn(0x7f5630078770 0x7f563007ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:37.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.877+0000 7f562ffff700 1 -- 172.21.15.175:0/3381644899 --> [v2:172.21.15.190:6824/3444881554,v1:172.21.15.190:6825/3444881554] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f563007b2e0 con 0x7f5630078770 2024-03-26T03:46:37.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.877+0000 7f562ffff700 1 -- 172.21.15.175:0/3381644899 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f5640095050 con 0x7f5648106530 2024-03-26T03:46:37.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.877+0000 7f5646d9d700 1 --2- 172.21.15.175:0/3381644899 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f563006e7e0 0x7f5630070ca0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5638009230 tx=0x7f5638005b40 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:46:37.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.877+0000 7f564659c700 1 --2- 172.21.15.175:0/3381644899 >> [v2:172.21.15.190:6824/3444881554,v1:172.21.15.190:6825/3444881554] conn(0x7f5630078770 0x7f563007ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:37.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.877+0000 7f564659c700 1 --2- 172.21.15.175:0/3381644899 >> [v2:172.21.15.190:6824/3444881554,v1:172.21.15.190:6825/3444881554] conn(0x7f5630078770 0x7f563007ac10 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:46:37.879 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.881+0000 7f562ffff700 1 -- 172.21.15.175:0/3381644899 <== osd.7 v2:172.21.15.190:6824/3444881554 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f563007b2e0 con 0x7f5630078770 2024-03-26T03:46:37.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.897+0000 7f564d258700 1 -- 172.21.15.175:0/3381644899 --> [v2:172.21.15.190:6824/3444881554,v1:172.21.15.190:6825/3444881554] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5648102c70 con 0x7f5630078770 2024-03-26T03:46:37.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.897+0000 7f562ffff700 1 -- 172.21.15.175:0/3381644899 <== osd.7 v2:172.21.15.190:6824/3444881554 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5648102c70 con 0x7f5630078770 2024-03-26T03:46:37.897 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.897+0000 7f564d258700 1 -- 172.21.15.175:0/3381644899 >> [v2:172.21.15.190:6824/3444881554,v1:172.21.15.190:6825/3444881554] conn(0x7f5630078770 msgr2=0x7f563007ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:37.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.897+0000 7f564d258700 1 --2- 172.21.15.175:0/3381644899 >> [v2:172.21.15.190:6824/3444881554,v1:172.21.15.190:6825/3444881554] conn(0x7f5630078770 0x7f563007ac10 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:37.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.897+0000 7f564d258700 1 -- 172.21.15.175:0/3381644899 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f563006e7e0 msgr2=0x7f5630070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:37.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.897+0000 7f564d258700 1 --2- 172.21.15.175:0/3381644899 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f563006e7e0 0x7f5630070ca0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5638009230 tx=0x7f5638005b40 comp rx=0 tx=0).stop 2024-03-26T03:46:37.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.897+0000 7f564d258700 1 -- 172.21.15.175:0/3381644899 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5648106530 msgr2=0x7f564819cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:37.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.897+0000 7f564d258700 1 --2- 172.21.15.175:0/3381644899 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5648106530 0x7f564819cd40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f564000b970 tx=0x7f564000bd30 comp rx=0 tx=0).stop 2024-03-26T03:46:37.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.897+0000 7f564759e700 1 -- 172.21.15.175:0/3381644899 reap_dead start 2024-03-26T03:46:37.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.897+0000 7f564d258700 1 -- 172.21.15.175:0/3381644899 shutdown_connections 2024-03-26T03:46:37.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.897+0000 7f564d258700 1 -- 172.21.15.175:0/3381644899 >> 172.21.15.175:0/3381644899 conn(0x7f56480784e0 msgr2=0x7f564806ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:46:37.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.897+0000 7f564d258700 1 -- 172.21.15.175:0/3381644899 shutdown_connections 2024-03-26T03:46:37.898 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:37.897+0000 7f564d258700 1 -- 172.21.15.175:0/3381644899 wait complete. 2024-03-26T03:46:37.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:37 smithi190 bash[23392]: cluster 2024-03-26T03:46:36.506267+0000 mgr.y (mgr.24431) 82 : cluster [DBG] pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:37.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:37 smithi175 bash[29284]: cluster 2024-03-26T03:46:36.506267+0000 mgr.y (mgr.24431) 82 : cluster [DBG] pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:37.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:37 smithi175 bash[21527]: cluster 2024-03-26T03:46:36.506267+0000 mgr.y (mgr.24431) 82 : cluster [DBG] pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:38.962 INFO:teuthology.orchestra.run.smithi175.stdout:77309411419 2024-03-26T03:46:38.963 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-26T03:46:39.209 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:46:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:46:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:46:39.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.577+0000 7f2783964700 1 -- 172.21.15.175:0/1032071874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f277c104570 msgr2=0x7f277c104950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:39.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.577+0000 7f2783964700 1 --2- 172.21.15.175:0/1032071874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f277c104570 0x7f277c104950 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f277800b0a0 tx=0x7f277800b3b0 comp rx=0 tx=0).stop 2024-03-26T03:46:39.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.577+0000 7f2783964700 1 -- 172.21.15.175:0/1032071874 shutdown_connections 2024-03-26T03:46:39.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.577+0000 7f2783964700 1 --2- 172.21.15.175:0/1032071874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f277c104570 0x7f277c104950 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:39.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.577+0000 7f2783964700 1 --2- 172.21.15.175:0/1032071874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f277c06b770 0x7f277c10f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:39.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.577+0000 7f2783964700 1 --2- 172.21.15.175:0/1032071874 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f277c06adb0 0x7f277c06b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:39.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.577+0000 7f2783964700 1 -- 172.21.15.175:0/1032071874 >> 172.21.15.175:0/1032071874 conn(0x7f277c0775e0 msgr2=0x7f277c0779f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:46:39.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.577+0000 7f2783964700 1 -- 172.21.15.175:0/1032071874 shutdown_connections 2024-03-26T03:46:39.579 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.577+0000 7f2783964700 1 -- 172.21.15.175:0/1032071874 wait complete. 2024-03-26T03:46:39.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.581+0000 7f2783964700 1 Processor -- start 2024-03-26T03:46:39.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.581+0000 7f2783964700 1 -- start start 2024-03-26T03:46:39.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.581+0000 7f2783964700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f277c06adb0 0x7f277c1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:39.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.581+0000 7f2783964700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f277c06b770 0x7f277c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:39.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.581+0000 7f2783964700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f277c104570 0x7f277c19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:39.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.581+0000 7f2783964700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f277c112060 con 0x7f277c06b770 2024-03-26T03:46:39.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.581+0000 7f2783964700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f277c111ee0 con 0x7f277c06adb0 2024-03-26T03:46:39.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.581+0000 7f2783964700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f277c1121e0 con 0x7f277c104570 2024-03-26T03:46:39.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.581+0000 7f2781f01700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f277c104570 0x7f277c19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:39.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.581+0000 7f2780eff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f277c06b770 0x7f277c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:39.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.581+0000 7f2781f01700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f277c104570 0x7f277c19cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:52630/0 (socket says 172.21.15.175:52630) 2024-03-26T03:46:39.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.581+0000 7f2780eff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f277c06b770 0x7f277c1989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:53180/0 (socket says 172.21.15.175:53180) 2024-03-26T03:46:39.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.581+0000 7f2781f01700 1 -- 172.21.15.175:0/3469014535 learned_addr learned my addr 172.21.15.175:0/3469014535 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:46:39.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.581+0000 7f2781700700 1 --2- 172.21.15.175:0/3469014535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f277c06adb0 0x7f277c1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:39.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.581+0000 7f2780eff700 1 -- 172.21.15.175:0/3469014535 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f277c104570 msgr2=0x7f277c19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:39.583 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.581+0000 7f2780eff700 1 --2- 172.21.15.175:0/3469014535 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f277c104570 0x7f277c19cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:39.583 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.581+0000 7f2780eff700 1 -- 172.21.15.175:0/3469014535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f277c06adb0 msgr2=0x7f277c1984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:39.583 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.581+0000 7f2780eff700 1 --2- 172.21.15.175:0/3469014535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f277c06adb0 0x7f277c1984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:39.583 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.581+0000 7f2780eff700 1 -- 172.21.15.175:0/3469014535 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f277c19d4c0 con 0x7f277c06b770 2024-03-26T03:46:39.583 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.581+0000 7f2781f01700 1 --2- 172.21.15.175:0/3469014535 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f277c104570 0x7f277c19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T03:46:39.583 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.581+0000 7f2780eff700 1 --2- 172.21.15.175:0/3469014535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f277c06b770 0x7f277c1989f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f276c00b450 tx=0x7f276c00b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:46:39.583 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.581+0000 7f27727fc700 1 -- 172.21.15.175:0/3469014535 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c015020 con 0x7f277c06b770 2024-03-26T03:46:39.583 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.581+0000 7f27727fc700 1 -- 172.21.15.175:0/3469014535 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f276c011cb0 con 0x7f277c06b770 2024-03-26T03:46:39.583 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.581+0000 7f2783964700 1 -- 172.21.15.175:0/3469014535 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f277c19d7b0 con 0x7f277c06b770 2024-03-26T03:46:39.583 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.581+0000 7f27727fc700 1 -- 172.21.15.175:0/3469014535 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c010bc0 con 0x7f277c06b770 2024-03-26T03:46:39.583 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.581+0000 7f2783964700 1 -- 172.21.15.175:0/3469014535 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f277c1a5030 con 0x7f277c06b770 2024-03-26T03:46:39.583 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.581+0000 7f2783964700 1 -- 172.21.15.175:0/3469014535 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f277c062d10 con 0x7f277c06b770 2024-03-26T03:46:39.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.581+0000 7f27727fc700 1 -- 172.21.15.175:0/3469014535 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7f276c018020 con 0x7f277c06b770 2024-03-26T03:46:39.584 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.581+0000 7f27727fc700 1 --2- 172.21.15.175:0/3469014535 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f276806e7e0 0x7f2768070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:39.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.585+0000 7f27727fc700 1 -- 172.21.15.175:0/3469014535 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7537+0+0 (secure 0 0 0) 0x7f276c08ec40 con 0x7f277c06b770 2024-03-26T03:46:39.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.585+0000 7f2781700700 1 --2- 172.21.15.175:0/3469014535 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f276806e7e0 0x7f2768070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:39.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.585+0000 7f27727fc700 1 --2- 172.21.15.175:0/3469014535 >> [v2:172.21.15.190:6808/4089468657,v1:172.21.15.190:6809/4089468657] conn(0x7f27680786f0 0x7f276807ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:39.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.585+0000 7f27727fc700 1 -- 172.21.15.175:0/3469014535 --> [v2:172.21.15.190:6808/4089468657,v1:172.21.15.190:6809/4089468657] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f276c01deb0 con 0x7f27680786f0 2024-03-26T03:46:39.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.585+0000 7f27727fc700 1 -- 172.21.15.175:0/3469014535 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f276c08f0d0 con 0x7f277c06b770 2024-03-26T03:46:39.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.585+0000 7f2781700700 1 --2- 172.21.15.175:0/3469014535 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f276806e7e0 0x7f2768070ca0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f277800b780 tx=0x7f2778009be0 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:46:39.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.585+0000 7f2781f01700 1 --2- 172.21.15.175:0/3469014535 >> [v2:172.21.15.190:6808/4089468657,v1:172.21.15.190:6809/4089468657] conn(0x7f27680786f0 0x7f276807ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:39.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.585+0000 7f2781f01700 1 --2- 172.21.15.175:0/3469014535 >> [v2:172.21.15.190:6808/4089468657,v1:172.21.15.190:6809/4089468657] conn(0x7f27680786f0 0x7f276807ab90 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:46:39.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.585+0000 7f27727fc700 1 -- 172.21.15.175:0/3469014535 <== osd.5 v2:172.21.15.190:6808/4089468657 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f276c01deb0 con 0x7f27680786f0 2024-03-26T03:46:39.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.605+0000 7f2783964700 1 -- 172.21.15.175:0/3469014535 --> [v2:172.21.15.190:6808/4089468657,v1:172.21.15.190:6809/4089468657] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f277c063110 con 0x7f27680786f0 2024-03-26T03:46:39.603 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.605+0000 7f27727fc700 1 -- 172.21.15.175:0/3469014535 <== osd.5 v2:172.21.15.190:6808/4089468657 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f277c063110 con 0x7f27680786f0 2024-03-26T03:46:39.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.605+0000 7f2783964700 1 -- 172.21.15.175:0/3469014535 >> [v2:172.21.15.190:6808/4089468657,v1:172.21.15.190:6809/4089468657] conn(0x7f27680786f0 msgr2=0x7f276807ab90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:39.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.605+0000 7f2783964700 1 --2- 172.21.15.175:0/3469014535 >> [v2:172.21.15.190:6808/4089468657,v1:172.21.15.190:6809/4089468657] conn(0x7f27680786f0 0x7f276807ab90 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:39.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.605+0000 7f2783964700 1 -- 172.21.15.175:0/3469014535 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f276806e7e0 msgr2=0x7f2768070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:39.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.605+0000 7f2783964700 1 --2- 172.21.15.175:0/3469014535 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f276806e7e0 0x7f2768070ca0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f277800b780 tx=0x7f2778009be0 comp rx=0 tx=0).stop 2024-03-26T03:46:39.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.605+0000 7f2783964700 1 -- 172.21.15.175:0/3469014535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f277c06b770 msgr2=0x7f277c1989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:39.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.605+0000 7f2783964700 1 --2- 172.21.15.175:0/3469014535 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f277c06b770 0x7f277c1989f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f276c00b450 tx=0x7f276c00b760 comp rx=0 tx=0).stop 2024-03-26T03:46:39.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.605+0000 7f2781f01700 1 -- 172.21.15.175:0/3469014535 reap_dead start 2024-03-26T03:46:39.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.605+0000 7f2783964700 1 -- 172.21.15.175:0/3469014535 shutdown_connections 2024-03-26T03:46:39.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.605+0000 7f2783964700 1 -- 172.21.15.175:0/3469014535 >> 172.21.15.175:0/3469014535 conn(0x7f277c0775e0 msgr2=0x7f277c0ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:46:39.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.605+0000 7f2783964700 1 -- 172.21.15.175:0/3469014535 shutdown_connections 2024-03-26T03:46:39.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:39.605+0000 7f2783964700 1 -- 172.21.15.175:0/3469014535 wait complete. 2024-03-26T03:46:39.868 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:39 smithi175 bash[29284]: cluster 2024-03-26T03:46:38.507189+0000 mgr.y (mgr.24431) 83 : cluster [DBG] pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:39.869 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:39 smithi175 bash[21527]: cluster 2024-03-26T03:46:38.507189+0000 mgr.y (mgr.24431) 83 : cluster [DBG] pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:39.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:39 smithi190 bash[23392]: cluster 2024-03-26T03:46:38.507189+0000 mgr.y (mgr.24431) 83 : cluster [DBG] pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:39.990 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:46:39 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:46:40.429 INFO:teuthology.orchestra.run.smithi175.stdout:236223201318 2024-03-26T03:46:40.430 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-26T03:46:40.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.585+0000 7efd798b8700 1 -- 172.21.15.175:0/760107735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd741018f0 msgr2=0x7efd74105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:40.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.585+0000 7efd798b8700 1 --2- 172.21.15.175:0/760107735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd741018f0 0x7efd74105940 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efd680099c0 tx=0x7efd68009cd0 comp rx=0 tx=0).stop 2024-03-26T03:46:40.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.585+0000 7efd798b8700 1 -- 172.21.15.175:0/760107735 shutdown_connections 2024-03-26T03:46:40.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.585+0000 7efd798b8700 1 --2- 172.21.15.175:0/760107735 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd74100f40 0x7efd74101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:40.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.585+0000 7efd798b8700 1 --2- 172.21.15.175:0/760107735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd74106020 0x7efd7410dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:40.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.585+0000 7efd798b8700 1 --2- 172.21.15.175:0/760107735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd741018f0 0x7efd74105940 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:40.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.585+0000 7efd798b8700 1 -- 172.21.15.175:0/760107735 >> 172.21.15.175:0/760107735 conn(0x7efd74077b40 msgr2=0x7efd740feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:46:40.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.585+0000 7efd798b8700 1 -- 172.21.15.175:0/760107735 shutdown_connections 2024-03-26T03:46:40.586 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.585+0000 7efd798b8700 1 -- 172.21.15.175:0/760107735 wait complete. 2024-03-26T03:46:40.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.585+0000 7efd798b8700 1 Processor -- start 2024-03-26T03:46:40.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.585+0000 7efd798b8700 1 -- start start 2024-03-26T03:46:40.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.585+0000 7efd798b8700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd74100f40 0x7efd741983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:40.587 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.585+0000 7efd798b8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd741018f0 0x7efd741988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:40.588 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.585+0000 7efd798b8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd74106020 0x7efd7419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:40.588 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.585+0000 7efd798b8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efd7410fca0 con 0x7efd74106020 2024-03-26T03:46:40.588 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.585+0000 7efd798b8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7efd7410fb20 con 0x7efd741018f0 2024-03-26T03:46:40.588 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.585+0000 7efd798b8700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efd7410fe20 con 0x7efd74100f40 2024-03-26T03:46:40.588 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.585+0000 7efd72ffd700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd74100f40 0x7efd741983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:40.588 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.585+0000 7efd72ffd700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd74100f40 0x7efd741983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:52656/0 (socket says 172.21.15.175:52656) 2024-03-26T03:46:40.588 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.585+0000 7efd737fe700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd74106020 0x7efd7419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:40.588 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.585+0000 7efd72ffd700 1 -- 172.21.15.175:0/3069049253 learned_addr learned my addr 172.21.15.175:0/3069049253 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:46:40.588 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.589+0000 7efd727fc700 1 --2- 172.21.15.175:0/3069049253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd741018f0 0x7efd741988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:40.588 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.589+0000 7efd72ffd700 1 -- 172.21.15.175:0/3069049253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd741018f0 msgr2=0x7efd741988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:40.588 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.589+0000 7efd72ffd700 1 --2- 172.21.15.175:0/3069049253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd741018f0 0x7efd741988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:40.588 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.589+0000 7efd72ffd700 1 -- 172.21.15.175:0/3069049253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd74106020 msgr2=0x7efd7419cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:40.588 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.589+0000 7efd72ffd700 1 --2- 172.21.15.175:0/3069049253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd74106020 0x7efd7419cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:40.588 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.589+0000 7efd72ffd700 1 -- 172.21.15.175:0/3069049253 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7419d3c0 con 0x7efd74100f40 2024-03-26T03:46:40.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.589+0000 7efd737fe700 1 --2- 172.21.15.175:0/3069049253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd74106020 0x7efd7419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T03:46:40.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.589+0000 7efd727fc700 1 --2- 172.21.15.175:0/3069049253 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd741018f0 0x7efd741988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T03:46:40.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.589+0000 7efd72ffd700 1 --2- 172.21.15.175:0/3069049253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd74100f40 0x7efd741983b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efd6400eba0 tx=0x7efd6400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:46:40.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.589+0000 7efd788b6700 1 -- 172.21.15.175:0/3069049253 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6400cbc0 con 0x7efd74100f40 2024-03-26T03:46:40.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.589+0000 7efd788b6700 1 -- 172.21.15.175:0/3069049253 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7efd640044a0 con 0x7efd74100f40 2024-03-26T03:46:40.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.589+0000 7efd798b8700 1 -- 172.21.15.175:0/3069049253 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd7419d6b0 con 0x7efd74100f40 2024-03-26T03:46:40.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.589+0000 7efd788b6700 1 -- 172.21.15.175:0/3069049253 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd64003dc0 con 0x7efd74100f40 2024-03-26T03:46:40.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.589+0000 7efd798b8700 1 -- 172.21.15.175:0/3069049253 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efd74195ee0 con 0x7efd74100f40 2024-03-26T03:46:40.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.589+0000 7efd798b8700 1 -- 172.21.15.175:0/3069049253 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efd74104520 con 0x7efd74100f40 2024-03-26T03:46:40.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.589+0000 7efd788b6700 1 -- 172.21.15.175:0/3069049253 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7efd6401d020 con 0x7efd74100f40 2024-03-26T03:46:40.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.589+0000 7efd788b6700 1 --2- 172.21.15.175:0/3069049253 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7efd5c06e7e0 0x7efd5c070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:40.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.589+0000 7efd788b6700 1 -- 172.21.15.175:0/3069049253 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(75..75 src has 1..75) ==== 7537+0+0 (secure 0 0 0) 0x7efd6408e900 con 0x7efd74100f40 2024-03-26T03:46:40.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.589+0000 7efd727fc700 1 --2- 172.21.15.175:0/3069049253 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7efd5c06e7e0 0x7efd5c070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:40.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.589+0000 7efd788b6700 1 --2- 172.21.15.175:0/3069049253 >> [v2:172.21.15.175:6826/672326154,v1:172.21.15.175:6827/672326154] conn(0x7efd5c0786f0 0x7efd5c07ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:40.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.589+0000 7efd788b6700 1 -- 172.21.15.175:0/3069049253 --> [v2:172.21.15.175:6826/672326154,v1:172.21.15.175:6827/672326154] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efd6400a610 con 0x7efd5c0786f0 2024-03-26T03:46:40.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.589+0000 7efd788b6700 1 -- 172.21.15.175:0/3069049253 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7efd640932c0 con 0x7efd74100f40 2024-03-26T03:46:40.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.589+0000 7efd737fe700 1 --2- 172.21.15.175:0/3069049253 >> [v2:172.21.15.175:6826/672326154,v1:172.21.15.175:6827/672326154] conn(0x7efd5c0786f0 0x7efd5c07ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:40.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.589+0000 7efd727fc700 1 --2- 172.21.15.175:0/3069049253 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7efd5c06e7e0 0x7efd5c070ca0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efd68006010 tx=0x7efd68005e80 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:46:40.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.589+0000 7efd737fe700 1 --2- 172.21.15.175:0/3069049253 >> [v2:172.21.15.175:6826/672326154,v1:172.21.15.175:6827/672326154] conn(0x7efd5c0786f0 0x7efd5c07ab90 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:46:40.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.593+0000 7efd788b6700 1 -- 172.21.15.175:0/3069049253 <== osd.3 v2:172.21.15.175:6826/672326154 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efd6400a610 con 0x7efd5c0786f0 2024-03-26T03:46:40.610 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.609+0000 7efd798b8700 1 -- 172.21.15.175:0/3069049253 --> [v2:172.21.15.175:6826/672326154,v1:172.21.15.175:6827/672326154] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efd74102420 con 0x7efd5c0786f0 2024-03-26T03:46:40.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.609+0000 7efd788b6700 1 -- 172.21.15.175:0/3069049253 <== osd.3 v2:172.21.15.175:6826/672326154 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7efd74102420 con 0x7efd5c0786f0 2024-03-26T03:46:40.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.613+0000 7efd798b8700 1 -- 172.21.15.175:0/3069049253 >> [v2:172.21.15.175:6826/672326154,v1:172.21.15.175:6827/672326154] conn(0x7efd5c0786f0 msgr2=0x7efd5c07ab90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:40.611 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.613+0000 7efd798b8700 1 --2- 172.21.15.175:0/3069049253 >> [v2:172.21.15.175:6826/672326154,v1:172.21.15.175:6827/672326154] conn(0x7efd5c0786f0 0x7efd5c07ab90 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:40.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.613+0000 7efd798b8700 1 -- 172.21.15.175:0/3069049253 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7efd5c06e7e0 msgr2=0x7efd5c070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:40.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.613+0000 7efd798b8700 1 --2- 172.21.15.175:0/3069049253 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7efd5c06e7e0 0x7efd5c070ca0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efd68006010 tx=0x7efd68005e80 comp rx=0 tx=0).stop 2024-03-26T03:46:40.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.613+0000 7efd798b8700 1 -- 172.21.15.175:0/3069049253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd74100f40 msgr2=0x7efd741983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:40.616 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.613+0000 7efd798b8700 1 --2- 172.21.15.175:0/3069049253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd74100f40 0x7efd741983b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efd6400eba0 tx=0x7efd6400eeb0 comp rx=0 tx=0).stop 2024-03-26T03:46:40.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.613+0000 7efd737fe700 1 -- 172.21.15.175:0/3069049253 reap_dead start 2024-03-26T03:46:40.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.613+0000 7efd798b8700 1 -- 172.21.15.175:0/3069049253 shutdown_connections 2024-03-26T03:46:40.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.613+0000 7efd798b8700 1 -- 172.21.15.175:0/3069049253 >> 172.21.15.175:0/3069049253 conn(0x7efd74077b40 msgr2=0x7efd74103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:46:40.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.613+0000 7efd798b8700 1 -- 172.21.15.175:0/3069049253 shutdown_connections 2024-03-26T03:46:40.617 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.613+0000 7efd798b8700 1 -- 172.21.15.175:0/3069049253 wait complete. 2024-03-26T03:46:40.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.805+0000 7ffa17db7700 1 -- 172.21.15.175:0/2558893988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa10101550 msgr2=0x7ffa1010f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:40.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.805+0000 7ffa17db7700 1 --2- 172.21.15.175:0/2558893988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa10101550 0x7ffa1010f530 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800b0a0 tx=0x7ffa0800b3b0 comp rx=0 tx=0).stop 2024-03-26T03:46:40.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.805+0000 7ffa17db7700 1 -- 172.21.15.175:0/2558893988 shutdown_connections 2024-03-26T03:46:40.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.805+0000 7ffa17db7700 1 --2- 172.21.15.175:0/2558893988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa10101550 0x7ffa1010f530 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:40.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.805+0000 7ffa17db7700 1 --2- 172.21.15.175:0/2558893988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa10108660 0x7ffa10108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:40.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.805+0000 7ffa17db7700 1 --2- 172.21.15.175:0/2558893988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa10100b90 0x7ffa10101010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:40.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.805+0000 7ffa17db7700 1 -- 172.21.15.175:0/2558893988 >> 172.21.15.175:0/2558893988 conn(0x7ffa10077b40 msgr2=0x7ffa100feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:46:40.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.809+0000 7ffa17db7700 1 -- 172.21.15.175:0/2558893988 shutdown_connections 2024-03-26T03:46:40.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.809+0000 7ffa17db7700 1 -- 172.21.15.175:0/2558893988 wait complete. 2024-03-26T03:46:40.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.809+0000 7ffa17db7700 1 Processor -- start 2024-03-26T03:46:40.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.809+0000 7ffa17db7700 1 -- start start 2024-03-26T03:46:40.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.809+0000 7ffa17db7700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa10100b90 0x7ffa10198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:40.808 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.809+0000 7ffa17db7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa10101550 0x7ffa10198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:40.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.809+0000 7ffa17db7700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa10108660 0x7ffa1019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:40.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.809+0000 7ffa17db7700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffa10111ec0 con 0x7ffa10100b90 2024-03-26T03:46:40.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.809+0000 7ffa17db7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ffa10111d40 con 0x7ffa10101550 2024-03-26T03:46:40.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.809+0000 7ffa17db7700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ffa10112040 con 0x7ffa10108660 2024-03-26T03:46:40.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.809+0000 7ffa16354700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa10108660 0x7ffa1019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:40.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.809+0000 7ffa16354700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa10108660 0x7ffa1019cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:52664/0 (socket says 172.21.15.175:52664) 2024-03-26T03:46:40.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.809+0000 7ffa16354700 1 -- 172.21.15.175:0/2048731506 learned_addr learned my addr 172.21.15.175:0/2048731506 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:46:40.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.809+0000 7ffa15b53700 1 --2- 172.21.15.175:0/2048731506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa10100b90 0x7ffa10198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:40.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.809+0000 7ffa15352700 1 --2- 172.21.15.175:0/2048731506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa10101550 0x7ffa10198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:40.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.809+0000 7ffa16354700 1 -- 172.21.15.175:0/2048731506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa10101550 msgr2=0x7ffa10198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:40.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.809+0000 7ffa16354700 1 --2- 172.21.15.175:0/2048731506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa10101550 0x7ffa10198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:40.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.809+0000 7ffa16354700 1 -- 172.21.15.175:0/2048731506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa10100b90 msgr2=0x7ffa10198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:40.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.809+0000 7ffa16354700 1 --2- 172.21.15.175:0/2048731506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa10100b90 0x7ffa10198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:40.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.809+0000 7ffa16354700 1 -- 172.21.15.175:0/2048731506 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1019d330 con 0x7ffa10108660 2024-03-26T03:46:40.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.809+0000 7ffa15352700 1 --2- 172.21.15.175:0/2048731506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa10101550 0x7ffa10198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T03:46:40.809 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.809+0000 7ffa16354700 1 --2- 172.21.15.175:0/2048731506 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa10108660 0x7ffa1019cbf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800b070 tx=0x7ffa08007a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:46:40.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.809+0000 7ffa02ffd700 1 -- 172.21.15.175:0/2048731506 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa08021020 con 0x7ffa10108660 2024-03-26T03:46:40.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.809+0000 7ffa02ffd700 1 -- 172.21.15.175:0/2048731506 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7ffa08004c50 con 0x7ffa10108660 2024-03-26T03:46:40.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.809+0000 7ffa02ffd700 1 -- 172.21.15.175:0/2048731506 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa08005050 con 0x7ffa10108660 2024-03-26T03:46:40.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.809+0000 7ffa17db7700 1 -- 172.21.15.175:0/2048731506 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa1019d5c0 con 0x7ffa10108660 2024-03-26T03:46:40.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.809+0000 7ffa17db7700 1 -- 172.21.15.175:0/2048731506 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa101a4ea0 con 0x7ffa10108660 2024-03-26T03:46:40.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.809+0000 7ffa17db7700 1 -- 172.21.15.175:0/2048731506 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffa10107b60 con 0x7ffa10108660 2024-03-26T03:46:40.811 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:40 smithi175 bash[29284]: audit 2024-03-26T03:46:39.548400+0000 mgr.y (mgr.24431) 84 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:46:40.811 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:40 smithi175 bash[21527]: audit 2024-03-26T03:46:39.548400+0000 mgr.y (mgr.24431) 84 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:46:40.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.813+0000 7ffa02ffd700 1 -- 172.21.15.175:0/2048731506 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7ffa08024020 con 0x7ffa10108660 2024-03-26T03:46:40.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.813+0000 7ffa02ffd700 1 --2- 172.21.15.175:0/2048731506 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7ff9fc06e7e0 0x7ff9fc070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:40.812 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.813+0000 7ffa02ffd700 1 -- 172.21.15.175:0/2048731506 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(75..75 src has 1..75) ==== 7537+0+0 (secure 0 0 0) 0x7ffa0800f070 con 0x7ffa10108660 2024-03-26T03:46:40.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.813+0000 7ffa15b53700 1 --2- 172.21.15.175:0/2048731506 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7ff9fc06e7e0 0x7ff9fc070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:40.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.813+0000 7ffa02ffd700 1 --2- 172.21.15.175:0/2048731506 >> [v2:172.21.15.175:6810/931168929,v1:172.21.15.175:6811/931168929] conn(0x7ff9fc078770 0x7ff9fc07ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:40.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.813+0000 7ffa02ffd700 1 -- 172.21.15.175:0/2048731506 --> [v2:172.21.15.175:6810/931168929,v1:172.21.15.175:6811/931168929] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa08013e70 con 0x7ff9fc078770 2024-03-26T03:46:40.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.813+0000 7ffa02ffd700 1 -- 172.21.15.175:0/2048731506 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7ffa080942c0 con 0x7ffa10108660 2024-03-26T03:46:40.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.813+0000 7ffa15352700 1 --2- 172.21.15.175:0/2048731506 >> [v2:172.21.15.175:6810/931168929,v1:172.21.15.175:6811/931168929] conn(0x7ff9fc078770 0x7ff9fc07ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:40.813 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.813+0000 7ffa15b53700 1 --2- 172.21.15.175:0/2048731506 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7ff9fc06e7e0 0x7ff9fc070ca0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009c60 tx=0x7ffa0c009260 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:46:40.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.813+0000 7ffa15352700 1 --2- 172.21.15.175:0/2048731506 >> [v2:172.21.15.175:6810/931168929,v1:172.21.15.175:6811/931168929] conn(0x7ff9fc078770 0x7ff9fc07ac10 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:46:40.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.813+0000 7ffa02ffd700 1 -- 172.21.15.175:0/2048731506 <== osd.1 v2:172.21.15.175:6810/931168929 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa08013e70 con 0x7ff9fc078770 2024-03-26T03:46:40.831 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.829+0000 7ffa17db7700 1 -- 172.21.15.175:0/2048731506 --> [v2:172.21.15.175:6810/931168929,v1:172.21.15.175:6811/931168929] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffa10103eb0 con 0x7ff9fc078770 2024-03-26T03:46:40.831 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.833+0000 7ffa02ffd700 1 -- 172.21.15.175:0/2048731506 <== osd.1 v2:172.21.15.175:6810/931168929 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ffa10103eb0 con 0x7ff9fc078770 2024-03-26T03:46:40.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.833+0000 7ffa17db7700 1 -- 172.21.15.175:0/2048731506 >> [v2:172.21.15.175:6810/931168929,v1:172.21.15.175:6811/931168929] conn(0x7ff9fc078770 msgr2=0x7ff9fc07ac10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:40.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.833+0000 7ffa17db7700 1 --2- 172.21.15.175:0/2048731506 >> [v2:172.21.15.175:6810/931168929,v1:172.21.15.175:6811/931168929] conn(0x7ff9fc078770 0x7ff9fc07ac10 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:40.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.833+0000 7ffa17db7700 1 -- 172.21.15.175:0/2048731506 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7ff9fc06e7e0 msgr2=0x7ff9fc070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:40.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.833+0000 7ffa17db7700 1 --2- 172.21.15.175:0/2048731506 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7ff9fc06e7e0 0x7ff9fc070ca0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009c60 tx=0x7ffa0c009260 comp rx=0 tx=0).stop 2024-03-26T03:46:40.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.833+0000 7ffa17db7700 1 -- 172.21.15.175:0/2048731506 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa10108660 msgr2=0x7ffa1019cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:40.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.833+0000 7ffa17db7700 1 --2- 172.21.15.175:0/2048731506 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa10108660 0x7ffa1019cbf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800b070 tx=0x7ffa08007a20 comp rx=0 tx=0).stop 2024-03-26T03:46:40.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.833+0000 7ffa16354700 1 -- 172.21.15.175:0/2048731506 reap_dead start 2024-03-26T03:46:40.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.833+0000 7ffa17db7700 1 -- 172.21.15.175:0/2048731506 shutdown_connections 2024-03-26T03:46:40.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.833+0000 7ffa17db7700 1 -- 172.21.15.175:0/2048731506 >> 172.21.15.175:0/2048731506 conn(0x7ffa10077b40 msgr2=0x7ffa101061b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:46:40.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.833+0000 7ffa17db7700 1 -- 172.21.15.175:0/2048731506 shutdown_connections 2024-03-26T03:46:40.833 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:40.833+0000 7ffa17db7700 1 -- 172.21.15.175:0/2048731506 wait complete. 2024-03-26T03:46:40.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:40 smithi190 bash[23392]: audit 2024-03-26T03:46:39.548400+0000 mgr.y (mgr.24431) 84 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:46:41.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd306cd0700 1 -- 172.21.15.175:0/725180705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd30006a3a0 msgr2=0x7fd30006a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:41.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd306cd0700 1 --2- 172.21.15.175:0/725180705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd30006a3a0 0x7fd30006a820 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec009990 tx=0x7fd2ec009ca0 comp rx=0 tx=0).stop 2024-03-26T03:46:41.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd306cd0700 1 -- 172.21.15.175:0/725180705 shutdown_connections 2024-03-26T03:46:41.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd306cd0700 1 --2- 172.21.15.175:0/725180705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd30006ad60 0x7fd30010f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:41.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd306cd0700 1 --2- 172.21.15.175:0/725180705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd300108650 0x7fd300108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:41.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd306cd0700 1 --2- 172.21.15.175:0/725180705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd30006a3a0 0x7fd30006a820 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:41.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd306cd0700 1 -- 172.21.15.175:0/725180705 >> 172.21.15.175:0/725180705 conn(0x7fd3000775e0 msgr2=0x7fd3000779f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:46:41.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd306cd0700 1 -- 172.21.15.175:0/725180705 shutdown_connections 2024-03-26T03:46:41.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd306cd0700 1 -- 172.21.15.175:0/725180705 wait complete. 2024-03-26T03:46:41.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd306cd0700 1 Processor -- start 2024-03-26T03:46:41.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd306cd0700 1 -- start start 2024-03-26T03:46:41.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd306cd0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd30006a3a0 0x7fd3001984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:41.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd306cd0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd30006ad60 0x7fd3001989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:41.709 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd306cd0700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd300108650 0x7fd30019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:41.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd306cd0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd300111eb0 con 0x7fd30006ad60 2024-03-26T03:46:41.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd306cd0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd300111d30 con 0x7fd30006a3a0 2024-03-26T03:46:41.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd306cd0700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd300112030 con 0x7fd300108650 2024-03-26T03:46:41.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd2fffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd30006ad60 0x7fd3001989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:41.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd2fffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd30006ad60 0x7fd3001989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:44354/0 (socket says 172.21.15.175:44354) 2024-03-26T03:46:41.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd2fffff700 1 -- 172.21.15.175:0/4165018336 learned_addr learned my addr 172.21.15.175:0/4165018336 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:46:41.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd30526d700 1 --2- 172.21.15.175:0/4165018336 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd300108650 0x7fd30019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:41.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd304a6c700 1 --2- 172.21.15.175:0/4165018336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd30006a3a0 0x7fd3001984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:41.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd2fffff700 1 -- 172.21.15.175:0/4165018336 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd300108650 msgr2=0x7fd30019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:41.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd2fffff700 1 --2- 172.21.15.175:0/4165018336 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd300108650 0x7fd30019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:41.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd2fffff700 1 -- 172.21.15.175:0/4165018336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd30006a3a0 msgr2=0x7fd3001984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:41.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd2fffff700 1 --2- 172.21.15.175:0/4165018336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd30006a3a0 0x7fd3001984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:41.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd2fffff700 1 -- 172.21.15.175:0/4165018336 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd30019d4c0 con 0x7fd30006ad60 2024-03-26T03:46:41.710 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd2fffff700 1 --2- 172.21.15.175:0/4165018336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd30006ad60 0x7fd3001989f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec0057c0 tx=0x7fd2ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:46:41.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd2fdffb700 1 -- 172.21.15.175:0/4165018336 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec026070 con 0x7fd30006ad60 2024-03-26T03:46:41.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd2fdffb700 1 -- 172.21.15.175:0/4165018336 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fd2ec021410 con 0x7fd30006ad60 2024-03-26T03:46:41.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd306cd0700 1 -- 172.21.15.175:0/4165018336 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd30019d750 con 0x7fd30006ad60 2024-03-26T03:46:41.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd2fdffb700 1 -- 172.21.15.175:0/4165018336 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec0173f0 con 0x7fd30006ad60 2024-03-26T03:46:41.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd306cd0700 1 -- 172.21.15.175:0/4165018336 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3001a5030 con 0x7fd30006ad60 2024-03-26T03:46:41.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd306cd0700 1 -- 172.21.15.175:0/4165018336 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd300107b90 con 0x7fd30006ad60 2024-03-26T03:46:41.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.705+0000 7fd2fdffb700 1 -- 172.21.15.175:0/4165018336 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7fd2ec029070 con 0x7fd30006ad60 2024-03-26T03:46:41.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.709+0000 7fd2fdffb700 1 --2- 172.21.15.175:0/4165018336 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fd2f006e7e0 0x7fd2f0070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:41.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.709+0000 7fd2fdffb700 1 -- 172.21.15.175:0/4165018336 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7537+0+0 (secure 0 0 0) 0x7fd2ec08fe70 con 0x7fd30006ad60 2024-03-26T03:46:41.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.709+0000 7fd304a6c700 1 --2- 172.21.15.175:0/4165018336 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fd2f006e7e0 0x7fd2f0070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:41.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.709+0000 7fd2fdffb700 1 --2- 172.21.15.175:0/4165018336 >> [v2:172.21.15.190:6800/874343325,v1:172.21.15.190:6801/874343325] conn(0x7fd2f00785c0 0x7fd2f007aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:41.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.709+0000 7fd2fdffb700 1 -- 172.21.15.175:0/4165018336 --> [v2:172.21.15.190:6800/874343325,v1:172.21.15.190:6801/874343325] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd2ec01fe80 con 0x7fd2f00785c0 2024-03-26T03:46:41.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.709+0000 7fd2fdffb700 1 -- 172.21.15.175:0/4165018336 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fd2ec090300 con 0x7fd30006ad60 2024-03-26T03:46:41.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.709+0000 7fd304a6c700 1 --2- 172.21.15.175:0/4165018336 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fd2f006e7e0 0x7fd2f0070ca0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4006fd0 tx=0x7fd2f4008040 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:46:41.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.709+0000 7fd30526d700 1 --2- 172.21.15.175:0/4165018336 >> [v2:172.21.15.190:6800/874343325,v1:172.21.15.190:6801/874343325] conn(0x7fd2f00785c0 0x7fd2f007aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:41.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.709+0000 7fd30526d700 1 --2- 172.21.15.175:0/4165018336 >> [v2:172.21.15.190:6800/874343325,v1:172.21.15.190:6801/874343325] conn(0x7fd2f00785c0 0x7fd2f007aa60 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:46:41.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.709+0000 7fd2fdffb700 1 -- 172.21.15.175:0/4165018336 <== osd.4 v2:172.21.15.190:6800/874343325 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd2ec01fe80 con 0x7fd2f00785c0 2024-03-26T03:46:41.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.729+0000 7fd306cd0700 1 -- 172.21.15.175:0/4165018336 --> [v2:172.21.15.190:6800/874343325,v1:172.21.15.190:6801/874343325] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd300103dd0 con 0x7fd2f00785c0 2024-03-26T03:46:41.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.729+0000 7fd2fdffb700 1 -- 172.21.15.175:0/4165018336 <== osd.4 v2:172.21.15.190:6800/874343325 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd300103dd0 con 0x7fd2f00785c0 2024-03-26T03:46:41.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.729+0000 7fd306cd0700 1 -- 172.21.15.175:0/4165018336 >> [v2:172.21.15.190:6800/874343325,v1:172.21.15.190:6801/874343325] conn(0x7fd2f00785c0 msgr2=0x7fd2f007aa60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:41.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.729+0000 7fd306cd0700 1 --2- 172.21.15.175:0/4165018336 >> [v2:172.21.15.190:6800/874343325,v1:172.21.15.190:6801/874343325] conn(0x7fd2f00785c0 0x7fd2f007aa60 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:41.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.729+0000 7fd306cd0700 1 -- 172.21.15.175:0/4165018336 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fd2f006e7e0 msgr2=0x7fd2f0070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:41.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.729+0000 7fd306cd0700 1 --2- 172.21.15.175:0/4165018336 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fd2f006e7e0 0x7fd2f0070ca0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4006fd0 tx=0x7fd2f4008040 comp rx=0 tx=0).stop 2024-03-26T03:46:41.730 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.729+0000 7fd306cd0700 1 -- 172.21.15.175:0/4165018336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd30006ad60 msgr2=0x7fd3001989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:41.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.729+0000 7fd306cd0700 1 --2- 172.21.15.175:0/4165018336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd30006ad60 0x7fd3001989f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec0057c0 tx=0x7fd2ec0057f0 comp rx=0 tx=0).stop 2024-03-26T03:46:41.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.729+0000 7fd30526d700 1 -- 172.21.15.175:0/4165018336 reap_dead start 2024-03-26T03:46:41.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.729+0000 7fd306cd0700 1 -- 172.21.15.175:0/4165018336 shutdown_connections 2024-03-26T03:46:41.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.729+0000 7fd306cd0700 1 -- 172.21.15.175:0/4165018336 >> 172.21.15.175:0/4165018336 conn(0x7fd3000775e0 msgr2=0x7fd3001061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:46:41.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.729+0000 7fd306cd0700 1 -- 172.21.15.175:0/4165018336 shutdown_connections 2024-03-26T03:46:41.731 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.729+0000 7fd306cd0700 1 -- 172.21.15.175:0/4165018336 wait complete. 2024-03-26T03:46:41.831 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:41 smithi175 bash[21527]: cluster 2024-03-26T03:46:40.508155+0000 mgr.y (mgr.24431) 85 : cluster [DBG] pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:41.832 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:41 smithi175 bash[29284]: cluster 2024-03-26T03:46:40.508155+0000 mgr.y (mgr.24431) 85 : cluster [DBG] pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:41.956 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa77d8b8700 1 -- 172.21.15.175:0/4258034774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa778108780 msgr2=0x7fa778108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:41.956 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa77d8b8700 1 --2- 172.21.15.175:0/4258034774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa778108780 0x7fa778108b60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa768009990 tx=0x7fa768009ca0 comp rx=0 tx=0).stop 2024-03-26T03:46:41.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa77d8b8700 1 -- 172.21.15.175:0/4258034774 shutdown_connections 2024-03-26T03:46:41.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa77d8b8700 1 --2- 172.21.15.175:0/4258034774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa778108780 0x7fa778108b60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:41.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa77d8b8700 1 --2- 172.21.15.175:0/4258034774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa77806b660 0x7fa77810f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:41.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa77d8b8700 1 --2- 172.21.15.175:0/4258034774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa77806aca0 0x7fa77806b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:41.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa77d8b8700 1 -- 172.21.15.175:0/4258034774 >> 172.21.15.175:0/4258034774 conn(0x7fa778078640 msgr2=0x7fa778078a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:46:41.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa77d8b8700 1 -- 172.21.15.175:0/4258034774 shutdown_connections 2024-03-26T03:46:41.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa77d8b8700 1 -- 172.21.15.175:0/4258034774 wait complete. 2024-03-26T03:46:41.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa77d8b8700 1 Processor -- start 2024-03-26T03:46:41.957 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa77d8b8700 1 -- start start 2024-03-26T03:46:41.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa77d8b8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa77806aca0 0x7fa7781983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:41.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa77d8b8700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa77806b660 0x7fa778198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:41.959 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa77d8b8700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa778108780 0x7fa77819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:41.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa77d8b8700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa778111ef0 con 0x7fa77806b660 2024-03-26T03:46:41.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa77d8b8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa778111d70 con 0x7fa77806aca0 2024-03-26T03:46:41.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa77d8b8700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa778112070 con 0x7fa778108780 2024-03-26T03:46:41.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa7767fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa77806b660 0x7fa778198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:41.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa7777fe700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa778108780 0x7fa77819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:41.960 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa7767fc700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa77806b660 0x7fa778198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:44372/0 (socket says 172.21.15.175:44372) 2024-03-26T03:46:41.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa7777fe700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa778108780 0x7fa77819cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:60922/0 (socket says 172.21.15.175:60922) 2024-03-26T03:46:41.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa7767fc700 1 -- 172.21.15.175:0/507322268 learned_addr learned my addr 172.21.15.175:0/507322268 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:46:41.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa776ffd700 1 --2- 172.21.15.175:0/507322268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa77806aca0 0x7fa7781983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:41.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa7767fc700 1 -- 172.21.15.175:0/507322268 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa778108780 msgr2=0x7fa77819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:41.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa7767fc700 1 --2- 172.21.15.175:0/507322268 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa778108780 0x7fa77819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:41.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa7767fc700 1 -- 172.21.15.175:0/507322268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa77806aca0 msgr2=0x7fa7781983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:41.961 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa7767fc700 1 --2- 172.21.15.175:0/507322268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa77806aca0 0x7fa7781983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:41.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa7767fc700 1 -- 172.21.15.175:0/507322268 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77819d3e0 con 0x7fa77806b660 2024-03-26T03:46:41.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa7767fc700 1 --2- 172.21.15.175:0/507322268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa77806b660 0x7fa778198910 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00eb30 tx=0x7fa76c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:46:41.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa77c8b6700 1 -- 172.21.15.175:0/507322268 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c00cc90 con 0x7fa77806b660 2024-03-26T03:46:41.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa77d8b8700 1 -- 172.21.15.175:0/507322268 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa77819d6d0 con 0x7fa77806b660 2024-03-26T03:46:41.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa77c8b6700 1 -- 172.21.15.175:0/507322268 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fa76c00ce30 con 0x7fa77806b660 2024-03-26T03:46:41.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa77c8b6700 1 -- 172.21.15.175:0/507322268 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c003d80 con 0x7fa77806b660 2024-03-26T03:46:41.962 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.957+0000 7fa77d8b8700 1 -- 172.21.15.175:0/507322268 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7781a4f50 con 0x7fa77806b660 2024-03-26T03:46:41.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.961+0000 7fa77d8b8700 1 -- 172.21.15.175:0/507322268 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa764000f80 con 0x7fa77806b660 2024-03-26T03:46:41.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.961+0000 7fa77c8b6700 1 -- 172.21.15.175:0/507322268 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7fa76c003f20 con 0x7fa77806b660 2024-03-26T03:46:41.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.961+0000 7fa77c8b6700 1 --2- 172.21.15.175:0/507322268 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fa76006e7e0 0x7fa760070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:41.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.961+0000 7fa77c8b6700 1 -- 172.21.15.175:0/507322268 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7537+0+0 (secure 0 0 0) 0x7fa76c08f0e0 con 0x7fa77806b660 2024-03-26T03:46:41.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.961+0000 7fa776ffd700 1 --2- 172.21.15.175:0/507322268 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fa76006e7e0 0x7fa760070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:41.965 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.961+0000 7fa77c8b6700 1 --2- 172.21.15.175:0/507322268 >> [v2:172.21.15.190:6816/4077005791,v1:172.21.15.190:6817/4077005791] conn(0x7fa7600786f0 0x7fa76007ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:41.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.961+0000 7fa77c8b6700 1 -- 172.21.15.175:0/507322268 --> [v2:172.21.15.190:6816/4077005791,v1:172.21.15.190:6817/4077005791] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa76c00be50 con 0x7fa7600786f0 2024-03-26T03:46:41.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.961+0000 7fa77c8b6700 1 -- 172.21.15.175:0/507322268 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fa76c096050 con 0x7fa77806b660 2024-03-26T03:46:41.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.961+0000 7fa776ffd700 1 --2- 172.21.15.175:0/507322268 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fa76006e7e0 0x7fa760070ca0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa76800b2e0 tx=0x7fa768000bc0 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:46:41.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.961+0000 7fa7777fe700 1 --2- 172.21.15.175:0/507322268 >> [v2:172.21.15.190:6816/4077005791,v1:172.21.15.190:6817/4077005791] conn(0x7fa7600786f0 0x7fa76007ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:41.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.961+0000 7fa7777fe700 1 --2- 172.21.15.175:0/507322268 >> [v2:172.21.15.190:6816/4077005791,v1:172.21.15.190:6817/4077005791] conn(0x7fa7600786f0 0x7fa76007ab90 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:46:41.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.965+0000 7fa77c8b6700 1 -- 172.21.15.175:0/507322268 <== osd.6 v2:172.21.15.190:6816/4077005791 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa76c00be50 con 0x7fa7600786f0 2024-03-26T03:46:41.981 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.981+0000 7fa77d8b8700 1 -- 172.21.15.175:0/507322268 --> [v2:172.21.15.190:6816/4077005791,v1:172.21.15.190:6817/4077005791] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa764002d00 con 0x7fa7600786f0 2024-03-26T03:46:41.982 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.981+0000 7fa77c8b6700 1 -- 172.21.15.175:0/507322268 <== osd.6 v2:172.21.15.190:6816/4077005791 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa764002d00 con 0x7fa7600786f0 2024-03-26T03:46:41.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.981+0000 7fa77d8b8700 1 -- 172.21.15.175:0/507322268 >> [v2:172.21.15.190:6816/4077005791,v1:172.21.15.190:6817/4077005791] conn(0x7fa7600786f0 msgr2=0x7fa76007ab90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:41.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.981+0000 7fa77d8b8700 1 --2- 172.21.15.175:0/507322268 >> [v2:172.21.15.190:6816/4077005791,v1:172.21.15.190:6817/4077005791] conn(0x7fa7600786f0 0x7fa76007ab90 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:41.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.985+0000 7fa77d8b8700 1 -- 172.21.15.175:0/507322268 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fa76006e7e0 msgr2=0x7fa760070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:41.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.985+0000 7fa77d8b8700 1 --2- 172.21.15.175:0/507322268 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fa76006e7e0 0x7fa760070ca0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa76800b2e0 tx=0x7fa768000bc0 comp rx=0 tx=0).stop 2024-03-26T03:46:41.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.985+0000 7fa77d8b8700 1 -- 172.21.15.175:0/507322268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa77806b660 msgr2=0x7fa778198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:41.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.985+0000 7fa77d8b8700 1 --2- 172.21.15.175:0/507322268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa77806b660 0x7fa778198910 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00eb30 tx=0x7fa76c00ee40 comp rx=0 tx=0).stop 2024-03-26T03:46:41.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.985+0000 7fa7777fe700 1 -- 172.21.15.175:0/507322268 reap_dead start 2024-03-26T03:46:41.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.985+0000 7fa77d8b8700 1 -- 172.21.15.175:0/507322268 shutdown_connections 2024-03-26T03:46:41.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.985+0000 7fa77d8b8700 1 -- 172.21.15.175:0/507322268 >> 172.21.15.175:0/507322268 conn(0x7fa778078640 msgr2=0x7fa7781061e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:46:41.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.985+0000 7fa77d8b8700 1 -- 172.21.15.175:0/507322268 shutdown_connections 2024-03-26T03:46:41.983 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:41.985+0000 7fa77d8b8700 1 -- 172.21.15.175:0/507322268 wait complete. 2024-03-26T03:46:41.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:41 smithi190 bash[23392]: cluster 2024-03-26T03:46:40.508155+0000 mgr.y (mgr.24431) 85 : cluster [DBG] pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:42.100 INFO:teuthology.orchestra.run.smithi175.stdout:167503724605 2024-03-26T03:46:42.101 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-26T03:46:42.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.465+0000 7f68aded5700 1 -- 172.21.15.175:0/2189597245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68a8106b40 msgr2=0x7f68a810e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:42.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.465+0000 7f68aded5700 1 --2- 172.21.15.175:0/2189597245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68a8106b40 0x7f68a810e450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6898009990 tx=0x7f6898009ca0 comp rx=0 tx=0).stop 2024-03-26T03:46:42.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.465+0000 7f68aded5700 1 -- 172.21.15.175:0/2189597245 shutdown_connections 2024-03-26T03:46:42.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.465+0000 7f68aded5700 1 --2- 172.21.15.175:0/2189597245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68a8106b40 0x7f68a810e450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:42.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.465+0000 7f68aded5700 1 --2- 172.21.15.175:0/2189597245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68a8101b20 0x7f68a8101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:42.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.465+0000 7f68aded5700 1 --2- 172.21.15.175:0/2189597245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68a8106180 0x7f68a8106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:42.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.465+0000 7f68aded5700 1 -- 172.21.15.175:0/2189597245 >> 172.21.15.175:0/2189597245 conn(0x7f68a80782a0 msgr2=0x7f68a80786b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:46:42.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.465+0000 7f68aded5700 1 -- 172.21.15.175:0/2189597245 shutdown_connections 2024-03-26T03:46:42.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.465+0000 7f68aded5700 1 -- 172.21.15.175:0/2189597245 wait complete. 2024-03-26T03:46:42.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.465+0000 7f68aded5700 1 Processor -- start 2024-03-26T03:46:42.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.465+0000 7f68aded5700 1 -- start start 2024-03-26T03:46:42.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.465+0000 7f68aded5700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68a8101b20 0x7f68a81990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:42.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.465+0000 7f68aded5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68a8106180 0x7f68a8199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:42.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.465+0000 7f68aded5700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68a8106b40 0x7f68a81a4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:42.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.469+0000 7f68aded5700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f68a8110d50 con 0x7f68a8106b40 2024-03-26T03:46:42.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.469+0000 7f68aded5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f68a8110bd0 con 0x7f68a8106180 2024-03-26T03:46:42.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.469+0000 7f68aded5700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f68a8110ed0 con 0x7f68a8101b20 2024-03-26T03:46:42.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.469+0000 7f68a7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68a8106b40 0x7f68a81a4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:42.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.469+0000 7f68a77fe700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68a8101b20 0x7f68a81990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:42.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.469+0000 7f68a7fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68a8106b40 0x7f68a81a4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:44386/0 (socket says 172.21.15.175:44386) 2024-03-26T03:46:42.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.469+0000 7f68a7fff700 1 -- 172.21.15.175:0/969593755 learned_addr learned my addr 172.21.15.175:0/969593755 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:46:42.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.469+0000 7f68a6ffd700 1 --2- 172.21.15.175:0/969593755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68a8106180 0x7f68a8199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:42.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.469+0000 7f68a7fff700 1 -- 172.21.15.175:0/969593755 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68a8101b20 msgr2=0x7f68a81990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:42.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.469+0000 7f68a7fff700 1 --2- 172.21.15.175:0/969593755 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68a8101b20 0x7f68a81990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:42.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.469+0000 7f68a7fff700 1 -- 172.21.15.175:0/969593755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68a8106180 msgr2=0x7f68a8199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:42.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.469+0000 7f68a7fff700 1 --2- 172.21.15.175:0/969593755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68a8106180 0x7f68a8199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:42.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.469+0000 7f68a7fff700 1 -- 172.21.15.175:0/969593755 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a81a4690 con 0x7f68a8106b40 2024-03-26T03:46:42.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.469+0000 7f68a7fff700 1 --2- 172.21.15.175:0/969593755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68a8106b40 0x7f68a81a4150 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f68980047b0 tx=0x7f6898009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:46:42.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.469+0000 7f68a4ff9700 1 -- 172.21.15.175:0/969593755 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6898027070 con 0x7f68a8106b40 2024-03-26T03:46:42.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.469+0000 7f68a4ff9700 1 -- 172.21.15.175:0/969593755 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f6898022470 con 0x7f68a8106b40 2024-03-26T03:46:42.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.469+0000 7f68aded5700 1 -- 172.21.15.175:0/969593755 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68a81a4860 con 0x7f68a8106b40 2024-03-26T03:46:42.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.469+0000 7f68a4ff9700 1 -- 172.21.15.175:0/969593755 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6898017650 con 0x7f68a8106b40 2024-03-26T03:46:42.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.469+0000 7f68aded5700 1 -- 172.21.15.175:0/969593755 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68a81a4cb0 con 0x7f68a8106b40 2024-03-26T03:46:42.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.469+0000 7f68aded5700 1 -- 172.21.15.175:0/969593755 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f68a8062d50 con 0x7f68a8106b40 2024-03-26T03:46:42.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.469+0000 7f68a4ff9700 1 -- 172.21.15.175:0/969593755 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7f689802a070 con 0x7f68a8106b40 2024-03-26T03:46:42.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.469+0000 7f68a4ff9700 1 --2- 172.21.15.175:0/969593755 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f689406e7e0 0x7f6894070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:42.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.469+0000 7f68a4ff9700 1 -- 172.21.15.175:0/969593755 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7537+0+0 (secure 0 0 0) 0x7f689808ffa0 con 0x7f68a8106b40 2024-03-26T03:46:42.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.469+0000 7f68a77fe700 1 --2- 172.21.15.175:0/969593755 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f689406e7e0 0x7f6894070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:42.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.469+0000 7f68a4ff9700 1 --2- 172.21.15.175:0/969593755 >> [v2:172.21.15.175:6802/241042466,v1:172.21.15.175:6803/241042466] conn(0x7f68940786f0 0x7f689407ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:42.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.469+0000 7f68a4ff9700 1 -- 172.21.15.175:0/969593755 --> [v2:172.21.15.175:6802/241042466,v1:172.21.15.175:6803/241042466] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6898018e50 con 0x7f68940786f0 2024-03-26T03:46:42.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.469+0000 7f68a4ff9700 1 -- 172.21.15.175:0/969593755 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f6898097050 con 0x7f68a8106b40 2024-03-26T03:46:42.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.469+0000 7f68a6ffd700 1 --2- 172.21.15.175:0/969593755 >> [v2:172.21.15.175:6802/241042466,v1:172.21.15.175:6803/241042466] conn(0x7f68940786f0 0x7f689407ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:42.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.469+0000 7f68a77fe700 1 --2- 172.21.15.175:0/969593755 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f689406e7e0 0x7f6894070ca0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f68a0009da0 tx=0x7f68a000b040 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:46:42.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.469+0000 7f68a6ffd700 1 --2- 172.21.15.175:0/969593755 >> [v2:172.21.15.175:6802/241042466,v1:172.21.15.175:6803/241042466] conn(0x7f68940786f0 0x7f689407ab90 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:46:42.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.473+0000 7f68a4ff9700 1 -- 172.21.15.175:0/969593755 <== osd.0 v2:172.21.15.175:6802/241042466 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6898018e50 con 0x7f68940786f0 2024-03-26T03:46:42.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.489+0000 7f68aded5700 1 -- 172.21.15.175:0/969593755 --> [v2:172.21.15.175:6802/241042466,v1:172.21.15.175:6803/241042466] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f68a8063150 con 0x7f68940786f0 2024-03-26T03:46:42.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.489+0000 7f68a4ff9700 1 -- 172.21.15.175:0/969593755 <== osd.0 v2:172.21.15.175:6802/241042466 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f68a8063150 con 0x7f68940786f0 2024-03-26T03:46:42.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.489+0000 7f68aded5700 1 -- 172.21.15.175:0/969593755 >> [v2:172.21.15.175:6802/241042466,v1:172.21.15.175:6803/241042466] conn(0x7f68940786f0 msgr2=0x7f689407ab90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:42.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.489+0000 7f68aded5700 1 --2- 172.21.15.175:0/969593755 >> [v2:172.21.15.175:6802/241042466,v1:172.21.15.175:6803/241042466] conn(0x7f68940786f0 0x7f689407ab90 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:42.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.489+0000 7f68aded5700 1 -- 172.21.15.175:0/969593755 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f689406e7e0 msgr2=0x7f6894070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:42.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.489+0000 7f68aded5700 1 --2- 172.21.15.175:0/969593755 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f689406e7e0 0x7f6894070ca0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f68a0009da0 tx=0x7f68a000b040 comp rx=0 tx=0).stop 2024-03-26T03:46:42.488 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.489+0000 7f68aded5700 1 -- 172.21.15.175:0/969593755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68a8106b40 msgr2=0x7f68a81a4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:42.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.489+0000 7f68aded5700 1 --2- 172.21.15.175:0/969593755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68a8106b40 0x7f68a81a4150 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f68980047b0 tx=0x7f6898009580 comp rx=0 tx=0).stop 2024-03-26T03:46:42.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.489+0000 7f68a7fff700 1 -- 172.21.15.175:0/969593755 reap_dead start 2024-03-26T03:46:42.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.489+0000 7f68aded5700 1 -- 172.21.15.175:0/969593755 shutdown_connections 2024-03-26T03:46:42.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.489+0000 7f68aded5700 1 -- 172.21.15.175:0/969593755 >> 172.21.15.175:0/969593755 conn(0x7f68a80782a0 msgr2=0x7f68a80ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:46:42.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.489+0000 7f68aded5700 1 -- 172.21.15.175:0/969593755 shutdown_connections 2024-03-26T03:46:42.489 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:42.489+0000 7f68aded5700 1 -- 172.21.15.175:0/969593755 wait complete. 2024-03-26T03:46:43.736 INFO:teuthology.orchestra.run.smithi175.stdout:107374182481 2024-03-26T03:46:43.736 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-26T03:46:43.895 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:43 smithi175 bash[29284]: cluster 2024-03-26T03:46:42.508753+0000 mgr.y (mgr.24431) 86 : cluster [DBG] pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 0 op/s 2024-03-26T03:46:43.896 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:43 smithi175 bash[21527]: cluster 2024-03-26T03:46:42.508753+0000 mgr.y (mgr.24431) 86 : cluster [DBG] pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 0 op/s 2024-03-26T03:46:43.896 INFO:teuthology.orchestra.run.smithi175.stdout:55834574952 2024-03-26T03:46:43.897 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-26T03:46:43.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:43 smithi190 bash[23392]: cluster 2024-03-26T03:46:42.508753+0000 mgr.y (mgr.24431) 86 : cluster [DBG] pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 0 op/s 2024-03-26T03:46:44.458 INFO:teuthology.orchestra.run.smithi175.stdout:128849018950 2024-03-26T03:46:44.459 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-26T03:46:45.189 INFO:teuthology.orchestra.run.smithi175.stdout:201863462961 2024-03-26T03:46:45.189 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-26T03:46:45.206 INFO:teuthology.orchestra.run.smithi175.stdout:34359738482 2024-03-26T03:46:45.206 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-26T03:46:45.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:45 smithi190 bash[23392]: cluster 2024-03-26T03:46:44.509725+0000 mgr.y (mgr.24431) 87 : cluster [DBG] pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:45.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:45 smithi175 bash[29284]: cluster 2024-03-26T03:46:44.509725+0000 mgr.y (mgr.24431) 87 : cluster [DBG] pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:45.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:45 smithi175 bash[21527]: cluster 2024-03-26T03:46:44.509725+0000 mgr.y (mgr.24431) 87 : cluster [DBG] pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:47.257 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:46:47.258 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:46:47.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:47 smithi190 bash[23392]: cluster 2024-03-26T03:46:46.510407+0000 mgr.y (mgr.24431) 88 : cluster [DBG] pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:47.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:47 smithi175 bash[29284]: cluster 2024-03-26T03:46:46.510407+0000 mgr.y (mgr.24431) 88 : cluster [DBG] pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:47.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:47 smithi175 bash[21527]: cluster 2024-03-26T03:46:46.510407+0000 mgr.y (mgr.24431) 88 : cluster [DBG] pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:48.271 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:46:48.272 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:46:48.272 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:46:48.272 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:46:49.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:48 smithi190 bash[23392]: audit 2024-03-26T03:46:47.847903+0000 mon.a (mon.0) 856 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:49.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:48 smithi190 bash[23392]: audit 2024-03-26T03:46:47.860238+0000 mon.a (mon.0) 857 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:49.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:48 smithi190 bash[23392]: audit 2024-03-26T03:46:48.430462+0000 mon.c (mon.2) 70 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:46:49.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:48 smithi190 bash[23392]: audit 2024-03-26T03:46:48.432041+0000 mon.c (mon.2) 71 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:46:49.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:48 smithi190 bash[23392]: audit 2024-03-26T03:46:48.442006+0000 mon.a (mon.0) 858 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:49.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:48 smithi190 bash[23392]: audit 2024-03-26T03:46:48.734055+0000 mon.c (mon.2) 72 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:46:49.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:46:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:46:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:46:49.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:48 smithi175 bash[21527]: audit 2024-03-26T03:46:47.847903+0000 mon.a (mon.0) 856 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:49.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:48 smithi175 bash[21527]: audit 2024-03-26T03:46:47.860238+0000 mon.a (mon.0) 857 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:49.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:48 smithi175 bash[21527]: audit 2024-03-26T03:46:48.430462+0000 mon.c (mon.2) 70 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:46:49.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:48 smithi175 bash[21527]: audit 2024-03-26T03:46:48.432041+0000 mon.c (mon.2) 71 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:46:49.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:48 smithi175 bash[21527]: audit 2024-03-26T03:46:48.442006+0000 mon.a (mon.0) 858 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:49.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:48 smithi175 bash[21527]: audit 2024-03-26T03:46:48.734055+0000 mon.c (mon.2) 72 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:46:49.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:48 smithi175 bash[29284]: audit 2024-03-26T03:46:47.847903+0000 mon.a (mon.0) 856 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:49.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:48 smithi175 bash[29284]: audit 2024-03-26T03:46:47.860238+0000 mon.a (mon.0) 857 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:49.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:48 smithi175 bash[29284]: audit 2024-03-26T03:46:48.430462+0000 mon.c (mon.2) 70 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:46:49.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:48 smithi175 bash[29284]: audit 2024-03-26T03:46:48.432041+0000 mon.c (mon.2) 71 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:46:49.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:48 smithi175 bash[29284]: audit 2024-03-26T03:46:48.442006+0000 mon.a (mon.0) 858 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:46:49.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:48 smithi175 bash[29284]: audit 2024-03-26T03:46:48.734055+0000 mon.c (mon.2) 72 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:46:49.287 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:46:49.288 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:46:49.844 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:46:49 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:46:49.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:49 smithi175 bash[29284]: cluster 2024-03-26T03:46:48.511040+0000 mgr.y (mgr.24431) 89 : cluster [DBG] pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:49.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:49 smithi175 bash[21527]: cluster 2024-03-26T03:46:48.511040+0000 mgr.y (mgr.24431) 89 : cluster [DBG] pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:50.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:49 smithi190 bash[23392]: cluster 2024-03-26T03:46:48.511040+0000 mgr.y (mgr.24431) 89 : cluster [DBG] pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:51.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:50 smithi190 bash[23392]: audit 2024-03-26T03:46:49.559695+0000 mgr.y (mgr.24431) 90 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:46:51.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:50 smithi175 bash[21527]: audit 2024-03-26T03:46:49.559695+0000 mgr.y (mgr.24431) 90 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:46:51.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:50 smithi175 bash[29284]: audit 2024-03-26T03:46:49.559695+0000 mgr.y (mgr.24431) 90 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:46:52.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:51 smithi190 bash[23392]: cluster 2024-03-26T03:46:50.512518+0000 mgr.y (mgr.24431) 91 : cluster [DBG] pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:52.265 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:51 smithi175 bash[29284]: cluster 2024-03-26T03:46:50.512518+0000 mgr.y (mgr.24431) 91 : cluster [DBG] pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:52.266 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:51 smithi175 bash[21527]: cluster 2024-03-26T03:46:50.512518+0000 mgr.y (mgr.24431) 91 : cluster [DBG] pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:54.176 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:53 smithi175 bash[21527]: cluster 2024-03-26T03:46:52.513233+0000 mgr.y (mgr.24431) 92 : cluster [DBG] pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:54.177 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:53 smithi175 bash[29284]: cluster 2024-03-26T03:46:52.513233+0000 mgr.y (mgr.24431) 92 : cluster [DBG] pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:54.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:53 smithi190 bash[23392]: cluster 2024-03-26T03:46:52.513233+0000 mgr.y (mgr.24431) 92 : cluster [DBG] pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:56.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:55 smithi190 bash[23392]: cluster 2024-03-26T03:46:54.514657+0000 mgr.y (mgr.24431) 93 : cluster [DBG] pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:56.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:55 smithi175 bash[29284]: cluster 2024-03-26T03:46:54.514657+0000 mgr.y (mgr.24431) 93 : cluster [DBG] pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:56.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:55 smithi175 bash[21527]: cluster 2024-03-26T03:46:54.514657+0000 mgr.y (mgr.24431) 93 : cluster [DBG] pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:46:58.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:57 smithi190 bash[23392]: cluster 2024-03-26T03:46:56.515298+0000 mgr.y (mgr.24431) 94 : cluster [DBG] pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:58.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:57 smithi175 bash[21527]: cluster 2024-03-26T03:46:56.515298+0000 mgr.y (mgr.24431) 94 : cluster [DBG] pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:58.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:57 smithi175 bash[29284]: cluster 2024-03-26T03:46:56.515298+0000 mgr.y (mgr.24431) 94 : cluster [DBG] pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:46:58.858 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:46:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:46:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:46:59.019 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.017+0000 7f4808344700 1 -- 172.21.15.175:0/1282880391 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4800103090 msgr2=0x7f480010f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:59.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.017+0000 7f4808344700 1 --2- 172.21.15.175:0/1282880391 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4800103090 0x7f480010f570 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f47f800b0a0 tx=0x7f47f800b3b0 comp rx=0 tx=0).stop 2024-03-26T03:46:59.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f4808344700 1 -- 172.21.15.175:0/1282880391 shutdown_connections 2024-03-26T03:46:59.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f4808344700 1 --2- 172.21.15.175:0/1282880391 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4800103090 0x7f480010f570 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:59.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f4808344700 1 --2- 172.21.15.175:0/1282880391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48001086a0 0x7f4800108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:59.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f4808344700 1 --2- 172.21.15.175:0/1282880391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48001026f0 0x7f4800102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:59.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f4808344700 1 -- 172.21.15.175:0/1282880391 >> 172.21.15.175:0/1282880391 conn(0x7f48000797f0 msgr2=0x7f4800079c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:46:59.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f4808344700 1 -- 172.21.15.175:0/1282880391 shutdown_connections 2024-03-26T03:46:59.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f4808344700 1 -- 172.21.15.175:0/1282880391 wait complete. 2024-03-26T03:46:59.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f4808344700 1 Processor -- start 2024-03-26T03:46:59.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f4808344700 1 -- start start 2024-03-26T03:46:59.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f4808344700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48001026f0 0x7f4800198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:59.020 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f4808344700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4800103090 0x7f48001988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:59.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f4808344700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48001086a0 0x7f480019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:59.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f4808344700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4800111f00 con 0x7f4800103090 2024-03-26T03:46:59.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f4808344700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4800111d80 con 0x7f48001026f0 2024-03-26T03:46:59.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f4808344700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4800112080 con 0x7f48001086a0 2024-03-26T03:46:59.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f48058df700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4800103090 0x7f48001988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:59.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f48068e1700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48001086a0 0x7f480019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:59.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f48058df700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4800103090 0x7f48001988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:59884/0 (socket says 172.21.15.175:59884) 2024-03-26T03:46:59.021 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f48058df700 1 -- 172.21.15.175:0/103956130 learned_addr learned my addr 172.21.15.175:0/103956130 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:46:59.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f48060e0700 1 --2- 172.21.15.175:0/103956130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48001026f0 0x7f4800198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:59.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f48068e1700 1 -- 172.21.15.175:0/103956130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48001026f0 msgr2=0x7f4800198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:59.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f48068e1700 1 --2- 172.21.15.175:0/103956130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48001026f0 0x7f4800198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:59.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f48068e1700 1 -- 172.21.15.175:0/103956130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4800103090 msgr2=0x7f48001988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:59.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f48068e1700 1 --2- 172.21.15.175:0/103956130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4800103090 0x7f48001988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:59.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f48068e1700 1 -- 172.21.15.175:0/103956130 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480019d330 con 0x7f48001086a0 2024-03-26T03:46:59.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f48060e0700 1 --2- 172.21.15.175:0/103956130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48001026f0 0x7f4800198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T03:46:59.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f48058df700 1 --2- 172.21.15.175:0/103956130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4800103090 0x7f48001988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T03:46:59.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f48068e1700 1 --2- 172.21.15.175:0/103956130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48001086a0 0x7f480019cbf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f47f800bfd0 tx=0x7f47f8007a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:46:59.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f47f77fe700 1 -- 172.21.15.175:0/103956130 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f8021020 con 0x7f48001086a0 2024-03-26T03:46:59.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f47f77fe700 1 -- 172.21.15.175:0/103956130 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f47f8004c50 con 0x7f48001086a0 2024-03-26T03:46:59.023 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f4808344700 1 -- 172.21.15.175:0/103956130 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f480019d5c0 con 0x7f48001086a0 2024-03-26T03:46:59.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f47f77fe700 1 -- 172.21.15.175:0/103956130 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f801de30 con 0x7f48001086a0 2024-03-26T03:46:59.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f4808344700 1 -- 172.21.15.175:0/103956130 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f48001a4ea0 con 0x7f48001086a0 2024-03-26T03:46:59.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f4808344700 1 -- 172.21.15.175:0/103956130 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4800050300 con 0x7f48001086a0 2024-03-26T03:46:59.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.021+0000 7f47f77fe700 1 -- 172.21.15.175:0/103956130 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7f47f8024020 con 0x7f48001086a0 2024-03-26T03:46:59.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.025+0000 7f47f77fe700 1 --2- 172.21.15.175:0/103956130 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f47ec06e7e0 0x7f47ec070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:59.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.025+0000 7f47f77fe700 1 -- 172.21.15.175:0/103956130 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(75..75 src has 1..75) ==== 7537+0+0 (secure 0 0 0) 0x7f47f800f020 con 0x7f48001086a0 2024-03-26T03:46:59.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.025+0000 7f48060e0700 1 --2- 172.21.15.175:0/103956130 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f47ec06e7e0 0x7f47ec070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:59.024 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.025+0000 7f48060e0700 1 --2- 172.21.15.175:0/103956130 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f47ec06e7e0 0x7f47ec070ca0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f47fc009b50 tx=0x7f47fc009550 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:46:59.028 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.029+0000 7f47f77fe700 1 -- 172.21.15.175:0/103956130 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f47f805d040 con 0x7f48001086a0 2024-03-26T03:46:59.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.209+0000 7f485c26b700 1 -- 172.21.15.175:0/1563215289 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f485406a3a0 msgr2=0x7f485406a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:59.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.209+0000 7f485c26b700 1 --2- 172.21.15.175:0/1563215289 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f485406a3a0 0x7f485406a820 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4844009990 tx=0x7f4844009ca0 comp rx=0 tx=0).stop 2024-03-26T03:46:59.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.209+0000 7f485c26b700 1 -- 172.21.15.175:0/1563215289 shutdown_connections 2024-03-26T03:46:59.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.209+0000 7f485c26b700 1 --2- 172.21.15.175:0/1563215289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f485406ad60 0x7f485410f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:59.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.209+0000 7f485c26b700 1 --2- 172.21.15.175:0/1563215289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4854108650 0x7f4854108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:59.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.209+0000 7f485c26b700 1 --2- 172.21.15.175:0/1563215289 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f485406a3a0 0x7f485406a820 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:59.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.209+0000 7f485c26b700 1 -- 172.21.15.175:0/1563215289 >> 172.21.15.175:0/1563215289 conn(0x7f48540775e0 msgr2=0x7f48540779f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:46:59.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.209+0000 7f485c26b700 1 -- 172.21.15.175:0/1563215289 shutdown_connections 2024-03-26T03:46:59.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.209+0000 7f485c26b700 1 -- 172.21.15.175:0/1563215289 wait complete. 2024-03-26T03:46:59.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.209+0000 7f485c26b700 1 Processor -- start 2024-03-26T03:46:59.210 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.209+0000 7f485c26b700 1 -- start start 2024-03-26T03:46:59.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.209+0000 7f485c26b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f485406a3a0 0x7f4854101670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:59.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.209+0000 7f485c26b700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f485406ad60 0x7f4854101bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:59.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.209+0000 7f485c26b700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4854108650 0x7f48541020f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:59.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.209+0000 7f485c26b700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4854111f00 con 0x7f4854108650 2024-03-26T03:46:59.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.209+0000 7f485c26b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4854111d80 con 0x7f485406a3a0 2024-03-26T03:46:59.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.209+0000 7f485c26b700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4854112080 con 0x7f485406ad60 2024-03-26T03:46:59.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.209+0000 7f4859806700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f485406ad60 0x7f4854101bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:59.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.209+0000 7f4859806700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f485406ad60 0x7f4854101bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:34790/0 (socket says 172.21.15.175:34790) 2024-03-26T03:46:59.212 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.209+0000 7f4859806700 1 -- 172.21.15.175:0/3422737476 learned_addr learned my addr 172.21.15.175:0/3422737476 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:46:59.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.209+0000 7f485a808700 1 --2- 172.21.15.175:0/3422737476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4854108650 0x7f48541020f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:59.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.213+0000 7f485a007700 1 --2- 172.21.15.175:0/3422737476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f485406a3a0 0x7f4854101670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:59.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.213+0000 7f4859806700 1 -- 172.21.15.175:0/3422737476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f485406a3a0 msgr2=0x7f4854101670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:59.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.213+0000 7f4859806700 1 --2- 172.21.15.175:0/3422737476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f485406a3a0 0x7f4854101670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:59.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.213+0000 7f4859806700 1 -- 172.21.15.175:0/3422737476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4854108650 msgr2=0x7f48541020f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:59.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.213+0000 7f4859806700 1 --2- 172.21.15.175:0/3422737476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4854108650 0x7f48541020f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:59.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.213+0000 7f4859806700 1 -- 172.21.15.175:0/3422737476 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48541a6980 con 0x7f485406ad60 2024-03-26T03:46:59.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.213+0000 7f4859806700 1 --2- 172.21.15.175:0/3422737476 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f485406ad60 0x7f4854101bb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4844005910 tx=0x7f48440047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:46:59.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.213+0000 7f484b7fe700 1 -- 172.21.15.175:0/3422737476 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4844027070 con 0x7f485406ad60 2024-03-26T03:46:59.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.213+0000 7f485c26b700 1 -- 172.21.15.175:0/3422737476 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48541a6c10 con 0x7f485406ad60 2024-03-26T03:46:59.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.213+0000 7f484b7fe700 1 -- 172.21.15.175:0/3422737476 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f4844022470 con 0x7f485406ad60 2024-03-26T03:46:59.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.213+0000 7f484b7fe700 1 -- 172.21.15.175:0/3422737476 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4844017650 con 0x7f485406ad60 2024-03-26T03:46:59.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.213+0000 7f485c26b700 1 -- 172.21.15.175:0/3422737476 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f48541a7060 con 0x7f485406ad60 2024-03-26T03:46:59.213 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.213+0000 7f485c26b700 1 -- 172.21.15.175:0/3422737476 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4854050300 con 0x7f485406ad60 2024-03-26T03:46:59.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.213+0000 7f484b7fe700 1 -- 172.21.15.175:0/3422737476 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7f484402a070 con 0x7f485406ad60 2024-03-26T03:46:59.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.213+0000 7f484b7fe700 1 --2- 172.21.15.175:0/3422737476 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f484006e7e0 0x7f4840070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:46:59.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.213+0000 7f484b7fe700 1 -- 172.21.15.175:0/3422737476 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(75..75 src has 1..75) ==== 7537+0+0 (secure 0 0 0) 0x7f484408ffc0 con 0x7f485406ad60 2024-03-26T03:46:59.214 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.213+0000 7f485a007700 1 --2- 172.21.15.175:0/3422737476 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f484006e7e0 0x7f4840070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:46:59.215 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.213+0000 7f485a007700 1 --2- 172.21.15.175:0/3422737476 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f484006e7e0 0x7f4840070ca0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4850006fd0 tx=0x7f4850008040 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:46:59.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.217+0000 7f484b7fe700 1 -- 172.21.15.175:0/3422737476 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f4844013070 con 0x7f485406ad60 2024-03-26T03:46:59.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.245+0000 7f4808344700 1 -- 172.21.15.175:0/103956130 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f480002dd30 con 0x7f48001086a0 2024-03-26T03:46:59.247 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.249+0000 7f47f77fe700 1 -- 172.21.15.175:0/103956130 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f47f8060cf0 con 0x7f48001086a0 2024-03-26T03:46:59.247 INFO:teuthology.orchestra.run.smithi175.stdout:236223201322 2024-03-26T03:46:59.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.249+0000 7f4808344700 1 -- 172.21.15.175:0/103956130 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f47ec06e7e0 msgr2=0x7f47ec070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:59.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.249+0000 7f4808344700 1 --2- 172.21.15.175:0/103956130 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f47ec06e7e0 0x7f47ec070ca0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f47fc009b50 tx=0x7f47fc009550 comp rx=0 tx=0).stop 2024-03-26T03:46:59.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.249+0000 7f4808344700 1 -- 172.21.15.175:0/103956130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48001086a0 msgr2=0x7f480019cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:59.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.249+0000 7f4808344700 1 --2- 172.21.15.175:0/103956130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48001086a0 0x7f480019cbf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f47f800bfd0 tx=0x7f47f8007a20 comp rx=0 tx=0).stop 2024-03-26T03:46:59.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.249+0000 7f4808344700 1 -- 172.21.15.175:0/103956130 shutdown_connections 2024-03-26T03:46:59.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.249+0000 7f4808344700 1 --2- 172.21.15.175:0/103956130 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f47ec06e7e0 0x7f47ec070ca0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:59.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.249+0000 7f4808344700 1 --2- 172.21.15.175:0/103956130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48001086a0 0x7f480019cbf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:59.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.249+0000 7f4808344700 1 --2- 172.21.15.175:0/103956130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48001026f0 0x7f4800198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:59.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.249+0000 7f4808344700 1 --2- 172.21.15.175:0/103956130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4800103090 0x7f48001988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:59.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.249+0000 7f4808344700 1 -- 172.21.15.175:0/103956130 >> 172.21.15.175:0/103956130 conn(0x7f48000797f0 msgr2=0x7f48001007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:46:59.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.249+0000 7f4808344700 1 -- 172.21.15.175:0/103956130 shutdown_connections 2024-03-26T03:46:59.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.249+0000 7f4808344700 1 -- 172.21.15.175:0/103956130 wait complete. 2024-03-26T03:46:59.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.433+0000 7f485c26b700 1 -- 172.21.15.175:0/3422737476 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f48541a72f0 con 0x7f485406ad60 2024-03-26T03:46:59.432 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.433+0000 7f484b7fe700 1 -- 172.21.15.175:0/3422737476 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f484405cfb0 con 0x7f485406ad60 2024-03-26T03:46:59.433 INFO:teuthology.orchestra.run.smithi175.stdout:77309411424 2024-03-26T03:46:59.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.433+0000 7f485c26b700 1 -- 172.21.15.175:0/3422737476 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f484006e7e0 msgr2=0x7f4840070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:59.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.433+0000 7f485c26b700 1 --2- 172.21.15.175:0/3422737476 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f484006e7e0 0x7f4840070ca0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4850006fd0 tx=0x7f4850008040 comp rx=0 tx=0).stop 2024-03-26T03:46:59.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.433+0000 7f485c26b700 1 -- 172.21.15.175:0/3422737476 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f485406ad60 msgr2=0x7f4854101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:46:59.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.433+0000 7f485c26b700 1 --2- 172.21.15.175:0/3422737476 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f485406ad60 0x7f4854101bb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4844005910 tx=0x7f48440047b0 comp rx=0 tx=0).stop 2024-03-26T03:46:59.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.433+0000 7f485c26b700 1 -- 172.21.15.175:0/3422737476 shutdown_connections 2024-03-26T03:46:59.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.433+0000 7f485c26b700 1 --2- 172.21.15.175:0/3422737476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4854108650 0x7f48541020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:59.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.437+0000 7f485c26b700 1 --2- 172.21.15.175:0/3422737476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f485406a3a0 0x7f4854101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:59.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.437+0000 7f485c26b700 1 --2- 172.21.15.175:0/3422737476 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f484006e7e0 0x7f4840070ca0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:59.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.437+0000 7f485c26b700 1 --2- 172.21.15.175:0/3422737476 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f485406ad60 0x7f4854101bb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:46:59.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.437+0000 7f485c26b700 1 -- 172.21.15.175:0/3422737476 >> 172.21.15.175:0/3422737476 conn(0x7f48540775e0 msgr2=0x7f4854109440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:46:59.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.437+0000 7f485c26b700 1 -- 172.21.15.175:0/3422737476 shutdown_connections 2024-03-26T03:46:59.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:46:59.437+0000 7f485c26b700 1 -- 172.21.15.175:0/3422737476 wait complete. 2024-03-26T03:46:59.873 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:46:59 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:47:00.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:59 smithi190 bash[23392]: cluster 2024-03-26T03:46:58.515765+0000 mgr.y (mgr.24431) 95 : cluster [DBG] pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:00.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:59 smithi190 bash[23392]: audit 2024-03-26T03:46:59.251569+0000 mon.c (mon.2) 73 : audit [DBG] from='client.? 172.21.15.175:0/103956130' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-26T03:47:00.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:46:59 smithi190 bash[23392]: audit 2024-03-26T03:46:59.437090+0000 mon.c (mon.2) 74 : audit [DBG] from='client.? 172.21.15.175:0/3422737476' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-26T03:47:00.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:59 smithi175 bash[29284]: cluster 2024-03-26T03:46:58.515765+0000 mgr.y (mgr.24431) 95 : cluster [DBG] pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:00.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:59 smithi175 bash[29284]: audit 2024-03-26T03:46:59.251569+0000 mon.c (mon.2) 73 : audit [DBG] from='client.? 172.21.15.175:0/103956130' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-26T03:47:00.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:46:59 smithi175 bash[29284]: audit 2024-03-26T03:46:59.437090+0000 mon.c (mon.2) 74 : audit [DBG] from='client.? 172.21.15.175:0/3422737476' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-26T03:47:00.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:59 smithi175 bash[21527]: cluster 2024-03-26T03:46:58.515765+0000 mgr.y (mgr.24431) 95 : cluster [DBG] pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:00.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:59 smithi175 bash[21527]: audit 2024-03-26T03:46:59.251569+0000 mon.c (mon.2) 73 : audit [DBG] from='client.? 172.21.15.175:0/103956130' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-26T03:47:00.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:46:59 smithi175 bash[21527]: audit 2024-03-26T03:46:59.437090+0000 mon.c (mon.2) 74 : audit [DBG] from='client.? 172.21.15.175:0/3422737476' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-26T03:47:01.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:00 smithi190 bash[23392]: audit 2024-03-26T03:46:59.570324+0000 mgr.y (mgr.24431) 96 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:47:01.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:00 smithi175 bash[29284]: audit 2024-03-26T03:46:59.570324+0000 mgr.y (mgr.24431) 96 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:47:01.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:00 smithi175 bash[21527]: audit 2024-03-26T03:46:59.570324+0000 mgr.y (mgr.24431) 96 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:47:02.006 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:01 smithi175 bash[29284]: cluster 2024-03-26T03:47:00.517152+0000 mgr.y (mgr.24431) 97 : cluster [DBG] pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:02.006 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:01 smithi175 bash[21527]: cluster 2024-03-26T03:47:00.517152+0000 mgr.y (mgr.24431) 97 : cluster [DBG] pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:02.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:01 smithi190 bash[23392]: cluster 2024-03-26T03:47:00.517152+0000 mgr.y (mgr.24431) 97 : cluster [DBG] pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:03.274 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411419 got 77309411424 for osd.2 2024-03-26T03:47:03.275 DEBUG:teuthology.parallel:result is None 2024-03-26T03:47:03.360 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201318 got 236223201322 for osd.7 2024-03-26T03:47:03.360 DEBUG:teuthology.parallel:result is None 2024-03-26T03:47:03.634 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.632+0000 7fd3aef1a700 1 -- 172.21.15.175:0/417249322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3a81076c0 msgr2=0x7fd3a8107aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:03.634 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.632+0000 7fd3aef1a700 1 --2- 172.21.15.175:0/417249322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3a81076c0 0x7fd3a8107aa0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd3a400b0d0 tx=0x7fd3a400b3e0 comp rx=0 tx=0).stop 2024-03-26T03:47:03.634 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.632+0000 7fd3aef1a700 1 -- 172.21.15.175:0/417249322 shutdown_connections 2024-03-26T03:47:03.634 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.632+0000 7fd3aef1a700 1 --2- 172.21.15.175:0/417249322 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3a81026a0 0x7fd3a810e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:03.634 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.632+0000 7fd3aef1a700 1 --2- 172.21.15.175:0/417249322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3a81076c0 0x7fd3a8107aa0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:03.634 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.632+0000 7fd3aef1a700 1 --2- 172.21.15.175:0/417249322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3a8101d00 0x7fd3a8102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:03.634 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.632+0000 7fd3aef1a700 1 -- 172.21.15.175:0/417249322 >> 172.21.15.175:0/417249322 conn(0x7fd3a8077b40 msgr2=0x7fd3a80feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:47:03.634 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.632+0000 7fd3aef1a700 1 -- 172.21.15.175:0/417249322 shutdown_connections 2024-03-26T03:47:03.635 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.632+0000 7fd3aef1a700 1 -- 172.21.15.175:0/417249322 wait complete. 2024-03-26T03:47:03.636 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.632+0000 7fd3aef1a700 1 Processor -- start 2024-03-26T03:47:03.636 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.632+0000 7fd3aef1a700 1 -- start start 2024-03-26T03:47:03.636 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.636+0000 7fd3aef1a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3a8101d00 0x7fd3a8198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:03.636 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.636+0000 7fd3aef1a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3a81026a0 0x7fd3a81988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:03.637 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.636+0000 7fd3aef1a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3a81076c0 0x7fd3a819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:03.637 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.636+0000 7fd3aef1a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a8110e20 con 0x7fd3a81026a0 2024-03-26T03:47:03.637 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.636+0000 7fd3aef1a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a8110ca0 con 0x7fd3a81076c0 2024-03-26T03:47:03.637 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.636+0000 7fd3aef1a700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd3a8110fa0 con 0x7fd3a8101d00 2024-03-26T03:47:03.637 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.636+0000 7fd39ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3a81026a0 0x7fd3a81988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:03.637 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.636+0000 7fd3accb6700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3a8101d00 0x7fd3a8198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:03.637 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.636+0000 7fd39ffff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3a81026a0 0x7fd3a81988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:34356/0 (socket says 172.21.15.175:34356) 2024-03-26T03:47:03.637 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.636+0000 7fd39ffff700 1 -- 172.21.15.175:0/1781157179 learned_addr learned my addr 172.21.15.175:0/1781157179 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:47:03.637 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.636+0000 7fd3ad4b7700 1 --2- 172.21.15.175:0/1781157179 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3a81076c0 0x7fd3a819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:03.637 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.636+0000 7fd3accb6700 1 -- 172.21.15.175:0/1781157179 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3a81076c0 msgr2=0x7fd3a819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:03.637 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.636+0000 7fd3accb6700 1 --2- 172.21.15.175:0/1781157179 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3a81076c0 0x7fd3a819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:03.637 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.636+0000 7fd3accb6700 1 -- 172.21.15.175:0/1781157179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3a81026a0 msgr2=0x7fd3a81988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:03.637 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.636+0000 7fd3accb6700 1 --2- 172.21.15.175:0/1781157179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3a81026a0 0x7fd3a81988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:03.638 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.636+0000 7fd3accb6700 1 -- 172.21.15.175:0/1781157179 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a819d390 con 0x7fd3a8101d00 2024-03-26T03:47:03.638 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.636+0000 7fd39ffff700 1 --2- 172.21.15.175:0/1781157179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3a81026a0 0x7fd3a81988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T03:47:03.638 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.636+0000 7fd3ad4b7700 1 --2- 172.21.15.175:0/1781157179 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3a81076c0 0x7fd3a819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T03:47:03.638 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.636+0000 7fd3accb6700 1 --2- 172.21.15.175:0/1781157179 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3a8101d00 0x7fd3a8198380 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd3a400b770 tx=0x7fd3a4007a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:47:03.638 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.636+0000 7fd39dffb700 1 -- 172.21.15.175:0/1781157179 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4021020 con 0x7fd3a8101d00 2024-03-26T03:47:03.638 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.636+0000 7fd39dffb700 1 -- 172.21.15.175:0/1781157179 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fd3a4004c50 con 0x7fd3a8101d00 2024-03-26T03:47:03.638 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.636+0000 7fd3aef1a700 1 -- 172.21.15.175:0/1781157179 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a819d620 con 0x7fd3a8101d00 2024-03-26T03:47:03.638 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.636+0000 7fd39dffb700 1 -- 172.21.15.175:0/1781157179 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4005050 con 0x7fd3a8101d00 2024-03-26T03:47:03.638 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.636+0000 7fd3aef1a700 1 -- 172.21.15.175:0/1781157179 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a81a4f00 con 0x7fd3a8101d00 2024-03-26T03:47:03.639 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.636+0000 7fd3aef1a700 1 -- 172.21.15.175:0/1781157179 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3a8050300 con 0x7fd3a8101d00 2024-03-26T03:47:03.639 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.636+0000 7fd39dffb700 1 -- 172.21.15.175:0/1781157179 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7fd3a4024020 con 0x7fd3a8101d00 2024-03-26T03:47:03.639 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.636+0000 7fd39dffb700 1 --2- 172.21.15.175:0/1781157179 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fd39006e7e0 0x7fd390070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:03.639 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.636+0000 7fd39ffff700 1 --2- 172.21.15.175:0/1781157179 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fd39006e7e0 0x7fd390070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:03.639 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.636+0000 7fd39dffb700 1 -- 172.21.15.175:0/1781157179 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(75..75 src has 1..75) ==== 7537+0+0 (secure 0 0 0) 0x7fd3a400f020 con 0x7fd3a8101d00 2024-03-26T03:47:03.639 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.640+0000 7fd39ffff700 1 --2- 172.21.15.175:0/1781157179 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fd39006e7e0 0x7fd390070ca0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd3a81000e0 tx=0x7fd39800b500 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:47:03.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.644+0000 7fd39dffb700 1 -- 172.21.15.175:0/1781157179 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7fd3a405cf90 con 0x7fd3a8101d00 2024-03-26T03:47:03.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.856+0000 7fd3aef1a700 1 -- 172.21.15.175:0/1781157179 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fd3a8103130 con 0x7fd3a8101d00 2024-03-26T03:47:03.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.856+0000 7fd39dffb700 1 -- 172.21.15.175:0/1781157179 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd3a4060c40 con 0x7fd3a8101d00 2024-03-26T03:47:03.857 INFO:teuthology.orchestra.run.smithi175.stdout:107374182485 2024-03-26T03:47:03.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.860+0000 7fd3aef1a700 1 -- 172.21.15.175:0/1781157179 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fd39006e7e0 msgr2=0x7fd390070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:03.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.860+0000 7fd3aef1a700 1 --2- 172.21.15.175:0/1781157179 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fd39006e7e0 0x7fd390070ca0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd3a81000e0 tx=0x7fd39800b500 comp rx=0 tx=0).stop 2024-03-26T03:47:03.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.860+0000 7fd3aef1a700 1 -- 172.21.15.175:0/1781157179 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3a8101d00 msgr2=0x7fd3a8198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:03.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.860+0000 7fd3aef1a700 1 --2- 172.21.15.175:0/1781157179 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3a8101d00 0x7fd3a8198380 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd3a400b770 tx=0x7fd3a4007a20 comp rx=0 tx=0).stop 2024-03-26T03:47:03.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.860+0000 7fd3aef1a700 1 -- 172.21.15.175:0/1781157179 shutdown_connections 2024-03-26T03:47:03.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.860+0000 7fd3aef1a700 1 --2- 172.21.15.175:0/1781157179 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fd39006e7e0 0x7fd390070ca0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:03.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.860+0000 7fd3aef1a700 1 --2- 172.21.15.175:0/1781157179 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3a8101d00 0x7fd3a8198380 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:03.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.860+0000 7fd3aef1a700 1 --2- 172.21.15.175:0/1781157179 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3a81076c0 0x7fd3a819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:03.860 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.860+0000 7fd3aef1a700 1 --2- 172.21.15.175:0/1781157179 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3a81026a0 0x7fd3a81988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:03.860 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.860+0000 7fd3aef1a700 1 -- 172.21.15.175:0/1781157179 >> 172.21.15.175:0/1781157179 conn(0x7fd3a8077b40 msgr2=0x7fd3a8100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:47:03.860 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.860+0000 7fd3aef1a700 1 -- 172.21.15.175:0/1781157179 shutdown_connections 2024-03-26T03:47:03.860 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:03.860+0000 7fd3aef1a700 1 -- 172.21.15.175:0/1781157179 wait complete. 2024-03-26T03:47:04.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:03 smithi190 bash[23392]: cluster 2024-03-26T03:47:02.517791+0000 mgr.y (mgr.24431) 98 : cluster [DBG] pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:04.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:03 smithi190 bash[23392]: audit 2024-03-26T03:47:03.764516+0000 mon.c (mon.2) 75 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:47:04.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:03 smithi190 bash[23392]: audit 2024-03-26T03:47:03.861059+0000 mon.c (mon.2) 76 : audit [DBG] from='client.? 172.21.15.175:0/1781157179' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-26T03:47:04.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:03 smithi175 bash[29284]: cluster 2024-03-26T03:47:02.517791+0000 mgr.y (mgr.24431) 98 : cluster [DBG] pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:04.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:03 smithi175 bash[29284]: audit 2024-03-26T03:47:03.764516+0000 mon.c (mon.2) 75 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:47:04.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:03 smithi175 bash[29284]: audit 2024-03-26T03:47:03.861059+0000 mon.c (mon.2) 76 : audit [DBG] from='client.? 172.21.15.175:0/1781157179' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-26T03:47:04.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:03 smithi175 bash[21527]: cluster 2024-03-26T03:47:02.517791+0000 mgr.y (mgr.24431) 98 : cluster [DBG] pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:04.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:03 smithi175 bash[21527]: audit 2024-03-26T03:47:03.764516+0000 mon.c (mon.2) 75 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:47:04.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:03 smithi175 bash[21527]: audit 2024-03-26T03:47:03.861059+0000 mon.c (mon.2) 76 : audit [DBG] from='client.? 172.21.15.175:0/1781157179' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-26T03:47:04.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.516+0000 7f9d6cb9a700 1 -- 172.21.15.175:0/727345236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d681015a0 msgr2=0x7f9d68101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:04.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.516+0000 7f9d6cb9a700 1 --2- 172.21.15.175:0/727345236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d681015a0 0x7f9d68101a00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9d500098f0 tx=0x7f9d50009c00 comp rx=0 tx=0).stop 2024-03-26T03:47:04.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.516+0000 7f9d6cb9a700 1 -- 172.21.15.175:0/727345236 shutdown_connections 2024-03-26T03:47:04.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.516+0000 7f9d6cb9a700 1 --2- 172.21.15.175:0/727345236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d68101f40 0x7f9d6810e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:04.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.516+0000 7f9d6cb9a700 1 --2- 172.21.15.175:0/727345236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d68106520 0x7f9d68106900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:04.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.516+0000 7f9d6cb9a700 1 --2- 172.21.15.175:0/727345236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d681015a0 0x7f9d68101a00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:04.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.516+0000 7f9d6cb9a700 1 -- 172.21.15.175:0/727345236 >> 172.21.15.175:0/727345236 conn(0x7f9d680775e0 msgr2=0x7f9d680779f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:47:04.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.516+0000 7f9d6cb9a700 1 -- 172.21.15.175:0/727345236 shutdown_connections 2024-03-26T03:47:04.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.516+0000 7f9d6cb9a700 1 -- 172.21.15.175:0/727345236 wait complete. 2024-03-26T03:47:04.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.520+0000 7f9d6cb9a700 1 Processor -- start 2024-03-26T03:47:04.519 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.520+0000 7f9d6cb9a700 1 -- start start 2024-03-26T03:47:04.520 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.520+0000 7f9d6cb9a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d681015a0 0x7f9d681984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:04.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.520+0000 7f9d6cb9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d68101f40 0x7f9d681989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:04.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.520+0000 7f9d6cb9a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d68106520 0x7f9d6819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:04.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.520+0000 7f9d6cb9a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9d68110d80 con 0x7f9d681015a0 2024-03-26T03:47:04.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.520+0000 7f9d6cb9a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9d68110c00 con 0x7f9d68101f40 2024-03-26T03:47:04.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.520+0000 7f9d6cb9a700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9d68110f00 con 0x7f9d68106520 2024-03-26T03:47:04.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.520+0000 7f9d6659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d681015a0 0x7f9d681984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:04.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.520+0000 7f9d6659c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d681015a0 0x7f9d681984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:34376/0 (socket says 172.21.15.175:34376) 2024-03-26T03:47:04.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.520+0000 7f9d6659c700 1 -- 172.21.15.175:0/1801200319 learned_addr learned my addr 172.21.15.175:0/1801200319 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:47:04.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.520+0000 7f9d66d9d700 1 --2- 172.21.15.175:0/1801200319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d68106520 0x7f9d6819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:04.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.520+0000 7f9d65d9b700 1 --2- 172.21.15.175:0/1801200319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d68101f40 0x7f9d681989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:04.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.520+0000 7f9d6659c700 1 -- 172.21.15.175:0/1801200319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d68106520 msgr2=0x7f9d6819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:04.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.520+0000 7f9d6659c700 1 --2- 172.21.15.175:0/1801200319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d68106520 0x7f9d6819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:04.521 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.520+0000 7f9d6659c700 1 -- 172.21.15.175:0/1801200319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d68101f40 msgr2=0x7f9d681989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:04.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.520+0000 7f9d6659c700 1 --2- 172.21.15.175:0/1801200319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d68101f40 0x7f9d681989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:04.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.520+0000 7f9d6659c700 1 -- 172.21.15.175:0/1801200319 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d6819d4c0 con 0x7f9d681015a0 2024-03-26T03:47:04.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.520+0000 7f9d6659c700 1 --2- 172.21.15.175:0/1801200319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d681015a0 0x7f9d681984b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000b9c0 tx=0x7f9d6000bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:47:04.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.520+0000 7f9d5f7fe700 1 -- 172.21.15.175:0/1801200319 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60019070 con 0x7f9d681015a0 2024-03-26T03:47:04.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.520+0000 7f9d6cb9a700 1 -- 172.21.15.175:0/1801200319 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d6819d7b0 con 0x7f9d681015a0 2024-03-26T03:47:04.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.520+0000 7f9d5f7fe700 1 -- 172.21.15.175:0/1801200319 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f9d6000cdf0 con 0x7f9d681015a0 2024-03-26T03:47:04.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.520+0000 7f9d5f7fe700 1 -- 172.21.15.175:0/1801200319 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60012340 con 0x7f9d681015a0 2024-03-26T03:47:04.522 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.520+0000 7f9d6cb9a700 1 -- 172.21.15.175:0/1801200319 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d681a5030 con 0x7f9d681015a0 2024-03-26T03:47:04.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.520+0000 7f9d5d7fa700 1 -- 172.21.15.175:0/1801200319 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d4c005120 con 0x7f9d681015a0 2024-03-26T03:47:04.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.520+0000 7f9d5f7fe700 1 -- 172.21.15.175:0/1801200319 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7f9d6001c070 con 0x7f9d681015a0 2024-03-26T03:47:04.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.520+0000 7f9d5f7fe700 1 --2- 172.21.15.175:0/1801200319 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f9d5406e7e0 0x7f9d54070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:04.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.524+0000 7f9d5f7fe700 1 -- 172.21.15.175:0/1801200319 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7537+0+0 (secure 0 0 0) 0x7f9d6008e1c0 con 0x7f9d681015a0 2024-03-26T03:47:04.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.524+0000 7f9d65d9b700 1 --2- 172.21.15.175:0/1801200319 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f9d5406e7e0 0x7f9d54070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:04.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.524+0000 7f9d65d9b700 1 --2- 172.21.15.175:0/1801200319 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f9d5406e7e0 0x7f9d54070ca0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9d50009fd0 tx=0x7f9d50019040 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:47:04.527 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.528+0000 7f9d5f7fe700 1 -- 172.21.15.175:0/1801200319 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f9d6005eee0 con 0x7f9d681015a0 2024-03-26T03:47:04.569 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.568+0000 7f65de3ef700 1 -- 172.21.15.175:0/659335260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65d806a3a0 msgr2=0x7f65d806a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:04.570 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.568+0000 7f65de3ef700 1 --2- 172.21.15.175:0/659335260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65d806a3a0 0x7f65d806a820 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009920 tx=0x7f65c8009c30 comp rx=0 tx=0).stop 2024-03-26T03:47:04.570 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.568+0000 7f65de3ef700 1 -- 172.21.15.175:0/659335260 shutdown_connections 2024-03-26T03:47:04.570 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.568+0000 7f65de3ef700 1 --2- 172.21.15.175:0/659335260 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65d8108650 0x7f65d8108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:04.570 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.568+0000 7f65de3ef700 1 --2- 172.21.15.175:0/659335260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65d806ad60 0x7f65d810f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:04.570 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.568+0000 7f65de3ef700 1 --2- 172.21.15.175:0/659335260 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65d806a3a0 0x7f65d806a820 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:04.570 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.568+0000 7f65de3ef700 1 -- 172.21.15.175:0/659335260 >> 172.21.15.175:0/659335260 conn(0x7f65d80775e0 msgr2=0x7f65d80779f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:47:04.570 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.568+0000 7f65de3ef700 1 -- 172.21.15.175:0/659335260 shutdown_connections 2024-03-26T03:47:04.570 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.568+0000 7f65de3ef700 1 -- 172.21.15.175:0/659335260 wait complete. 2024-03-26T03:47:04.570 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.568+0000 7f65de3ef700 1 Processor -- start 2024-03-26T03:47:04.570 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.568+0000 7f65de3ef700 1 -- start start 2024-03-26T03:47:04.572 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.568+0000 7f65de3ef700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65d806a3a0 0x7f65d8101670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:04.572 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.568+0000 7f65de3ef700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65d806ad60 0x7f65d8101bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:04.572 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.568+0000 7f65de3ef700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65d8108650 0x7f65d81020f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:04.572 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.568+0000 7f65de3ef700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f65d8111eb0 con 0x7f65d8108650 2024-03-26T03:47:04.572 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.568+0000 7f65de3ef700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f65d8111d30 con 0x7f65d806ad60 2024-03-26T03:47:04.572 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.568+0000 7f65de3ef700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f65d8112030 con 0x7f65d806a3a0 2024-03-26T03:47:04.572 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.568+0000 7f65d7fff700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65d806a3a0 0x7f65d8101670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:04.572 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.568+0000 7f65d7fff700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65d806a3a0 0x7f65d8101670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:51662/0 (socket says 172.21.15.175:51662) 2024-03-26T03:47:04.572 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.568+0000 7f65d7fff700 1 -- 172.21.15.175:0/23177331 learned_addr learned my addr 172.21.15.175:0/23177331 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:47:04.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.568+0000 7f65dc98c700 1 --2- 172.21.15.175:0/23177331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65d8108650 0x7f65d81020f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:04.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.568+0000 7f65d77fe700 1 --2- 172.21.15.175:0/23177331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65d806ad60 0x7f65d8101bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:04.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.568+0000 7f65d7fff700 1 -- 172.21.15.175:0/23177331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65d806ad60 msgr2=0x7f65d8101bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:04.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.568+0000 7f65d7fff700 1 --2- 172.21.15.175:0/23177331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65d806ad60 0x7f65d8101bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:04.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.568+0000 7f65d7fff700 1 -- 172.21.15.175:0/23177331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65d8108650 msgr2=0x7f65d81020f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:04.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.568+0000 7f65d7fff700 1 --2- 172.21.15.175:0/23177331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65d8108650 0x7f65d81020f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:04.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.572+0000 7f65d7fff700 1 -- 172.21.15.175:0/23177331 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d81a68f0 con 0x7f65d806a3a0 2024-03-26T03:47:04.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.572+0000 7f65d77fe700 1 --2- 172.21.15.175:0/23177331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65d806ad60 0x7f65d8101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T03:47:04.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.572+0000 7f65dc98c700 1 --2- 172.21.15.175:0/23177331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65d8108650 0x7f65d81020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T03:47:04.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.572+0000 7f65d7fff700 1 --2- 172.21.15.175:0/23177331 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65d806a3a0 0x7f65d8101670 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00ec10 tx=0x7f65cc00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:47:04.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.572+0000 7f65d57fa700 1 -- 172.21.15.175:0/23177331 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65cc00cd40 con 0x7f65d806a3a0 2024-03-26T03:47:04.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.572+0000 7f65d57fa700 1 -- 172.21.15.175:0/23177331 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f65cc0044a0 con 0x7f65d806a3a0 2024-03-26T03:47:04.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.572+0000 7f65d57fa700 1 -- 172.21.15.175:0/23177331 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65cc010430 con 0x7f65d806a3a0 2024-03-26T03:47:04.573 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.572+0000 7f65de3ef700 1 -- 172.21.15.175:0/23177331 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65d81a6be0 con 0x7f65d806a3a0 2024-03-26T03:47:04.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.572+0000 7f65de3ef700 1 -- 172.21.15.175:0/23177331 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f65d81a70d0 con 0x7f65d806a3a0 2024-03-26T03:47:04.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.572+0000 7f65d57fa700 1 -- 172.21.15.175:0/23177331 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7f65cc003fc0 con 0x7f65d806a3a0 2024-03-26T03:47:04.574 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.572+0000 7f65de3ef700 1 -- 172.21.15.175:0/23177331 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65d8050300 con 0x7f65d806a3a0 2024-03-26T03:47:04.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.576+0000 7f65d57fa700 1 --2- 172.21.15.175:0/23177331 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f65c006e7e0 0x7f65c0070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:04.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.576+0000 7f65d77fe700 1 --2- 172.21.15.175:0/23177331 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f65c006e7e0 0x7f65c0070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:04.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.576+0000 7f65d57fa700 1 -- 172.21.15.175:0/23177331 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(75..75 src has 1..75) ==== 7537+0+0 (secure 0 0 0) 0x7f65cc08f280 con 0x7f65d806a3a0 2024-03-26T03:47:04.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.576+0000 7f65d77fe700 1 --2- 172.21.15.175:0/23177331 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f65c006e7e0 0x7f65c0070ca0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009230 tx=0x7f65c8005b40 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:47:04.578 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.576+0000 7f65d57fa700 1 -- 172.21.15.175:0/23177331 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f65cc05c270 con 0x7f65d806a3a0 2024-03-26T03:47:04.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.780+0000 7f9d5d7fa700 1 -- 172.21.15.175:0/1801200319 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f9d4c0056e0 con 0x7f9d681015a0 2024-03-26T03:47:04.779 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.780+0000 7f9d5f7fe700 1 -- 172.21.15.175:0/1801200319 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9d60063710 con 0x7f9d681015a0 2024-03-26T03:47:04.780 INFO:teuthology.orchestra.run.smithi175.stdout:167503724609 2024-03-26T03:47:04.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.780+0000 7f9d5d7fa700 1 -- 172.21.15.175:0/1801200319 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f9d5406e7e0 msgr2=0x7f9d54070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:04.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.780+0000 7f9d5d7fa700 1 --2- 172.21.15.175:0/1801200319 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f9d5406e7e0 0x7f9d54070ca0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9d50009fd0 tx=0x7f9d50019040 comp rx=0 tx=0).stop 2024-03-26T03:47:04.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.780+0000 7f9d5d7fa700 1 -- 172.21.15.175:0/1801200319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d681015a0 msgr2=0x7f9d681984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:04.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.780+0000 7f9d5d7fa700 1 --2- 172.21.15.175:0/1801200319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d681015a0 0x7f9d681984b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000b9c0 tx=0x7f9d6000bd80 comp rx=0 tx=0).stop 2024-03-26T03:47:04.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.780+0000 7f9d5d7fa700 1 -- 172.21.15.175:0/1801200319 shutdown_connections 2024-03-26T03:47:04.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.780+0000 7f9d5d7fa700 1 --2- 172.21.15.175:0/1801200319 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f9d5406e7e0 0x7f9d54070ca0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:04.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.780+0000 7f9d5d7fa700 1 --2- 172.21.15.175:0/1801200319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d68106520 0x7f9d6819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:04.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.780+0000 7f9d5d7fa700 1 --2- 172.21.15.175:0/1801200319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d681015a0 0x7f9d681984b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:04.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.780+0000 7f9d5d7fa700 1 --2- 172.21.15.175:0/1801200319 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d68101f40 0x7f9d681989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:04.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.780+0000 7f9d5d7fa700 1 -- 172.21.15.175:0/1801200319 >> 172.21.15.175:0/1801200319 conn(0x7f9d680775e0 msgr2=0x7f9d681052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:47:04.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.780+0000 7f9d5d7fa700 1 -- 172.21.15.175:0/1801200319 shutdown_connections 2024-03-26T03:47:04.783 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.780+0000 7f9d5d7fa700 1 -- 172.21.15.175:0/1801200319 wait complete. 2024-03-26T03:47:04.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.808+0000 7f65de3ef700 1 -- 172.21.15.175:0/23177331 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f65d802dd30 con 0x7f65d806a3a0 2024-03-26T03:47:04.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.808+0000 7f65d57fa700 1 -- 172.21.15.175:0/23177331 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f65cc05ff20 con 0x7f65d806a3a0 2024-03-26T03:47:04.807 INFO:teuthology.orchestra.run.smithi175.stdout:201863462965 2024-03-26T03:47:04.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.808+0000 7f65de3ef700 1 -- 172.21.15.175:0/23177331 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f65c006e7e0 msgr2=0x7f65c0070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:04.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.808+0000 7f65de3ef700 1 --2- 172.21.15.175:0/23177331 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f65c006e7e0 0x7f65c0070ca0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009230 tx=0x7f65c8005b40 comp rx=0 tx=0).stop 2024-03-26T03:47:04.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.808+0000 7f65de3ef700 1 -- 172.21.15.175:0/23177331 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65d806a3a0 msgr2=0x7f65d8101670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:04.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.808+0000 7f65de3ef700 1 --2- 172.21.15.175:0/23177331 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65d806a3a0 0x7f65d8101670 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00ec10 tx=0x7f65cc00ef20 comp rx=0 tx=0).stop 2024-03-26T03:47:04.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.808+0000 7f65de3ef700 1 -- 172.21.15.175:0/23177331 shutdown_connections 2024-03-26T03:47:04.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.808+0000 7f65de3ef700 1 --2- 172.21.15.175:0/23177331 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f65c006e7e0 0x7f65c0070ca0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:04.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.808+0000 7f65de3ef700 1 --2- 172.21.15.175:0/23177331 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65d806a3a0 0x7f65d8101670 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:04.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.808+0000 7f65de3ef700 1 --2- 172.21.15.175:0/23177331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65d8108650 0x7f65d81020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:04.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.808+0000 7f65de3ef700 1 --2- 172.21.15.175:0/23177331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65d806ad60 0x7f65d8101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:04.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.808+0000 7f65de3ef700 1 -- 172.21.15.175:0/23177331 >> 172.21.15.175:0/23177331 conn(0x7f65d80775e0 msgr2=0x7f65d8100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:47:04.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.808+0000 7f65de3ef700 1 -- 172.21.15.175:0/23177331 shutdown_connections 2024-03-26T03:47:04.811 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:04.808+0000 7f65de3ef700 1 -- 172.21.15.175:0/23177331 wait complete. 2024-03-26T03:47:04.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:04 smithi175 bash[21527]: audit 2024-03-26T03:47:04.783951+0000 mon.a (mon.0) 859 : audit [DBG] from='client.? 172.21.15.175:0/1801200319' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-26T03:47:04.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:04 smithi175 bash[21527]: audit 2024-03-26T03:47:04.811699+0000 mon.c (mon.2) 77 : audit [DBG] from='client.? 172.21.15.175:0/23177331' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-26T03:47:04.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:04 smithi175 bash[29284]: audit 2024-03-26T03:47:04.783951+0000 mon.a (mon.0) 859 : audit [DBG] from='client.? 172.21.15.175:0/1801200319' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-26T03:47:04.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:04 smithi175 bash[29284]: audit 2024-03-26T03:47:04.811699+0000 mon.c (mon.2) 77 : audit [DBG] from='client.? 172.21.15.175:0/23177331' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-26T03:47:05.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:04 smithi190 bash[23392]: audit 2024-03-26T03:47:04.783951+0000 mon.a (mon.0) 859 : audit [DBG] from='client.? 172.21.15.175:0/1801200319' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-26T03:47:05.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:04 smithi190 bash[23392]: audit 2024-03-26T03:47:04.811699+0000 mon.c (mon.2) 77 : audit [DBG] from='client.? 172.21.15.175:0/23177331' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-26T03:47:05.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.848+0000 7f5feb407700 1 -- 172.21.15.175:0/1970744197 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fe4100b90 msgr2=0x7f5fe4101010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:05.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.848+0000 7f5feb407700 1 --2- 172.21.15.175:0/1970744197 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fe4100b90 0x7f5fe4101010 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40098f0 tx=0x7f5fd4009c00 comp rx=0 tx=0).stop 2024-03-26T03:47:05.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.848+0000 7f5feb407700 1 -- 172.21.15.175:0/1970744197 shutdown_connections 2024-03-26T03:47:05.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.848+0000 7f5feb407700 1 --2- 172.21.15.175:0/1970744197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fe4101550 0x7f5fe410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:05.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.848+0000 7f5feb407700 1 --2- 172.21.15.175:0/1970744197 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fe4108660 0x7f5fe4108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:05.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.848+0000 7f5feb407700 1 --2- 172.21.15.175:0/1970744197 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fe4100b90 0x7f5fe4101010 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:05.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.848+0000 7f5feb407700 1 -- 172.21.15.175:0/1970744197 >> 172.21.15.175:0/1970744197 conn(0x7f5fe4077b40 msgr2=0x7f5fe40feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:47:05.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.848+0000 7f5feb407700 1 -- 172.21.15.175:0/1970744197 shutdown_connections 2024-03-26T03:47:05.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.848+0000 7f5feb407700 1 -- 172.21.15.175:0/1970744197 wait complete. 2024-03-26T03:47:05.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.848+0000 7f5feb407700 1 Processor -- start 2024-03-26T03:47:05.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.848+0000 7f5feb407700 1 -- start start 2024-03-26T03:47:05.853 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.848+0000 7f5feb407700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fe4100b90 0x7f5fe4198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:05.853 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.848+0000 7f5feb407700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fe4101550 0x7f5fe41988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:05.853 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.848+0000 7f5feb407700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fe4108660 0x7f5fe419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:05.853 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.848+0000 7f5feb407700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe4111ec0 con 0x7f5fe4100b90 2024-03-26T03:47:05.853 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.848+0000 7f5feb407700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe4111d40 con 0x7f5fe4108660 2024-03-26T03:47:05.853 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.848+0000 7f5feb407700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5fe4112040 con 0x7f5fe4101550 2024-03-26T03:47:05.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.848+0000 7f5fe91a3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fe4100b90 0x7f5fe4198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:05.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.852+0000 7f5fe89a2700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fe4101550 0x7f5fe41988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:05.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.852+0000 7f5fe91a3700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fe4100b90 0x7f5fe4198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:34406/0 (socket says 172.21.15.175:34406) 2024-03-26T03:47:05.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.852+0000 7f5fe91a3700 1 -- 172.21.15.175:0/1441736920 learned_addr learned my addr 172.21.15.175:0/1441736920 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:47:05.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.852+0000 7f5fe99a4700 1 --2- 172.21.15.175:0/1441736920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fe4108660 0x7f5fe419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:05.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.852+0000 7f5fe91a3700 1 -- 172.21.15.175:0/1441736920 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fe4101550 msgr2=0x7f5fe41988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:05.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.852+0000 7f5fe91a3700 1 --2- 172.21.15.175:0/1441736920 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fe4101550 0x7f5fe41988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:05.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.852+0000 7f5fe91a3700 1 -- 172.21.15.175:0/1441736920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fe4108660 msgr2=0x7f5fe419cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:05.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.852+0000 7f5fe91a3700 1 --2- 172.21.15.175:0/1441736920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fe4108660 0x7f5fe419cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:05.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.852+0000 7f5fe91a3700 1 -- 172.21.15.175:0/1441736920 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe419d3a0 con 0x7f5fe4100b90 2024-03-26T03:47:05.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.852+0000 7f5fe89a2700 1 --2- 172.21.15.175:0/1441736920 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fe4101550 0x7f5fe41988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T03:47:05.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.852+0000 7f5fe91a3700 1 --2- 172.21.15.175:0/1441736920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fe4100b90 0x7f5fe4198390 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000c920 tx=0x7f5fe000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:47:05.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.852+0000 7f5fda7fc700 1 -- 172.21.15.175:0/1441736920 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0007b00 con 0x7f5fe4100b90 2024-03-26T03:47:05.854 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.852+0000 7f5fda7fc700 1 -- 172.21.15.175:0/1441736920 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f5fe0004cb0 con 0x7f5fe4100b90 2024-03-26T03:47:05.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.852+0000 7f5feb407700 1 -- 172.21.15.175:0/1441736920 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe419d690 con 0x7f5fe4100b90 2024-03-26T03:47:05.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.852+0000 7f5fda7fc700 1 -- 172.21.15.175:0/1441736920 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0005410 con 0x7f5fe4100b90 2024-03-26T03:47:05.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.852+0000 7f5feb407700 1 -- 172.21.15.175:0/1441736920 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe41a4f10 con 0x7f5fe4100b90 2024-03-26T03:47:05.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.852+0000 7f5feb407700 1 -- 172.21.15.175:0/1441736920 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fe4050300 con 0x7f5fe4100b90 2024-03-26T03:47:05.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.852+0000 7f5fda7fc700 1 -- 172.21.15.175:0/1441736920 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7f5fe0004750 con 0x7f5fe4100b90 2024-03-26T03:47:05.855 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.852+0000 7f5fda7fc700 1 --2- 172.21.15.175:0/1441736920 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f5fd006e7e0 0x7f5fd0070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:05.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.852+0000 7f5fda7fc700 1 -- 172.21.15.175:0/1441736920 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7537+0+0 (secure 0 0 0) 0x7f5fe008ed60 con 0x7f5fe4100b90 2024-03-26T03:47:05.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.852+0000 7f5fe89a2700 1 --2- 172.21.15.175:0/1441736920 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f5fd006e7e0 0x7f5fd0070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:05.856 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.852+0000 7f5fe89a2700 1 --2- 172.21.15.175:0/1441736920 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f5fd006e7e0 0x7f5fd0070ca0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009fd0 tx=0x7f5fd4019040 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:47:05.858 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:05.856+0000 7f5fda7fc700 1 -- 172.21.15.175:0/1441736920 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f5fe0014030 con 0x7f5fe4100b90 2024-03-26T03:47:05.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:05 smithi175 bash[29284]: cluster 2024-03-26T03:47:04.518588+0000 mgr.y (mgr.24431) 99 : cluster [DBG] pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:05.951 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:05 smithi175 bash[21527]: cluster 2024-03-26T03:47:04.518588+0000 mgr.y (mgr.24431) 99 : cluster [DBG] pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:06.008 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182481 got 107374182485 for osd.3 2024-03-26T03:47:06.008 DEBUG:teuthology.parallel:result is None 2024-03-26T03:47:06.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.084+0000 7f5feb407700 1 -- 172.21.15.175:0/1441736920 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f5fe41996f0 con 0x7f5fe4100b90 2024-03-26T03:47:06.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.084+0000 7f5fda7fc700 1 -- 172.21.15.175:0/1441736920 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5fe005bd30 con 0x7f5fe4100b90 2024-03-26T03:47:06.085 INFO:teuthology.orchestra.run.smithi175.stdout:55834574957 2024-03-26T03:47:06.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.088+0000 7f5feb407700 1 -- 172.21.15.175:0/1441736920 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f5fd006e7e0 msgr2=0x7f5fd0070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:06.087 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.088+0000 7f5feb407700 1 --2- 172.21.15.175:0/1441736920 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f5fd006e7e0 0x7f5fd0070ca0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009fd0 tx=0x7f5fd4019040 comp rx=0 tx=0).stop 2024-03-26T03:47:06.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.088+0000 7f5feb407700 1 -- 172.21.15.175:0/1441736920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fe4100b90 msgr2=0x7f5fe4198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:06.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.088+0000 7f5feb407700 1 --2- 172.21.15.175:0/1441736920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fe4100b90 0x7f5fe4198390 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000c920 tx=0x7f5fe000cc30 comp rx=0 tx=0).stop 2024-03-26T03:47:06.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.088+0000 7f5feb407700 1 -- 172.21.15.175:0/1441736920 shutdown_connections 2024-03-26T03:47:06.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.088+0000 7f5feb407700 1 --2- 172.21.15.175:0/1441736920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fe4108660 0x7f5fe419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:06.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.088+0000 7f5feb407700 1 --2- 172.21.15.175:0/1441736920 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fe4100b90 0x7f5fe4198390 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:06.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.088+0000 7f5feb407700 1 --2- 172.21.15.175:0/1441736920 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f5fd006e7e0 0x7f5fd0070ca0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:06.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.088+0000 7f5feb407700 1 --2- 172.21.15.175:0/1441736920 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fe4101550 0x7f5fe41988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:06.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.088+0000 7f5feb407700 1 -- 172.21.15.175:0/1441736920 >> 172.21.15.175:0/1441736920 conn(0x7f5fe4077b40 msgr2=0x7f5fe40ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:47:06.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.088+0000 7f5feb407700 1 -- 172.21.15.175:0/1441736920 shutdown_connections 2024-03-26T03:47:06.088 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.088+0000 7f5feb407700 1 -- 172.21.15.175:0/1441736920 wait complete. 2024-03-26T03:47:06.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.120+0000 7f71235b0700 1 -- 172.21.15.175:0/1836033824 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f711c1015a0 msgr2=0x7f711c101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:06.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.120+0000 7f71235b0700 1 --2- 172.21.15.175:0/1836033824 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f711c1015a0 0x7f711c101a00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f710c009990 tx=0x7f710c009ca0 comp rx=0 tx=0).stop 2024-03-26T03:47:06.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.120+0000 7f71235b0700 1 -- 172.21.15.175:0/1836033824 shutdown_connections 2024-03-26T03:47:06.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.120+0000 7f71235b0700 1 --2- 172.21.15.175:0/1836033824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f711c101f40 0x7f711c10e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:06.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.120+0000 7f71235b0700 1 --2- 172.21.15.175:0/1836033824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f711c106520 0x7f711c106900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:06.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.120+0000 7f71235b0700 1 --2- 172.21.15.175:0/1836033824 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f711c1015a0 0x7f711c101a00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:06.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.120+0000 7f71235b0700 1 -- 172.21.15.175:0/1836033824 >> 172.21.15.175:0/1836033824 conn(0x7f711c0775e0 msgr2=0x7f711c0779f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:47:06.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.120+0000 7f71235b0700 1 -- 172.21.15.175:0/1836033824 shutdown_connections 2024-03-26T03:47:06.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.120+0000 7f71235b0700 1 -- 172.21.15.175:0/1836033824 wait complete. 2024-03-26T03:47:06.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.124+0000 7f71235b0700 1 Processor -- start 2024-03-26T03:47:06.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.124+0000 7f71235b0700 1 -- start start 2024-03-26T03:47:06.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.124+0000 7f71235b0700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f711c1015a0 0x7f711c1031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:06.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.124+0000 7f71235b0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f711c101f40 0x7f711c1036f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:06.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.124+0000 7f71235b0700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f711c106520 0x7f711c103c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:06.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.124+0000 7f71235b0700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f711c110dd0 con 0x7f711c1015a0 2024-03-26T03:47:06.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.124+0000 7f71235b0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f711c110c50 con 0x7f711c101f40 2024-03-26T03:47:06.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.124+0000 7f71235b0700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f711c110f50 con 0x7f711c106520 2024-03-26T03:47:06.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.124+0000 7f712134c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f711c1015a0 0x7f711c1031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:06.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.124+0000 7f712134c700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f711c1015a0 0x7f711c1031b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:34424/0 (socket says 172.21.15.175:34424) 2024-03-26T03:47:06.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.124+0000 7f712134c700 1 -- 172.21.15.175:0/3411161185 learned_addr learned my addr 172.21.15.175:0/3411161185 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:47:06.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.124+0000 7f7121b4d700 1 --2- 172.21.15.175:0/3411161185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f711c106520 0x7f711c103c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:06.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.124+0000 7f712134c700 1 -- 172.21.15.175:0/3411161185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f711c106520 msgr2=0x7f711c103c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:06.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.124+0000 7f712134c700 1 --2- 172.21.15.175:0/3411161185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f711c106520 0x7f711c103c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:06.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.124+0000 7f7120b4b700 1 --2- 172.21.15.175:0/3411161185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f711c101f40 0x7f711c1036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:06.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.124+0000 7f712134c700 1 -- 172.21.15.175:0/3411161185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f711c101f40 msgr2=0x7f711c1036f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:06.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.124+0000 7f712134c700 1 --2- 172.21.15.175:0/3411161185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f711c101f40 0x7f711c1036f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:06.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.124+0000 7f712134c700 1 -- 172.21.15.175:0/3411161185 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711c1a6a90 con 0x7f711c1015a0 2024-03-26T03:47:06.125 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.124+0000 7f712134c700 1 --2- 172.21.15.175:0/3411161185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f711c1015a0 0x7f711c1031b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f711800e970 tx=0x7f711800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:47:06.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.124+0000 7f71127fc700 1 -- 172.21.15.175:0/3411161185 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711800cbd0 con 0x7f711c1015a0 2024-03-26T03:47:06.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.124+0000 7f71235b0700 1 -- 172.21.15.175:0/3411161185 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f711c1a6d80 con 0x7f711c1015a0 2024-03-26T03:47:06.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.124+0000 7f71127fc700 1 -- 172.21.15.175:0/3411161185 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f71180044a0 con 0x7f711c1015a0 2024-03-26T03:47:06.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.124+0000 7f71127fc700 1 -- 172.21.15.175:0/3411161185 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118003c70 con 0x7f711c1015a0 2024-03-26T03:47:06.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.124+0000 7f71235b0700 1 -- 172.21.15.175:0/3411161185 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f711c1a7260 con 0x7f711c1015a0 2024-03-26T03:47:06.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.124+0000 7f71235b0700 1 -- 172.21.15.175:0/3411161185 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f711c102c60 con 0x7f711c1015a0 2024-03-26T03:47:06.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.124+0000 7f71127fc700 1 -- 172.21.15.175:0/3411161185 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7f7118003e10 con 0x7f711c1015a0 2024-03-26T03:47:06.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.124+0000 7f71127fc700 1 --2- 172.21.15.175:0/3411161185 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f710806e6c0 0x7f7108070b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:06.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.124+0000 7f71127fc700 1 -- 172.21.15.175:0/3411161185 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7537+0+0 (secure 0 0 0) 0x7f711808f150 con 0x7f711c1015a0 2024-03-26T03:47:06.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.124+0000 7f7120b4b700 1 --2- 172.21.15.175:0/3411161185 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f710806e6c0 0x7f7108070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:06.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.128+0000 7f7120b4b700 1 --2- 172.21.15.175:0/3411161185 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f710806e6c0 0x7f7108070b80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f710c00b2e0 tx=0x7f710c000bc0 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:47:06.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.132+0000 7f71127fc700 1 -- 172.21.15.175:0/3411161185 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f711801fe20 con 0x7f711c1015a0 2024-03-26T03:47:06.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:05 smithi190 bash[23392]: cluster 2024-03-26T03:47:04.518588+0000 mgr.y (mgr.24431) 99 : cluster [DBG] pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:06.249 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9709f94700 1 -- 172.21.15.175:0/3645247710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97041015a0 msgr2=0x7f9704101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:06.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9709f94700 1 --2- 172.21.15.175:0/3645247710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97041015a0 0x7f9704101a00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f96f8009990 tx=0x7f96f8009ca0 comp rx=0 tx=0).stop 2024-03-26T03:47:06.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9709f94700 1 -- 172.21.15.175:0/3645247710 shutdown_connections 2024-03-26T03:47:06.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9709f94700 1 --2- 172.21.15.175:0/3645247710 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9704101f40 0x7f970410e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:06.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9709f94700 1 --2- 172.21.15.175:0/3645247710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9704106520 0x7f9704106900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:06.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9709f94700 1 --2- 172.21.15.175:0/3645247710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97041015a0 0x7f9704101a00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:06.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9709f94700 1 -- 172.21.15.175:0/3645247710 >> 172.21.15.175:0/3645247710 conn(0x7f97040775e0 msgr2=0x7f97040779f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:47:06.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9709f94700 1 -- 172.21.15.175:0/3645247710 shutdown_connections 2024-03-26T03:47:06.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9709f94700 1 -- 172.21.15.175:0/3645247710 wait complete. 2024-03-26T03:47:06.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9709f94700 1 Processor -- start 2024-03-26T03:47:06.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9709f94700 1 -- start start 2024-03-26T03:47:06.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9709f94700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97041015a0 0x7f97041031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:06.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9709f94700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9704101f40 0x7f97041036f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:06.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9709f94700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9704106520 0x7f9704103c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:06.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9709f94700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9704110d80 con 0x7f9704106520 2024-03-26T03:47:06.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9709f94700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9704110c00 con 0x7f97041015a0 2024-03-26T03:47:06.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9709f94700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9704110f00 con 0x7f9704101f40 2024-03-26T03:47:06.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9702ffd700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9704101f40 0x7f97041036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:06.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9702ffd700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9704101f40 0x7f97041036f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:51702/0 (socket says 172.21.15.175:51702) 2024-03-26T03:47:06.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9702ffd700 1 -- 172.21.15.175:0/2278860711 learned_addr learned my addr 172.21.15.175:0/2278860711 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:47:06.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9703fff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9704106520 0x7f9704103c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:06.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9702ffd700 1 -- 172.21.15.175:0/2278860711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97041015a0 msgr2=0x7f97041031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:06.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9702ffd700 1 --2- 172.21.15.175:0/2278860711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97041015a0 0x7f97041031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:06.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9702ffd700 1 -- 172.21.15.175:0/2278860711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9704106520 msgr2=0x7f9704103c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:06.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9702ffd700 1 --2- 172.21.15.175:0/2278860711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9704106520 0x7f9704103c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:06.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9702ffd700 1 -- 172.21.15.175:0/2278860711 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97041a68f0 con 0x7f9704101f40 2024-03-26T03:47:06.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9703fff700 1 --2- 172.21.15.175:0/2278860711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9704106520 0x7f9704103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T03:47:06.252 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9702ffd700 1 --2- 172.21.15.175:0/2278860711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9704101f40 0x7f97041036f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f96f80057c0 tx=0x7f96f80057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:47:06.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9700ff9700 1 -- 172.21.15.175:0/2278860711 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f8026070 con 0x7f9704101f40 2024-03-26T03:47:06.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9700ff9700 1 -- 172.21.15.175:0/2278860711 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f96f800fbf0 con 0x7f9704101f40 2024-03-26T03:47:06.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9709f94700 1 -- 172.21.15.175:0/2278860711 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97041a6b80 con 0x7f9704101f40 2024-03-26T03:47:06.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9709f94700 1 -- 172.21.15.175:0/2278860711 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f97041a70b0 con 0x7f9704101f40 2024-03-26T03:47:06.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.248+0000 7f9700ff9700 1 -- 172.21.15.175:0/2278860711 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f80173f0 con 0x7f9704101f40 2024-03-26T03:47:06.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.252+0000 7f9709f94700 1 -- 172.21.15.175:0/2278860711 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96f0005120 con 0x7f9704101f40 2024-03-26T03:47:06.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.252+0000 7f9700ff9700 1 -- 172.21.15.175:0/2278860711 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7f96f8029070 con 0x7f9704101f40 2024-03-26T03:47:06.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.252+0000 7f9700ff9700 1 --2- 172.21.15.175:0/2278860711 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f96ec06e7e0 0x7f96ec070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:06.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.252+0000 7f9700ff9700 1 -- 172.21.15.175:0/2278860711 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(75..75 src has 1..75) ==== 7537+0+0 (secure 0 0 0) 0x7f96f808fcc0 con 0x7f9704101f40 2024-03-26T03:47:06.253 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.252+0000 7f97037fe700 1 --2- 172.21.15.175:0/2278860711 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f96ec06e7e0 0x7f96ec070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:06.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.252+0000 7f97037fe700 1 --2- 172.21.15.175:0/2278860711 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f96ec06e7e0 0x7f96ec070ca0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9704102780 tx=0x7f96f4005c10 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:47:06.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.256+0000 7f9700ff9700 1 -- 172.21.15.175:0/2278860711 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f96f8013070 con 0x7f9704101f40 2024-03-26T03:47:06.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.344+0000 7f71235b0700 1 -- 172.21.15.175:0/3411161185 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f711c104690 con 0x7f711c1015a0 2024-03-26T03:47:06.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.344+0000 7f71127fc700 1 -- 172.21.15.175:0/3411161185 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f711805c140 con 0x7f711c1015a0 2024-03-26T03:47:06.347 INFO:teuthology.orchestra.run.smithi175.stdout:128849018955 2024-03-26T03:47:06.349 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.348+0000 7f71235b0700 1 -- 172.21.15.175:0/3411161185 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f710806e6c0 msgr2=0x7f7108070b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:06.349 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.348+0000 7f71235b0700 1 --2- 172.21.15.175:0/3411161185 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f710806e6c0 0x7f7108070b80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f710c00b2e0 tx=0x7f710c000bc0 comp rx=0 tx=0).stop 2024-03-26T03:47:06.349 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.348+0000 7f71235b0700 1 -- 172.21.15.175:0/3411161185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f711c1015a0 msgr2=0x7f711c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:06.349 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.348+0000 7f71235b0700 1 --2- 172.21.15.175:0/3411161185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f711c1015a0 0x7f711c1031b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f711800e970 tx=0x7f711800ec80 comp rx=0 tx=0).stop 2024-03-26T03:47:06.349 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.348+0000 7f71235b0700 1 -- 172.21.15.175:0/3411161185 shutdown_connections 2024-03-26T03:47:06.349 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.348+0000 7f71235b0700 1 --2- 172.21.15.175:0/3411161185 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f710806e6c0 0x7f7108070b80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:06.349 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.348+0000 7f71235b0700 1 --2- 172.21.15.175:0/3411161185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f711c106520 0x7f711c103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:06.349 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.348+0000 7f71235b0700 1 --2- 172.21.15.175:0/3411161185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f711c1015a0 0x7f711c1031b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:06.349 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.348+0000 7f71235b0700 1 --2- 172.21.15.175:0/3411161185 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f711c101f40 0x7f711c1036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:06.349 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.348+0000 7f71235b0700 1 -- 172.21.15.175:0/3411161185 >> 172.21.15.175:0/3411161185 conn(0x7f711c0775e0 msgr2=0x7f711c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:47:06.349 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.348+0000 7f71235b0700 1 -- 172.21.15.175:0/3411161185 shutdown_connections 2024-03-26T03:47:06.350 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.348+0000 7f71235b0700 1 -- 172.21.15.175:0/3411161185 wait complete. 2024-03-26T03:47:06.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.472+0000 7f9709f94700 1 -- 172.21.15.175:0/2278860711 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f96f00056e0 con 0x7f9704101f40 2024-03-26T03:47:06.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.472+0000 7f9700ff9700 1 -- 172.21.15.175:0/2278860711 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f96f805cd30 con 0x7f9704101f40 2024-03-26T03:47:06.472 INFO:teuthology.orchestra.run.smithi175.stdout:34359738487 2024-03-26T03:47:06.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.472+0000 7f9709f94700 1 -- 172.21.15.175:0/2278860711 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f96ec06e7e0 msgr2=0x7f96ec070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:06.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.472+0000 7f9709f94700 1 --2- 172.21.15.175:0/2278860711 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f96ec06e7e0 0x7f96ec070ca0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9704102780 tx=0x7f96f4005c10 comp rx=0 tx=0).stop 2024-03-26T03:47:06.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.472+0000 7f9709f94700 1 -- 172.21.15.175:0/2278860711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9704101f40 msgr2=0x7f97041036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:06.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.472+0000 7f9709f94700 1 --2- 172.21.15.175:0/2278860711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9704101f40 0x7f97041036f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f96f80057c0 tx=0x7f96f80057f0 comp rx=0 tx=0).stop 2024-03-26T03:47:06.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.472+0000 7f9709f94700 1 -- 172.21.15.175:0/2278860711 shutdown_connections 2024-03-26T03:47:06.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.472+0000 7f9709f94700 1 --2- 172.21.15.175:0/2278860711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9704106520 0x7f9704103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:06.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.472+0000 7f9709f94700 1 --2- 172.21.15.175:0/2278860711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97041015a0 0x7f97041031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:06.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.472+0000 7f9709f94700 1 --2- 172.21.15.175:0/2278860711 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f96ec06e7e0 0x7f96ec070ca0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:06.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.472+0000 7f9709f94700 1 --2- 172.21.15.175:0/2278860711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9704101f40 0x7f97041036f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:06.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.472+0000 7f9709f94700 1 -- 172.21.15.175:0/2278860711 >> 172.21.15.175:0/2278860711 conn(0x7f97040775e0 msgr2=0x7f970406a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:47:06.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.472+0000 7f9709f94700 1 -- 172.21.15.175:0/2278860711 shutdown_connections 2024-03-26T03:47:06.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:06.472+0000 7f9709f94700 1 -- 172.21.15.175:0/2278860711 wait complete. 2024-03-26T03:47:07.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:06 smithi190 bash[23392]: audit 2024-03-26T03:47:06.089945+0000 mon.a (mon.0) 860 : audit [DBG] from='client.? 172.21.15.175:0/1441736920' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-26T03:47:07.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:06 smithi190 bash[23392]: audit 2024-03-26T03:47:06.350843+0000 mon.a (mon.0) 861 : audit [DBG] from='client.? 172.21.15.175:0/3411161185' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-26T03:47:07.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:06 smithi190 bash[23392]: audit 2024-03-26T03:47:06.476165+0000 mon.c (mon.2) 78 : audit [DBG] from='client.? 172.21.15.175:0/2278860711' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-26T03:47:07.273 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:06 smithi175 bash[29284]: audit 2024-03-26T03:47:06.089945+0000 mon.a (mon.0) 860 : audit [DBG] from='client.? 172.21.15.175:0/1441736920' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-26T03:47:07.273 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:06 smithi175 bash[29284]: audit 2024-03-26T03:47:06.350843+0000 mon.a (mon.0) 861 : audit [DBG] from='client.? 172.21.15.175:0/3411161185' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-26T03:47:07.273 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:06 smithi175 bash[29284]: audit 2024-03-26T03:47:06.476165+0000 mon.c (mon.2) 78 : audit [DBG] from='client.? 172.21.15.175:0/2278860711' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-26T03:47:07.274 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:06 smithi175 bash[21527]: audit 2024-03-26T03:47:06.089945+0000 mon.a (mon.0) 860 : audit [DBG] from='client.? 172.21.15.175:0/1441736920' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-26T03:47:07.274 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:06 smithi175 bash[21527]: audit 2024-03-26T03:47:06.350843+0000 mon.a (mon.0) 861 : audit [DBG] from='client.? 172.21.15.175:0/3411161185' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-26T03:47:07.274 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:06 smithi175 bash[21527]: audit 2024-03-26T03:47:06.476165+0000 mon.c (mon.2) 78 : audit [DBG] from='client.? 172.21.15.175:0/2278860711' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-26T03:47:07.637 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724605 got 167503724609 for osd.5 2024-03-26T03:47:07.637 DEBUG:teuthology.parallel:result is None 2024-03-26T03:47:08.015 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462961 got 201863462965 for osd.6 2024-03-26T03:47:08.015 DEBUG:teuthology.parallel:result is None 2024-03-26T03:47:08.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:07 smithi190 bash[23392]: cluster 2024-03-26T03:47:06.519112+0000 mgr.y (mgr.24431) 100 : cluster [DBG] pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:08.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:07 smithi175 bash[21527]: cluster 2024-03-26T03:47:06.519112+0000 mgr.y (mgr.24431) 100 : cluster [DBG] pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:08.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:07 smithi175 bash[29284]: cluster 2024-03-26T03:47:06.519112+0000 mgr.y (mgr.24431) 100 : cluster [DBG] pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:09.181 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:47:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:47:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:47:09.182 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738482 got 34359738487 for osd.0 2024-03-26T03:47:09.182 DEBUG:teuthology.parallel:result is None 2024-03-26T03:47:09.322 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574952 got 55834574957 for osd.1 2024-03-26T03:47:09.322 DEBUG:teuthology.parallel:result is None 2024-03-26T03:47:09.344 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018950 got 128849018955 for osd.4 2024-03-26T03:47:09.344 DEBUG:teuthology.parallel:result is None 2024-03-26T03:47:09.344 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-26T03:47:09.344 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph pg dump --format=json 2024-03-26T03:47:09.913 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:47:09 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:47:10.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:09 smithi190 bash[23392]: cluster 2024-03-26T03:47:08.519776+0000 mgr.y (mgr.24431) 101 : cluster [DBG] pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-26T03:47:10.278 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:09 smithi175 bash[29284]: cluster 2024-03-26T03:47:08.519776+0000 mgr.y (mgr.24431) 101 : cluster [DBG] pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-26T03:47:10.278 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:09 smithi175 bash[21527]: cluster 2024-03-26T03:47:08.519776+0000 mgr.y (mgr.24431) 101 : cluster [DBG] pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-26T03:47:11.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:10 smithi175 bash[29284]: audit 2024-03-26T03:47:09.581644+0000 mgr.y (mgr.24431) 102 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:47:11.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:10 smithi190 bash[23392]: audit 2024-03-26T03:47:09.581644+0000 mgr.y (mgr.24431) 102 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:47:11.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:10 smithi175 bash[21527]: audit 2024-03-26T03:47:09.581644+0000 mgr.y (mgr.24431) 102 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:47:12.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:11 smithi190 bash[23392]: cluster 2024-03-26T03:47:10.521286+0000 mgr.y (mgr.24431) 103 : cluster [DBG] pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:12.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:11 smithi175 bash[21527]: cluster 2024-03-26T03:47:10.521286+0000 mgr.y (mgr.24431) 103 : cluster [DBG] pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:12.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:11 smithi175 bash[29284]: cluster 2024-03-26T03:47:10.521286+0000 mgr.y (mgr.24431) 103 : cluster [DBG] pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:14.176 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:47:14.197 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:13 smithi175 bash[21527]: cluster 2024-03-26T03:47:12.522003+0000 mgr.y (mgr.24431) 104 : cluster [DBG] pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:14.198 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:13 smithi175 bash[29284]: cluster 2024-03-26T03:47:12.522003+0000 mgr.y (mgr.24431) 104 : cluster [DBG] pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:14.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:13 smithi190 bash[23392]: cluster 2024-03-26T03:47:12.522003+0000 mgr.y (mgr.24431) 104 : cluster [DBG] pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:16.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:15 smithi190 bash[23392]: cluster 2024-03-26T03:47:14.523350+0000 mgr.y (mgr.24431) 105 : cluster [DBG] pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:16.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:15 smithi175 bash[21527]: cluster 2024-03-26T03:47:14.523350+0000 mgr.y (mgr.24431) 105 : cluster [DBG] pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:16.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:15 smithi175 bash[29284]: cluster 2024-03-26T03:47:14.523350+0000 mgr.y (mgr.24431) 105 : cluster [DBG] pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:17.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e520ae700 1 -- 172.21.15.175:0/2904509743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e4c06adb0 msgr2=0x7f2e4c06b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:17.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e520ae700 1 --2- 172.21.15.175:0/2904509743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e4c06adb0 0x7f2e4c06b230 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c009990 tx=0x7f2e3c009ca0 comp rx=0 tx=0).stop 2024-03-26T03:47:17.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e520ae700 1 -- 172.21.15.175:0/2904509743 shutdown_connections 2024-03-26T03:47:17.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e520ae700 1 --2- 172.21.15.175:0/2904509743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e4c104570 0x7f2e4c104950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:17.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e520ae700 1 --2- 172.21.15.175:0/2904509743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e4c06b770 0x7f2e4c10f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:17.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e520ae700 1 --2- 172.21.15.175:0/2904509743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e4c06adb0 0x7f2e4c06b230 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:17.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e520ae700 1 -- 172.21.15.175:0/2904509743 >> 172.21.15.175:0/2904509743 conn(0x7f2e4c0775e0 msgr2=0x7f2e4c0779f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:47:17.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e520ae700 1 -- 172.21.15.175:0/2904509743 shutdown_connections 2024-03-26T03:47:17.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e520ae700 1 -- 172.21.15.175:0/2904509743 wait complete. 2024-03-26T03:47:17.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e520ae700 1 Processor -- start 2024-03-26T03:47:17.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e520ae700 1 -- start start 2024-03-26T03:47:17.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e520ae700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e4c06adb0 0x7f2e4c1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:17.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e520ae700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e4c06b770 0x7f2e4c198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:17.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e520ae700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e4c104570 0x7f2e4c19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:17.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e520ae700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c112060 con 0x7f2e4c06b770 2024-03-26T03:47:17.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e520ae700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c111ee0 con 0x7f2e4c06adb0 2024-03-26T03:47:17.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e520ae700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2e4c1121e0 con 0x7f2e4c104570 2024-03-26T03:47:17.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e4affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e4c06b770 0x7f2e4c198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:17.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e4affd700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e4c06b770 0x7f2e4c198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:56532/0 (socket says 172.21.15.175:56532) 2024-03-26T03:47:17.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e4affd700 1 -- 172.21.15.175:0/1287746396 learned_addr learned my addr 172.21.15.175:0/1287746396 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:47:17.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e4bfff700 1 --2- 172.21.15.175:0/1287746396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e4c104570 0x7f2e4c19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:17.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e4b7fe700 1 --2- 172.21.15.175:0/1287746396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e4c06adb0 0x7f2e4c1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:17.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e4affd700 1 -- 172.21.15.175:0/1287746396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e4c104570 msgr2=0x7f2e4c19cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:17.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e4affd700 1 --2- 172.21.15.175:0/1287746396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e4c104570 0x7f2e4c19cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:17.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e4affd700 1 -- 172.21.15.175:0/1287746396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e4c06adb0 msgr2=0x7f2e4c1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:17.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e4affd700 1 --2- 172.21.15.175:0/1287746396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e4c06adb0 0x7f2e4c1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:17.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e4affd700 1 -- 172.21.15.175:0/1287746396 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4c19d4b0 con 0x7f2e4c06b770 2024-03-26T03:47:17.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e4affd700 1 --2- 172.21.15.175:0/1287746396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e4c06b770 0x7f2e4c198a30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c000c00 tx=0x7f2e3c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:47:17.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e48ff9700 1 -- 172.21.15.175:0/1287746396 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c027070 con 0x7f2e4c06b770 2024-03-26T03:47:17.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e48ff9700 1 -- 172.21.15.175:0/1287746396 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f2e3c022470 con 0x7f2e4c06b770 2024-03-26T03:47:17.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e520ae700 1 -- 172.21.15.175:0/1287746396 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e4c19d740 con 0x7f2e4c06b770 2024-03-26T03:47:17.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e48ff9700 1 -- 172.21.15.175:0/1287746396 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c017650 con 0x7f2e4c06b770 2024-03-26T03:47:17.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.040+0000 7f2e520ae700 1 -- 172.21.15.175:0/1287746396 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e4c1a5020 con 0x7f2e4c06b770 2024-03-26T03:47:17.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.044+0000 7f2e520ae700 1 -- 172.21.15.175:0/1287746396 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e4c050300 con 0x7f2e4c06b770 2024-03-26T03:47:17.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.044+0000 7f2e48ff9700 1 -- 172.21.15.175:0/1287746396 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7f2e3c02a070 con 0x7f2e4c06b770 2024-03-26T03:47:17.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.044+0000 7f2e48ff9700 1 --2- 172.21.15.175:0/1287746396 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f2e3406e7e0 0x7f2e34070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:17.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.044+0000 7f2e48ff9700 1 -- 172.21.15.175:0/1287746396 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7537+0+0 (secure 0 0 0) 0x7f2e3c08fe90 con 0x7f2e4c06b770 2024-03-26T03:47:17.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.044+0000 7f2e4b7fe700 1 --2- 172.21.15.175:0/1287746396 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f2e3406e7e0 0x7f2e34070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:17.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.044+0000 7f2e4b7fe700 1 --2- 172.21.15.175:0/1287746396 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f2e3406e7e0 0x7f2e34070ca0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2e40009680 tx=0x7f2e40009260 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:47:17.049 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.048+0000 7f2e48ff9700 1 -- 172.21.15.175:0/1287746396 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f2e3c013070 con 0x7f2e4c06b770 2024-03-26T03:47:17.251 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.252+0000 7f2e520ae700 1 -- 172.21.15.175:0/1287746396 --> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2e4c199800 con 0x7f2e3406e7e0 2024-03-26T03:47:17.264 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.264+0000 7f2e48ff9700 1 -- 172.21.15.175:0/1287746396 <== mgr.24431 v2:172.21.15.175:6800/1140903233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+346530 (secure 0 0 0) 0x7f2e4c199800 con 0x7f2e3406e7e0 2024-03-26T03:47:17.268 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-26T03:47:17.270 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.268+0000 7f2e520ae700 1 -- 172.21.15.175:0/1287746396 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f2e3406e7e0 msgr2=0x7f2e34070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:17.270 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.268+0000 7f2e520ae700 1 --2- 172.21.15.175:0/1287746396 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f2e3406e7e0 0x7f2e34070ca0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2e40009680 tx=0x7f2e40009260 comp rx=0 tx=0).stop 2024-03-26T03:47:17.270 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.268+0000 7f2e520ae700 1 -- 172.21.15.175:0/1287746396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e4c06b770 msgr2=0x7f2e4c198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:17.270 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.268+0000 7f2e520ae700 1 --2- 172.21.15.175:0/1287746396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e4c06b770 0x7f2e4c198a30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c000c00 tx=0x7f2e3c0047b0 comp rx=0 tx=0).stop 2024-03-26T03:47:17.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.268+0000 7f2e520ae700 1 -- 172.21.15.175:0/1287746396 shutdown_connections 2024-03-26T03:47:17.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.268+0000 7f2e520ae700 1 --2- 172.21.15.175:0/1287746396 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f2e3406e7e0 0x7f2e34070ca0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:17.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.268+0000 7f2e520ae700 1 --2- 172.21.15.175:0/1287746396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e4c104570 0x7f2e4c19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:17.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.268+0000 7f2e520ae700 1 --2- 172.21.15.175:0/1287746396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e4c06adb0 0x7f2e4c1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:17.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.268+0000 7f2e520ae700 1 --2- 172.21.15.175:0/1287746396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e4c06b770 0x7f2e4c198a30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:17.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.268+0000 7f2e520ae700 1 -- 172.21.15.175:0/1287746396 >> 172.21.15.175:0/1287746396 conn(0x7f2e4c0775e0 msgr2=0x7f2e4c100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:47:17.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.268+0000 7f2e520ae700 1 -- 172.21.15.175:0/1287746396 shutdown_connections 2024-03-26T03:47:17.271 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:17.268+0000 7f2e520ae700 1 -- 172.21.15.175:0/1287746396 wait complete. 2024-03-26T03:47:17.271 INFO:teuthology.orchestra.run.smithi175.stderr:dumped all 2024-03-26T03:47:18.059 INFO:teuthology.orchestra.run.smithi175.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2024-03-26T03:47:16.523609+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596392,"num_objects":230,"num_object_clones":0,"num_object_copies":690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":0,"num_read":1391,"num_read_kb":1205,"num_write":807,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":823,"ondisk_log_size":823,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":237008,"kb_used_data":8380,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":749724208,"statfs":{"total":767960285184,"available":767717588992,"internally_reserved":0,"allocated":8581120,"data_stored":4630896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005545"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.454446+0000","last_change":"2024-03-26T03:44:22.338146+0000","last_active":"2024-03-26T03:45:18.454446+0000","last_peered":"2024-03-26T03:45:18.454446+0000","last_clean":"2024-03-26T03:45:18.454446+0000","last_became_active":"2024-03-26T03:44:22.336866+0000","last_became_peered":"2024-03-26T03:44:22.336866+0000","last_unstale":"2024-03-26T03:45:18.454446+0000","last_undegraded":"2024-03-26T03:45:18.454446+0000","last_fullsized":"2024-03-26T03:45:18.454446+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:55:45.204177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.453136+0000","last_change":"2024-03-26T03:44:20.335088+0000","last_active":"2024-03-26T03:45:18.453136+0000","last_peered":"2024-03-26T03:45:18.453136+0000","last_clean":"2024-03-26T03:45:18.453136+0000","last_became_active":"2024-03-26T03:44:20.334476+0000","last_became_peered":"2024-03-26T03:44:20.334476+0000","last_unstale":"2024-03-26T03:45:18.453136+0000","last_undegraded":"2024-03-26T03:45:18.453136+0000","last_fullsized":"2024-03-26T03:45:18.453136+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:34:55.744404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"61'1","reported_seq":48,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.517319+0000","last_change":"2024-03-26T03:44:16.272258+0000","last_active":"2024-03-26T03:45:18.517319+0000","last_peered":"2024-03-26T03:45:18.517319+0000","last_clean":"2024-03-26T03:45:18.517319+0000","last_became_active":"2024-03-26T03:44:16.270381+0000","last_became_peered":"2024-03-26T03:44:16.270381+0000","last_unstale":"2024-03-26T03:45:18.517319+0000","last_undegraded":"2024-03-26T03:45:18.517319+0000","last_fullsized":"2024-03-26T03:45:18.517319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:22:13.789751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"68'16","reported_seq":62,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.456478+0000","last_change":"2024-03-26T03:44:19.013489+0000","last_active":"2024-03-26T03:45:18.456478+0000","last_peered":"2024-03-26T03:45:18.456478+0000","last_clean":"2024-03-26T03:45:18.456478+0000","last_became_active":"2024-03-26T03:44:19.012559+0000","last_became_peered":"2024-03-26T03:44:19.012559+0000","last_unstale":"2024-03-26T03:45:18.456478+0000","last_undegraded":"2024-03-26T03:45:18.456478+0000","last_fullsized":"2024-03-26T03:45:18.456478+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:10:50.905039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.455666+0000","last_change":"2024-03-26T03:44:22.342750+0000","last_active":"2024-03-26T03:45:18.455666+0000","last_peered":"2024-03-26T03:45:18.455666+0000","last_clean":"2024-03-26T03:45:18.455666+0000","last_became_active":"2024-03-26T03:44:22.341960+0000","last_became_peered":"2024-03-26T03:44:22.341960+0000","last_unstale":"2024-03-26T03:45:18.455666+0000","last_undegraded":"2024-03-26T03:45:18.455666+0000","last_fullsized":"2024-03-26T03:45:18.455666+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:13:09.847546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.455357+0000","last_change":"2024-03-26T03:44:20.326832+0000","last_active":"2024-03-26T03:45:18.455357+0000","last_peered":"2024-03-26T03:45:18.455357+0000","last_clean":"2024-03-26T03:45:18.455357+0000","last_became_active":"2024-03-26T03:44:20.326497+0000","last_became_peered":"2024-03-26T03:44:20.326497+0000","last_unstale":"2024-03-26T03:45:18.455357+0000","last_undegraded":"2024-03-26T03:45:18.455357+0000","last_fullsized":"2024-03-26T03:45:18.455357+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:22:54.583875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.456242+0000","last_change":"2024-03-26T03:44:16.287994+0000","last_active":"2024-03-26T03:45:18.456242+0000","last_peered":"2024-03-26T03:45:18.456242+0000","last_clean":"2024-03-26T03:45:18.456242+0000","last_became_active":"2024-03-26T03:44:16.287505+0000","last_became_peered":"2024-03-26T03:44:16.287505+0000","last_unstale":"2024-03-26T03:45:18.456242+0000","last_undegraded":"2024-03-26T03:45:18.456242+0000","last_fullsized":"2024-03-26T03:45:18.456242+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:34:38.566037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"68'17","reported_seq":67,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.517128+0000","last_change":"2024-03-26T03:44:19.018727+0000","last_active":"2024-03-26T03:45:18.517128+0000","last_peered":"2024-03-26T03:45:18.517128+0000","last_clean":"2024-03-26T03:45:18.517128+0000","last_became_active":"2024-03-26T03:44:19.017370+0000","last_became_peered":"2024-03-26T03:44:19.017370+0000","last_unstale":"2024-03-26T03:45:18.517128+0000","last_undegraded":"2024-03-26T03:45:18.517128+0000","last_fullsized":"2024-03-26T03:45:18.517128+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:25:05.655395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":29,"num_read_kb":21,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.521511+0000","last_change":"2024-03-26T03:44:22.325358+0000","last_active":"2024-03-26T03:45:18.521511+0000","last_peered":"2024-03-26T03:45:18.521511+0000","last_clean":"2024-03-26T03:45:18.521511+0000","last_became_active":"2024-03-26T03:44:22.324426+0000","last_became_peered":"2024-03-26T03:44:22.324426+0000","last_unstale":"2024-03-26T03:45:18.521511+0000","last_undegraded":"2024-03-26T03:45:18.521511+0000","last_fullsized":"2024-03-26T03:45:18.521511+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:26:08.476795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.456463+0000","last_change":"2024-03-26T03:44:20.336269+0000","last_active":"2024-03-26T03:45:18.456463+0000","last_peered":"2024-03-26T03:45:18.456463+0000","last_clean":"2024-03-26T03:45:18.456463+0000","last_became_active":"2024-03-26T03:44:20.335949+0000","last_became_peered":"2024-03-26T03:44:20.335949+0000","last_unstale":"2024-03-26T03:45:18.456463+0000","last_undegraded":"2024-03-26T03:45:18.456463+0000","last_fullsized":"2024-03-26T03:45:18.456463+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:33:19.079896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.524056+0000","last_change":"2024-03-26T03:44:16.289648+0000","last_active":"2024-03-26T03:45:18.524056+0000","last_peered":"2024-03-26T03:45:18.524056+0000","last_clean":"2024-03-26T03:45:18.524056+0000","last_became_active":"2024-03-26T03:44:16.289164+0000","last_became_peered":"2024-03-26T03:44:16.289164+0000","last_unstale":"2024-03-26T03:45:18.524056+0000","last_undegraded":"2024-03-26T03:45:18.524056+0000","last_fullsized":"2024-03-26T03:45:18.524056+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T05:40:46.986870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"68'15","reported_seq":61,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.523864+0000","last_change":"2024-03-26T03:44:18.299144+0000","last_active":"2024-03-26T03:45:18.523864+0000","last_peered":"2024-03-26T03:45:18.523864+0000","last_clean":"2024-03-26T03:45:18.523864+0000","last_became_active":"2024-03-26T03:44:18.298663+0000","last_became_peered":"2024-03-26T03:44:18.298663+0000","last_unstale":"2024-03-26T03:45:18.523864+0000","last_undegraded":"2024-03-26T03:45:18.523864+0000","last_fullsized":"2024-03-26T03:45:18.523864+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T04:00:42.524427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516056+0000","last_change":"2024-03-26T03:44:22.345208+0000","last_active":"2024-03-26T03:45:18.516056+0000","last_peered":"2024-03-26T03:45:18.516056+0000","last_clean":"2024-03-26T03:45:18.516056+0000","last_became_active":"2024-03-26T03:44:22.344467+0000","last_became_peered":"2024-03-26T03:44:22.344467+0000","last_unstale":"2024-03-26T03:45:18.516056+0000","last_undegraded":"2024-03-26T03:45:18.516056+0000","last_fullsized":"2024-03-26T03:45:18.516056+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:09:31.902007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.456905+0000","last_change":"2024-03-26T03:44:20.334321+0000","last_active":"2024-03-26T03:45:18.456905+0000","last_peered":"2024-03-26T03:45:18.456905+0000","last_clean":"2024-03-26T03:45:18.456905+0000","last_became_active":"2024-03-26T03:44:20.333957+0000","last_became_peered":"2024-03-26T03:44:20.333957+0000","last_unstale":"2024-03-26T03:45:18.456905+0000","last_undegraded":"2024-03-26T03:45:18.456905+0000","last_fullsized":"2024-03-26T03:45:18.456905+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:57:51.564301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.522478+0000","last_change":"2024-03-26T03:44:16.279125+0000","last_active":"2024-03-26T03:45:18.522478+0000","last_peered":"2024-03-26T03:45:18.522478+0000","last_clean":"2024-03-26T03:45:18.522478+0000","last_became_active":"2024-03-26T03:44:16.278739+0000","last_became_peered":"2024-03-26T03:44:16.278739+0000","last_unstale":"2024-03-26T03:45:18.522478+0000","last_undegraded":"2024-03-26T03:45:18.522478+0000","last_fullsized":"2024-03-26T03:45:18.522478+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T04:09:02.885420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"68'18","reported_seq":71,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.523311+0000","last_change":"2024-03-26T03:44:18.300417+0000","last_active":"2024-03-26T03:45:18.523311+0000","last_peered":"2024-03-26T03:45:18.523311+0000","last_clean":"2024-03-26T03:45:18.523311+0000","last_became_active":"2024-03-26T03:44:18.300098+0000","last_became_peered":"2024-03-26T03:44:18.300098+0000","last_unstale":"2024-03-26T03:45:18.523311+0000","last_undegraded":"2024-03-26T03:45:18.523311+0000","last_fullsized":"2024-03-26T03:45:18.523311+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:24:01.922756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.456990+0000","last_change":"2024-03-26T03:44:22.330778+0000","last_active":"2024-03-26T03:45:18.456990+0000","last_peered":"2024-03-26T03:45:18.456990+0000","last_clean":"2024-03-26T03:45:18.456990+0000","last_became_active":"2024-03-26T03:44:22.328134+0000","last_became_peered":"2024-03-26T03:44:22.328134+0000","last_unstale":"2024-03-26T03:45:18.456990+0000","last_undegraded":"2024-03-26T03:45:18.456990+0000","last_fullsized":"2024-03-26T03:45:18.456990+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:42:54.583750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.456019+0000","last_change":"2024-03-26T03:44:54.230091+0000","last_active":"2024-03-26T03:45:18.456019+0000","last_peered":"2024-03-26T03:45:18.456019+0000","last_clean":"2024-03-26T03:45:18.456019+0000","last_became_active":"2024-03-26T03:44:54.229613+0000","last_became_peered":"2024-03-26T03:44:54.229613+0000","last_unstale":"2024-03-26T03:45:18.456019+0000","last_undegraded":"2024-03-26T03:45:18.456019+0000","last_fullsized":"2024-03-26T03:45:18.456019+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T05:02:08.657168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.455001+0000","last_change":"2024-03-26T03:44:16.272846+0000","last_active":"2024-03-26T03:45:18.455001+0000","last_peered":"2024-03-26T03:45:18.455001+0000","last_clean":"2024-03-26T03:45:18.455001+0000","last_became_active":"2024-03-26T03:44:16.272358+0000","last_became_peered":"2024-03-26T03:44:16.272358+0000","last_unstale":"2024-03-26T03:45:18.455001+0000","last_undegraded":"2024-03-26T03:45:18.455001+0000","last_fullsized":"2024-03-26T03:45:18.455001+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:47:16.998660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"68'21","reported_seq":75,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.455966+0000","last_change":"2024-03-26T03:44:19.019867+0000","last_active":"2024-03-26T03:45:18.455966+0000","last_peered":"2024-03-26T03:45:18.455966+0000","last_clean":"2024-03-26T03:45:18.455966+0000","last_became_active":"2024-03-26T03:44:19.016823+0000","last_became_peered":"2024-03-26T03:44:19.016823+0000","last_unstale":"2024-03-26T03:45:18.455966+0000","last_undegraded":"2024-03-26T03:45:18.455966+0000","last_fullsized":"2024-03-26T03:45:18.455966+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:57:04.191719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.458127+0000","last_change":"2024-03-26T03:44:22.332867+0000","last_active":"2024-03-26T03:45:18.458127+0000","last_peered":"2024-03-26T03:45:18.458127+0000","last_clean":"2024-03-26T03:45:18.458127+0000","last_became_active":"2024-03-26T03:44:22.331837+0000","last_became_peered":"2024-03-26T03:44:22.331837+0000","last_unstale":"2024-03-26T03:45:18.458127+0000","last_undegraded":"2024-03-26T03:45:18.458127+0000","last_fullsized":"2024-03-26T03:45:18.458127+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:34:43.858175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.450968+0000","last_change":"2024-03-26T03:44:20.325006+0000","last_active":"2024-03-26T03:45:18.450968+0000","last_peered":"2024-03-26T03:45:18.450968+0000","last_clean":"2024-03-26T03:45:18.450968+0000","last_became_active":"2024-03-26T03:44:20.324098+0000","last_became_peered":"2024-03-26T03:44:20.324098+0000","last_unstale":"2024-03-26T03:45:18.450968+0000","last_undegraded":"2024-03-26T03:45:18.450968+0000","last_fullsized":"2024-03-26T03:45:18.450968+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T04:07:48.295994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.521873+0000","last_change":"2024-03-26T03:44:16.275277+0000","last_active":"2024-03-26T03:45:18.521873+0000","last_peered":"2024-03-26T03:45:18.521873+0000","last_clean":"2024-03-26T03:45:18.521873+0000","last_became_active":"2024-03-26T03:44:16.274798+0000","last_became_peered":"2024-03-26T03:44:16.274798+0000","last_unstale":"2024-03-26T03:45:18.521873+0000","last_undegraded":"2024-03-26T03:45:18.521873+0000","last_fullsized":"2024-03-26T03:45:18.521873+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T05:23:28.829957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"68'11","reported_seq":58,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516670+0000","last_change":"2024-03-26T03:44:18.307189+0000","last_active":"2024-03-26T03:45:18.516670+0000","last_peered":"2024-03-26T03:45:18.516670+0000","last_clean":"2024-03-26T03:45:18.516670+0000","last_became_active":"2024-03-26T03:44:18.302665+0000","last_became_peered":"2024-03-26T03:44:18.302665+0000","last_unstale":"2024-03-26T03:45:18.516670+0000","last_undegraded":"2024-03-26T03:45:18.516670+0000","last_fullsized":"2024-03-26T03:45:18.516670+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:59:21.617776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":15,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516668+0000","last_change":"2024-03-26T03:44:22.346745+0000","last_active":"2024-03-26T03:45:18.516668+0000","last_peered":"2024-03-26T03:45:18.516668+0000","last_clean":"2024-03-26T03:45:18.516668+0000","last_became_active":"2024-03-26T03:44:22.345901+0000","last_became_peered":"2024-03-26T03:44:22.345901+0000","last_unstale":"2024-03-26T03:45:18.516668+0000","last_undegraded":"2024-03-26T03:45:18.516668+0000","last_fullsized":"2024-03-26T03:45:18.516668+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T05:18:00.556703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1f","version":"68'11","reported_seq":81,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:47:06.498765+0000","last_change":"2024-03-26T03:44:20.326524+0000","last_active":"2024-03-26T03:47:06.498765+0000","last_peered":"2024-03-26T03:47:06.498765+0000","last_clean":"2024-03-26T03:47:06.498765+0000","last_became_active":"2024-03-26T03:44:20.325453+0000","last_became_peered":"2024-03-26T03:44:20.325453+0000","last_unstale":"2024-03-26T03:47:06.498765+0000","last_undegraded":"2024-03-26T03:47:06.498765+0000","last_fullsized":"2024-03-26T03:47:06.498765+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:54:38.703763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"61'1","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.453285+0000","last_change":"2024-03-26T03:44:16.275315+0000","last_active":"2024-03-26T03:45:18.453285+0000","last_peered":"2024-03-26T03:45:18.453285+0000","last_clean":"2024-03-26T03:45:18.453285+0000","last_became_active":"2024-03-26T03:44:16.270628+0000","last_became_peered":"2024-03-26T03:44:16.270628+0000","last_unstale":"2024-03-26T03:45:18.453285+0000","last_undegraded":"2024-03-26T03:45:18.453285+0000","last_fullsized":"2024-03-26T03:45:18.453285+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:16:12.845364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"68'15","reported_seq":63,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.452608+0000","last_change":"2024-03-26T03:44:18.299981+0000","last_active":"2024-03-26T03:45:18.452608+0000","last_peered":"2024-03-26T03:45:18.452608+0000","last_clean":"2024-03-26T03:45:18.452608+0000","last_became_active":"2024-03-26T03:44:18.299360+0000","last_became_peered":"2024-03-26T03:44:18.299360+0000","last_unstale":"2024-03-26T03:45:18.452608+0000","last_undegraded":"2024-03-26T03:45:18.452608+0000","last_fullsized":"2024-03-26T03:45:18.452608+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:20:48.240896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":26,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"68'1","reported_seq":30,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.522239+0000","last_change":"2024-03-26T03:44:22.337569+0000","last_active":"2024-03-26T03:45:18.522239+0000","last_peered":"2024-03-26T03:45:18.522239+0000","last_clean":"2024-03-26T03:45:18.522239+0000","last_became_active":"2024-03-26T03:44:22.334116+0000","last_became_peered":"2024-03-26T03:44:22.334116+0000","last_unstale":"2024-03-26T03:45:18.522239+0000","last_undegraded":"2024-03-26T03:45:18.522239+0000","last_fullsized":"2024-03-26T03:45:18.522239+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T05:09:37.347925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":370,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.517082+0000","last_change":"2024-03-26T03:44:20.336266+0000","last_active":"2024-03-26T03:45:18.517082+0000","last_peered":"2024-03-26T03:45:18.517082+0000","last_clean":"2024-03-26T03:45:18.517082+0000","last_became_active":"2024-03-26T03:44:20.333977+0000","last_became_peered":"2024-03-26T03:44:20.333977+0000","last_unstale":"2024-03-26T03:45:18.517082+0000","last_undegraded":"2024-03-26T03:45:18.517082+0000","last_fullsized":"2024-03-26T03:45:18.517082+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:56:13.373173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.521324+0000","last_change":"2024-03-26T03:44:16.270706+0000","last_active":"2024-03-26T03:45:18.521324+0000","last_peered":"2024-03-26T03:45:18.521324+0000","last_clean":"2024-03-26T03:45:18.521324+0000","last_became_active":"2024-03-26T03:44:16.270241+0000","last_became_peered":"2024-03-26T03:44:16.270241+0000","last_unstale":"2024-03-26T03:45:18.521324+0000","last_undegraded":"2024-03-26T03:45:18.521324+0000","last_fullsized":"2024-03-26T03:45:18.521324+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T03:53:12.241914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"68'39","reported_seq":109,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516738+0000","last_change":"2024-03-26T03:44:18.294448+0000","last_active":"2024-03-26T03:45:18.516738+0000","last_peered":"2024-03-26T03:45:18.516738+0000","last_clean":"2024-03-26T03:45:18.516738+0000","last_became_active":"2024-03-26T03:44:18.293402+0000","last_became_peered":"2024-03-26T03:44:18.293402+0000","last_unstale":"2024-03-26T03:45:18.516738+0000","last_undegraded":"2024-03-26T03:45:18.516738+0000","last_fullsized":"2024-03-26T03:45:18.516738+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:31:07.948096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":62,"num_read_kb":49,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.455809+0000","last_change":"2024-03-26T03:44:22.331526+0000","last_active":"2024-03-26T03:45:18.455809+0000","last_peered":"2024-03-26T03:45:18.455809+0000","last_clean":"2024-03-26T03:45:18.455809+0000","last_became_active":"2024-03-26T03:44:22.329577+0000","last_became_peered":"2024-03-26T03:44:22.329577+0000","last_unstale":"2024-03-26T03:45:18.455809+0000","last_undegraded":"2024-03-26T03:45:18.455809+0000","last_fullsized":"2024-03-26T03:45:18.455809+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:07:43.723662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.453955+0000","last_change":"2024-03-26T03:44:20.309417+0000","last_active":"2024-03-26T03:45:18.453955+0000","last_peered":"2024-03-26T03:45:18.453955+0000","last_clean":"2024-03-26T03:45:18.453955+0000","last_became_active":"2024-03-26T03:44:20.308558+0000","last_became_peered":"2024-03-26T03:44:20.308558+0000","last_unstale":"2024-03-26T03:45:18.453955+0000","last_undegraded":"2024-03-26T03:45:18.453955+0000","last_fullsized":"2024-03-26T03:45:18.453955+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T05:24:24.768168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.523189+0000","last_change":"2024-03-26T03:44:16.282348+0000","last_active":"2024-03-26T03:45:18.523189+0000","last_peered":"2024-03-26T03:45:18.523189+0000","last_clean":"2024-03-26T03:45:18.523189+0000","last_became_active":"2024-03-26T03:44:16.282081+0000","last_became_peered":"2024-03-26T03:44:16.282081+0000","last_unstale":"2024-03-26T03:45:18.523189+0000","last_undegraded":"2024-03-26T03:45:18.523189+0000","last_fullsized":"2024-03-26T03:45:18.523189+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:35:09.555003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"68'22","reported_seq":77,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.521891+0000","last_change":"2024-03-26T03:44:18.306611+0000","last_active":"2024-03-26T03:45:18.521891+0000","last_peered":"2024-03-26T03:45:18.521891+0000","last_clean":"2024-03-26T03:45:18.521891+0000","last_became_active":"2024-03-26T03:44:18.306237+0000","last_became_peered":"2024-03-26T03:44:18.306237+0000","last_unstale":"2024-03-26T03:45:18.521891+0000","last_undegraded":"2024-03-26T03:45:18.521891+0000","last_fullsized":"2024-03-26T03:45:18.521891+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T05:59:47.316294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.521398+0000","last_change":"2024-03-26T03:44:22.334585+0000","last_active":"2024-03-26T03:45:18.521398+0000","last_peered":"2024-03-26T03:45:18.521398+0000","last_clean":"2024-03-26T03:45:18.521398+0000","last_became_active":"2024-03-26T03:44:22.333208+0000","last_became_peered":"2024-03-26T03:44:22.333208+0000","last_unstale":"2024-03-26T03:45:18.521398+0000","last_undegraded":"2024-03-26T03:45:18.521398+0000","last_fullsized":"2024-03-26T03:45:18.521398+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:26:52.325903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.452820+0000","last_change":"2024-03-26T03:44:20.327091+0000","last_active":"2024-03-26T03:45:18.452820+0000","last_peered":"2024-03-26T03:45:18.452820+0000","last_clean":"2024-03-26T03:45:18.452820+0000","last_became_active":"2024-03-26T03:44:20.326360+0000","last_became_peered":"2024-03-26T03:44:20.326360+0000","last_unstale":"2024-03-26T03:45:18.452820+0000","last_undegraded":"2024-03-26T03:45:18.452820+0000","last_fullsized":"2024-03-26T03:45:18.452820+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:26:54.380294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.521787+0000","last_change":"2024-03-26T03:44:16.286716+0000","last_active":"2024-03-26T03:45:18.521787+0000","last_peered":"2024-03-26T03:45:18.521787+0000","last_clean":"2024-03-26T03:45:18.521787+0000","last_became_active":"2024-03-26T03:44:16.286436+0000","last_became_peered":"2024-03-26T03:44:16.286436+0000","last_unstale":"2024-03-26T03:45:18.521787+0000","last_undegraded":"2024-03-26T03:45:18.521787+0000","last_fullsized":"2024-03-26T03:45:18.521787+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:25:37.685854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"68'12","reported_seq":57,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.517010+0000","last_change":"2024-03-26T03:44:18.300971+0000","last_active":"2024-03-26T03:45:18.517010+0000","last_peered":"2024-03-26T03:45:18.517010+0000","last_clean":"2024-03-26T03:45:18.517010+0000","last_became_active":"2024-03-26T03:44:18.299992+0000","last_became_peered":"2024-03-26T03:44:18.299992+0000","last_unstale":"2024-03-26T03:45:18.517010+0000","last_undegraded":"2024-03-26T03:45:18.517010+0000","last_fullsized":"2024-03-26T03:45:18.517010+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:28:04.014349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":18,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.452927+0000","last_change":"2024-03-26T03:44:22.330528+0000","last_active":"2024-03-26T03:45:18.452927+0000","last_peered":"2024-03-26T03:45:18.452927+0000","last_clean":"2024-03-26T03:45:18.452927+0000","last_became_active":"2024-03-26T03:44:22.328066+0000","last_became_peered":"2024-03-26T03:44:22.328066+0000","last_unstale":"2024-03-26T03:45:18.452927+0000","last_undegraded":"2024-03-26T03:45:18.452927+0000","last_fullsized":"2024-03-26T03:45:18.452927+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:28:18.405902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.462694+0000","last_change":"2024-03-26T03:44:20.336209+0000","last_active":"2024-03-26T03:45:18.462694+0000","last_peered":"2024-03-26T03:45:18.462694+0000","last_clean":"2024-03-26T03:45:18.462694+0000","last_became_active":"2024-03-26T03:44:20.335886+0000","last_became_peered":"2024-03-26T03:44:20.335886+0000","last_unstale":"2024-03-26T03:45:18.462694+0000","last_undegraded":"2024-03-26T03:45:18.462694+0000","last_fullsized":"2024-03-26T03:45:18.462694+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:53:46.045429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.517253+0000","last_change":"2024-03-26T03:44:16.291755+0000","last_active":"2024-03-26T03:45:18.517253+0000","last_peered":"2024-03-26T03:45:18.517253+0000","last_clean":"2024-03-26T03:45:18.517253+0000","last_became_active":"2024-03-26T03:44:16.291278+0000","last_became_peered":"2024-03-26T03:44:16.291278+0000","last_unstale":"2024-03-26T03:45:18.517253+0000","last_undegraded":"2024-03-26T03:45:18.517253+0000","last_fullsized":"2024-03-26T03:45:18.517253+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:29:26.307443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"68'22","reported_seq":74,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.456619+0000","last_change":"2024-03-26T03:44:18.303537+0000","last_active":"2024-03-26T03:45:18.456619+0000","last_peered":"2024-03-26T03:45:18.456619+0000","last_clean":"2024-03-26T03:45:18.456619+0000","last_became_active":"2024-03-26T03:44:18.303090+0000","last_became_peered":"2024-03-26T03:44:18.303090+0000","last_unstale":"2024-03-26T03:45:18.456619+0000","last_undegraded":"2024-03-26T03:45:18.456619+0000","last_fullsized":"2024-03-26T03:45:18.456619+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:15:38.958251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":33,"num_read_kb":26,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516080+0000","last_change":"2024-03-26T03:44:22.345312+0000","last_active":"2024-03-26T03:45:18.516080+0000","last_peered":"2024-03-26T03:45:18.516080+0000","last_clean":"2024-03-26T03:45:18.516080+0000","last_became_active":"2024-03-26T03:44:22.343501+0000","last_became_peered":"2024-03-26T03:44:22.343501+0000","last_unstale":"2024-03-26T03:45:18.516080+0000","last_undegraded":"2024-03-26T03:45:18.516080+0000","last_fullsized":"2024-03-26T03:45:18.516080+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:25:08.571989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.521578+0000","last_change":"2024-03-26T03:44:54.367176+0000","last_active":"2024-03-26T03:45:18.521578+0000","last_peered":"2024-03-26T03:45:18.521578+0000","last_clean":"2024-03-26T03:45:18.521578+0000","last_became_active":"2024-03-26T03:44:54.366318+0000","last_became_peered":"2024-03-26T03:44:54.366318+0000","last_unstale":"2024-03-26T03:45:18.521578+0000","last_undegraded":"2024-03-26T03:45:18.521578+0000","last_fullsized":"2024-03-26T03:45:18.521578+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:41:46.663430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.522469+0000","last_change":"2024-03-26T03:44:16.276071+0000","last_active":"2024-03-26T03:45:18.522469+0000","last_peered":"2024-03-26T03:45:18.522469+0000","last_clean":"2024-03-26T03:45:18.522469+0000","last_became_active":"2024-03-26T03:44:16.275777+0000","last_became_peered":"2024-03-26T03:44:16.275777+0000","last_unstale":"2024-03-26T03:45:18.522469+0000","last_undegraded":"2024-03-26T03:45:18.522469+0000","last_fullsized":"2024-03-26T03:45:18.522469+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:48:56.798635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"68'15","reported_seq":64,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.524202+0000","last_change":"2024-03-26T03:44:18.304098+0000","last_active":"2024-03-26T03:45:18.524202+0000","last_peered":"2024-03-26T03:45:18.524202+0000","last_clean":"2024-03-26T03:45:18.524202+0000","last_became_active":"2024-03-26T03:44:18.303805+0000","last_became_peered":"2024-03-26T03:44:18.303805+0000","last_unstale":"2024-03-26T03:45:18.524202+0000","last_undegraded":"2024-03-26T03:45:18.524202+0000","last_fullsized":"2024-03-26T03:45:18.524202+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T05:30:20.868582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":26,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.462331+0000","last_change":"2024-03-26T03:44:22.345201+0000","last_active":"2024-03-26T03:45:18.462331+0000","last_peered":"2024-03-26T03:45:18.462331+0000","last_clean":"2024-03-26T03:45:18.462331+0000","last_became_active":"2024-03-26T03:44:22.344643+0000","last_became_peered":"2024-03-26T03:44:22.344643+0000","last_unstale":"2024-03-26T03:45:18.462331+0000","last_undegraded":"2024-03-26T03:45:18.462331+0000","last_fullsized":"2024-03-26T03:45:18.462331+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:28:57.773904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"68'11","reported_seq":81,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:47:06.498749+0000","last_change":"2024-03-26T03:44:20.315194+0000","last_active":"2024-03-26T03:47:06.498749+0000","last_peered":"2024-03-26T03:47:06.498749+0000","last_clean":"2024-03-26T03:47:06.498749+0000","last_became_active":"2024-03-26T03:44:20.314772+0000","last_became_peered":"2024-03-26T03:44:20.314772+0000","last_unstale":"2024-03-26T03:47:06.498749+0000","last_undegraded":"2024-03-26T03:47:06.498749+0000","last_fullsized":"2024-03-26T03:47:06.498749+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:13:53.715741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516502+0000","last_change":"2024-03-26T03:44:16.273273+0000","last_active":"2024-03-26T03:45:18.516502+0000","last_peered":"2024-03-26T03:45:18.516502+0000","last_clean":"2024-03-26T03:45:18.516502+0000","last_became_active":"2024-03-26T03:44:16.272418+0000","last_became_peered":"2024-03-26T03:44:16.272418+0000","last_unstale":"2024-03-26T03:45:18.516502+0000","last_undegraded":"2024-03-26T03:45:18.516502+0000","last_fullsized":"2024-03-26T03:45:18.516502+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:00:36.966560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"68'9","reported_seq":52,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516571+0000","last_change":"2024-03-26T03:44:18.302502+0000","last_active":"2024-03-26T03:45:18.516571+0000","last_peered":"2024-03-26T03:45:18.516571+0000","last_clean":"2024-03-26T03:45:18.516571+0000","last_became_active":"2024-03-26T03:44:18.301521+0000","last_became_peered":"2024-03-26T03:44:18.301521+0000","last_unstale":"2024-03-26T03:45:18.516571+0000","last_undegraded":"2024-03-26T03:45:18.516571+0000","last_fullsized":"2024-03-26T03:45:18.516571+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:52:56.388079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.16","version":"68'1","reported_seq":29,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516845+0000","last_change":"2024-03-26T03:44:22.341037+0000","last_active":"2024-03-26T03:45:18.516845+0000","last_peered":"2024-03-26T03:45:18.516845+0000","last_clean":"2024-03-26T03:45:18.516845+0000","last_became_active":"2024-03-26T03:44:22.337576+0000","last_became_peered":"2024-03-26T03:44:22.337576+0000","last_unstale":"2024-03-26T03:45:18.516845+0000","last_undegraded":"2024-03-26T03:45:18.516845+0000","last_fullsized":"2024-03-26T03:45:18.516845+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:12:01.087443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"68'11","reported_seq":77,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:47:06.492245+0000","last_change":"2024-03-26T03:44:20.324528+0000","last_active":"2024-03-26T03:47:06.492245+0000","last_peered":"2024-03-26T03:47:06.492245+0000","last_clean":"2024-03-26T03:47:06.492245+0000","last_became_active":"2024-03-26T03:44:20.322642+0000","last_became_peered":"2024-03-26T03:44:20.322642+0000","last_unstale":"2024-03-26T03:47:06.492245+0000","last_undegraded":"2024-03-26T03:47:06.492245+0000","last_fullsized":"2024-03-26T03:47:06.492245+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:50:29.666007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.521846+0000","last_change":"2024-03-26T03:44:16.275480+0000","last_active":"2024-03-26T03:45:18.521846+0000","last_peered":"2024-03-26T03:45:18.521846+0000","last_clean":"2024-03-26T03:45:18.521846+0000","last_became_active":"2024-03-26T03:44:16.275108+0000","last_became_peered":"2024-03-26T03:44:16.275108+0000","last_unstale":"2024-03-26T03:45:18.521846+0000","last_undegraded":"2024-03-26T03:45:18.521846+0000","last_fullsized":"2024-03-26T03:45:18.521846+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:12:16.770696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"68'11","reported_seq":55,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.523092+0000","last_change":"2024-03-26T03:44:19.018499+0000","last_active":"2024-03-26T03:45:18.523092+0000","last_peered":"2024-03-26T03:45:18.523092+0000","last_clean":"2024-03-26T03:45:18.523092+0000","last_became_active":"2024-03-26T03:44:19.016703+0000","last_became_peered":"2024-03-26T03:44:19.016703+0000","last_unstale":"2024-03-26T03:45:18.523092+0000","last_undegraded":"2024-03-26T03:45:18.523092+0000","last_fullsized":"2024-03-26T03:45:18.523092+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:12:12.881881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.523701+0000","last_change":"2024-03-26T03:44:22.336554+0000","last_active":"2024-03-26T03:45:18.523701+0000","last_peered":"2024-03-26T03:45:18.523701+0000","last_clean":"2024-03-26T03:45:18.523701+0000","last_became_active":"2024-03-26T03:44:22.335615+0000","last_became_peered":"2024-03-26T03:44:22.335615+0000","last_unstale":"2024-03-26T03:45:18.523701+0000","last_undegraded":"2024-03-26T03:45:18.523701+0000","last_fullsized":"2024-03-26T03:45:18.523701+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:48:46.055161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.457036+0000","last_change":"2024-03-26T03:44:20.331738+0000","last_active":"2024-03-26T03:45:18.457036+0000","last_peered":"2024-03-26T03:45:18.457036+0000","last_clean":"2024-03-26T03:45:18.457036+0000","last_became_active":"2024-03-26T03:44:20.330112+0000","last_became_peered":"2024-03-26T03:44:20.330112+0000","last_unstale":"2024-03-26T03:45:18.457036+0000","last_undegraded":"2024-03-26T03:45:18.457036+0000","last_fullsized":"2024-03-26T03:45:18.457036+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:32:02.214724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.523660+0000","last_change":"2024-03-26T03:44:16.295711+0000","last_active":"2024-03-26T03:45:18.523660+0000","last_peered":"2024-03-26T03:45:18.523660+0000","last_clean":"2024-03-26T03:45:18.523660+0000","last_became_active":"2024-03-26T03:44:16.295341+0000","last_became_peered":"2024-03-26T03:44:16.295341+0000","last_unstale":"2024-03-26T03:45:18.523660+0000","last_undegraded":"2024-03-26T03:45:18.523660+0000","last_fullsized":"2024-03-26T03:45:18.523660+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:32:47.007119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"68'4","reported_seq":42,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.522401+0000","last_change":"2024-03-26T03:44:18.303208+0000","last_active":"2024-03-26T03:45:18.522401+0000","last_peered":"2024-03-26T03:45:18.522401+0000","last_clean":"2024-03-26T03:45:18.522401+0000","last_became_active":"2024-03-26T03:44:18.302934+0000","last_became_peered":"2024-03-26T03:44:18.302934+0000","last_unstale":"2024-03-26T03:45:18.522401+0000","last_undegraded":"2024-03-26T03:45:18.522401+0000","last_fullsized":"2024-03-26T03:45:18.522401+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T04:51:12.374238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.452196+0000","last_change":"2024-03-26T03:44:22.332841+0000","last_active":"2024-03-26T03:45:18.452196+0000","last_peered":"2024-03-26T03:45:18.452196+0000","last_clean":"2024-03-26T03:45:18.452196+0000","last_became_active":"2024-03-26T03:44:22.329041+0000","last_became_peered":"2024-03-26T03:44:22.329041+0000","last_unstale":"2024-03-26T03:45:18.452196+0000","last_undegraded":"2024-03-26T03:45:18.452196+0000","last_fullsized":"2024-03-26T03:45:18.452196+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:05:26.957647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516737+0000","last_change":"2024-03-26T03:44:20.342958+0000","last_active":"2024-03-26T03:45:18.516737+0000","last_peered":"2024-03-26T03:45:18.516737+0000","last_clean":"2024-03-26T03:45:18.516737+0000","last_became_active":"2024-03-26T03:44:20.339202+0000","last_became_peered":"2024-03-26T03:44:20.339202+0000","last_unstale":"2024-03-26T03:45:18.516737+0000","last_undegraded":"2024-03-26T03:45:18.516737+0000","last_fullsized":"2024-03-26T03:45:18.516737+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:33:52.804681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.450294+0000","last_change":"2024-03-26T03:44:16.282930+0000","last_active":"2024-03-26T03:45:18.450294+0000","last_peered":"2024-03-26T03:45:18.450294+0000","last_clean":"2024-03-26T03:45:18.450294+0000","last_became_active":"2024-03-26T03:44:16.282446+0000","last_became_peered":"2024-03-26T03:44:16.282446+0000","last_unstale":"2024-03-26T03:45:18.450294+0000","last_undegraded":"2024-03-26T03:45:18.450294+0000","last_fullsized":"2024-03-26T03:45:18.450294+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:57:27.460578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"68'17","reported_seq":67,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.521760+0000","last_change":"2024-03-26T03:44:18.310536+0000","last_active":"2024-03-26T03:45:18.521760+0000","last_peered":"2024-03-26T03:45:18.521760+0000","last_clean":"2024-03-26T03:45:18.521760+0000","last_became_active":"2024-03-26T03:44:18.310232+0000","last_became_peered":"2024-03-26T03:44:18.310232+0000","last_unstale":"2024-03-26T03:45:18.521760+0000","last_undegraded":"2024-03-26T03:45:18.521760+0000","last_fullsized":"2024-03-26T03:45:18.521760+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:15:46.155308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":29,"num_read_kb":21,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.455583+0000","last_change":"2024-03-26T03:44:22.342056+0000","last_active":"2024-03-26T03:45:18.455583+0000","last_peered":"2024-03-26T03:45:18.455583+0000","last_clean":"2024-03-26T03:45:18.455583+0000","last_became_active":"2024-03-26T03:44:22.341147+0000","last_became_peered":"2024-03-26T03:44:22.341147+0000","last_unstale":"2024-03-26T03:45:18.455583+0000","last_undegraded":"2024-03-26T03:45:18.455583+0000","last_fullsized":"2024-03-26T03:45:18.455583+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:22:23.943685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"61'2","reported_seq":55,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.459281+0000","last_change":"2024-03-26T03:44:16.274598+0000","last_active":"2024-03-26T03:45:18.459281+0000","last_peered":"2024-03-26T03:45:18.459281+0000","last_clean":"2024-03-26T03:45:18.459281+0000","last_became_active":"2024-03-26T03:44:16.274227+0000","last_became_peered":"2024-03-26T03:44:16.274227+0000","last_unstale":"2024-03-26T03:45:18.459281+0000","last_undegraded":"2024-03-26T03:45:18.459281+0000","last_fullsized":"2024-03-26T03:45:18.459281+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T04:02:56.201913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"68'17","reported_seq":67,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.523290+0000","last_change":"2024-03-26T03:44:18.300703+0000","last_active":"2024-03-26T03:45:18.523290+0000","last_peered":"2024-03-26T03:45:18.523290+0000","last_clean":"2024-03-26T03:45:18.523290+0000","last_became_active":"2024-03-26T03:44:18.300172+0000","last_became_peered":"2024-03-26T03:44:18.300172+0000","last_unstale":"2024-03-26T03:45:18.523290+0000","last_undegraded":"2024-03-26T03:45:18.523290+0000","last_fullsized":"2024-03-26T03:45:18.523290+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:34:14.174703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":29,"num_read_kb":21,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"68'11","reported_seq":77,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:47:06.499408+0000","last_change":"2024-03-26T03:44:20.327412+0000","last_active":"2024-03-26T03:47:06.499408+0000","last_peered":"2024-03-26T03:47:06.499408+0000","last_clean":"2024-03-26T03:47:06.499408+0000","last_became_active":"2024-03-26T03:44:20.326865+0000","last_became_peered":"2024-03-26T03:44:20.326865+0000","last_unstale":"2024-03-26T03:47:06.499408+0000","last_undegraded":"2024-03-26T03:47:06.499408+0000","last_fullsized":"2024-03-26T03:47:06.499408+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:10:11.113954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516163+0000","last_change":"2024-03-26T03:44:22.335472+0000","last_active":"2024-03-26T03:45:18.516163+0000","last_peered":"2024-03-26T03:45:18.516163+0000","last_clean":"2024-03-26T03:45:18.516163+0000","last_became_active":"2024-03-26T03:44:22.333780+0000","last_became_peered":"2024-03-26T03:44:22.333780+0000","last_unstale":"2024-03-26T03:45:18.516163+0000","last_undegraded":"2024-03-26T03:45:18.516163+0000","last_fullsized":"2024-03-26T03:45:18.516163+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:18:51.806269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.517549+0000","last_change":"2024-03-26T03:44:16.292414+0000","last_active":"2024-03-26T03:45:18.517549+0000","last_peered":"2024-03-26T03:45:18.517549+0000","last_clean":"2024-03-26T03:45:18.517549+0000","last_became_active":"2024-03-26T03:44:16.291660+0000","last_became_peered":"2024-03-26T03:44:16.291660+0000","last_unstale":"2024-03-26T03:45:18.517549+0000","last_undegraded":"2024-03-26T03:45:18.517549+0000","last_fullsized":"2024-03-26T03:45:18.517549+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:38:18.552119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"68'16","reported_seq":94,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:47:06.498797+0000","last_change":"2024-03-26T03:44:19.017721+0000","last_active":"2024-03-26T03:47:06.498797+0000","last_peered":"2024-03-26T03:47:06.498797+0000","last_clean":"2024-03-26T03:47:06.498797+0000","last_became_active":"2024-03-26T03:44:19.016758+0000","last_became_peered":"2024-03-26T03:44:19.016758+0000","last_unstale":"2024-03-26T03:47:06.498797+0000","last_undegraded":"2024-03-26T03:47:06.498797+0000","last_fullsized":"2024-03-26T03:47:06.498797+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:19:02.572817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.451509+0000","last_change":"2024-03-26T03:44:54.228617+0000","last_active":"2024-03-26T03:45:18.451509+0000","last_peered":"2024-03-26T03:45:18.451509+0000","last_clean":"2024-03-26T03:45:18.451509+0000","last_became_active":"2024-03-26T03:44:54.228073+0000","last_became_peered":"2024-03-26T03:44:54.228073+0000","last_unstale":"2024-03-26T03:45:18.451509+0000","last_undegraded":"2024-03-26T03:45:18.451509+0000","last_fullsized":"2024-03-26T03:45:18.451509+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:48:43.797931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.456331+0000","last_change":"2024-03-26T03:44:22.351278+0000","last_active":"2024-03-26T03:45:18.456331+0000","last_peered":"2024-03-26T03:45:18.456331+0000","last_clean":"2024-03-26T03:45:18.456331+0000","last_became_active":"2024-03-26T03:44:22.350716+0000","last_became_peered":"2024-03-26T03:44:22.350716+0000","last_unstale":"2024-03-26T03:45:18.456331+0000","last_undegraded":"2024-03-26T03:45:18.456331+0000","last_fullsized":"2024-03-26T03:45:18.456331+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:21:38.950129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.521942+0000","last_change":"2024-03-26T03:44:16.286765+0000","last_active":"2024-03-26T03:45:18.521942+0000","last_peered":"2024-03-26T03:45:18.521942+0000","last_clean":"2024-03-26T03:45:18.521942+0000","last_became_active":"2024-03-26T03:44:16.286370+0000","last_became_peered":"2024-03-26T03:44:16.286370+0000","last_unstale":"2024-03-26T03:45:18.521942+0000","last_undegraded":"2024-03-26T03:45:18.521942+0000","last_fullsized":"2024-03-26T03:45:18.521942+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:24:41.103950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"68'27","reported_seq":85,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.462515+0000","last_change":"2024-03-26T03:44:18.313370+0000","last_active":"2024-03-26T03:45:18.462515+0000","last_peered":"2024-03-26T03:45:18.462515+0000","last_clean":"2024-03-26T03:45:18.462515+0000","last_became_active":"2024-03-26T03:44:18.312850+0000","last_became_peered":"2024-03-26T03:44:18.312850+0000","last_unstale":"2024-03-26T03:45:18.462515+0000","last_undegraded":"2024-03-26T03:45:18.462515+0000","last_fullsized":"2024-03-26T03:45:18.462515+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T03:50:35.499073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.517354+0000","last_change":"2024-03-26T03:44:20.341293+0000","last_active":"2024-03-26T03:45:18.517354+0000","last_peered":"2024-03-26T03:45:18.517354+0000","last_clean":"2024-03-26T03:45:18.517354+0000","last_became_active":"2024-03-26T03:44:20.340439+0000","last_became_peered":"2024-03-26T03:44:20.340439+0000","last_unstale":"2024-03-26T03:45:18.517354+0000","last_undegraded":"2024-03-26T03:45:18.517354+0000","last_fullsized":"2024-03-26T03:45:18.517354+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:32:57.309782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.523836+0000","last_change":"2024-03-26T03:44:22.346981+0000","last_active":"2024-03-26T03:45:18.523836+0000","last_peered":"2024-03-26T03:45:18.523836+0000","last_clean":"2024-03-26T03:45:18.523836+0000","last_became_active":"2024-03-26T03:44:22.346271+0000","last_became_peered":"2024-03-26T03:44:22.346271+0000","last_unstale":"2024-03-26T03:45:18.523836+0000","last_undegraded":"2024-03-26T03:45:18.523836+0000","last_fullsized":"2024-03-26T03:45:18.523836+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:13:33.700704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"68'30","reported_seq":92,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.517104+0000","last_change":"2024-03-26T03:44:19.013703+0000","last_active":"2024-03-26T03:45:18.517104+0000","last_peered":"2024-03-26T03:45:18.517104+0000","last_clean":"2024-03-26T03:45:18.517104+0000","last_became_active":"2024-03-26T03:44:19.012601+0000","last_became_peered":"2024-03-26T03:44:19.012601+0000","last_unstale":"2024-03-26T03:45:18.517104+0000","last_undegraded":"2024-03-26T03:45:18.517104+0000","last_fullsized":"2024-03-26T03:45:18.517104+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:52:15.198255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":52,"num_read_kb":38,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.450630+0000","last_change":"2024-03-26T03:44:16.279025+0000","last_active":"2024-03-26T03:45:18.450630+0000","last_peered":"2024-03-26T03:45:18.450630+0000","last_clean":"2024-03-26T03:45:18.450630+0000","last_became_active":"2024-03-26T03:44:16.278503+0000","last_became_peered":"2024-03-26T03:44:16.278503+0000","last_unstale":"2024-03-26T03:45:18.450630+0000","last_undegraded":"2024-03-26T03:45:18.450630+0000","last_fullsized":"2024-03-26T03:45:18.450630+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:04:38.695284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.517207+0000","last_change":"2024-03-26T03:44:20.345993+0000","last_active":"2024-03-26T03:45:18.517207+0000","last_peered":"2024-03-26T03:45:18.517207+0000","last_clean":"2024-03-26T03:45:18.517207+0000","last_became_active":"2024-03-26T03:44:20.345481+0000","last_became_peered":"2024-03-26T03:44:20.345481+0000","last_unstale":"2024-03-26T03:45:18.517207+0000","last_undegraded":"2024-03-26T03:45:18.517207+0000","last_fullsized":"2024-03-26T03:45:18.517207+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T05:52:21.574020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.523932+0000","last_change":"2024-03-26T03:44:22.339804+0000","last_active":"2024-03-26T03:45:18.523932+0000","last_peered":"2024-03-26T03:45:18.523932+0000","last_clean":"2024-03-26T03:45:18.523932+0000","last_became_active":"2024-03-26T03:44:22.339031+0000","last_became_peered":"2024-03-26T03:44:22.339031+0000","last_unstale":"2024-03-26T03:45:18.523932+0000","last_undegraded":"2024-03-26T03:45:18.523932+0000","last_fullsized":"2024-03-26T03:45:18.523932+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:16:37.844926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"68'16","reported_seq":62,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.454914+0000","last_change":"2024-03-26T03:44:18.310573+0000","last_active":"2024-03-26T03:45:18.454914+0000","last_peered":"2024-03-26T03:45:18.454914+0000","last_clean":"2024-03-26T03:45:18.454914+0000","last_became_active":"2024-03-26T03:44:18.309925+0000","last_became_peered":"2024-03-26T03:44:18.309925+0000","last_unstale":"2024-03-26T03:45:18.454914+0000","last_undegraded":"2024-03-26T03:45:18.454914+0000","last_fullsized":"2024-03-26T03:45:18.454914+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:16:50.453724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.521578+0000","last_change":"2024-03-26T03:44:16.286853+0000","last_active":"2024-03-26T03:45:18.521578+0000","last_peered":"2024-03-26T03:45:18.521578+0000","last_clean":"2024-03-26T03:45:18.521578+0000","last_became_active":"2024-03-26T03:44:16.286474+0000","last_became_peered":"2024-03-26T03:44:16.286474+0000","last_unstale":"2024-03-26T03:45:18.521578+0000","last_undegraded":"2024-03-26T03:45:18.521578+0000","last_fullsized":"2024-03-26T03:45:18.521578+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:51:15.008303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":280,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:20.636665+0000","last_change":"2024-03-26T03:43:38.041937+0000","last_active":"2024-03-26T03:45:20.636665+0000","last_peered":"2024-03-26T03:45:20.636665+0000","last_clean":"2024-03-26T03:45:20.636665+0000","last_became_active":"2024-03-26T03:43:38.041066+0000","last_became_peered":"2024-03-26T03:43:38.041066+0000","last_unstale":"2024-03-26T03:45:20.636665+0000","last_undegraded":"2024-03-26T03:45:20.636665+0000","last_fullsized":"2024-03-26T03:45:20.636665+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:39:09.344827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:39:09.344827+0000","last_clean_scrub_stamp":"2024-03-26T03:39:09.344827+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:31:47.546805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.523270+0000","last_change":"2024-03-26T03:44:20.313558+0000","last_active":"2024-03-26T03:45:18.523270+0000","last_peered":"2024-03-26T03:45:18.523270+0000","last_clean":"2024-03-26T03:45:18.523270+0000","last_became_active":"2024-03-26T03:44:20.313183+0000","last_became_peered":"2024-03-26T03:44:20.313183+0000","last_unstale":"2024-03-26T03:45:18.523270+0000","last_undegraded":"2024-03-26T03:45:18.523270+0000","last_fullsized":"2024-03-26T03:45:18.523270+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:39:31.962065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.517266+0000","last_change":"2024-03-26T03:44:22.344155+0000","last_active":"2024-03-26T03:45:18.517266+0000","last_peered":"2024-03-26T03:45:18.517266+0000","last_clean":"2024-03-26T03:45:18.517266+0000","last_became_active":"2024-03-26T03:44:22.343297+0000","last_became_peered":"2024-03-26T03:44:22.343297+0000","last_unstale":"2024-03-26T03:45:18.517266+0000","last_undegraded":"2024-03-26T03:45:18.517266+0000","last_fullsized":"2024-03-26T03:45:18.517266+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:36:56.757960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"68'20","reported_seq":69,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516338+0000","last_change":"2024-03-26T03:44:18.307439+0000","last_active":"2024-03-26T03:45:18.516338+0000","last_peered":"2024-03-26T03:45:18.516338+0000","last_clean":"2024-03-26T03:45:18.516338+0000","last_became_active":"2024-03-26T03:44:18.306771+0000","last_became_peered":"2024-03-26T03:44:18.306771+0000","last_unstale":"2024-03-26T03:45:18.516338+0000","last_undegraded":"2024-03-26T03:45:18.516338+0000","last_fullsized":"2024-03-26T03:45:18.516338+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:37:31.727137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":30,"num_read_kb":22,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"61'1","reported_seq":48,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.523351+0000","last_change":"2024-03-26T03:44:16.288851+0000","last_active":"2024-03-26T03:45:18.523351+0000","last_peered":"2024-03-26T03:45:18.523351+0000","last_clean":"2024-03-26T03:45:18.523351+0000","last_became_active":"2024-03-26T03:44:16.288487+0000","last_became_peered":"2024-03-26T03:44:16.288487+0000","last_unstale":"2024-03-26T03:45:18.523351+0000","last_undegraded":"2024-03-26T03:45:18.523351+0000","last_fullsized":"2024-03-26T03:45:18.523351+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:16:09.492975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":911,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516947+0000","last_change":"2024-03-26T03:44:20.335072+0000","last_active":"2024-03-26T03:45:18.516947+0000","last_peered":"2024-03-26T03:45:18.516947+0000","last_clean":"2024-03-26T03:45:18.516947+0000","last_became_active":"2024-03-26T03:44:20.332784+0000","last_became_peered":"2024-03-26T03:44:20.332784+0000","last_unstale":"2024-03-26T03:45:18.516947+0000","last_undegraded":"2024-03-26T03:45:18.516947+0000","last_fullsized":"2024-03-26T03:45:18.516947+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:00:05.533230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516481+0000","last_change":"2024-03-26T03:44:22.342615+0000","last_active":"2024-03-26T03:45:18.516481+0000","last_peered":"2024-03-26T03:45:18.516481+0000","last_clean":"2024-03-26T03:45:18.516481+0000","last_became_active":"2024-03-26T03:44:22.341693+0000","last_became_peered":"2024-03-26T03:44:22.341693+0000","last_unstale":"2024-03-26T03:45:18.516481+0000","last_undegraded":"2024-03-26T03:45:18.516481+0000","last_fullsized":"2024-03-26T03:45:18.516481+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:10:59.487539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"71'2","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.521766+0000","last_change":"2024-03-26T03:44:32.022302+0000","last_active":"2024-03-26T03:45:18.521766+0000","last_peered":"2024-03-26T03:45:18.521766+0000","last_clean":"2024-03-26T03:45:18.521766+0000","last_became_active":"2024-03-26T03:44:21.310582+0000","last_became_peered":"2024-03-26T03:44:21.310582+0000","last_unstale":"2024-03-26T03:45:18.521766+0000","last_undegraded":"2024-03-26T03:45:18.521766+0000","last_fullsized":"2024-03-26T03:45:18.521766+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:20.280962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:20.280962+0000","last_clean_scrub_stamp":"2024-03-26T03:44:20.280962+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:36:49.816834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029300670000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.524018+0000","last_change":"2024-03-26T03:44:16.289675+0000","last_active":"2024-03-26T03:45:18.524018+0000","last_peered":"2024-03-26T03:45:18.524018+0000","last_clean":"2024-03-26T03:45:18.524018+0000","last_became_active":"2024-03-26T03:44:16.289167+0000","last_became_peered":"2024-03-26T03:44:16.289167+0000","last_unstale":"2024-03-26T03:45:18.524018+0000","last_undegraded":"2024-03-26T03:45:18.524018+0000","last_fullsized":"2024-03-26T03:45:18.524018+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:18:03.137915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"68'36","reported_seq":134,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:47:06.492012+0000","last_change":"2024-03-26T03:44:19.013341+0000","last_active":"2024-03-26T03:47:06.492012+0000","last_peered":"2024-03-26T03:47:06.492012+0000","last_clean":"2024-03-26T03:47:06.492012+0000","last_became_active":"2024-03-26T03:44:19.012639+0000","last_became_peered":"2024-03-26T03:44:19.012639+0000","last_unstale":"2024-03-26T03:47:06.492012+0000","last_undegraded":"2024-03-26T03:47:06.492012+0000","last_fullsized":"2024-03-26T03:47:06.492012+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:30:11.804581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"68'11","reported_seq":78,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:47:06.491997+0000","last_change":"2024-03-26T03:44:20.322594+0000","last_active":"2024-03-26T03:47:06.491997+0000","last_peered":"2024-03-26T03:47:06.491997+0000","last_clean":"2024-03-26T03:47:06.491997+0000","last_became_active":"2024-03-26T03:44:20.321578+0000","last_became_peered":"2024-03-26T03:44:20.321578+0000","last_unstale":"2024-03-26T03:47:06.491997+0000","last_undegraded":"2024-03-26T03:47:06.491997+0000","last_fullsized":"2024-03-26T03:47:06.491997+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:28:56.174543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.456340+0000","last_change":"2024-03-26T03:44:22.353904+0000","last_active":"2024-03-26T03:45:18.456340+0000","last_peered":"2024-03-26T03:45:18.456340+0000","last_clean":"2024-03-26T03:45:18.456340+0000","last_became_active":"2024-03-26T03:44:22.353455+0000","last_became_peered":"2024-03-26T03:44:22.353455+0000","last_unstale":"2024-03-26T03:45:18.456340+0000","last_undegraded":"2024-03-26T03:45:18.456340+0000","last_fullsized":"2024-03-26T03:45:18.456340+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:22:39.526810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":35,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.455421+0000","last_change":"2024-03-26T03:44:32.012604+0000","last_active":"2024-03-26T03:45:18.455421+0000","last_peered":"2024-03-26T03:45:18.455421+0000","last_clean":"2024-03-26T03:45:18.455421+0000","last_became_active":"2024-03-26T03:44:21.320994+0000","last_became_peered":"2024-03-26T03:44:21.320994+0000","last_unstale":"2024-03-26T03:45:18.455421+0000","last_undegraded":"2024-03-26T03:45:18.455421+0000","last_fullsized":"2024-03-26T03:45:18.455421+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:20.280962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:20.280962+0000","last_clean_scrub_stamp":"2024-03-26T03:44:20.280962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:33:31.555209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001946038,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"61'1","reported_seq":41,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516869+0000","last_change":"2024-03-26T03:44:16.285263+0000","last_active":"2024-03-26T03:45:18.516869+0000","last_peered":"2024-03-26T03:45:18.516869+0000","last_clean":"2024-03-26T03:45:18.516869+0000","last_became_active":"2024-03-26T03:44:16.282901+0000","last_became_peered":"2024-03-26T03:44:16.282901+0000","last_unstale":"2024-03-26T03:45:18.516869+0000","last_undegraded":"2024-03-26T03:45:18.516869+0000","last_fullsized":"2024-03-26T03:45:18.516869+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:27:33.040539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"68'21","reported_seq":76,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.456862+0000","last_change":"2024-03-26T03:44:18.301027+0000","last_active":"2024-03-26T03:45:18.456862+0000","last_peered":"2024-03-26T03:45:18.456862+0000","last_clean":"2024-03-26T03:45:18.456862+0000","last_became_active":"2024-03-26T03:44:18.299512+0000","last_became_peered":"2024-03-26T03:44:18.299512+0000","last_unstale":"2024-03-26T03:45:18.456862+0000","last_undegraded":"2024-03-26T03:45:18.456862+0000","last_fullsized":"2024-03-26T03:45:18.456862+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T05:58:12.478446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":39,"num_read_kb":27,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"68'11","reported_seq":77,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:47:06.491985+0000","last_change":"2024-03-26T03:44:20.334110+0000","last_active":"2024-03-26T03:47:06.491985+0000","last_peered":"2024-03-26T03:47:06.491985+0000","last_clean":"2024-03-26T03:47:06.491985+0000","last_became_active":"2024-03-26T03:44:20.333495+0000","last_became_peered":"2024-03-26T03:44:20.333495+0000","last_unstale":"2024-03-26T03:47:06.491985+0000","last_undegraded":"2024-03-26T03:47:06.491985+0000","last_fullsized":"2024-03-26T03:47:06.491985+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T05:06:09.792091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.522862+0000","last_change":"2024-03-26T03:44:22.322794+0000","last_active":"2024-03-26T03:45:18.522862+0000","last_peered":"2024-03-26T03:45:18.522862+0000","last_clean":"2024-03-26T03:45:18.522862+0000","last_became_active":"2024-03-26T03:44:22.321990+0000","last_became_peered":"2024-03-26T03:44:22.321990+0000","last_unstale":"2024-03-26T03:45:18.522862+0000","last_undegraded":"2024-03-26T03:45:18.522862+0000","last_fullsized":"2024-03-26T03:45:18.522862+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:07:16.553113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"72'5","reported_seq":191,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:47:13.064102+0000","last_change":"2024-03-26T03:44:32.013497+0000","last_active":"2024-03-26T03:47:13.064102+0000","last_peered":"2024-03-26T03:47:13.064102+0000","last_clean":"2024-03-26T03:47:13.064102+0000","last_became_active":"2024-03-26T03:44:21.320574+0000","last_became_peered":"2024-03-26T03:44:21.320574+0000","last_unstale":"2024-03-26T03:47:13.064102+0000","last_undegraded":"2024-03-26T03:47:13.064102+0000","last_fullsized":"2024-03-26T03:47:13.064102+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:20.280962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:20.280962+0000","last_clean_scrub_stamp":"2024-03-26T03:44:20.280962+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:46:30.807315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012228269999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":153,"num_read_kb":148,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.523091+0000","last_change":"2024-03-26T03:44:16.281630+0000","last_active":"2024-03-26T03:45:18.523091+0000","last_peered":"2024-03-26T03:45:18.523091+0000","last_clean":"2024-03-26T03:45:18.523091+0000","last_became_active":"2024-03-26T03:44:16.281164+0000","last_became_peered":"2024-03-26T03:44:16.281164+0000","last_unstale":"2024-03-26T03:45:18.523091+0000","last_undegraded":"2024-03-26T03:45:18.523091+0000","last_fullsized":"2024-03-26T03:45:18.523091+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T04:37:32.669790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"68'12","reported_seq":54,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.517178+0000","last_change":"2024-03-26T03:44:18.303063+0000","last_active":"2024-03-26T03:45:18.517178+0000","last_peered":"2024-03-26T03:45:18.517178+0000","last_clean":"2024-03-26T03:45:18.517178+0000","last_became_active":"2024-03-26T03:44:18.302311+0000","last_became_peered":"2024-03-26T03:44:18.302311+0000","last_unstale":"2024-03-26T03:45:18.517178+0000","last_undegraded":"2024-03-26T03:45:18.517178+0000","last_fullsized":"2024-03-26T03:45:18.517178+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T04:19:23.091634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.455791+0000","last_change":"2024-03-26T03:44:20.310584+0000","last_active":"2024-03-26T03:45:18.455791+0000","last_peered":"2024-03-26T03:45:18.455791+0000","last_clean":"2024-03-26T03:45:18.455791+0000","last_became_active":"2024-03-26T03:44:20.310221+0000","last_became_peered":"2024-03-26T03:44:20.310221+0000","last_unstale":"2024-03-26T03:45:18.455791+0000","last_undegraded":"2024-03-26T03:45:18.455791+0000","last_fullsized":"2024-03-26T03:45:18.455791+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:31:39.989077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.455716+0000","last_change":"2024-03-26T03:44:22.332468+0000","last_active":"2024-03-26T03:45:18.455716+0000","last_peered":"2024-03-26T03:45:18.455716+0000","last_clean":"2024-03-26T03:45:18.455716+0000","last_became_active":"2024-03-26T03:44:22.323935+0000","last_became_peered":"2024-03-26T03:44:22.323935+0000","last_unstale":"2024-03-26T03:45:18.455716+0000","last_undegraded":"2024-03-26T03:45:18.455716+0000","last_fullsized":"2024-03-26T03:45:18.455716+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T03:48:57.596725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"68'11","reported_seq":77,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:47:06.498885+0000","last_change":"2024-03-26T03:44:20.328056+0000","last_active":"2024-03-26T03:47:06.498885+0000","last_peered":"2024-03-26T03:47:06.498885+0000","last_clean":"2024-03-26T03:47:06.498885+0000","last_became_active":"2024-03-26T03:44:20.327625+0000","last_became_peered":"2024-03-26T03:44:20.327625+0000","last_unstale":"2024-03-26T03:47:06.498885+0000","last_undegraded":"2024-03-26T03:47:06.498885+0000","last_fullsized":"2024-03-26T03:47:06.498885+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:06:58.148564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.523464+0000","last_change":"2024-03-26T03:44:16.288581+0000","last_active":"2024-03-26T03:45:18.523464+0000","last_peered":"2024-03-26T03:45:18.523464+0000","last_clean":"2024-03-26T03:45:18.523464+0000","last_became_active":"2024-03-26T03:44:16.288390+0000","last_became_peered":"2024-03-26T03:44:16.288390+0000","last_unstale":"2024-03-26T03:45:18.523464+0000","last_undegraded":"2024-03-26T03:45:18.523464+0000","last_fullsized":"2024-03-26T03:45:18.523464+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:27:00.261919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"68'24","reported_seq":82,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.455494+0000","last_change":"2024-03-26T03:44:19.017777+0000","last_active":"2024-03-26T03:45:18.455494+0000","last_peered":"2024-03-26T03:45:18.455494+0000","last_clean":"2024-03-26T03:45:18.455494+0000","last_became_active":"2024-03-26T03:44:19.016777+0000","last_became_peered":"2024-03-26T03:44:19.016777+0000","last_unstale":"2024-03-26T03:45:18.455494+0000","last_undegraded":"2024-03-26T03:45:18.455494+0000","last_fullsized":"2024-03-26T03:45:18.455494+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:45:32.513405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":32,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.523646+0000","last_change":"2024-03-26T03:44:22.342790+0000","last_active":"2024-03-26T03:45:18.523646+0000","last_peered":"2024-03-26T03:45:18.523646+0000","last_clean":"2024-03-26T03:45:18.523646+0000","last_became_active":"2024-03-26T03:44:22.342014+0000","last_became_peered":"2024-03-26T03:44:22.342014+0000","last_unstale":"2024-03-26T03:45:18.523646+0000","last_undegraded":"2024-03-26T03:45:18.523646+0000","last_fullsized":"2024-03-26T03:45:18.523646+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:04:51.280399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.517147+0000","last_change":"2024-03-26T03:44:20.340671+0000","last_active":"2024-03-26T03:45:18.517147+0000","last_peered":"2024-03-26T03:45:18.517147+0000","last_clean":"2024-03-26T03:45:18.517147+0000","last_became_active":"2024-03-26T03:44:20.339960+0000","last_became_peered":"2024-03-26T03:44:20.339960+0000","last_unstale":"2024-03-26T03:45:18.517147+0000","last_undegraded":"2024-03-26T03:45:18.517147+0000","last_fullsized":"2024-03-26T03:45:18.517147+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T05:38:43.478175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":46,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.517059+0000","last_change":"2024-03-26T03:44:54.227425+0000","last_active":"2024-03-26T03:45:18.517059+0000","last_peered":"2024-03-26T03:45:18.517059+0000","last_clean":"2024-03-26T03:45:18.517059+0000","last_became_active":"2024-03-26T03:44:54.226552+0000","last_became_peered":"2024-03-26T03:44:54.226552+0000","last_unstale":"2024-03-26T03:45:18.517059+0000","last_undegraded":"2024-03-26T03:45:18.517059+0000","last_fullsized":"2024-03-26T03:45:18.517059+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T04:49:56.267326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"68'27","reported_seq":84,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.452798+0000","last_change":"2024-03-26T03:44:18.299021+0000","last_active":"2024-03-26T03:45:18.452798+0000","last_peered":"2024-03-26T03:45:18.452798+0000","last_clean":"2024-03-26T03:45:18.452798+0000","last_became_active":"2024-03-26T03:44:18.298313+0000","last_became_peered":"2024-03-26T03:44:18.298313+0000","last_unstale":"2024-03-26T03:45:18.452798+0000","last_undegraded":"2024-03-26T03:45:18.452798+0000","last_fullsized":"2024-03-26T03:45:18.452798+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T05:54:07.169738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":44,"num_read_kb":33,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.458380+0000","last_change":"2024-03-26T03:44:22.346835+0000","last_active":"2024-03-26T03:45:18.458380+0000","last_peered":"2024-03-26T03:45:18.458380+0000","last_clean":"2024-03-26T03:45:18.458380+0000","last_became_active":"2024-03-26T03:44:22.346199+0000","last_became_peered":"2024-03-26T03:44:22.346199+0000","last_unstale":"2024-03-26T03:45:18.458380+0000","last_undegraded":"2024-03-26T03:45:18.458380+0000","last_fullsized":"2024-03-26T03:45:18.458380+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:25:56.787286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516383+0000","last_change":"2024-03-26T03:44:16.284156+0000","last_active":"2024-03-26T03:45:18.516383+0000","last_peered":"2024-03-26T03:45:18.516383+0000","last_clean":"2024-03-26T03:45:18.516383+0000","last_became_active":"2024-03-26T03:44:16.281845+0000","last_became_peered":"2024-03-26T03:44:16.281845+0000","last_unstale":"2024-03-26T03:45:18.516383+0000","last_undegraded":"2024-03-26T03:45:18.516383+0000","last_fullsized":"2024-03-26T03:45:18.516383+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T04:38:36.984770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"68'15","reported_seq":63,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.455278+0000","last_change":"2024-03-26T03:44:18.301601+0000","last_active":"2024-03-26T03:45:18.455278+0000","last_peered":"2024-03-26T03:45:18.455278+0000","last_clean":"2024-03-26T03:45:18.455278+0000","last_became_active":"2024-03-26T03:44:18.300175+0000","last_became_peered":"2024-03-26T03:44:18.300175+0000","last_unstale":"2024-03-26T03:45:18.455278+0000","last_undegraded":"2024-03-26T03:45:18.455278+0000","last_fullsized":"2024-03-26T03:45:18.455278+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T05:56:12.520657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":26,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.456358+0000","last_change":"2024-03-26T03:44:20.313591+0000","last_active":"2024-03-26T03:45:18.456358+0000","last_peered":"2024-03-26T03:45:18.456358+0000","last_clean":"2024-03-26T03:45:18.456358+0000","last_became_active":"2024-03-26T03:44:20.313252+0000","last_became_peered":"2024-03-26T03:44:20.313252+0000","last_unstale":"2024-03-26T03:45:18.456358+0000","last_undegraded":"2024-03-26T03:45:18.456358+0000","last_fullsized":"2024-03-26T03:45:18.456358+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:30:59.072644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.453013+0000","last_change":"2024-03-26T03:44:22.349373+0000","last_active":"2024-03-26T03:45:18.453013+0000","last_peered":"2024-03-26T03:45:18.453013+0000","last_clean":"2024-03-26T03:45:18.453013+0000","last_became_active":"2024-03-26T03:44:22.348628+0000","last_became_peered":"2024-03-26T03:44:22.348628+0000","last_unstale":"2024-03-26T03:45:18.453013+0000","last_undegraded":"2024-03-26T03:45:18.453013+0000","last_fullsized":"2024-03-26T03:45:18.453013+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:37:58.792925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"68'11","reported_seq":77,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:47:06.498894+0000","last_change":"2024-03-26T03:44:20.327014+0000","last_active":"2024-03-26T03:47:06.498894+0000","last_peered":"2024-03-26T03:47:06.498894+0000","last_clean":"2024-03-26T03:47:06.498894+0000","last_became_active":"2024-03-26T03:44:20.326494+0000","last_became_peered":"2024-03-26T03:44:20.326494+0000","last_unstale":"2024-03-26T03:47:06.498894+0000","last_undegraded":"2024-03-26T03:47:06.498894+0000","last_fullsized":"2024-03-26T03:47:06.498894+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:50:24.943742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.522984+0000","last_change":"2024-03-26T03:44:16.279613+0000","last_active":"2024-03-26T03:45:18.522984+0000","last_peered":"2024-03-26T03:45:18.522984+0000","last_clean":"2024-03-26T03:45:18.522984+0000","last_became_active":"2024-03-26T03:44:16.279300+0000","last_became_peered":"2024-03-26T03:44:16.279300+0000","last_unstale":"2024-03-26T03:45:18.522984+0000","last_undegraded":"2024-03-26T03:45:18.522984+0000","last_fullsized":"2024-03-26T03:45:18.522984+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:26:33.600103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"68'25","reported_seq":79,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.521979+0000","last_change":"2024-03-26T03:44:18.308685+0000","last_active":"2024-03-26T03:45:18.521979+0000","last_peered":"2024-03-26T03:45:18.521979+0000","last_clean":"2024-03-26T03:45:18.521979+0000","last_became_active":"2024-03-26T03:44:18.307821+0000","last_became_peered":"2024-03-26T03:44:18.307821+0000","last_unstale":"2024-03-26T03:45:18.521979+0000","last_undegraded":"2024-03-26T03:45:18.521979+0000","last_fullsized":"2024-03-26T03:45:18.521979+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:44:54.216486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":41,"num_read_kb":29,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.523110+0000","last_change":"2024-03-26T03:44:22.321149+0000","last_active":"2024-03-26T03:45:18.523110+0000","last_peered":"2024-03-26T03:45:18.523110+0000","last_clean":"2024-03-26T03:45:18.523110+0000","last_became_active":"2024-03-26T03:44:22.320317+0000","last_became_peered":"2024-03-26T03:44:22.320317+0000","last_unstale":"2024-03-26T03:45:18.523110+0000","last_undegraded":"2024-03-26T03:45:18.523110+0000","last_fullsized":"2024-03-26T03:45:18.523110+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:51:33.516980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.450825+0000","last_change":"2024-03-26T03:44:16.269172+0000","last_active":"2024-03-26T03:45:18.450825+0000","last_peered":"2024-03-26T03:45:18.450825+0000","last_clean":"2024-03-26T03:45:18.450825+0000","last_became_active":"2024-03-26T03:44:16.267794+0000","last_became_peered":"2024-03-26T03:44:16.267794+0000","last_unstale":"2024-03-26T03:45:18.450825+0000","last_undegraded":"2024-03-26T03:45:18.450825+0000","last_fullsized":"2024-03-26T03:45:18.450825+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:31:51.866705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"68'17","reported_seq":64,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.524139+0000","last_change":"2024-03-26T03:44:18.311470+0000","last_active":"2024-03-26T03:45:18.524139+0000","last_peered":"2024-03-26T03:45:18.524139+0000","last_clean":"2024-03-26T03:45:18.524139+0000","last_became_active":"2024-03-26T03:44:18.311152+0000","last_became_peered":"2024-03-26T03:44:18.311152+0000","last_unstale":"2024-03-26T03:45:18.524139+0000","last_undegraded":"2024-03-26T03:45:18.524139+0000","last_fullsized":"2024-03-26T03:45:18.524139+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:46:30.965334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.521664+0000","last_change":"2024-03-26T03:44:20.325898+0000","last_active":"2024-03-26T03:45:18.521664+0000","last_peered":"2024-03-26T03:45:18.521664+0000","last_clean":"2024-03-26T03:45:18.521664+0000","last_became_active":"2024-03-26T03:44:20.325470+0000","last_became_peered":"2024-03-26T03:44:20.325470+0000","last_unstale":"2024-03-26T03:45:18.521664+0000","last_undegraded":"2024-03-26T03:45:18.521664+0000","last_fullsized":"2024-03-26T03:45:18.521664+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T04:32:18.072371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516136+0000","last_change":"2024-03-26T03:44:22.326248+0000","last_active":"2024-03-26T03:45:18.516136+0000","last_peered":"2024-03-26T03:45:18.516136+0000","last_clean":"2024-03-26T03:45:18.516136+0000","last_became_active":"2024-03-26T03:44:22.325062+0000","last_became_peered":"2024-03-26T03:44:22.325062+0000","last_unstale":"2024-03-26T03:45:18.516136+0000","last_undegraded":"2024-03-26T03:45:18.516136+0000","last_fullsized":"2024-03-26T03:45:18.516136+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:29:28.977976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516813+0000","last_change":"2024-03-26T03:44:16.286186+0000","last_active":"2024-03-26T03:45:18.516813+0000","last_peered":"2024-03-26T03:45:18.516813+0000","last_clean":"2024-03-26T03:45:18.516813+0000","last_became_active":"2024-03-26T03:44:16.283812+0000","last_became_peered":"2024-03-26T03:44:16.283812+0000","last_unstale":"2024-03-26T03:45:18.516813+0000","last_undegraded":"2024-03-26T03:45:18.516813+0000","last_fullsized":"2024-03-26T03:45:18.516813+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:17:43.468208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"68'10","reported_seq":51,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.522641+0000","last_change":"2024-03-26T03:44:18.306687+0000","last_active":"2024-03-26T03:45:18.522641+0000","last_peered":"2024-03-26T03:45:18.522641+0000","last_clean":"2024-03-26T03:45:18.522641+0000","last_became_active":"2024-03-26T03:44:18.306356+0000","last_became_peered":"2024-03-26T03:44:18.306356+0000","last_unstale":"2024-03-26T03:45:18.522641+0000","last_undegraded":"2024-03-26T03:45:18.522641+0000","last_fullsized":"2024-03-26T03:45:18.522641+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T04:22:32.044623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516386+0000","last_change":"2024-03-26T03:44:20.333134+0000","last_active":"2024-03-26T03:45:18.516386+0000","last_peered":"2024-03-26T03:45:18.516386+0000","last_clean":"2024-03-26T03:45:18.516386+0000","last_became_active":"2024-03-26T03:44:20.332218+0000","last_became_peered":"2024-03-26T03:44:20.332218+0000","last_unstale":"2024-03-26T03:45:18.516386+0000","last_undegraded":"2024-03-26T03:45:18.516386+0000","last_fullsized":"2024-03-26T03:45:18.516386+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:06:27.081828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.522173+0000","last_change":"2024-03-26T03:44:22.344707+0000","last_active":"2024-03-26T03:45:18.522173+0000","last_peered":"2024-03-26T03:45:18.522173+0000","last_clean":"2024-03-26T03:45:18.522173+0000","last_became_active":"2024-03-26T03:44:22.343429+0000","last_became_peered":"2024-03-26T03:44:22.343429+0000","last_unstale":"2024-03-26T03:45:18.522173+0000","last_undegraded":"2024-03-26T03:45:18.522173+0000","last_fullsized":"2024-03-26T03:45:18.522173+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T04:59:57.540509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.452681+0000","last_change":"2024-03-26T03:44:16.278056+0000","last_active":"2024-03-26T03:45:18.452681+0000","last_peered":"2024-03-26T03:45:18.452681+0000","last_clean":"2024-03-26T03:45:18.452681+0000","last_became_active":"2024-03-26T03:44:16.277225+0000","last_became_peered":"2024-03-26T03:44:16.277225+0000","last_unstale":"2024-03-26T03:45:18.452681+0000","last_undegraded":"2024-03-26T03:45:18.452681+0000","last_fullsized":"2024-03-26T03:45:18.452681+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:29:38.976272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"68'15","reported_seq":61,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.522397+0000","last_change":"2024-03-26T03:44:18.300839+0000","last_active":"2024-03-26T03:45:18.522397+0000","last_peered":"2024-03-26T03:45:18.522397+0000","last_clean":"2024-03-26T03:45:18.522397+0000","last_became_active":"2024-03-26T03:44:18.300559+0000","last_became_peered":"2024-03-26T03:44:18.300559+0000","last_unstale":"2024-03-26T03:45:18.522397+0000","last_undegraded":"2024-03-26T03:45:18.522397+0000","last_fullsized":"2024-03-26T03:45:18.522397+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:38:24.454193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.521223+0000","last_change":"2024-03-26T03:44:20.310647+0000","last_active":"2024-03-26T03:45:18.521223+0000","last_peered":"2024-03-26T03:45:18.521223+0000","last_clean":"2024-03-26T03:45:18.521223+0000","last_became_active":"2024-03-26T03:44:20.310294+0000","last_became_peered":"2024-03-26T03:44:20.310294+0000","last_unstale":"2024-03-26T03:45:18.521223+0000","last_undegraded":"2024-03-26T03:45:18.521223+0000","last_fullsized":"2024-03-26T03:45:18.521223+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:07:21.204615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":383,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":153,"num_read_kb":148,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":9,"acting":9,"num_store_stats":7},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88,"ondisk_log_size":88,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":210,"num_object_clones":0,"num_object_copies":630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":210,"num_whiteouts":0,"num_read":989,"num_read_kb":714,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":592,"ondisk_log_size":592,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1531,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201326,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28444,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716708,"statfs":{"total":95995035648,"available":95965908992,"internally_reserved":0,"allocated":1462272,"data_stored":949055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462968,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28412,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716740,"statfs":{"total":95995035648,"available":95965941760,"internally_reserved":0,"allocated":1429504,"data_stored":946794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574959,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32012,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713140,"statfs":{"total":95995035648,"available":95962255360,"internally_reserved":0,"allocated":851968,"data_stored":358019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738489,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28436,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716716,"statfs":{"total":95995035648,"available":95965917184,"internally_reserved":0,"allocated":1454080,"data_stored":948420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411428,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":835584,"data_stored":356867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182488,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":835584,"data_stored":356807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018957,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":876544,"data_stored":358158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724612,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":835584,"data_stored":356776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T03:47:18.066 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph pg dump --format=json 2024-03-26T03:47:18.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:17 smithi190 bash[23392]: cluster 2024-03-26T03:47:16.524000+0000 mgr.y (mgr.24431) 106 : cluster [DBG] pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:18.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:17 smithi190 bash[23392]: audit 2024-03-26T03:47:17.256143+0000 mgr.y (mgr.24431) 107 : audit [DBG] from='client.14712 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T03:47:18.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:17 smithi175 bash[29284]: cluster 2024-03-26T03:47:16.524000+0000 mgr.y (mgr.24431) 106 : cluster [DBG] pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:18.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:17 smithi175 bash[29284]: audit 2024-03-26T03:47:17.256143+0000 mgr.y (mgr.24431) 107 : audit [DBG] from='client.14712 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T03:47:18.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:17 smithi175 bash[21527]: cluster 2024-03-26T03:47:16.524000+0000 mgr.y (mgr.24431) 106 : cluster [DBG] pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:18.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:17 smithi175 bash[21527]: audit 2024-03-26T03:47:17.256143+0000 mgr.y (mgr.24431) 107 : audit [DBG] from='client.14712 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T03:47:19.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:18 smithi190 bash[23392]: audit 2024-03-26T03:47:18.795090+0000 mon.c (mon.2) 79 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:47:19.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:18 smithi175 bash[21527]: audit 2024-03-26T03:47:18.795090+0000 mon.c (mon.2) 79 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:47:19.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:47:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:47:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:47:19.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:18 smithi175 bash[29284]: audit 2024-03-26T03:47:18.795090+0000 mon.c (mon.2) 79 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:47:19.947 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:47:19 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:47:20.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:19 smithi190 bash[23392]: cluster 2024-03-26T03:47:18.524703+0000 mgr.y (mgr.24431) 108 : cluster [DBG] pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:20.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:19 smithi175 bash[29284]: cluster 2024-03-26T03:47:18.524703+0000 mgr.y (mgr.24431) 108 : cluster [DBG] pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:20.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:19 smithi175 bash[21527]: cluster 2024-03-26T03:47:18.524703+0000 mgr.y (mgr.24431) 108 : cluster [DBG] pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:21.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:20 smithi190 bash[23392]: audit 2024-03-26T03:47:19.592996+0000 mgr.y (mgr.24431) 109 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:47:21.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:20 smithi175 bash[21527]: audit 2024-03-26T03:47:19.592996+0000 mgr.y (mgr.24431) 109 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:47:21.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:20 smithi175 bash[29284]: audit 2024-03-26T03:47:19.592996+0000 mgr.y (mgr.24431) 109 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:47:22.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:21 smithi190 bash[23392]: cluster 2024-03-26T03:47:20.526116+0000 mgr.y (mgr.24431) 110 : cluster [DBG] pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:22.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:21 smithi175 bash[21527]: cluster 2024-03-26T03:47:20.526116+0000 mgr.y (mgr.24431) 110 : cluster [DBG] pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:22.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:21 smithi175 bash[29284]: cluster 2024-03-26T03:47:20.526116+0000 mgr.y (mgr.24431) 110 : cluster [DBG] pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:22.850 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:47:23.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:23 smithi175 bash[29284]: cluster 2024-03-26T03:47:22.526818+0000 mgr.y (mgr.24431) 111 : cluster [DBG] pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:23.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:23 smithi175 bash[21527]: cluster 2024-03-26T03:47:22.526818+0000 mgr.y (mgr.24431) 111 : cluster [DBG] pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:24.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:23 smithi190 bash[23392]: cluster 2024-03-26T03:47:22.526818+0000 mgr.y (mgr.24431) 111 : cluster [DBG] pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:25.545 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.540+0000 7fa6d631a700 1 -- 172.21.15.175:0/2599309558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6d0104870 msgr2=0x7fa6d0104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:25.545 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.540+0000 7fa6d631a700 1 --2- 172.21.15.175:0/2599309558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6d0104870 0x7fa6d0104cf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0009990 tx=0x7fa6c0009ca0 comp rx=0 tx=0).stop 2024-03-26T03:47:25.545 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.540+0000 7fa6d631a700 1 -- 172.21.15.175:0/2599309558 shutdown_connections 2024-03-26T03:47:25.545 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.540+0000 7fa6d631a700 1 --2- 172.21.15.175:0/2599309558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6d0105230 0x7fa6d0111740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:25.545 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.540+0000 7fa6d631a700 1 --2- 172.21.15.175:0/2599309558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6d010a870 0x7fa6d010ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:25.545 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.540+0000 7fa6d631a700 1 --2- 172.21.15.175:0/2599309558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6d0104870 0x7fa6d0104cf0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:25.545 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.540+0000 7fa6d631a700 1 -- 172.21.15.175:0/2599309558 >> 172.21.15.175:0/2599309558 conn(0x7fa6d0100360 msgr2=0x7fa6d0102780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:47:25.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.540+0000 7fa6d631a700 1 -- 172.21.15.175:0/2599309558 shutdown_connections 2024-03-26T03:47:25.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.540+0000 7fa6d631a700 1 -- 172.21.15.175:0/2599309558 wait complete. 2024-03-26T03:47:25.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.540+0000 7fa6d631a700 1 Processor -- start 2024-03-26T03:47:25.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.540+0000 7fa6d631a700 1 -- start start 2024-03-26T03:47:25.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.540+0000 7fa6d631a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6d0104870 0x7fa6d019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:25.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.540+0000 7fa6d631a700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6d0105230 0x7fa6d019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:25.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.540+0000 7fa6d631a700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6d010a870 0x7fa6d019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:25.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.540+0000 7fa6d631a700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d01140d0 con 0x7fa6d010a870 2024-03-26T03:47:25.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.540+0000 7fa6d631a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d0113f50 con 0x7fa6d0104870 2024-03-26T03:47:25.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.540+0000 7fa6d631a700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa6d0114250 con 0x7fa6d0105230 2024-03-26T03:47:25.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.540+0000 7fa6cf7fe700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6d0105230 0x7fa6d019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:25.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.540+0000 7fa6cf7fe700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6d0105230 0x7fa6d019aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:38208/0 (socket says 172.21.15.175:38208) 2024-03-26T03:47:25.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.540+0000 7fa6cf7fe700 1 -- 172.21.15.175:0/3223729842 learned_addr learned my addr 172.21.15.175:0/3223729842 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:47:25.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.540+0000 7fa6d48b7700 1 --2- 172.21.15.175:0/3223729842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6d010a870 0x7fa6d019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:25.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.540+0000 7fa6cffff700 1 --2- 172.21.15.175:0/3223729842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6d0104870 0x7fa6d019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:25.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.540+0000 7fa6cf7fe700 1 -- 172.21.15.175:0/3223729842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6d0104870 msgr2=0x7fa6d019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:25.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.540+0000 7fa6cf7fe700 1 --2- 172.21.15.175:0/3223729842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6d0104870 0x7fa6d019a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:25.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.540+0000 7fa6cf7fe700 1 -- 172.21.15.175:0/3223729842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6d010a870 msgr2=0x7fa6d019ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:25.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.540+0000 7fa6cf7fe700 1 --2- 172.21.15.175:0/3223729842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6d010a870 0x7fa6d019ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:25.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.540+0000 7fa6cf7fe700 1 -- 172.21.15.175:0/3223729842 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d019f570 con 0x7fa6d0105230 2024-03-26T03:47:25.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.540+0000 7fa6cf7fe700 1 --2- 172.21.15.175:0/3223729842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6d0105230 0x7fa6d019aaa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0000c00 tx=0x7fa6c00047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:47:25.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.544+0000 7fa6cd7fa700 1 -- 172.21.15.175:0/3223729842 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0027070 con 0x7fa6d0105230 2024-03-26T03:47:25.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.544+0000 7fa6cd7fa700 1 -- 172.21.15.175:0/3223729842 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fa6c0022470 con 0x7fa6d0105230 2024-03-26T03:47:25.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.544+0000 7fa6d631a700 1 -- 172.21.15.175:0/3223729842 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d019f800 con 0x7fa6d0105230 2024-03-26T03:47:25.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.544+0000 7fa6cd7fa700 1 -- 172.21.15.175:0/3223729842 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0017650 con 0x7fa6d0105230 2024-03-26T03:47:25.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.544+0000 7fa6d631a700 1 -- 172.21.15.175:0/3223729842 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d01a70e0 con 0x7fa6d0105230 2024-03-26T03:47:25.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.544+0000 7fa6d631a700 1 -- 172.21.15.175:0/3223729842 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6d0105fb0 con 0x7fa6d0105230 2024-03-26T03:47:25.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.544+0000 7fa6cd7fa700 1 -- 172.21.15.175:0/3223729842 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7fa6c002a070 con 0x7fa6d0105230 2024-03-26T03:47:25.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.544+0000 7fa6cd7fa700 1 --2- 172.21.15.175:0/3223729842 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fa6bc06e7e0 0x7fa6bc070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:25.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.544+0000 7fa6cd7fa700 1 -- 172.21.15.175:0/3223729842 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(75..75 src has 1..75) ==== 7537+0+0 (secure 0 0 0) 0x7fa6c008ffc0 con 0x7fa6d0105230 2024-03-26T03:47:25.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.544+0000 7fa6cffff700 1 --2- 172.21.15.175:0/3223729842 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fa6bc06e7e0 0x7fa6bc070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:25.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.544+0000 7fa6cffff700 1 --2- 172.21.15.175:0/3223729842 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fa6bc06e7e0 0x7fa6bc070ca0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8005fd0 tx=0x7fa6b8005db0 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:47:25.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.548+0000 7fa6cd7fa700 1 -- 172.21.15.175:0/3223729842 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7fa6c0013070 con 0x7fa6d0105230 2024-03-26T03:47:25.754 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.752+0000 7fa6d631a700 1 -- 172.21.15.175:0/3223729842 --> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa6d0050300 con 0x7fa6bc06e7e0 2024-03-26T03:47:25.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.768+0000 7fa6cd7fa700 1 -- 172.21.15.175:0/3223729842 <== mgr.24431 v2:172.21.15.175:6800/1140903233 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+346530 (secure 0 0 0) 0x7fa6d0050300 con 0x7fa6bc06e7e0 2024-03-26T03:47:25.771 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-26T03:47:25.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.772+0000 7fa6d631a700 1 -- 172.21.15.175:0/3223729842 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fa6bc06e7e0 msgr2=0x7fa6bc070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:25.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.772+0000 7fa6d631a700 1 --2- 172.21.15.175:0/3223729842 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fa6bc06e7e0 0x7fa6bc070ca0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8005fd0 tx=0x7fa6b8005db0 comp rx=0 tx=0).stop 2024-03-26T03:47:25.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.772+0000 7fa6d631a700 1 -- 172.21.15.175:0/3223729842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6d0105230 msgr2=0x7fa6d019aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:25.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.772+0000 7fa6d631a700 1 --2- 172.21.15.175:0/3223729842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6d0105230 0x7fa6d019aaa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0000c00 tx=0x7fa6c00047b0 comp rx=0 tx=0).stop 2024-03-26T03:47:25.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.772+0000 7fa6d631a700 1 -- 172.21.15.175:0/3223729842 shutdown_connections 2024-03-26T03:47:25.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.776+0000 7fa6d631a700 1 --2- 172.21.15.175:0/3223729842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6d010a870 0x7fa6d019ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:25.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.776+0000 7fa6d631a700 1 --2- 172.21.15.175:0/3223729842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6d0104870 0x7fa6d019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:25.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.776+0000 7fa6d631a700 1 --2- 172.21.15.175:0/3223729842 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fa6bc06e7e0 0x7fa6bc070ca0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:25.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.776+0000 7fa6d631a700 1 --2- 172.21.15.175:0/3223729842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6d0105230 0x7fa6d019aaa0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:25.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.776+0000 7fa6d631a700 1 -- 172.21.15.175:0/3223729842 >> 172.21.15.175:0/3223729842 conn(0x7fa6d0100360 msgr2=0x7fa6d0100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:47:25.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.776+0000 7fa6d631a700 1 -- 172.21.15.175:0/3223729842 shutdown_connections 2024-03-26T03:47:25.777 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:25.776+0000 7fa6d631a700 1 -- 172.21.15.175:0/3223729842 wait complete. 2024-03-26T03:47:25.777 INFO:teuthology.orchestra.run.smithi175.stderr:dumped all 2024-03-26T03:47:26.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:25 smithi190 bash[23392]: cluster 2024-03-26T03:47:24.528323+0000 mgr.y (mgr.24431) 112 : cluster [DBG] pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:26.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:25 smithi175 bash[21527]: cluster 2024-03-26T03:47:24.528323+0000 mgr.y (mgr.24431) 112 : cluster [DBG] pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:26.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:25 smithi175 bash[29284]: cluster 2024-03-26T03:47:24.528323+0000 mgr.y (mgr.24431) 112 : cluster [DBG] pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:26.557 INFO:teuthology.orchestra.run.smithi175.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-03-26T03:47:24.527121+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596392,"num_objects":230,"num_object_clones":0,"num_object_copies":690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":0,"num_read":1401,"num_read_kb":1215,"num_write":807,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":823,"ondisk_log_size":823,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":237008,"kb_used_data":8380,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":749724208,"statfs":{"total":767960285184,"available":767717588992,"internally_reserved":0,"allocated":8581120,"data_stored":4630896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005580"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.454446+0000","last_change":"2024-03-26T03:44:22.338146+0000","last_active":"2024-03-26T03:45:18.454446+0000","last_peered":"2024-03-26T03:45:18.454446+0000","last_clean":"2024-03-26T03:45:18.454446+0000","last_became_active":"2024-03-26T03:44:22.336866+0000","last_became_peered":"2024-03-26T03:44:22.336866+0000","last_unstale":"2024-03-26T03:45:18.454446+0000","last_undegraded":"2024-03-26T03:45:18.454446+0000","last_fullsized":"2024-03-26T03:45:18.454446+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:55:45.204177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.453136+0000","last_change":"2024-03-26T03:44:20.335088+0000","last_active":"2024-03-26T03:45:18.453136+0000","last_peered":"2024-03-26T03:45:18.453136+0000","last_clean":"2024-03-26T03:45:18.453136+0000","last_became_active":"2024-03-26T03:44:20.334476+0000","last_became_peered":"2024-03-26T03:44:20.334476+0000","last_unstale":"2024-03-26T03:45:18.453136+0000","last_undegraded":"2024-03-26T03:45:18.453136+0000","last_fullsized":"2024-03-26T03:45:18.453136+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:34:55.744404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"61'1","reported_seq":48,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.517319+0000","last_change":"2024-03-26T03:44:16.272258+0000","last_active":"2024-03-26T03:45:18.517319+0000","last_peered":"2024-03-26T03:45:18.517319+0000","last_clean":"2024-03-26T03:45:18.517319+0000","last_became_active":"2024-03-26T03:44:16.270381+0000","last_became_peered":"2024-03-26T03:44:16.270381+0000","last_unstale":"2024-03-26T03:45:18.517319+0000","last_undegraded":"2024-03-26T03:45:18.517319+0000","last_fullsized":"2024-03-26T03:45:18.517319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:22:13.789751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"68'16","reported_seq":62,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.456478+0000","last_change":"2024-03-26T03:44:19.013489+0000","last_active":"2024-03-26T03:45:18.456478+0000","last_peered":"2024-03-26T03:45:18.456478+0000","last_clean":"2024-03-26T03:45:18.456478+0000","last_became_active":"2024-03-26T03:44:19.012559+0000","last_became_peered":"2024-03-26T03:44:19.012559+0000","last_unstale":"2024-03-26T03:45:18.456478+0000","last_undegraded":"2024-03-26T03:45:18.456478+0000","last_fullsized":"2024-03-26T03:45:18.456478+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:10:50.905039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.455666+0000","last_change":"2024-03-26T03:44:22.342750+0000","last_active":"2024-03-26T03:45:18.455666+0000","last_peered":"2024-03-26T03:45:18.455666+0000","last_clean":"2024-03-26T03:45:18.455666+0000","last_became_active":"2024-03-26T03:44:22.341960+0000","last_became_peered":"2024-03-26T03:44:22.341960+0000","last_unstale":"2024-03-26T03:45:18.455666+0000","last_undegraded":"2024-03-26T03:45:18.455666+0000","last_fullsized":"2024-03-26T03:45:18.455666+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:13:09.847546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.455357+0000","last_change":"2024-03-26T03:44:20.326832+0000","last_active":"2024-03-26T03:45:18.455357+0000","last_peered":"2024-03-26T03:45:18.455357+0000","last_clean":"2024-03-26T03:45:18.455357+0000","last_became_active":"2024-03-26T03:44:20.326497+0000","last_became_peered":"2024-03-26T03:44:20.326497+0000","last_unstale":"2024-03-26T03:45:18.455357+0000","last_undegraded":"2024-03-26T03:45:18.455357+0000","last_fullsized":"2024-03-26T03:45:18.455357+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:22:54.583875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.456242+0000","last_change":"2024-03-26T03:44:16.287994+0000","last_active":"2024-03-26T03:45:18.456242+0000","last_peered":"2024-03-26T03:45:18.456242+0000","last_clean":"2024-03-26T03:45:18.456242+0000","last_became_active":"2024-03-26T03:44:16.287505+0000","last_became_peered":"2024-03-26T03:44:16.287505+0000","last_unstale":"2024-03-26T03:45:18.456242+0000","last_undegraded":"2024-03-26T03:45:18.456242+0000","last_fullsized":"2024-03-26T03:45:18.456242+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:34:38.566037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"68'17","reported_seq":67,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.517128+0000","last_change":"2024-03-26T03:44:19.018727+0000","last_active":"2024-03-26T03:45:18.517128+0000","last_peered":"2024-03-26T03:45:18.517128+0000","last_clean":"2024-03-26T03:45:18.517128+0000","last_became_active":"2024-03-26T03:44:19.017370+0000","last_became_peered":"2024-03-26T03:44:19.017370+0000","last_unstale":"2024-03-26T03:45:18.517128+0000","last_undegraded":"2024-03-26T03:45:18.517128+0000","last_fullsized":"2024-03-26T03:45:18.517128+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:25:05.655395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":29,"num_read_kb":21,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.521511+0000","last_change":"2024-03-26T03:44:22.325358+0000","last_active":"2024-03-26T03:45:18.521511+0000","last_peered":"2024-03-26T03:45:18.521511+0000","last_clean":"2024-03-26T03:45:18.521511+0000","last_became_active":"2024-03-26T03:44:22.324426+0000","last_became_peered":"2024-03-26T03:44:22.324426+0000","last_unstale":"2024-03-26T03:45:18.521511+0000","last_undegraded":"2024-03-26T03:45:18.521511+0000","last_fullsized":"2024-03-26T03:45:18.521511+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:26:08.476795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.456463+0000","last_change":"2024-03-26T03:44:20.336269+0000","last_active":"2024-03-26T03:45:18.456463+0000","last_peered":"2024-03-26T03:45:18.456463+0000","last_clean":"2024-03-26T03:45:18.456463+0000","last_became_active":"2024-03-26T03:44:20.335949+0000","last_became_peered":"2024-03-26T03:44:20.335949+0000","last_unstale":"2024-03-26T03:45:18.456463+0000","last_undegraded":"2024-03-26T03:45:18.456463+0000","last_fullsized":"2024-03-26T03:45:18.456463+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:33:19.079896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.524056+0000","last_change":"2024-03-26T03:44:16.289648+0000","last_active":"2024-03-26T03:45:18.524056+0000","last_peered":"2024-03-26T03:45:18.524056+0000","last_clean":"2024-03-26T03:45:18.524056+0000","last_became_active":"2024-03-26T03:44:16.289164+0000","last_became_peered":"2024-03-26T03:44:16.289164+0000","last_unstale":"2024-03-26T03:45:18.524056+0000","last_undegraded":"2024-03-26T03:45:18.524056+0000","last_fullsized":"2024-03-26T03:45:18.524056+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T05:40:46.986870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"68'15","reported_seq":61,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.523864+0000","last_change":"2024-03-26T03:44:18.299144+0000","last_active":"2024-03-26T03:45:18.523864+0000","last_peered":"2024-03-26T03:45:18.523864+0000","last_clean":"2024-03-26T03:45:18.523864+0000","last_became_active":"2024-03-26T03:44:18.298663+0000","last_became_peered":"2024-03-26T03:44:18.298663+0000","last_unstale":"2024-03-26T03:45:18.523864+0000","last_undegraded":"2024-03-26T03:45:18.523864+0000","last_fullsized":"2024-03-26T03:45:18.523864+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T04:00:42.524427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516056+0000","last_change":"2024-03-26T03:44:22.345208+0000","last_active":"2024-03-26T03:45:18.516056+0000","last_peered":"2024-03-26T03:45:18.516056+0000","last_clean":"2024-03-26T03:45:18.516056+0000","last_became_active":"2024-03-26T03:44:22.344467+0000","last_became_peered":"2024-03-26T03:44:22.344467+0000","last_unstale":"2024-03-26T03:45:18.516056+0000","last_undegraded":"2024-03-26T03:45:18.516056+0000","last_fullsized":"2024-03-26T03:45:18.516056+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:09:31.902007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.456905+0000","last_change":"2024-03-26T03:44:20.334321+0000","last_active":"2024-03-26T03:45:18.456905+0000","last_peered":"2024-03-26T03:45:18.456905+0000","last_clean":"2024-03-26T03:45:18.456905+0000","last_became_active":"2024-03-26T03:44:20.333957+0000","last_became_peered":"2024-03-26T03:44:20.333957+0000","last_unstale":"2024-03-26T03:45:18.456905+0000","last_undegraded":"2024-03-26T03:45:18.456905+0000","last_fullsized":"2024-03-26T03:45:18.456905+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:57:51.564301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.522478+0000","last_change":"2024-03-26T03:44:16.279125+0000","last_active":"2024-03-26T03:45:18.522478+0000","last_peered":"2024-03-26T03:45:18.522478+0000","last_clean":"2024-03-26T03:45:18.522478+0000","last_became_active":"2024-03-26T03:44:16.278739+0000","last_became_peered":"2024-03-26T03:44:16.278739+0000","last_unstale":"2024-03-26T03:45:18.522478+0000","last_undegraded":"2024-03-26T03:45:18.522478+0000","last_fullsized":"2024-03-26T03:45:18.522478+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T04:09:02.885420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"68'18","reported_seq":71,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.523311+0000","last_change":"2024-03-26T03:44:18.300417+0000","last_active":"2024-03-26T03:45:18.523311+0000","last_peered":"2024-03-26T03:45:18.523311+0000","last_clean":"2024-03-26T03:45:18.523311+0000","last_became_active":"2024-03-26T03:44:18.300098+0000","last_became_peered":"2024-03-26T03:44:18.300098+0000","last_unstale":"2024-03-26T03:45:18.523311+0000","last_undegraded":"2024-03-26T03:45:18.523311+0000","last_fullsized":"2024-03-26T03:45:18.523311+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:24:01.922756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.456990+0000","last_change":"2024-03-26T03:44:22.330778+0000","last_active":"2024-03-26T03:45:18.456990+0000","last_peered":"2024-03-26T03:45:18.456990+0000","last_clean":"2024-03-26T03:45:18.456990+0000","last_became_active":"2024-03-26T03:44:22.328134+0000","last_became_peered":"2024-03-26T03:44:22.328134+0000","last_unstale":"2024-03-26T03:45:18.456990+0000","last_undegraded":"2024-03-26T03:45:18.456990+0000","last_fullsized":"2024-03-26T03:45:18.456990+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:42:54.583750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.456019+0000","last_change":"2024-03-26T03:44:54.230091+0000","last_active":"2024-03-26T03:45:18.456019+0000","last_peered":"2024-03-26T03:45:18.456019+0000","last_clean":"2024-03-26T03:45:18.456019+0000","last_became_active":"2024-03-26T03:44:54.229613+0000","last_became_peered":"2024-03-26T03:44:54.229613+0000","last_unstale":"2024-03-26T03:45:18.456019+0000","last_undegraded":"2024-03-26T03:45:18.456019+0000","last_fullsized":"2024-03-26T03:45:18.456019+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T05:02:08.657168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.455001+0000","last_change":"2024-03-26T03:44:16.272846+0000","last_active":"2024-03-26T03:45:18.455001+0000","last_peered":"2024-03-26T03:45:18.455001+0000","last_clean":"2024-03-26T03:45:18.455001+0000","last_became_active":"2024-03-26T03:44:16.272358+0000","last_became_peered":"2024-03-26T03:44:16.272358+0000","last_unstale":"2024-03-26T03:45:18.455001+0000","last_undegraded":"2024-03-26T03:45:18.455001+0000","last_fullsized":"2024-03-26T03:45:18.455001+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:47:16.998660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"68'21","reported_seq":75,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.455966+0000","last_change":"2024-03-26T03:44:19.019867+0000","last_active":"2024-03-26T03:45:18.455966+0000","last_peered":"2024-03-26T03:45:18.455966+0000","last_clean":"2024-03-26T03:45:18.455966+0000","last_became_active":"2024-03-26T03:44:19.016823+0000","last_became_peered":"2024-03-26T03:44:19.016823+0000","last_unstale":"2024-03-26T03:45:18.455966+0000","last_undegraded":"2024-03-26T03:45:18.455966+0000","last_fullsized":"2024-03-26T03:45:18.455966+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:57:04.191719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.458127+0000","last_change":"2024-03-26T03:44:22.332867+0000","last_active":"2024-03-26T03:45:18.458127+0000","last_peered":"2024-03-26T03:45:18.458127+0000","last_clean":"2024-03-26T03:45:18.458127+0000","last_became_active":"2024-03-26T03:44:22.331837+0000","last_became_peered":"2024-03-26T03:44:22.331837+0000","last_unstale":"2024-03-26T03:45:18.458127+0000","last_undegraded":"2024-03-26T03:45:18.458127+0000","last_fullsized":"2024-03-26T03:45:18.458127+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:34:43.858175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.450968+0000","last_change":"2024-03-26T03:44:20.325006+0000","last_active":"2024-03-26T03:45:18.450968+0000","last_peered":"2024-03-26T03:45:18.450968+0000","last_clean":"2024-03-26T03:45:18.450968+0000","last_became_active":"2024-03-26T03:44:20.324098+0000","last_became_peered":"2024-03-26T03:44:20.324098+0000","last_unstale":"2024-03-26T03:45:18.450968+0000","last_undegraded":"2024-03-26T03:45:18.450968+0000","last_fullsized":"2024-03-26T03:45:18.450968+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T04:07:48.295994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.521873+0000","last_change":"2024-03-26T03:44:16.275277+0000","last_active":"2024-03-26T03:45:18.521873+0000","last_peered":"2024-03-26T03:45:18.521873+0000","last_clean":"2024-03-26T03:45:18.521873+0000","last_became_active":"2024-03-26T03:44:16.274798+0000","last_became_peered":"2024-03-26T03:44:16.274798+0000","last_unstale":"2024-03-26T03:45:18.521873+0000","last_undegraded":"2024-03-26T03:45:18.521873+0000","last_fullsized":"2024-03-26T03:45:18.521873+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T05:23:28.829957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"68'11","reported_seq":58,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516670+0000","last_change":"2024-03-26T03:44:18.307189+0000","last_active":"2024-03-26T03:45:18.516670+0000","last_peered":"2024-03-26T03:45:18.516670+0000","last_clean":"2024-03-26T03:45:18.516670+0000","last_became_active":"2024-03-26T03:44:18.302665+0000","last_became_peered":"2024-03-26T03:44:18.302665+0000","last_unstale":"2024-03-26T03:45:18.516670+0000","last_undegraded":"2024-03-26T03:45:18.516670+0000","last_fullsized":"2024-03-26T03:45:18.516670+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:59:21.617776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":15,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516668+0000","last_change":"2024-03-26T03:44:22.346745+0000","last_active":"2024-03-26T03:45:18.516668+0000","last_peered":"2024-03-26T03:45:18.516668+0000","last_clean":"2024-03-26T03:45:18.516668+0000","last_became_active":"2024-03-26T03:44:22.345901+0000","last_became_peered":"2024-03-26T03:44:22.345901+0000","last_unstale":"2024-03-26T03:45:18.516668+0000","last_undegraded":"2024-03-26T03:45:18.516668+0000","last_fullsized":"2024-03-26T03:45:18.516668+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T05:18:00.556703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1f","version":"68'11","reported_seq":83,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:47:16.507045+0000","last_change":"2024-03-26T03:44:20.326524+0000","last_active":"2024-03-26T03:47:16.507045+0000","last_peered":"2024-03-26T03:47:16.507045+0000","last_clean":"2024-03-26T03:47:16.507045+0000","last_became_active":"2024-03-26T03:44:20.325453+0000","last_became_peered":"2024-03-26T03:44:20.325453+0000","last_unstale":"2024-03-26T03:47:16.507045+0000","last_undegraded":"2024-03-26T03:47:16.507045+0000","last_fullsized":"2024-03-26T03:47:16.507045+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:54:38.703763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"61'1","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.453285+0000","last_change":"2024-03-26T03:44:16.275315+0000","last_active":"2024-03-26T03:45:18.453285+0000","last_peered":"2024-03-26T03:45:18.453285+0000","last_clean":"2024-03-26T03:45:18.453285+0000","last_became_active":"2024-03-26T03:44:16.270628+0000","last_became_peered":"2024-03-26T03:44:16.270628+0000","last_unstale":"2024-03-26T03:45:18.453285+0000","last_undegraded":"2024-03-26T03:45:18.453285+0000","last_fullsized":"2024-03-26T03:45:18.453285+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:16:12.845364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"68'15","reported_seq":63,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.452608+0000","last_change":"2024-03-26T03:44:18.299981+0000","last_active":"2024-03-26T03:45:18.452608+0000","last_peered":"2024-03-26T03:45:18.452608+0000","last_clean":"2024-03-26T03:45:18.452608+0000","last_became_active":"2024-03-26T03:44:18.299360+0000","last_became_peered":"2024-03-26T03:44:18.299360+0000","last_unstale":"2024-03-26T03:45:18.452608+0000","last_undegraded":"2024-03-26T03:45:18.452608+0000","last_fullsized":"2024-03-26T03:45:18.452608+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:20:48.240896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":26,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"68'1","reported_seq":30,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.522239+0000","last_change":"2024-03-26T03:44:22.337569+0000","last_active":"2024-03-26T03:45:18.522239+0000","last_peered":"2024-03-26T03:45:18.522239+0000","last_clean":"2024-03-26T03:45:18.522239+0000","last_became_active":"2024-03-26T03:44:22.334116+0000","last_became_peered":"2024-03-26T03:44:22.334116+0000","last_unstale":"2024-03-26T03:45:18.522239+0000","last_undegraded":"2024-03-26T03:45:18.522239+0000","last_fullsized":"2024-03-26T03:45:18.522239+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T05:09:37.347925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":370,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.517082+0000","last_change":"2024-03-26T03:44:20.336266+0000","last_active":"2024-03-26T03:45:18.517082+0000","last_peered":"2024-03-26T03:45:18.517082+0000","last_clean":"2024-03-26T03:45:18.517082+0000","last_became_active":"2024-03-26T03:44:20.333977+0000","last_became_peered":"2024-03-26T03:44:20.333977+0000","last_unstale":"2024-03-26T03:45:18.517082+0000","last_undegraded":"2024-03-26T03:45:18.517082+0000","last_fullsized":"2024-03-26T03:45:18.517082+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:56:13.373173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.521324+0000","last_change":"2024-03-26T03:44:16.270706+0000","last_active":"2024-03-26T03:45:18.521324+0000","last_peered":"2024-03-26T03:45:18.521324+0000","last_clean":"2024-03-26T03:45:18.521324+0000","last_became_active":"2024-03-26T03:44:16.270241+0000","last_became_peered":"2024-03-26T03:44:16.270241+0000","last_unstale":"2024-03-26T03:45:18.521324+0000","last_undegraded":"2024-03-26T03:45:18.521324+0000","last_fullsized":"2024-03-26T03:45:18.521324+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T03:53:12.241914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"68'39","reported_seq":109,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516738+0000","last_change":"2024-03-26T03:44:18.294448+0000","last_active":"2024-03-26T03:45:18.516738+0000","last_peered":"2024-03-26T03:45:18.516738+0000","last_clean":"2024-03-26T03:45:18.516738+0000","last_became_active":"2024-03-26T03:44:18.293402+0000","last_became_peered":"2024-03-26T03:44:18.293402+0000","last_unstale":"2024-03-26T03:45:18.516738+0000","last_undegraded":"2024-03-26T03:45:18.516738+0000","last_fullsized":"2024-03-26T03:45:18.516738+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:31:07.948096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":62,"num_read_kb":49,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.455809+0000","last_change":"2024-03-26T03:44:22.331526+0000","last_active":"2024-03-26T03:45:18.455809+0000","last_peered":"2024-03-26T03:45:18.455809+0000","last_clean":"2024-03-26T03:45:18.455809+0000","last_became_active":"2024-03-26T03:44:22.329577+0000","last_became_peered":"2024-03-26T03:44:22.329577+0000","last_unstale":"2024-03-26T03:45:18.455809+0000","last_undegraded":"2024-03-26T03:45:18.455809+0000","last_fullsized":"2024-03-26T03:45:18.455809+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:07:43.723662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.453955+0000","last_change":"2024-03-26T03:44:20.309417+0000","last_active":"2024-03-26T03:45:18.453955+0000","last_peered":"2024-03-26T03:45:18.453955+0000","last_clean":"2024-03-26T03:45:18.453955+0000","last_became_active":"2024-03-26T03:44:20.308558+0000","last_became_peered":"2024-03-26T03:44:20.308558+0000","last_unstale":"2024-03-26T03:45:18.453955+0000","last_undegraded":"2024-03-26T03:45:18.453955+0000","last_fullsized":"2024-03-26T03:45:18.453955+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T05:24:24.768168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.523189+0000","last_change":"2024-03-26T03:44:16.282348+0000","last_active":"2024-03-26T03:45:18.523189+0000","last_peered":"2024-03-26T03:45:18.523189+0000","last_clean":"2024-03-26T03:45:18.523189+0000","last_became_active":"2024-03-26T03:44:16.282081+0000","last_became_peered":"2024-03-26T03:44:16.282081+0000","last_unstale":"2024-03-26T03:45:18.523189+0000","last_undegraded":"2024-03-26T03:45:18.523189+0000","last_fullsized":"2024-03-26T03:45:18.523189+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:35:09.555003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"68'22","reported_seq":77,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.521891+0000","last_change":"2024-03-26T03:44:18.306611+0000","last_active":"2024-03-26T03:45:18.521891+0000","last_peered":"2024-03-26T03:45:18.521891+0000","last_clean":"2024-03-26T03:45:18.521891+0000","last_became_active":"2024-03-26T03:44:18.306237+0000","last_became_peered":"2024-03-26T03:44:18.306237+0000","last_unstale":"2024-03-26T03:45:18.521891+0000","last_undegraded":"2024-03-26T03:45:18.521891+0000","last_fullsized":"2024-03-26T03:45:18.521891+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T05:59:47.316294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.521398+0000","last_change":"2024-03-26T03:44:22.334585+0000","last_active":"2024-03-26T03:45:18.521398+0000","last_peered":"2024-03-26T03:45:18.521398+0000","last_clean":"2024-03-26T03:45:18.521398+0000","last_became_active":"2024-03-26T03:44:22.333208+0000","last_became_peered":"2024-03-26T03:44:22.333208+0000","last_unstale":"2024-03-26T03:45:18.521398+0000","last_undegraded":"2024-03-26T03:45:18.521398+0000","last_fullsized":"2024-03-26T03:45:18.521398+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:26:52.325903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.452820+0000","last_change":"2024-03-26T03:44:20.327091+0000","last_active":"2024-03-26T03:45:18.452820+0000","last_peered":"2024-03-26T03:45:18.452820+0000","last_clean":"2024-03-26T03:45:18.452820+0000","last_became_active":"2024-03-26T03:44:20.326360+0000","last_became_peered":"2024-03-26T03:44:20.326360+0000","last_unstale":"2024-03-26T03:45:18.452820+0000","last_undegraded":"2024-03-26T03:45:18.452820+0000","last_fullsized":"2024-03-26T03:45:18.452820+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:26:54.380294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.521787+0000","last_change":"2024-03-26T03:44:16.286716+0000","last_active":"2024-03-26T03:45:18.521787+0000","last_peered":"2024-03-26T03:45:18.521787+0000","last_clean":"2024-03-26T03:45:18.521787+0000","last_became_active":"2024-03-26T03:44:16.286436+0000","last_became_peered":"2024-03-26T03:44:16.286436+0000","last_unstale":"2024-03-26T03:45:18.521787+0000","last_undegraded":"2024-03-26T03:45:18.521787+0000","last_fullsized":"2024-03-26T03:45:18.521787+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:25:37.685854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"68'12","reported_seq":57,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.517010+0000","last_change":"2024-03-26T03:44:18.300971+0000","last_active":"2024-03-26T03:45:18.517010+0000","last_peered":"2024-03-26T03:45:18.517010+0000","last_clean":"2024-03-26T03:45:18.517010+0000","last_became_active":"2024-03-26T03:44:18.299992+0000","last_became_peered":"2024-03-26T03:44:18.299992+0000","last_unstale":"2024-03-26T03:45:18.517010+0000","last_undegraded":"2024-03-26T03:45:18.517010+0000","last_fullsized":"2024-03-26T03:45:18.517010+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:28:04.014349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":18,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.452927+0000","last_change":"2024-03-26T03:44:22.330528+0000","last_active":"2024-03-26T03:45:18.452927+0000","last_peered":"2024-03-26T03:45:18.452927+0000","last_clean":"2024-03-26T03:45:18.452927+0000","last_became_active":"2024-03-26T03:44:22.328066+0000","last_became_peered":"2024-03-26T03:44:22.328066+0000","last_unstale":"2024-03-26T03:45:18.452927+0000","last_undegraded":"2024-03-26T03:45:18.452927+0000","last_fullsized":"2024-03-26T03:45:18.452927+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:28:18.405902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.462694+0000","last_change":"2024-03-26T03:44:20.336209+0000","last_active":"2024-03-26T03:45:18.462694+0000","last_peered":"2024-03-26T03:45:18.462694+0000","last_clean":"2024-03-26T03:45:18.462694+0000","last_became_active":"2024-03-26T03:44:20.335886+0000","last_became_peered":"2024-03-26T03:44:20.335886+0000","last_unstale":"2024-03-26T03:45:18.462694+0000","last_undegraded":"2024-03-26T03:45:18.462694+0000","last_fullsized":"2024-03-26T03:45:18.462694+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:53:46.045429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.517253+0000","last_change":"2024-03-26T03:44:16.291755+0000","last_active":"2024-03-26T03:45:18.517253+0000","last_peered":"2024-03-26T03:45:18.517253+0000","last_clean":"2024-03-26T03:45:18.517253+0000","last_became_active":"2024-03-26T03:44:16.291278+0000","last_became_peered":"2024-03-26T03:44:16.291278+0000","last_unstale":"2024-03-26T03:45:18.517253+0000","last_undegraded":"2024-03-26T03:45:18.517253+0000","last_fullsized":"2024-03-26T03:45:18.517253+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:29:26.307443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"68'22","reported_seq":74,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.456619+0000","last_change":"2024-03-26T03:44:18.303537+0000","last_active":"2024-03-26T03:45:18.456619+0000","last_peered":"2024-03-26T03:45:18.456619+0000","last_clean":"2024-03-26T03:45:18.456619+0000","last_became_active":"2024-03-26T03:44:18.303090+0000","last_became_peered":"2024-03-26T03:44:18.303090+0000","last_unstale":"2024-03-26T03:45:18.456619+0000","last_undegraded":"2024-03-26T03:45:18.456619+0000","last_fullsized":"2024-03-26T03:45:18.456619+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:15:38.958251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":33,"num_read_kb":26,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516080+0000","last_change":"2024-03-26T03:44:22.345312+0000","last_active":"2024-03-26T03:45:18.516080+0000","last_peered":"2024-03-26T03:45:18.516080+0000","last_clean":"2024-03-26T03:45:18.516080+0000","last_became_active":"2024-03-26T03:44:22.343501+0000","last_became_peered":"2024-03-26T03:44:22.343501+0000","last_unstale":"2024-03-26T03:45:18.516080+0000","last_undegraded":"2024-03-26T03:45:18.516080+0000","last_fullsized":"2024-03-26T03:45:18.516080+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:25:08.571989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.521623+0000","last_change":"2024-03-26T03:44:54.367176+0000","last_active":"2024-03-26T03:45:18.521623+0000","last_peered":"2024-03-26T03:45:18.521623+0000","last_clean":"2024-03-26T03:45:18.521623+0000","last_became_active":"2024-03-26T03:44:54.366318+0000","last_became_peered":"2024-03-26T03:44:54.366318+0000","last_unstale":"2024-03-26T03:45:18.521623+0000","last_undegraded":"2024-03-26T03:45:18.521623+0000","last_fullsized":"2024-03-26T03:45:18.521623+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:41:46.663430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.522469+0000","last_change":"2024-03-26T03:44:16.276071+0000","last_active":"2024-03-26T03:45:18.522469+0000","last_peered":"2024-03-26T03:45:18.522469+0000","last_clean":"2024-03-26T03:45:18.522469+0000","last_became_active":"2024-03-26T03:44:16.275777+0000","last_became_peered":"2024-03-26T03:44:16.275777+0000","last_unstale":"2024-03-26T03:45:18.522469+0000","last_undegraded":"2024-03-26T03:45:18.522469+0000","last_fullsized":"2024-03-26T03:45:18.522469+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:48:56.798635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"68'15","reported_seq":64,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.524202+0000","last_change":"2024-03-26T03:44:18.304098+0000","last_active":"2024-03-26T03:45:18.524202+0000","last_peered":"2024-03-26T03:45:18.524202+0000","last_clean":"2024-03-26T03:45:18.524202+0000","last_became_active":"2024-03-26T03:44:18.303805+0000","last_became_peered":"2024-03-26T03:44:18.303805+0000","last_unstale":"2024-03-26T03:45:18.524202+0000","last_undegraded":"2024-03-26T03:45:18.524202+0000","last_fullsized":"2024-03-26T03:45:18.524202+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T05:30:20.868582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":26,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.462331+0000","last_change":"2024-03-26T03:44:22.345201+0000","last_active":"2024-03-26T03:45:18.462331+0000","last_peered":"2024-03-26T03:45:18.462331+0000","last_clean":"2024-03-26T03:45:18.462331+0000","last_became_active":"2024-03-26T03:44:22.344643+0000","last_became_peered":"2024-03-26T03:44:22.344643+0000","last_unstale":"2024-03-26T03:45:18.462331+0000","last_undegraded":"2024-03-26T03:45:18.462331+0000","last_fullsized":"2024-03-26T03:45:18.462331+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:28:57.773904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"68'11","reported_seq":83,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:47:16.507361+0000","last_change":"2024-03-26T03:44:20.315194+0000","last_active":"2024-03-26T03:47:16.507361+0000","last_peered":"2024-03-26T03:47:16.507361+0000","last_clean":"2024-03-26T03:47:16.507361+0000","last_became_active":"2024-03-26T03:44:20.314772+0000","last_became_peered":"2024-03-26T03:44:20.314772+0000","last_unstale":"2024-03-26T03:47:16.507361+0000","last_undegraded":"2024-03-26T03:47:16.507361+0000","last_fullsized":"2024-03-26T03:47:16.507361+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:13:53.715741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516502+0000","last_change":"2024-03-26T03:44:16.273273+0000","last_active":"2024-03-26T03:45:18.516502+0000","last_peered":"2024-03-26T03:45:18.516502+0000","last_clean":"2024-03-26T03:45:18.516502+0000","last_became_active":"2024-03-26T03:44:16.272418+0000","last_became_peered":"2024-03-26T03:44:16.272418+0000","last_unstale":"2024-03-26T03:45:18.516502+0000","last_undegraded":"2024-03-26T03:45:18.516502+0000","last_fullsized":"2024-03-26T03:45:18.516502+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:00:36.966560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"68'9","reported_seq":52,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516571+0000","last_change":"2024-03-26T03:44:18.302502+0000","last_active":"2024-03-26T03:45:18.516571+0000","last_peered":"2024-03-26T03:45:18.516571+0000","last_clean":"2024-03-26T03:45:18.516571+0000","last_became_active":"2024-03-26T03:44:18.301521+0000","last_became_peered":"2024-03-26T03:44:18.301521+0000","last_unstale":"2024-03-26T03:45:18.516571+0000","last_undegraded":"2024-03-26T03:45:18.516571+0000","last_fullsized":"2024-03-26T03:45:18.516571+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:52:56.388079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.16","version":"68'1","reported_seq":29,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516845+0000","last_change":"2024-03-26T03:44:22.341037+0000","last_active":"2024-03-26T03:45:18.516845+0000","last_peered":"2024-03-26T03:45:18.516845+0000","last_clean":"2024-03-26T03:45:18.516845+0000","last_became_active":"2024-03-26T03:44:22.337576+0000","last_became_peered":"2024-03-26T03:44:22.337576+0000","last_unstale":"2024-03-26T03:45:18.516845+0000","last_undegraded":"2024-03-26T03:45:18.516845+0000","last_fullsized":"2024-03-26T03:45:18.516845+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:12:01.087443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"68'11","reported_seq":79,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:47:16.501139+0000","last_change":"2024-03-26T03:44:20.324528+0000","last_active":"2024-03-26T03:47:16.501139+0000","last_peered":"2024-03-26T03:47:16.501139+0000","last_clean":"2024-03-26T03:47:16.501139+0000","last_became_active":"2024-03-26T03:44:20.322642+0000","last_became_peered":"2024-03-26T03:44:20.322642+0000","last_unstale":"2024-03-26T03:47:16.501139+0000","last_undegraded":"2024-03-26T03:47:16.501139+0000","last_fullsized":"2024-03-26T03:47:16.501139+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:50:29.666007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.521846+0000","last_change":"2024-03-26T03:44:16.275480+0000","last_active":"2024-03-26T03:45:18.521846+0000","last_peered":"2024-03-26T03:45:18.521846+0000","last_clean":"2024-03-26T03:45:18.521846+0000","last_became_active":"2024-03-26T03:44:16.275108+0000","last_became_peered":"2024-03-26T03:44:16.275108+0000","last_unstale":"2024-03-26T03:45:18.521846+0000","last_undegraded":"2024-03-26T03:45:18.521846+0000","last_fullsized":"2024-03-26T03:45:18.521846+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:12:16.770696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"68'11","reported_seq":55,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.523092+0000","last_change":"2024-03-26T03:44:19.018499+0000","last_active":"2024-03-26T03:45:18.523092+0000","last_peered":"2024-03-26T03:45:18.523092+0000","last_clean":"2024-03-26T03:45:18.523092+0000","last_became_active":"2024-03-26T03:44:19.016703+0000","last_became_peered":"2024-03-26T03:44:19.016703+0000","last_unstale":"2024-03-26T03:45:18.523092+0000","last_undegraded":"2024-03-26T03:45:18.523092+0000","last_fullsized":"2024-03-26T03:45:18.523092+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:12:12.881881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.523701+0000","last_change":"2024-03-26T03:44:22.336554+0000","last_active":"2024-03-26T03:45:18.523701+0000","last_peered":"2024-03-26T03:45:18.523701+0000","last_clean":"2024-03-26T03:45:18.523701+0000","last_became_active":"2024-03-26T03:44:22.335615+0000","last_became_peered":"2024-03-26T03:44:22.335615+0000","last_unstale":"2024-03-26T03:45:18.523701+0000","last_undegraded":"2024-03-26T03:45:18.523701+0000","last_fullsized":"2024-03-26T03:45:18.523701+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:48:46.055161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.457036+0000","last_change":"2024-03-26T03:44:20.331738+0000","last_active":"2024-03-26T03:45:18.457036+0000","last_peered":"2024-03-26T03:45:18.457036+0000","last_clean":"2024-03-26T03:45:18.457036+0000","last_became_active":"2024-03-26T03:44:20.330112+0000","last_became_peered":"2024-03-26T03:44:20.330112+0000","last_unstale":"2024-03-26T03:45:18.457036+0000","last_undegraded":"2024-03-26T03:45:18.457036+0000","last_fullsized":"2024-03-26T03:45:18.457036+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:32:02.214724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.523660+0000","last_change":"2024-03-26T03:44:16.295711+0000","last_active":"2024-03-26T03:45:18.523660+0000","last_peered":"2024-03-26T03:45:18.523660+0000","last_clean":"2024-03-26T03:45:18.523660+0000","last_became_active":"2024-03-26T03:44:16.295341+0000","last_became_peered":"2024-03-26T03:44:16.295341+0000","last_unstale":"2024-03-26T03:45:18.523660+0000","last_undegraded":"2024-03-26T03:45:18.523660+0000","last_fullsized":"2024-03-26T03:45:18.523660+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:32:47.007119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"68'4","reported_seq":42,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.522401+0000","last_change":"2024-03-26T03:44:18.303208+0000","last_active":"2024-03-26T03:45:18.522401+0000","last_peered":"2024-03-26T03:45:18.522401+0000","last_clean":"2024-03-26T03:45:18.522401+0000","last_became_active":"2024-03-26T03:44:18.302934+0000","last_became_peered":"2024-03-26T03:44:18.302934+0000","last_unstale":"2024-03-26T03:45:18.522401+0000","last_undegraded":"2024-03-26T03:45:18.522401+0000","last_fullsized":"2024-03-26T03:45:18.522401+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T04:51:12.374238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.452196+0000","last_change":"2024-03-26T03:44:22.332841+0000","last_active":"2024-03-26T03:45:18.452196+0000","last_peered":"2024-03-26T03:45:18.452196+0000","last_clean":"2024-03-26T03:45:18.452196+0000","last_became_active":"2024-03-26T03:44:22.329041+0000","last_became_peered":"2024-03-26T03:44:22.329041+0000","last_unstale":"2024-03-26T03:45:18.452196+0000","last_undegraded":"2024-03-26T03:45:18.452196+0000","last_fullsized":"2024-03-26T03:45:18.452196+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:05:26.957647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516737+0000","last_change":"2024-03-26T03:44:20.342958+0000","last_active":"2024-03-26T03:45:18.516737+0000","last_peered":"2024-03-26T03:45:18.516737+0000","last_clean":"2024-03-26T03:45:18.516737+0000","last_became_active":"2024-03-26T03:44:20.339202+0000","last_became_peered":"2024-03-26T03:44:20.339202+0000","last_unstale":"2024-03-26T03:45:18.516737+0000","last_undegraded":"2024-03-26T03:45:18.516737+0000","last_fullsized":"2024-03-26T03:45:18.516737+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:33:52.804681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.450294+0000","last_change":"2024-03-26T03:44:16.282930+0000","last_active":"2024-03-26T03:45:18.450294+0000","last_peered":"2024-03-26T03:45:18.450294+0000","last_clean":"2024-03-26T03:45:18.450294+0000","last_became_active":"2024-03-26T03:44:16.282446+0000","last_became_peered":"2024-03-26T03:44:16.282446+0000","last_unstale":"2024-03-26T03:45:18.450294+0000","last_undegraded":"2024-03-26T03:45:18.450294+0000","last_fullsized":"2024-03-26T03:45:18.450294+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:57:27.460578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"68'17","reported_seq":67,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.521760+0000","last_change":"2024-03-26T03:44:18.310536+0000","last_active":"2024-03-26T03:45:18.521760+0000","last_peered":"2024-03-26T03:45:18.521760+0000","last_clean":"2024-03-26T03:45:18.521760+0000","last_became_active":"2024-03-26T03:44:18.310232+0000","last_became_peered":"2024-03-26T03:44:18.310232+0000","last_unstale":"2024-03-26T03:45:18.521760+0000","last_undegraded":"2024-03-26T03:45:18.521760+0000","last_fullsized":"2024-03-26T03:45:18.521760+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:15:46.155308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":29,"num_read_kb":21,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.455583+0000","last_change":"2024-03-26T03:44:22.342056+0000","last_active":"2024-03-26T03:45:18.455583+0000","last_peered":"2024-03-26T03:45:18.455583+0000","last_clean":"2024-03-26T03:45:18.455583+0000","last_became_active":"2024-03-26T03:44:22.341147+0000","last_became_peered":"2024-03-26T03:44:22.341147+0000","last_unstale":"2024-03-26T03:45:18.455583+0000","last_undegraded":"2024-03-26T03:45:18.455583+0000","last_fullsized":"2024-03-26T03:45:18.455583+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:22:23.943685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"61'2","reported_seq":55,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.459281+0000","last_change":"2024-03-26T03:44:16.274598+0000","last_active":"2024-03-26T03:45:18.459281+0000","last_peered":"2024-03-26T03:45:18.459281+0000","last_clean":"2024-03-26T03:45:18.459281+0000","last_became_active":"2024-03-26T03:44:16.274227+0000","last_became_peered":"2024-03-26T03:44:16.274227+0000","last_unstale":"2024-03-26T03:45:18.459281+0000","last_undegraded":"2024-03-26T03:45:18.459281+0000","last_fullsized":"2024-03-26T03:45:18.459281+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T04:02:56.201913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"68'17","reported_seq":67,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.523290+0000","last_change":"2024-03-26T03:44:18.300703+0000","last_active":"2024-03-26T03:45:18.523290+0000","last_peered":"2024-03-26T03:45:18.523290+0000","last_clean":"2024-03-26T03:45:18.523290+0000","last_became_active":"2024-03-26T03:44:18.300172+0000","last_became_peered":"2024-03-26T03:44:18.300172+0000","last_unstale":"2024-03-26T03:45:18.523290+0000","last_undegraded":"2024-03-26T03:45:18.523290+0000","last_fullsized":"2024-03-26T03:45:18.523290+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:34:14.174703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":29,"num_read_kb":21,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"68'11","reported_seq":79,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:47:16.507550+0000","last_change":"2024-03-26T03:44:20.327412+0000","last_active":"2024-03-26T03:47:16.507550+0000","last_peered":"2024-03-26T03:47:16.507550+0000","last_clean":"2024-03-26T03:47:16.507550+0000","last_became_active":"2024-03-26T03:44:20.326865+0000","last_became_peered":"2024-03-26T03:44:20.326865+0000","last_unstale":"2024-03-26T03:47:16.507550+0000","last_undegraded":"2024-03-26T03:47:16.507550+0000","last_fullsized":"2024-03-26T03:47:16.507550+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:10:11.113954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516163+0000","last_change":"2024-03-26T03:44:22.335472+0000","last_active":"2024-03-26T03:45:18.516163+0000","last_peered":"2024-03-26T03:45:18.516163+0000","last_clean":"2024-03-26T03:45:18.516163+0000","last_became_active":"2024-03-26T03:44:22.333780+0000","last_became_peered":"2024-03-26T03:44:22.333780+0000","last_unstale":"2024-03-26T03:45:18.516163+0000","last_undegraded":"2024-03-26T03:45:18.516163+0000","last_fullsized":"2024-03-26T03:45:18.516163+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:18:51.806269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.517549+0000","last_change":"2024-03-26T03:44:16.292414+0000","last_active":"2024-03-26T03:45:18.517549+0000","last_peered":"2024-03-26T03:45:18.517549+0000","last_clean":"2024-03-26T03:45:18.517549+0000","last_became_active":"2024-03-26T03:44:16.291660+0000","last_became_peered":"2024-03-26T03:44:16.291660+0000","last_unstale":"2024-03-26T03:45:18.517549+0000","last_undegraded":"2024-03-26T03:45:18.517549+0000","last_fullsized":"2024-03-26T03:45:18.517549+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:38:18.552119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"68'16","reported_seq":96,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:47:16.507441+0000","last_change":"2024-03-26T03:44:19.017721+0000","last_active":"2024-03-26T03:47:16.507441+0000","last_peered":"2024-03-26T03:47:16.507441+0000","last_clean":"2024-03-26T03:47:16.507441+0000","last_became_active":"2024-03-26T03:44:19.016758+0000","last_became_peered":"2024-03-26T03:44:19.016758+0000","last_unstale":"2024-03-26T03:47:16.507441+0000","last_undegraded":"2024-03-26T03:47:16.507441+0000","last_fullsized":"2024-03-26T03:47:16.507441+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:19:02.572817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.451509+0000","last_change":"2024-03-26T03:44:54.228617+0000","last_active":"2024-03-26T03:45:18.451509+0000","last_peered":"2024-03-26T03:45:18.451509+0000","last_clean":"2024-03-26T03:45:18.451509+0000","last_became_active":"2024-03-26T03:44:54.228073+0000","last_became_peered":"2024-03-26T03:44:54.228073+0000","last_unstale":"2024-03-26T03:45:18.451509+0000","last_undegraded":"2024-03-26T03:45:18.451509+0000","last_fullsized":"2024-03-26T03:45:18.451509+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:48:43.797931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.456331+0000","last_change":"2024-03-26T03:44:22.351278+0000","last_active":"2024-03-26T03:45:18.456331+0000","last_peered":"2024-03-26T03:45:18.456331+0000","last_clean":"2024-03-26T03:45:18.456331+0000","last_became_active":"2024-03-26T03:44:22.350716+0000","last_became_peered":"2024-03-26T03:44:22.350716+0000","last_unstale":"2024-03-26T03:45:18.456331+0000","last_undegraded":"2024-03-26T03:45:18.456331+0000","last_fullsized":"2024-03-26T03:45:18.456331+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:21:38.950129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.521942+0000","last_change":"2024-03-26T03:44:16.286765+0000","last_active":"2024-03-26T03:45:18.521942+0000","last_peered":"2024-03-26T03:45:18.521942+0000","last_clean":"2024-03-26T03:45:18.521942+0000","last_became_active":"2024-03-26T03:44:16.286370+0000","last_became_peered":"2024-03-26T03:44:16.286370+0000","last_unstale":"2024-03-26T03:45:18.521942+0000","last_undegraded":"2024-03-26T03:45:18.521942+0000","last_fullsized":"2024-03-26T03:45:18.521942+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:24:41.103950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"68'27","reported_seq":85,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.462515+0000","last_change":"2024-03-26T03:44:18.313370+0000","last_active":"2024-03-26T03:45:18.462515+0000","last_peered":"2024-03-26T03:45:18.462515+0000","last_clean":"2024-03-26T03:45:18.462515+0000","last_became_active":"2024-03-26T03:44:18.312850+0000","last_became_peered":"2024-03-26T03:44:18.312850+0000","last_unstale":"2024-03-26T03:45:18.462515+0000","last_undegraded":"2024-03-26T03:45:18.462515+0000","last_fullsized":"2024-03-26T03:45:18.462515+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T03:50:35.499073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.517354+0000","last_change":"2024-03-26T03:44:20.341293+0000","last_active":"2024-03-26T03:45:18.517354+0000","last_peered":"2024-03-26T03:45:18.517354+0000","last_clean":"2024-03-26T03:45:18.517354+0000","last_became_active":"2024-03-26T03:44:20.340439+0000","last_became_peered":"2024-03-26T03:44:20.340439+0000","last_unstale":"2024-03-26T03:45:18.517354+0000","last_undegraded":"2024-03-26T03:45:18.517354+0000","last_fullsized":"2024-03-26T03:45:18.517354+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:32:57.309782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.523836+0000","last_change":"2024-03-26T03:44:22.346981+0000","last_active":"2024-03-26T03:45:18.523836+0000","last_peered":"2024-03-26T03:45:18.523836+0000","last_clean":"2024-03-26T03:45:18.523836+0000","last_became_active":"2024-03-26T03:44:22.346271+0000","last_became_peered":"2024-03-26T03:44:22.346271+0000","last_unstale":"2024-03-26T03:45:18.523836+0000","last_undegraded":"2024-03-26T03:45:18.523836+0000","last_fullsized":"2024-03-26T03:45:18.523836+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:13:33.700704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"68'30","reported_seq":92,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.517104+0000","last_change":"2024-03-26T03:44:19.013703+0000","last_active":"2024-03-26T03:45:18.517104+0000","last_peered":"2024-03-26T03:45:18.517104+0000","last_clean":"2024-03-26T03:45:18.517104+0000","last_became_active":"2024-03-26T03:44:19.012601+0000","last_became_peered":"2024-03-26T03:44:19.012601+0000","last_unstale":"2024-03-26T03:45:18.517104+0000","last_undegraded":"2024-03-26T03:45:18.517104+0000","last_fullsized":"2024-03-26T03:45:18.517104+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:52:15.198255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":52,"num_read_kb":38,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.450630+0000","last_change":"2024-03-26T03:44:16.279025+0000","last_active":"2024-03-26T03:45:18.450630+0000","last_peered":"2024-03-26T03:45:18.450630+0000","last_clean":"2024-03-26T03:45:18.450630+0000","last_became_active":"2024-03-26T03:44:16.278503+0000","last_became_peered":"2024-03-26T03:44:16.278503+0000","last_unstale":"2024-03-26T03:45:18.450630+0000","last_undegraded":"2024-03-26T03:45:18.450630+0000","last_fullsized":"2024-03-26T03:45:18.450630+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:04:38.695284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.517207+0000","last_change":"2024-03-26T03:44:20.345993+0000","last_active":"2024-03-26T03:45:18.517207+0000","last_peered":"2024-03-26T03:45:18.517207+0000","last_clean":"2024-03-26T03:45:18.517207+0000","last_became_active":"2024-03-26T03:44:20.345481+0000","last_became_peered":"2024-03-26T03:44:20.345481+0000","last_unstale":"2024-03-26T03:45:18.517207+0000","last_undegraded":"2024-03-26T03:45:18.517207+0000","last_fullsized":"2024-03-26T03:45:18.517207+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T05:52:21.574020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.523932+0000","last_change":"2024-03-26T03:44:22.339804+0000","last_active":"2024-03-26T03:45:18.523932+0000","last_peered":"2024-03-26T03:45:18.523932+0000","last_clean":"2024-03-26T03:45:18.523932+0000","last_became_active":"2024-03-26T03:44:22.339031+0000","last_became_peered":"2024-03-26T03:44:22.339031+0000","last_unstale":"2024-03-26T03:45:18.523932+0000","last_undegraded":"2024-03-26T03:45:18.523932+0000","last_fullsized":"2024-03-26T03:45:18.523932+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:16:37.844926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"68'16","reported_seq":62,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.454914+0000","last_change":"2024-03-26T03:44:18.310573+0000","last_active":"2024-03-26T03:45:18.454914+0000","last_peered":"2024-03-26T03:45:18.454914+0000","last_clean":"2024-03-26T03:45:18.454914+0000","last_became_active":"2024-03-26T03:44:18.309925+0000","last_became_peered":"2024-03-26T03:44:18.309925+0000","last_unstale":"2024-03-26T03:45:18.454914+0000","last_undegraded":"2024-03-26T03:45:18.454914+0000","last_fullsized":"2024-03-26T03:45:18.454914+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:16:50.453724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.521578+0000","last_change":"2024-03-26T03:44:16.286853+0000","last_active":"2024-03-26T03:45:18.521578+0000","last_peered":"2024-03-26T03:45:18.521578+0000","last_clean":"2024-03-26T03:45:18.521578+0000","last_became_active":"2024-03-26T03:44:16.286474+0000","last_became_peered":"2024-03-26T03:44:16.286474+0000","last_unstale":"2024-03-26T03:45:18.521578+0000","last_undegraded":"2024-03-26T03:45:18.521578+0000","last_fullsized":"2024-03-26T03:45:18.521578+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:51:15.008303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":280,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:20.636665+0000","last_change":"2024-03-26T03:43:38.041937+0000","last_active":"2024-03-26T03:45:20.636665+0000","last_peered":"2024-03-26T03:45:20.636665+0000","last_clean":"2024-03-26T03:45:20.636665+0000","last_became_active":"2024-03-26T03:43:38.041066+0000","last_became_peered":"2024-03-26T03:43:38.041066+0000","last_unstale":"2024-03-26T03:45:20.636665+0000","last_undegraded":"2024-03-26T03:45:20.636665+0000","last_fullsized":"2024-03-26T03:45:20.636665+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:39:09.344827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:39:09.344827+0000","last_clean_scrub_stamp":"2024-03-26T03:39:09.344827+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:31:47.546805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.523270+0000","last_change":"2024-03-26T03:44:20.313558+0000","last_active":"2024-03-26T03:45:18.523270+0000","last_peered":"2024-03-26T03:45:18.523270+0000","last_clean":"2024-03-26T03:45:18.523270+0000","last_became_active":"2024-03-26T03:44:20.313183+0000","last_became_peered":"2024-03-26T03:44:20.313183+0000","last_unstale":"2024-03-26T03:45:18.523270+0000","last_undegraded":"2024-03-26T03:45:18.523270+0000","last_fullsized":"2024-03-26T03:45:18.523270+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:39:31.962065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.517266+0000","last_change":"2024-03-26T03:44:22.344155+0000","last_active":"2024-03-26T03:45:18.517266+0000","last_peered":"2024-03-26T03:45:18.517266+0000","last_clean":"2024-03-26T03:45:18.517266+0000","last_became_active":"2024-03-26T03:44:22.343297+0000","last_became_peered":"2024-03-26T03:44:22.343297+0000","last_unstale":"2024-03-26T03:45:18.517266+0000","last_undegraded":"2024-03-26T03:45:18.517266+0000","last_fullsized":"2024-03-26T03:45:18.517266+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:36:56.757960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"68'20","reported_seq":69,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516338+0000","last_change":"2024-03-26T03:44:18.307439+0000","last_active":"2024-03-26T03:45:18.516338+0000","last_peered":"2024-03-26T03:45:18.516338+0000","last_clean":"2024-03-26T03:45:18.516338+0000","last_became_active":"2024-03-26T03:44:18.306771+0000","last_became_peered":"2024-03-26T03:44:18.306771+0000","last_unstale":"2024-03-26T03:45:18.516338+0000","last_undegraded":"2024-03-26T03:45:18.516338+0000","last_fullsized":"2024-03-26T03:45:18.516338+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:37:31.727137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":30,"num_read_kb":22,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"61'1","reported_seq":48,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.523351+0000","last_change":"2024-03-26T03:44:16.288851+0000","last_active":"2024-03-26T03:45:18.523351+0000","last_peered":"2024-03-26T03:45:18.523351+0000","last_clean":"2024-03-26T03:45:18.523351+0000","last_became_active":"2024-03-26T03:44:16.288487+0000","last_became_peered":"2024-03-26T03:44:16.288487+0000","last_unstale":"2024-03-26T03:45:18.523351+0000","last_undegraded":"2024-03-26T03:45:18.523351+0000","last_fullsized":"2024-03-26T03:45:18.523351+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:16:09.492975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":911,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516947+0000","last_change":"2024-03-26T03:44:20.335072+0000","last_active":"2024-03-26T03:45:18.516947+0000","last_peered":"2024-03-26T03:45:18.516947+0000","last_clean":"2024-03-26T03:45:18.516947+0000","last_became_active":"2024-03-26T03:44:20.332784+0000","last_became_peered":"2024-03-26T03:44:20.332784+0000","last_unstale":"2024-03-26T03:45:18.516947+0000","last_undegraded":"2024-03-26T03:45:18.516947+0000","last_fullsized":"2024-03-26T03:45:18.516947+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:00:05.533230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516481+0000","last_change":"2024-03-26T03:44:22.342615+0000","last_active":"2024-03-26T03:45:18.516481+0000","last_peered":"2024-03-26T03:45:18.516481+0000","last_clean":"2024-03-26T03:45:18.516481+0000","last_became_active":"2024-03-26T03:44:22.341693+0000","last_became_peered":"2024-03-26T03:44:22.341693+0000","last_unstale":"2024-03-26T03:45:18.516481+0000","last_undegraded":"2024-03-26T03:45:18.516481+0000","last_fullsized":"2024-03-26T03:45:18.516481+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:10:59.487539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"71'2","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.521766+0000","last_change":"2024-03-26T03:44:32.022302+0000","last_active":"2024-03-26T03:45:18.521766+0000","last_peered":"2024-03-26T03:45:18.521766+0000","last_clean":"2024-03-26T03:45:18.521766+0000","last_became_active":"2024-03-26T03:44:21.310582+0000","last_became_peered":"2024-03-26T03:44:21.310582+0000","last_unstale":"2024-03-26T03:45:18.521766+0000","last_undegraded":"2024-03-26T03:45:18.521766+0000","last_fullsized":"2024-03-26T03:45:18.521766+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:20.280962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:20.280962+0000","last_clean_scrub_stamp":"2024-03-26T03:44:20.280962+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:36:49.816834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029300670000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.524018+0000","last_change":"2024-03-26T03:44:16.289675+0000","last_active":"2024-03-26T03:45:18.524018+0000","last_peered":"2024-03-26T03:45:18.524018+0000","last_clean":"2024-03-26T03:45:18.524018+0000","last_became_active":"2024-03-26T03:44:16.289167+0000","last_became_peered":"2024-03-26T03:44:16.289167+0000","last_unstale":"2024-03-26T03:45:18.524018+0000","last_undegraded":"2024-03-26T03:45:18.524018+0000","last_fullsized":"2024-03-26T03:45:18.524018+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:18:03.137915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"68'36","reported_seq":136,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:47:16.501137+0000","last_change":"2024-03-26T03:44:19.013341+0000","last_active":"2024-03-26T03:47:16.501137+0000","last_peered":"2024-03-26T03:47:16.501137+0000","last_clean":"2024-03-26T03:47:16.501137+0000","last_became_active":"2024-03-26T03:44:19.012639+0000","last_became_peered":"2024-03-26T03:44:19.012639+0000","last_unstale":"2024-03-26T03:47:16.501137+0000","last_undegraded":"2024-03-26T03:47:16.501137+0000","last_fullsized":"2024-03-26T03:47:16.501137+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:30:11.804581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"68'11","reported_seq":80,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:47:16.500925+0000","last_change":"2024-03-26T03:44:20.322594+0000","last_active":"2024-03-26T03:47:16.500925+0000","last_peered":"2024-03-26T03:47:16.500925+0000","last_clean":"2024-03-26T03:47:16.500925+0000","last_became_active":"2024-03-26T03:44:20.321578+0000","last_became_peered":"2024-03-26T03:44:20.321578+0000","last_unstale":"2024-03-26T03:47:16.500925+0000","last_undegraded":"2024-03-26T03:47:16.500925+0000","last_fullsized":"2024-03-26T03:47:16.500925+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:28:56.174543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.456340+0000","last_change":"2024-03-26T03:44:22.353904+0000","last_active":"2024-03-26T03:45:18.456340+0000","last_peered":"2024-03-26T03:45:18.456340+0000","last_clean":"2024-03-26T03:45:18.456340+0000","last_became_active":"2024-03-26T03:44:22.353455+0000","last_became_peered":"2024-03-26T03:44:22.353455+0000","last_unstale":"2024-03-26T03:45:18.456340+0000","last_undegraded":"2024-03-26T03:45:18.456340+0000","last_fullsized":"2024-03-26T03:45:18.456340+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:22:39.526810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":35,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.455421+0000","last_change":"2024-03-26T03:44:32.012604+0000","last_active":"2024-03-26T03:45:18.455421+0000","last_peered":"2024-03-26T03:45:18.455421+0000","last_clean":"2024-03-26T03:45:18.455421+0000","last_became_active":"2024-03-26T03:44:21.320994+0000","last_became_peered":"2024-03-26T03:44:21.320994+0000","last_unstale":"2024-03-26T03:45:18.455421+0000","last_undegraded":"2024-03-26T03:45:18.455421+0000","last_fullsized":"2024-03-26T03:45:18.455421+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:20.280962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:20.280962+0000","last_clean_scrub_stamp":"2024-03-26T03:44:20.280962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:33:31.555209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001946038,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"61'1","reported_seq":41,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516869+0000","last_change":"2024-03-26T03:44:16.285263+0000","last_active":"2024-03-26T03:45:18.516869+0000","last_peered":"2024-03-26T03:45:18.516869+0000","last_clean":"2024-03-26T03:45:18.516869+0000","last_became_active":"2024-03-26T03:44:16.282901+0000","last_became_peered":"2024-03-26T03:44:16.282901+0000","last_unstale":"2024-03-26T03:45:18.516869+0000","last_undegraded":"2024-03-26T03:45:18.516869+0000","last_fullsized":"2024-03-26T03:45:18.516869+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:27:33.040539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"68'21","reported_seq":76,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.456862+0000","last_change":"2024-03-26T03:44:18.301027+0000","last_active":"2024-03-26T03:45:18.456862+0000","last_peered":"2024-03-26T03:45:18.456862+0000","last_clean":"2024-03-26T03:45:18.456862+0000","last_became_active":"2024-03-26T03:44:18.299512+0000","last_became_peered":"2024-03-26T03:44:18.299512+0000","last_unstale":"2024-03-26T03:45:18.456862+0000","last_undegraded":"2024-03-26T03:45:18.456862+0000","last_fullsized":"2024-03-26T03:45:18.456862+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T05:58:12.478446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":39,"num_read_kb":27,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"68'11","reported_seq":79,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:47:16.500850+0000","last_change":"2024-03-26T03:44:20.334110+0000","last_active":"2024-03-26T03:47:16.500850+0000","last_peered":"2024-03-26T03:47:16.500850+0000","last_clean":"2024-03-26T03:47:16.500850+0000","last_became_active":"2024-03-26T03:44:20.333495+0000","last_became_peered":"2024-03-26T03:44:20.333495+0000","last_unstale":"2024-03-26T03:47:16.500850+0000","last_undegraded":"2024-03-26T03:47:16.500850+0000","last_fullsized":"2024-03-26T03:47:16.500850+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T05:06:09.792091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.522862+0000","last_change":"2024-03-26T03:44:22.322794+0000","last_active":"2024-03-26T03:45:18.522862+0000","last_peered":"2024-03-26T03:45:18.522862+0000","last_clean":"2024-03-26T03:45:18.522862+0000","last_became_active":"2024-03-26T03:44:22.321990+0000","last_became_peered":"2024-03-26T03:44:22.321990+0000","last_unstale":"2024-03-26T03:45:18.522862+0000","last_undegraded":"2024-03-26T03:45:18.522862+0000","last_fullsized":"2024-03-26T03:45:18.522862+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:07:16.553113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"72'5","reported_seq":201,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:47:23.088027+0000","last_change":"2024-03-26T03:44:32.013497+0000","last_active":"2024-03-26T03:47:23.088027+0000","last_peered":"2024-03-26T03:47:23.088027+0000","last_clean":"2024-03-26T03:47:23.088027+0000","last_became_active":"2024-03-26T03:44:21.320574+0000","last_became_peered":"2024-03-26T03:44:21.320574+0000","last_unstale":"2024-03-26T03:47:23.088027+0000","last_undegraded":"2024-03-26T03:47:23.088027+0000","last_fullsized":"2024-03-26T03:47:23.088027+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:20.280962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:20.280962+0000","last_clean_scrub_stamp":"2024-03-26T03:44:20.280962+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:46:30.807315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012228269999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":163,"num_read_kb":158,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.523091+0000","last_change":"2024-03-26T03:44:16.281630+0000","last_active":"2024-03-26T03:45:18.523091+0000","last_peered":"2024-03-26T03:45:18.523091+0000","last_clean":"2024-03-26T03:45:18.523091+0000","last_became_active":"2024-03-26T03:44:16.281164+0000","last_became_peered":"2024-03-26T03:44:16.281164+0000","last_unstale":"2024-03-26T03:45:18.523091+0000","last_undegraded":"2024-03-26T03:45:18.523091+0000","last_fullsized":"2024-03-26T03:45:18.523091+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T04:37:32.669790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"68'12","reported_seq":54,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.517178+0000","last_change":"2024-03-26T03:44:18.303063+0000","last_active":"2024-03-26T03:45:18.517178+0000","last_peered":"2024-03-26T03:45:18.517178+0000","last_clean":"2024-03-26T03:45:18.517178+0000","last_became_active":"2024-03-26T03:44:18.302311+0000","last_became_peered":"2024-03-26T03:44:18.302311+0000","last_unstale":"2024-03-26T03:45:18.517178+0000","last_undegraded":"2024-03-26T03:45:18.517178+0000","last_fullsized":"2024-03-26T03:45:18.517178+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T04:19:23.091634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.455791+0000","last_change":"2024-03-26T03:44:20.310584+0000","last_active":"2024-03-26T03:45:18.455791+0000","last_peered":"2024-03-26T03:45:18.455791+0000","last_clean":"2024-03-26T03:45:18.455791+0000","last_became_active":"2024-03-26T03:44:20.310221+0000","last_became_peered":"2024-03-26T03:44:20.310221+0000","last_unstale":"2024-03-26T03:45:18.455791+0000","last_undegraded":"2024-03-26T03:45:18.455791+0000","last_fullsized":"2024-03-26T03:45:18.455791+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:31:39.989077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.455716+0000","last_change":"2024-03-26T03:44:22.332468+0000","last_active":"2024-03-26T03:45:18.455716+0000","last_peered":"2024-03-26T03:45:18.455716+0000","last_clean":"2024-03-26T03:45:18.455716+0000","last_became_active":"2024-03-26T03:44:22.323935+0000","last_became_peered":"2024-03-26T03:44:22.323935+0000","last_unstale":"2024-03-26T03:45:18.455716+0000","last_undegraded":"2024-03-26T03:45:18.455716+0000","last_fullsized":"2024-03-26T03:45:18.455716+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T03:48:57.596725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"68'11","reported_seq":79,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:47:16.507343+0000","last_change":"2024-03-26T03:44:20.328056+0000","last_active":"2024-03-26T03:47:16.507343+0000","last_peered":"2024-03-26T03:47:16.507343+0000","last_clean":"2024-03-26T03:47:16.507343+0000","last_became_active":"2024-03-26T03:44:20.327625+0000","last_became_peered":"2024-03-26T03:44:20.327625+0000","last_unstale":"2024-03-26T03:47:16.507343+0000","last_undegraded":"2024-03-26T03:47:16.507343+0000","last_fullsized":"2024-03-26T03:47:16.507343+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:06:58.148564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.523464+0000","last_change":"2024-03-26T03:44:16.288581+0000","last_active":"2024-03-26T03:45:18.523464+0000","last_peered":"2024-03-26T03:45:18.523464+0000","last_clean":"2024-03-26T03:45:18.523464+0000","last_became_active":"2024-03-26T03:44:16.288390+0000","last_became_peered":"2024-03-26T03:44:16.288390+0000","last_unstale":"2024-03-26T03:45:18.523464+0000","last_undegraded":"2024-03-26T03:45:18.523464+0000","last_fullsized":"2024-03-26T03:45:18.523464+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:27:00.261919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"68'24","reported_seq":82,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.455494+0000","last_change":"2024-03-26T03:44:19.017777+0000","last_active":"2024-03-26T03:45:18.455494+0000","last_peered":"2024-03-26T03:45:18.455494+0000","last_clean":"2024-03-26T03:45:18.455494+0000","last_became_active":"2024-03-26T03:44:19.016777+0000","last_became_peered":"2024-03-26T03:44:19.016777+0000","last_unstale":"2024-03-26T03:45:18.455494+0000","last_undegraded":"2024-03-26T03:45:18.455494+0000","last_fullsized":"2024-03-26T03:45:18.455494+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:45:32.513405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":32,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.523646+0000","last_change":"2024-03-26T03:44:22.342790+0000","last_active":"2024-03-26T03:45:18.523646+0000","last_peered":"2024-03-26T03:45:18.523646+0000","last_clean":"2024-03-26T03:45:18.523646+0000","last_became_active":"2024-03-26T03:44:22.342014+0000","last_became_peered":"2024-03-26T03:44:22.342014+0000","last_unstale":"2024-03-26T03:45:18.523646+0000","last_undegraded":"2024-03-26T03:45:18.523646+0000","last_fullsized":"2024-03-26T03:45:18.523646+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:04:51.280399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.517147+0000","last_change":"2024-03-26T03:44:20.340671+0000","last_active":"2024-03-26T03:45:18.517147+0000","last_peered":"2024-03-26T03:45:18.517147+0000","last_clean":"2024-03-26T03:45:18.517147+0000","last_became_active":"2024-03-26T03:44:20.339960+0000","last_became_peered":"2024-03-26T03:44:20.339960+0000","last_unstale":"2024-03-26T03:45:18.517147+0000","last_undegraded":"2024-03-26T03:45:18.517147+0000","last_fullsized":"2024-03-26T03:45:18.517147+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T05:38:43.478175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":46,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.517059+0000","last_change":"2024-03-26T03:44:54.227425+0000","last_active":"2024-03-26T03:45:18.517059+0000","last_peered":"2024-03-26T03:45:18.517059+0000","last_clean":"2024-03-26T03:45:18.517059+0000","last_became_active":"2024-03-26T03:44:54.226552+0000","last_became_peered":"2024-03-26T03:44:54.226552+0000","last_unstale":"2024-03-26T03:45:18.517059+0000","last_undegraded":"2024-03-26T03:45:18.517059+0000","last_fullsized":"2024-03-26T03:45:18.517059+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T04:49:56.267326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"68'27","reported_seq":84,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.452798+0000","last_change":"2024-03-26T03:44:18.299021+0000","last_active":"2024-03-26T03:45:18.452798+0000","last_peered":"2024-03-26T03:45:18.452798+0000","last_clean":"2024-03-26T03:45:18.452798+0000","last_became_active":"2024-03-26T03:44:18.298313+0000","last_became_peered":"2024-03-26T03:44:18.298313+0000","last_unstale":"2024-03-26T03:45:18.452798+0000","last_undegraded":"2024-03-26T03:45:18.452798+0000","last_fullsized":"2024-03-26T03:45:18.452798+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T05:54:07.169738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":44,"num_read_kb":33,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.458380+0000","last_change":"2024-03-26T03:44:22.346835+0000","last_active":"2024-03-26T03:45:18.458380+0000","last_peered":"2024-03-26T03:45:18.458380+0000","last_clean":"2024-03-26T03:45:18.458380+0000","last_became_active":"2024-03-26T03:44:22.346199+0000","last_became_peered":"2024-03-26T03:44:22.346199+0000","last_unstale":"2024-03-26T03:45:18.458380+0000","last_undegraded":"2024-03-26T03:45:18.458380+0000","last_fullsized":"2024-03-26T03:45:18.458380+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:25:56.787286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516383+0000","last_change":"2024-03-26T03:44:16.284156+0000","last_active":"2024-03-26T03:45:18.516383+0000","last_peered":"2024-03-26T03:45:18.516383+0000","last_clean":"2024-03-26T03:45:18.516383+0000","last_became_active":"2024-03-26T03:44:16.281845+0000","last_became_peered":"2024-03-26T03:44:16.281845+0000","last_unstale":"2024-03-26T03:45:18.516383+0000","last_undegraded":"2024-03-26T03:45:18.516383+0000","last_fullsized":"2024-03-26T03:45:18.516383+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T04:38:36.984770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"68'15","reported_seq":63,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.455278+0000","last_change":"2024-03-26T03:44:18.301601+0000","last_active":"2024-03-26T03:45:18.455278+0000","last_peered":"2024-03-26T03:45:18.455278+0000","last_clean":"2024-03-26T03:45:18.455278+0000","last_became_active":"2024-03-26T03:44:18.300175+0000","last_became_peered":"2024-03-26T03:44:18.300175+0000","last_unstale":"2024-03-26T03:45:18.455278+0000","last_undegraded":"2024-03-26T03:45:18.455278+0000","last_fullsized":"2024-03-26T03:45:18.455278+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T05:56:12.520657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":26,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.456358+0000","last_change":"2024-03-26T03:44:20.313591+0000","last_active":"2024-03-26T03:45:18.456358+0000","last_peered":"2024-03-26T03:45:18.456358+0000","last_clean":"2024-03-26T03:45:18.456358+0000","last_became_active":"2024-03-26T03:44:20.313252+0000","last_became_peered":"2024-03-26T03:44:20.313252+0000","last_unstale":"2024-03-26T03:45:18.456358+0000","last_undegraded":"2024-03-26T03:45:18.456358+0000","last_fullsized":"2024-03-26T03:45:18.456358+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:30:59.072644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.453013+0000","last_change":"2024-03-26T03:44:22.349373+0000","last_active":"2024-03-26T03:45:18.453013+0000","last_peered":"2024-03-26T03:45:18.453013+0000","last_clean":"2024-03-26T03:45:18.453013+0000","last_became_active":"2024-03-26T03:44:22.348628+0000","last_became_peered":"2024-03-26T03:44:22.348628+0000","last_unstale":"2024-03-26T03:45:18.453013+0000","last_undegraded":"2024-03-26T03:45:18.453013+0000","last_fullsized":"2024-03-26T03:45:18.453013+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:37:58.792925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"68'11","reported_seq":79,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:47:16.507413+0000","last_change":"2024-03-26T03:44:20.327014+0000","last_active":"2024-03-26T03:47:16.507413+0000","last_peered":"2024-03-26T03:47:16.507413+0000","last_clean":"2024-03-26T03:47:16.507413+0000","last_became_active":"2024-03-26T03:44:20.326494+0000","last_became_peered":"2024-03-26T03:44:20.326494+0000","last_unstale":"2024-03-26T03:47:16.507413+0000","last_undegraded":"2024-03-26T03:47:16.507413+0000","last_fullsized":"2024-03-26T03:47:16.507413+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T06:50:24.943742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.522984+0000","last_change":"2024-03-26T03:44:16.279613+0000","last_active":"2024-03-26T03:45:18.522984+0000","last_peered":"2024-03-26T03:45:18.522984+0000","last_clean":"2024-03-26T03:45:18.522984+0000","last_became_active":"2024-03-26T03:44:16.279300+0000","last_became_peered":"2024-03-26T03:44:16.279300+0000","last_unstale":"2024-03-26T03:45:18.522984+0000","last_undegraded":"2024-03-26T03:45:18.522984+0000","last_fullsized":"2024-03-26T03:45:18.522984+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:26:33.600103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"68'25","reported_seq":79,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.521979+0000","last_change":"2024-03-26T03:44:18.308685+0000","last_active":"2024-03-26T03:45:18.521979+0000","last_peered":"2024-03-26T03:45:18.521979+0000","last_clean":"2024-03-26T03:45:18.521979+0000","last_became_active":"2024-03-26T03:44:18.307821+0000","last_became_peered":"2024-03-26T03:44:18.307821+0000","last_unstale":"2024-03-26T03:45:18.521979+0000","last_undegraded":"2024-03-26T03:45:18.521979+0000","last_fullsized":"2024-03-26T03:45:18.521979+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:44:54.216486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":41,"num_read_kb":29,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.523110+0000","last_change":"2024-03-26T03:44:22.321149+0000","last_active":"2024-03-26T03:45:18.523110+0000","last_peered":"2024-03-26T03:45:18.523110+0000","last_clean":"2024-03-26T03:45:18.523110+0000","last_became_active":"2024-03-26T03:44:22.320317+0000","last_became_peered":"2024-03-26T03:44:22.320317+0000","last_unstale":"2024-03-26T03:45:18.523110+0000","last_undegraded":"2024-03-26T03:45:18.523110+0000","last_fullsized":"2024-03-26T03:45:18.523110+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:51:33.516980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.450825+0000","last_change":"2024-03-26T03:44:16.269172+0000","last_active":"2024-03-26T03:45:18.450825+0000","last_peered":"2024-03-26T03:45:18.450825+0000","last_clean":"2024-03-26T03:45:18.450825+0000","last_became_active":"2024-03-26T03:44:16.267794+0000","last_became_peered":"2024-03-26T03:44:16.267794+0000","last_unstale":"2024-03-26T03:45:18.450825+0000","last_undegraded":"2024-03-26T03:45:18.450825+0000","last_fullsized":"2024-03-26T03:45:18.450825+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:31:51.866705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"68'17","reported_seq":64,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.524139+0000","last_change":"2024-03-26T03:44:18.311470+0000","last_active":"2024-03-26T03:45:18.524139+0000","last_peered":"2024-03-26T03:45:18.524139+0000","last_clean":"2024-03-26T03:45:18.524139+0000","last_became_active":"2024-03-26T03:44:18.311152+0000","last_became_peered":"2024-03-26T03:44:18.311152+0000","last_unstale":"2024-03-26T03:45:18.524139+0000","last_undegraded":"2024-03-26T03:45:18.524139+0000","last_fullsized":"2024-03-26T03:45:18.524139+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:46:30.965334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.521664+0000","last_change":"2024-03-26T03:44:20.325898+0000","last_active":"2024-03-26T03:45:18.521664+0000","last_peered":"2024-03-26T03:45:18.521664+0000","last_clean":"2024-03-26T03:45:18.521664+0000","last_became_active":"2024-03-26T03:44:20.325470+0000","last_became_peered":"2024-03-26T03:44:20.325470+0000","last_unstale":"2024-03-26T03:45:18.521664+0000","last_undegraded":"2024-03-26T03:45:18.521664+0000","last_fullsized":"2024-03-26T03:45:18.521664+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T04:32:18.072371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516136+0000","last_change":"2024-03-26T03:44:22.326248+0000","last_active":"2024-03-26T03:45:18.516136+0000","last_peered":"2024-03-26T03:45:18.516136+0000","last_clean":"2024-03-26T03:45:18.516136+0000","last_became_active":"2024-03-26T03:44:22.325062+0000","last_became_peered":"2024-03-26T03:44:22.325062+0000","last_unstale":"2024-03-26T03:45:18.516136+0000","last_undegraded":"2024-03-26T03:45:18.516136+0000","last_fullsized":"2024-03-26T03:45:18.516136+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:29:28.977976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516813+0000","last_change":"2024-03-26T03:44:16.286186+0000","last_active":"2024-03-26T03:45:18.516813+0000","last_peered":"2024-03-26T03:45:18.516813+0000","last_clean":"2024-03-26T03:45:18.516813+0000","last_became_active":"2024-03-26T03:44:16.283812+0000","last_became_peered":"2024-03-26T03:44:16.283812+0000","last_unstale":"2024-03-26T03:45:18.516813+0000","last_undegraded":"2024-03-26T03:45:18.516813+0000","last_fullsized":"2024-03-26T03:45:18.516813+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:17:43.468208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"68'10","reported_seq":51,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.522641+0000","last_change":"2024-03-26T03:44:18.306687+0000","last_active":"2024-03-26T03:45:18.522641+0000","last_peered":"2024-03-26T03:45:18.522641+0000","last_clean":"2024-03-26T03:45:18.522641+0000","last_became_active":"2024-03-26T03:44:18.306356+0000","last_became_peered":"2024-03-26T03:44:18.306356+0000","last_unstale":"2024-03-26T03:45:18.522641+0000","last_undegraded":"2024-03-26T03:45:18.522641+0000","last_fullsized":"2024-03-26T03:45:18.522641+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T04:22:32.044623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.516386+0000","last_change":"2024-03-26T03:44:20.333134+0000","last_active":"2024-03-26T03:45:18.516386+0000","last_peered":"2024-03-26T03:45:18.516386+0000","last_clean":"2024-03-26T03:45:18.516386+0000","last_became_active":"2024-03-26T03:44:20.332218+0000","last_became_peered":"2024-03-26T03:44:20.332218+0000","last_unstale":"2024-03-26T03:45:18.516386+0000","last_undegraded":"2024-03-26T03:45:18.516386+0000","last_fullsized":"2024-03-26T03:45:18.516386+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:06:27.081828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.522173+0000","last_change":"2024-03-26T03:44:22.344707+0000","last_active":"2024-03-26T03:45:18.522173+0000","last_peered":"2024-03-26T03:45:18.522173+0000","last_clean":"2024-03-26T03:45:18.522173+0000","last_became_active":"2024-03-26T03:44:22.343429+0000","last_became_peered":"2024-03-26T03:44:22.343429+0000","last_unstale":"2024-03-26T03:45:18.522173+0000","last_undegraded":"2024-03-26T03:45:18.522173+0000","last_fullsized":"2024-03-26T03:45:18.522173+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:21.284283+0000","last_clean_scrub_stamp":"2024-03-26T03:44:21.284283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T04:59:57.540509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.452681+0000","last_change":"2024-03-26T03:44:16.278056+0000","last_active":"2024-03-26T03:45:18.452681+0000","last_peered":"2024-03-26T03:45:18.452681+0000","last_clean":"2024-03-26T03:45:18.452681+0000","last_became_active":"2024-03-26T03:44:16.277225+0000","last_became_peered":"2024-03-26T03:44:16.277225+0000","last_unstale":"2024-03-26T03:45:18.452681+0000","last_undegraded":"2024-03-26T03:45:18.452681+0000","last_fullsized":"2024-03-26T03:45:18.452681+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:15.239878+0000","last_clean_scrub_stamp":"2024-03-26T03:44:15.239878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:29:38.976272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"68'15","reported_seq":61,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.522397+0000","last_change":"2024-03-26T03:44:18.300839+0000","last_active":"2024-03-26T03:45:18.522397+0000","last_peered":"2024-03-26T03:45:18.522397+0000","last_clean":"2024-03-26T03:45:18.522397+0000","last_became_active":"2024-03-26T03:44:18.300559+0000","last_became_peered":"2024-03-26T03:44:18.300559+0000","last_unstale":"2024-03-26T03:45:18.522397+0000","last_undegraded":"2024-03-26T03:45:18.522397+0000","last_fullsized":"2024-03-26T03:45:18.522397+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:17.251979+0000","last_clean_scrub_stamp":"2024-03-26T03:44:17.251979+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:38:24.454193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-26T03:45:18.521223+0000","last_change":"2024-03-26T03:44:20.310647+0000","last_active":"2024-03-26T03:45:18.521223+0000","last_peered":"2024-03-26T03:45:18.521223+0000","last_clean":"2024-03-26T03:45:18.521223+0000","last_became_active":"2024-03-26T03:44:20.310294+0000","last_became_peered":"2024-03-26T03:44:20.310294+0000","last_unstale":"2024-03-26T03:45:18.521223+0000","last_undegraded":"2024-03-26T03:45:18.521223+0000","last_fullsized":"2024-03-26T03:45:18.521223+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T03:44:19.274909+0000","last_clean_scrub_stamp":"2024-03-26T03:44:19.274909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T07:07:21.204615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":383,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":163,"num_read_kb":158,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":9,"acting":9,"num_store_stats":7},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88,"ondisk_log_size":88,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":210,"num_object_clones":0,"num_object_copies":630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":210,"num_whiteouts":0,"num_read":989,"num_read_kb":714,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":592,"ondisk_log_size":592,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1531,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201328,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28444,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716708,"statfs":{"total":95995035648,"available":95965908992,"internally_reserved":0,"allocated":1462272,"data_stored":949055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462970,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28412,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716740,"statfs":{"total":95995035648,"available":95965941760,"internally_reserved":0,"allocated":1429504,"data_stored":946794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574961,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32012,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713140,"statfs":{"total":95995035648,"available":95962255360,"internally_reserved":0,"allocated":851968,"data_stored":358019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738491,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28436,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716716,"statfs":{"total":95995035648,"available":95965917184,"internally_reserved":0,"allocated":1454080,"data_stored":948420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411430,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":835584,"data_stored":356867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182490,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":835584,"data_stored":356807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018959,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":876544,"data_stored":358158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724614,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":835584,"data_stored":356776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T03:47:26.563 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-26T03:47:26.563 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-26T03:47:26.563 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-26T03:47:26.564 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 shell --fsid 762e930c-eb21-11ee-95d0-87774f69a715 -- ceph health --format=json 2024-03-26T03:47:27.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:26 smithi190 bash[23392]: audit 2024-03-26T03:47:25.758960+0000 mgr.y (mgr.24431) 113 : audit [DBG] from='client.24608 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T03:47:27.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:26 smithi175 bash[29284]: audit 2024-03-26T03:47:25.758960+0000 mgr.y (mgr.24431) 113 : audit [DBG] from='client.24608 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T03:47:27.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:26 smithi175 bash[21527]: audit 2024-03-26T03:47:25.758960+0000 mgr.y (mgr.24431) 113 : audit [DBG] from='client.24608 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T03:47:28.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:27 smithi190 bash[23392]: cluster 2024-03-26T03:47:26.529121+0000 mgr.y (mgr.24431) 114 : cluster [DBG] pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:28.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:27 smithi175 bash[21527]: cluster 2024-03-26T03:47:26.529121+0000 mgr.y (mgr.24431) 114 : cluster [DBG] pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:28.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:27 smithi175 bash[29284]: cluster 2024-03-26T03:47:26.529121+0000 mgr.y (mgr.24431) 114 : cluster [DBG] pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:29.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:47:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:47:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:47:29.981 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:47:29 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:47:30.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:29 smithi190 bash[23392]: cluster 2024-03-26T03:47:28.529816+0000 mgr.y (mgr.24431) 115 : cluster [DBG] pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:30.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:29 smithi175 bash[21527]: cluster 2024-03-26T03:47:28.529816+0000 mgr.y (mgr.24431) 115 : cluster [DBG] pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:30.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:29 smithi175 bash[29284]: cluster 2024-03-26T03:47:28.529816+0000 mgr.y (mgr.24431) 115 : cluster [DBG] pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:31.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:30 smithi175 bash[21527]: audit 2024-03-26T03:47:29.604537+0000 mgr.y (mgr.24431) 116 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:47:31.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:30 smithi175 bash[29284]: audit 2024-03-26T03:47:29.604537+0000 mgr.y (mgr.24431) 116 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:47:31.391 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T03:47:31.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:31 smithi190 bash[23392]: audit 2024-03-26T03:47:29.604537+0000 mgr.y (mgr.24431) 116 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:47:32.381 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:31 smithi175 bash[21527]: cluster 2024-03-26T03:47:30.531254+0000 mgr.y (mgr.24431) 117 : cluster [DBG] pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:32.381 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:31 smithi175 bash[29284]: cluster 2024-03-26T03:47:30.531254+0000 mgr.y (mgr.24431) 117 : cluster [DBG] pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:32.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:32 smithi190 bash[23392]: cluster 2024-03-26T03:47:30.531254+0000 mgr.y (mgr.24431) 117 : cluster [DBG] pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:34.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.252+0000 7f12629ad700 1 -- 172.21.15.175:0/3245606579 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f125c104890 msgr2=0x7f125c104d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:34.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.252+0000 7f12629ad700 1 --2- 172.21.15.175:0/3245606579 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f125c104890 0x7f125c104d10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1250009990 tx=0x7f1250009ca0 comp rx=0 tx=0).stop 2024-03-26T03:47:34.254 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.252+0000 7f12629ad700 1 -- 172.21.15.175:0/3245606579 shutdown_connections 2024-03-26T03:47:34.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.252+0000 7f12629ad700 1 --2- 172.21.15.175:0/3245606579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f125c105250 0x7f125c111760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:34.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.252+0000 7f12629ad700 1 --2- 172.21.15.175:0/3245606579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f125c10a890 0x7f125c10ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:34.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.252+0000 7f12629ad700 1 --2- 172.21.15.175:0/3245606579 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f125c104890 0x7f125c104d10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:34.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.252+0000 7f12629ad700 1 -- 172.21.15.175:0/3245606579 >> 172.21.15.175:0/3245606579 conn(0x7f125c1003a0 msgr2=0x7f125c1027c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:47:34.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.252+0000 7f12629ad700 1 -- 172.21.15.175:0/3245606579 shutdown_connections 2024-03-26T03:47:34.255 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.252+0000 7f12629ad700 1 -- 172.21.15.175:0/3245606579 wait complete. 2024-03-26T03:47:34.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.256+0000 7f12629ad700 1 Processor -- start 2024-03-26T03:47:34.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.256+0000 7f12629ad700 1 -- start start 2024-03-26T03:47:34.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.256+0000 7f12629ad700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f125c104890 0x7f125c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:34.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.256+0000 7f12629ad700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f125c105250 0x7f125c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:34.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.256+0000 7f12629ad700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f125c10a890 0x7f125c19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:34.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.256+0000 7f12629ad700 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f125c1140f0 con 0x7f125c104890 2024-03-26T03:47:34.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.256+0000 7f12629ad700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f125c113f70 con 0x7f125c10a890 2024-03-26T03:47:34.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.256+0000 7f12629ad700 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f125c114270 con 0x7f125c105250 2024-03-26T03:47:34.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.256+0000 7f125b7fe700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f125c105250 0x7f125c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:34.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.256+0000 7f125bfff700 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f125c104890 0x7f125c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:34.257 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.256+0000 7f125b7fe700 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f125c105250 0x7f125c19ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:60096/0 (socket says 172.21.15.175:60096) 2024-03-26T03:47:34.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.256+0000 7f125b7fe700 1 -- 172.21.15.175:0/1238665556 learned_addr learned my addr 172.21.15.175:0/1238665556 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T03:47:34.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.256+0000 7f1260f4a700 1 --2- 172.21.15.175:0/1238665556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f125c10a890 0x7f125c19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:34.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.256+0000 7f125b7fe700 1 -- 172.21.15.175:0/1238665556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f125c10a890 msgr2=0x7f125c19ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:34.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.256+0000 7f125b7fe700 1 --2- 172.21.15.175:0/1238665556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f125c10a890 0x7f125c19ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:34.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.256+0000 7f125b7fe700 1 -- 172.21.15.175:0/1238665556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f125c104890 msgr2=0x7f125c19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:34.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.256+0000 7f125b7fe700 1 --2- 172.21.15.175:0/1238665556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f125c104890 0x7f125c19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:34.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.256+0000 7f125b7fe700 1 -- 172.21.15.175:0/1238665556 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f125c19f5d0 con 0x7f125c105250 2024-03-26T03:47:34.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.256+0000 7f125bfff700 1 --2- 172.21.15.175:0/1238665556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f125c104890 0x7f125c19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T03:47:34.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.256+0000 7f1260f4a700 1 --2- 172.21.15.175:0/1238665556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f125c10a890 0x7f125c19ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T03:47:34.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.256+0000 7f125b7fe700 1 --2- 172.21.15.175:0/1238665556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f125c105250 0x7f125c19ab50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f12500057c0 tx=0x7f12500057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:47:34.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.256+0000 7f12597fa700 1 -- 172.21.15.175:0/1238665556 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1250026070 con 0x7f125c105250 2024-03-26T03:47:34.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.256+0000 7f12629ad700 1 -- 172.21.15.175:0/1238665556 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f125c19f860 con 0x7f125c105250 2024-03-26T03:47:34.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.256+0000 7f12597fa700 1 -- 172.21.15.175:0/1238665556 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f1250021410 con 0x7f125c105250 2024-03-26T03:47:34.258 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.256+0000 7f12629ad700 1 -- 172.21.15.175:0/1238665556 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f125c1a7140 con 0x7f125c105250 2024-03-26T03:47:34.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.256+0000 7f12597fa700 1 -- 172.21.15.175:0/1238665556 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12500173f0 con 0x7f125c105250 2024-03-26T03:47:34.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.256+0000 7f12629ad700 1 -- 172.21.15.175:0/1238665556 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f125c050300 con 0x7f125c105250 2024-03-26T03:47:34.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.256+0000 7f12597fa700 1 -- 172.21.15.175:0/1238665556 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7f1250029070 con 0x7f125c105250 2024-03-26T03:47:34.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.256+0000 7f12597fa700 1 --2- 172.21.15.175:0/1238665556 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f124406e7e0 0x7f1244070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T03:47:34.259 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.256+0000 7f12597fa700 1 -- 172.21.15.175:0/1238665556 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(75..75 src has 1..75) ==== 7537+0+0 (secure 0 0 0) 0x7f125008fb60 con 0x7f125c105250 2024-03-26T03:47:34.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.256+0000 7f125bfff700 1 --2- 172.21.15.175:0/1238665556 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f124406e7e0 0x7f1244070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T03:47:34.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.260+0000 7f125bfff700 1 --2- 172.21.15.175:0/1238665556 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f124406e7e0 0x7f1244070ca0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f124c009de0 tx=0x7f124c009190 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T03:47:34.263 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.264+0000 7f12597fa700 1 -- 172.21.15.175:0/1238665556 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f1250013070 con 0x7f125c105250 2024-03-26T03:47:34.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:34 smithi175 bash[29284]: cluster 2024-03-26T03:47:32.531761+0000 mgr.y (mgr.24431) 118 : cluster [DBG] pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:34.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:34 smithi175 bash[29284]: audit 2024-03-26T03:47:33.825600+0000 mon.c (mon.2) 80 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:47:34.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:34 smithi190 bash[23392]: cluster 2024-03-26T03:47:32.531761+0000 mgr.y (mgr.24431) 118 : cluster [DBG] pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:34.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:34 smithi190 bash[23392]: audit 2024-03-26T03:47:33.825600+0000 mon.c (mon.2) 80 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:47:34.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:34 smithi175 bash[21527]: cluster 2024-03-26T03:47:32.531761+0000 mgr.y (mgr.24431) 118 : cluster [DBG] pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:34.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:34 smithi175 bash[21527]: audit 2024-03-26T03:47:33.825600+0000 mon.c (mon.2) 80 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:47:34.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.528+0000 7f12629ad700 1 -- 172.21.15.175:0/1238665556 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f125c1a73d0 con 0x7f125c105250 2024-03-26T03:47:34.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.528+0000 7f12597fa700 1 -- 172.21.15.175:0/1238665556 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f1250060880 con 0x7f125c105250 2024-03-26T03:47:34.530 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-03-26T03:47:34.530 INFO:teuthology.orchestra.run.smithi175.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-26T03:47:34.532 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.532+0000 7f12629ad700 1 -- 172.21.15.175:0/1238665556 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f124406e7e0 msgr2=0x7f1244070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:34.532 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.532+0000 7f12629ad700 1 --2- 172.21.15.175:0/1238665556 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f124406e7e0 0x7f1244070ca0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f124c009de0 tx=0x7f124c009190 comp rx=0 tx=0).stop 2024-03-26T03:47:34.532 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.532+0000 7f12629ad700 1 -- 172.21.15.175:0/1238665556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f125c105250 msgr2=0x7f125c19ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T03:47:34.532 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.532+0000 7f12629ad700 1 --2- 172.21.15.175:0/1238665556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f125c105250 0x7f125c19ab50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f12500057c0 tx=0x7f12500057f0 comp rx=0 tx=0).stop 2024-03-26T03:47:34.532 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.532+0000 7f12629ad700 1 -- 172.21.15.175:0/1238665556 shutdown_connections 2024-03-26T03:47:34.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.532+0000 7f12629ad700 1 --2- 172.21.15.175:0/1238665556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f125c10a890 0x7f125c19ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:34.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.532+0000 7f12629ad700 1 --2- 172.21.15.175:0/1238665556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f125c104890 0x7f125c19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:34.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.532+0000 7f12629ad700 1 --2- 172.21.15.175:0/1238665556 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f124406e7e0 0x7f1244070ca0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:34.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.532+0000 7f12629ad700 1 --2- 172.21.15.175:0/1238665556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f125c105250 0x7f125c19ab50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T03:47:34.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.532+0000 7f12629ad700 1 -- 172.21.15.175:0/1238665556 >> 172.21.15.175:0/1238665556 conn(0x7f125c1003a0 msgr2=0x7f125c101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T03:47:34.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.532+0000 7f12629ad700 1 -- 172.21.15.175:0/1238665556 shutdown_connections 2024-03-26T03:47:34.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-03-26T03:47:34.532+0000 7f12629ad700 1 -- 172.21.15.175:0/1238665556 wait complete. 2024-03-26T03:47:35.304 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:35 smithi175 bash[29284]: audit 2024-03-26T03:47:34.534233+0000 mon.c (mon.2) 81 : audit [DBG] from='client.? 172.21.15.175:0/1238665556' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-26T03:47:35.304 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:35 smithi175 bash[21527]: audit 2024-03-26T03:47:34.534233+0000 mon.c (mon.2) 81 : audit [DBG] from='client.? 172.21.15.175:0/1238665556' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-26T03:47:35.306 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-26T03:47:35.306 INFO:tasks.cephadm:Setup complete, yielding 2024-03-26T03:47:35.306 INFO:teuthology.run_tasks:Running task workunit... 2024-03-26T03:47:35.327 INFO:tasks.workunit:Pulling workunits from ref e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 2024-03-26T03:47:35.329 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-26T03:47:35.329 DEBUG:teuthology.orchestra.run.smithi175:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-26T03:47:35.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T03:47:35.337 INFO:teuthology.orchestra.run.smithi175.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-26T03:47:35.337 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-03-26T03:47:35.387 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-03-26T03:47:35.388 DEBUG:teuthology.orchestra.run.smithi175:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-03-26T03:47:35.435 INFO:tasks.workunit:timeout=3h 2024-03-26T03:47:35.436 INFO:tasks.workunit:cleanup=True 2024-03-26T03:47:35.436 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 2024-03-26T03:47:35.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:35 smithi190 bash[23392]: audit 2024-03-26T03:47:34.534233+0000 mon.c (mon.2) 81 : audit [DBG] from='client.? 172.21.15.175:0/1238665556' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-26T03:47:35.490 INFO:tasks.workunit.client.0.smithi175.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-26T03:47:36.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:36 smithi190 bash[23392]: cluster 2024-03-26T03:47:34.533070+0000 mgr.y (mgr.24431) 119 : cluster [DBG] pgmap v71: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:36.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:36 smithi175 bash[21527]: cluster 2024-03-26T03:47:34.533070+0000 mgr.y (mgr.24431) 119 : cluster [DBG] pgmap v71: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:36.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:36 smithi175 bash[29284]: cluster 2024-03-26T03:47:34.533070+0000 mgr.y (mgr.24431) 119 : cluster [DBG] pgmap v71: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:38.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:38 smithi190 bash[23392]: cluster 2024-03-26T03:47:36.533727+0000 mgr.y (mgr.24431) 120 : cluster [DBG] pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:38.490 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:38 smithi175 bash[29284]: cluster 2024-03-26T03:47:36.533727+0000 mgr.y (mgr.24431) 120 : cluster [DBG] pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:38.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:38 smithi175 bash[21527]: cluster 2024-03-26T03:47:36.533727+0000 mgr.y (mgr.24431) 120 : cluster [DBG] pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:39.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:47:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:47:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:47:39.990 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:47:39 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:47:40.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:40 smithi190 bash[23392]: cluster 2024-03-26T03:47:38.534390+0000 mgr.y (mgr.24431) 121 : cluster [DBG] pgmap v73: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:40.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:40 smithi175 bash[21527]: cluster 2024-03-26T03:47:38.534390+0000 mgr.y (mgr.24431) 121 : cluster [DBG] pgmap v73: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:40.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:40 smithi175 bash[29284]: cluster 2024-03-26T03:47:38.534390+0000 mgr.y (mgr.24431) 121 : cluster [DBG] pgmap v73: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:41.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:41 smithi190 bash[23392]: audit 2024-03-26T03:47:39.615974+0000 mgr.y (mgr.24431) 122 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:47:41.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:41 smithi175 bash[21527]: audit 2024-03-26T03:47:39.615974+0000 mgr.y (mgr.24431) 122 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:47:41.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:41 smithi175 bash[29284]: audit 2024-03-26T03:47:39.615974+0000 mgr.y (mgr.24431) 122 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:47:42.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:42 smithi190 bash[23392]: cluster 2024-03-26T03:47:40.535987+0000 mgr.y (mgr.24431) 123 : cluster [DBG] pgmap v74: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:42.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:42 smithi175 bash[21527]: cluster 2024-03-26T03:47:40.535987+0000 mgr.y (mgr.24431) 123 : cluster [DBG] pgmap v74: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:42.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:42 smithi175 bash[29284]: cluster 2024-03-26T03:47:40.535987+0000 mgr.y (mgr.24431) 123 : cluster [DBG] pgmap v74: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:44.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:44 smithi190 bash[23392]: cluster 2024-03-26T03:47:42.536788+0000 mgr.y (mgr.24431) 124 : cluster [DBG] pgmap v75: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:44.490 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:44 smithi175 bash[29284]: cluster 2024-03-26T03:47:42.536788+0000 mgr.y (mgr.24431) 124 : cluster [DBG] pgmap v75: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:44.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:44 smithi175 bash[21527]: cluster 2024-03-26T03:47:42.536788+0000 mgr.y (mgr.24431) 124 : cluster [DBG] pgmap v75: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:46.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:46 smithi190 bash[23392]: cluster 2024-03-26T03:47:44.538216+0000 mgr.y (mgr.24431) 125 : cluster [DBG] pgmap v76: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:46.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:46 smithi175 bash[21527]: cluster 2024-03-26T03:47:44.538216+0000 mgr.y (mgr.24431) 125 : cluster [DBG] pgmap v76: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:46.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:46 smithi175 bash[29284]: cluster 2024-03-26T03:47:44.538216+0000 mgr.y (mgr.24431) 125 : cluster [DBG] pgmap v76: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:48.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:48 smithi190 bash[23392]: cluster 2024-03-26T03:47:46.538999+0000 mgr.y (mgr.24431) 126 : cluster [DBG] pgmap v77: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:48.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:48 smithi175 bash[21527]: cluster 2024-03-26T03:47:46.538999+0000 mgr.y (mgr.24431) 126 : cluster [DBG] pgmap v77: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:48.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:48 smithi175 bash[29284]: cluster 2024-03-26T03:47:46.538999+0000 mgr.y (mgr.24431) 126 : cluster [DBG] pgmap v77: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:49.055 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:47:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:47:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:47:49.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:49 smithi190 bash[23392]: audit 2024-03-26T03:47:48.534862+0000 mon.c (mon.2) 82 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:47:49.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:49 smithi190 bash[23392]: audit 2024-03-26T03:47:48.856852+0000 mon.c (mon.2) 83 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:47:49.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:49 smithi175 bash[21527]: audit 2024-03-26T03:47:48.534862+0000 mon.c (mon.2) 82 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:47:49.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:49 smithi175 bash[21527]: audit 2024-03-26T03:47:48.856852+0000 mon.c (mon.2) 83 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:47:49.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:49 smithi175 bash[29284]: audit 2024-03-26T03:47:48.534862+0000 mon.c (mon.2) 82 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:47:49.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:49 smithi175 bash[29284]: audit 2024-03-26T03:47:48.856852+0000 mon.c (mon.2) 83 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:47:49.990 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:47:49 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:47:50.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:50 smithi190 bash[23392]: cluster 2024-03-26T03:47:48.539698+0000 mgr.y (mgr.24431) 127 : cluster [DBG] pgmap v78: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:50.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:50 smithi190 bash[23392]: audit 2024-03-26T03:47:49.326964+0000 mon.c (mon.2) 84 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:47:50.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:50 smithi190 bash[23392]: audit 2024-03-26T03:47:49.328789+0000 mon.c (mon.2) 85 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:47:50.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:50 smithi190 bash[23392]: audit 2024-03-26T03:47:49.339570+0000 mon.a (mon.0) 862 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:47:50.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:50 smithi175 bash[29284]: cluster 2024-03-26T03:47:48.539698+0000 mgr.y (mgr.24431) 127 : cluster [DBG] pgmap v78: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:50.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:50 smithi175 bash[29284]: audit 2024-03-26T03:47:49.326964+0000 mon.c (mon.2) 84 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:47:50.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:50 smithi175 bash[29284]: audit 2024-03-26T03:47:49.328789+0000 mon.c (mon.2) 85 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:47:50.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:50 smithi175 bash[29284]: audit 2024-03-26T03:47:49.339570+0000 mon.a (mon.0) 862 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:47:50.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:50 smithi175 bash[21527]: cluster 2024-03-26T03:47:48.539698+0000 mgr.y (mgr.24431) 127 : cluster [DBG] pgmap v78: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:50.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:50 smithi175 bash[21527]: audit 2024-03-26T03:47:49.326964+0000 mon.c (mon.2) 84 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:47:50.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:50 smithi175 bash[21527]: audit 2024-03-26T03:47:49.328789+0000 mon.c (mon.2) 85 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:47:50.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:50 smithi175 bash[21527]: audit 2024-03-26T03:47:49.339570+0000 mon.a (mon.0) 862 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:47:51.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:51 smithi190 bash[23392]: audit 2024-03-26T03:47:49.627485+0000 mgr.y (mgr.24431) 128 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:47:51.490 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:51 smithi175 bash[29284]: audit 2024-03-26T03:47:49.627485+0000 mgr.y (mgr.24431) 128 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:47:51.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:51 smithi175 bash[21527]: audit 2024-03-26T03:47:49.627485+0000 mgr.y (mgr.24431) 128 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:47:52.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:52 smithi190 bash[23392]: cluster 2024-03-26T03:47:50.541196+0000 mgr.y (mgr.24431) 129 : cluster [DBG] pgmap v79: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:52.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:52 smithi175 bash[21527]: cluster 2024-03-26T03:47:50.541196+0000 mgr.y (mgr.24431) 129 : cluster [DBG] pgmap v79: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:52.490 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:52 smithi175 bash[29284]: cluster 2024-03-26T03:47:50.541196+0000 mgr.y (mgr.24431) 129 : cluster [DBG] pgmap v79: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:54.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:54 smithi190 bash[23392]: cluster 2024-03-26T03:47:52.542044+0000 mgr.y (mgr.24431) 130 : cluster [DBG] pgmap v80: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:54.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:54 smithi175 bash[21527]: cluster 2024-03-26T03:47:52.542044+0000 mgr.y (mgr.24431) 130 : cluster [DBG] pgmap v80: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:54.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:54 smithi175 bash[29284]: cluster 2024-03-26T03:47:52.542044+0000 mgr.y (mgr.24431) 130 : cluster [DBG] pgmap v80: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:56.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:56 smithi190 bash[23392]: cluster 2024-03-26T03:47:54.543615+0000 mgr.y (mgr.24431) 131 : cluster [DBG] pgmap v81: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:56.490 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:56 smithi175 bash[29284]: cluster 2024-03-26T03:47:54.543615+0000 mgr.y (mgr.24431) 131 : cluster [DBG] pgmap v81: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:56.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:56 smithi175 bash[21527]: cluster 2024-03-26T03:47:54.543615+0000 mgr.y (mgr.24431) 131 : cluster [DBG] pgmap v81: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:47:58.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:47:58 smithi190 bash[23392]: cluster 2024-03-26T03:47:56.544452+0000 mgr.y (mgr.24431) 132 : cluster [DBG] pgmap v82: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:58.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:47:58 smithi175 bash[21527]: cluster 2024-03-26T03:47:56.544452+0000 mgr.y (mgr.24431) 132 : cluster [DBG] pgmap v82: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:58.490 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:47:58 smithi175 bash[29284]: cluster 2024-03-26T03:47:56.544452+0000 mgr.y (mgr.24431) 132 : cluster [DBG] pgmap v82: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:47:59.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:47:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:47:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:47:59.990 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:47:59 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:48:00.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:00 smithi190 bash[23392]: cluster 2024-03-26T03:47:58.545268+0000 mgr.y (mgr.24431) 133 : cluster [DBG] pgmap v83: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:00.490 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:00 smithi175 bash[29284]: cluster 2024-03-26T03:47:58.545268+0000 mgr.y (mgr.24431) 133 : cluster [DBG] pgmap v83: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:00.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:00 smithi175 bash[21527]: cluster 2024-03-26T03:47:58.545268+0000 mgr.y (mgr.24431) 133 : cluster [DBG] pgmap v83: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:01.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:01 smithi190 bash[23392]: audit 2024-03-26T03:47:59.638913+0000 mgr.y (mgr.24431) 134 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:48:01.490 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:01 smithi175 bash[29284]: audit 2024-03-26T03:47:59.638913+0000 mgr.y (mgr.24431) 134 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:48:01.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:01 smithi175 bash[21527]: audit 2024-03-26T03:47:59.638913+0000 mgr.y (mgr.24431) 134 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:48:02.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:02 smithi190 bash[23392]: cluster 2024-03-26T03:48:00.546779+0000 mgr.y (mgr.24431) 135 : cluster [DBG] pgmap v84: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:02.490 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:02 smithi175 bash[29284]: cluster 2024-03-26T03:48:00.546779+0000 mgr.y (mgr.24431) 135 : cluster [DBG] pgmap v84: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:02.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:02 smithi175 bash[21527]: cluster 2024-03-26T03:48:00.546779+0000 mgr.y (mgr.24431) 135 : cluster [DBG] pgmap v84: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:04.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:04 smithi190 bash[23392]: cluster 2024-03-26T03:48:02.547569+0000 mgr.y (mgr.24431) 136 : cluster [DBG] pgmap v85: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:04.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:04 smithi190 bash[23392]: audit 2024-03-26T03:48:03.886495+0000 mon.c (mon.2) 86 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:48:04.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:04 smithi175 bash[29284]: cluster 2024-03-26T03:48:02.547569+0000 mgr.y (mgr.24431) 136 : cluster [DBG] pgmap v85: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:04.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:04 smithi175 bash[29284]: audit 2024-03-26T03:48:03.886495+0000 mon.c (mon.2) 86 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:48:04.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:04 smithi175 bash[21527]: cluster 2024-03-26T03:48:02.547569+0000 mgr.y (mgr.24431) 136 : cluster [DBG] pgmap v85: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:04.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:04 smithi175 bash[21527]: audit 2024-03-26T03:48:03.886495+0000 mon.c (mon.2) 86 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:48:06.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:06 smithi190 bash[23392]: cluster 2024-03-26T03:48:04.548977+0000 mgr.y (mgr.24431) 137 : cluster [DBG] pgmap v86: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:06.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:06 smithi175 bash[21527]: cluster 2024-03-26T03:48:04.548977+0000 mgr.y (mgr.24431) 137 : cluster [DBG] pgmap v86: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:06.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:06 smithi175 bash[29284]: cluster 2024-03-26T03:48:04.548977+0000 mgr.y (mgr.24431) 137 : cluster [DBG] pgmap v86: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:08.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:08 smithi190 bash[23392]: cluster 2024-03-26T03:48:06.549705+0000 mgr.y (mgr.24431) 138 : cluster [DBG] pgmap v87: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:08.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:08 smithi175 bash[21527]: cluster 2024-03-26T03:48:06.549705+0000 mgr.y (mgr.24431) 138 : cluster [DBG] pgmap v87: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:08.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:08 smithi175 bash[29284]: cluster 2024-03-26T03:48:06.549705+0000 mgr.y (mgr.24431) 138 : cluster [DBG] pgmap v87: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:09.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:48:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:48:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:48:09.990 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:48:09 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:48:10.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:10 smithi190 bash[23392]: cluster 2024-03-26T03:48:08.550410+0000 mgr.y (mgr.24431) 139 : cluster [DBG] pgmap v88: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:10.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:10 smithi175 bash[21527]: cluster 2024-03-26T03:48:08.550410+0000 mgr.y (mgr.24431) 139 : cluster [DBG] pgmap v88: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:10.490 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:10 smithi175 bash[29284]: cluster 2024-03-26T03:48:08.550410+0000 mgr.y (mgr.24431) 139 : cluster [DBG] pgmap v88: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:11.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:11 smithi190 bash[23392]: audit 2024-03-26T03:48:09.646838+0000 mgr.y (mgr.24431) 140 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:48:11.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:11 smithi175 bash[21527]: audit 2024-03-26T03:48:09.646838+0000 mgr.y (mgr.24431) 140 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:48:11.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:11 smithi175 bash[29284]: audit 2024-03-26T03:48:09.646838+0000 mgr.y (mgr.24431) 140 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:48:12.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:12 smithi190 bash[23392]: cluster 2024-03-26T03:48:10.551879+0000 mgr.y (mgr.24431) 141 : cluster [DBG] pgmap v89: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:12.490 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:12 smithi175 bash[29284]: cluster 2024-03-26T03:48:10.551879+0000 mgr.y (mgr.24431) 141 : cluster [DBG] pgmap v89: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:12.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:12 smithi175 bash[21527]: cluster 2024-03-26T03:48:10.551879+0000 mgr.y (mgr.24431) 141 : cluster [DBG] pgmap v89: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:14.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:14 smithi190 bash[23392]: cluster 2024-03-26T03:48:12.552527+0000 mgr.y (mgr.24431) 142 : cluster [DBG] pgmap v90: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:14.490 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:14 smithi175 bash[29284]: cluster 2024-03-26T03:48:12.552527+0000 mgr.y (mgr.24431) 142 : cluster [DBG] pgmap v90: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:14.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:14 smithi175 bash[21527]: cluster 2024-03-26T03:48:12.552527+0000 mgr.y (mgr.24431) 142 : cluster [DBG] pgmap v90: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:16.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:16 smithi190 bash[23392]: cluster 2024-03-26T03:48:14.553925+0000 mgr.y (mgr.24431) 143 : cluster [DBG] pgmap v91: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:16.490 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:16 smithi175 bash[29284]: cluster 2024-03-26T03:48:14.553925+0000 mgr.y (mgr.24431) 143 : cluster [DBG] pgmap v91: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:16.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:16 smithi175 bash[21527]: cluster 2024-03-26T03:48:14.553925+0000 mgr.y (mgr.24431) 143 : cluster [DBG] pgmap v91: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:18.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:18 smithi190 bash[23392]: cluster 2024-03-26T03:48:16.554425+0000 mgr.y (mgr.24431) 144 : cluster [DBG] pgmap v92: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:18.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:18 smithi175 bash[21527]: cluster 2024-03-26T03:48:16.554425+0000 mgr.y (mgr.24431) 144 : cluster [DBG] pgmap v92: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:18.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:18 smithi175 bash[29284]: cluster 2024-03-26T03:48:16.554425+0000 mgr.y (mgr.24431) 144 : cluster [DBG] pgmap v92: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:19.146 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:48:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:48:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:48:19.147 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:19 smithi175 bash[21527]: audit 2024-03-26T03:48:18.917692+0000 mon.c (mon.2) 87 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:48:19.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:19 smithi190 bash[23392]: audit 2024-03-26T03:48:18.917692+0000 mon.c (mon.2) 87 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:48:19.490 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:19 smithi175 bash[29284]: audit 2024-03-26T03:48:18.917692+0000 mon.c (mon.2) 87 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:48:19.990 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:48:19 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:48:20.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:20 smithi190 bash[23392]: cluster 2024-03-26T03:48:18.555033+0000 mgr.y (mgr.24431) 145 : cluster [DBG] pgmap v93: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:20.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:20 smithi175 bash[21527]: cluster 2024-03-26T03:48:18.555033+0000 mgr.y (mgr.24431) 145 : cluster [DBG] pgmap v93: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:20.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:20 smithi175 bash[29284]: cluster 2024-03-26T03:48:18.555033+0000 mgr.y (mgr.24431) 145 : cluster [DBG] pgmap v93: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:21.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:21 smithi190 bash[23392]: audit 2024-03-26T03:48:19.658257+0000 mgr.y (mgr.24431) 146 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:48:21.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:21 smithi175 bash[21527]: audit 2024-03-26T03:48:19.658257+0000 mgr.y (mgr.24431) 146 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:48:21.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:21 smithi175 bash[29284]: audit 2024-03-26T03:48:19.658257+0000 mgr.y (mgr.24431) 146 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:48:22.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:22 smithi190 bash[23392]: cluster 2024-03-26T03:48:20.556413+0000 mgr.y (mgr.24431) 147 : cluster [DBG] pgmap v94: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:22.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:22 smithi175 bash[21527]: cluster 2024-03-26T03:48:20.556413+0000 mgr.y (mgr.24431) 147 : cluster [DBG] pgmap v94: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:22.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:22 smithi175 bash[29284]: cluster 2024-03-26T03:48:20.556413+0000 mgr.y (mgr.24431) 147 : cluster [DBG] pgmap v94: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:24.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:24 smithi190 bash[23392]: cluster 2024-03-26T03:48:22.557019+0000 mgr.y (mgr.24431) 148 : cluster [DBG] pgmap v95: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:24.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:24 smithi175 bash[29284]: cluster 2024-03-26T03:48:22.557019+0000 mgr.y (mgr.24431) 148 : cluster [DBG] pgmap v95: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:24.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:24 smithi175 bash[21527]: cluster 2024-03-26T03:48:22.557019+0000 mgr.y (mgr.24431) 148 : cluster [DBG] pgmap v95: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:26.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:26 smithi190 bash[23392]: cluster 2024-03-26T03:48:24.558482+0000 mgr.y (mgr.24431) 149 : cluster [DBG] pgmap v96: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:26.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:26 smithi175 bash[21527]: cluster 2024-03-26T03:48:24.558482+0000 mgr.y (mgr.24431) 149 : cluster [DBG] pgmap v96: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:26.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:26 smithi175 bash[29284]: cluster 2024-03-26T03:48:24.558482+0000 mgr.y (mgr.24431) 149 : cluster [DBG] pgmap v96: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:28.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:28 smithi190 bash[23392]: cluster 2024-03-26T03:48:26.558881+0000 mgr.y (mgr.24431) 150 : cluster [DBG] pgmap v97: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:28.490 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:28 smithi175 bash[29284]: cluster 2024-03-26T03:48:26.558881+0000 mgr.y (mgr.24431) 150 : cluster [DBG] pgmap v97: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:28.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:28 smithi175 bash[21527]: cluster 2024-03-26T03:48:26.558881+0000 mgr.y (mgr.24431) 150 : cluster [DBG] pgmap v97: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:29.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:48:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:48:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:48:29.990 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:48:29 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:48:30.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:30 smithi190 bash[23392]: cluster 2024-03-26T03:48:28.559198+0000 mgr.y (mgr.24431) 151 : cluster [DBG] pgmap v98: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:30.490 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:30 smithi175 bash[29284]: cluster 2024-03-26T03:48:28.559198+0000 mgr.y (mgr.24431) 151 : cluster [DBG] pgmap v98: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:30.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:30 smithi175 bash[21527]: cluster 2024-03-26T03:48:28.559198+0000 mgr.y (mgr.24431) 151 : cluster [DBG] pgmap v98: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:31.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:31 smithi190 bash[23392]: audit 2024-03-26T03:48:29.665727+0000 mgr.y (mgr.24431) 152 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:48:31.490 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:31 smithi175 bash[29284]: audit 2024-03-26T03:48:29.665727+0000 mgr.y (mgr.24431) 152 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:48:31.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:31 smithi175 bash[21527]: audit 2024-03-26T03:48:29.665727+0000 mgr.y (mgr.24431) 152 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:48:32.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:32 smithi190 bash[23392]: cluster 2024-03-26T03:48:30.559994+0000 mgr.y (mgr.24431) 153 : cluster [DBG] pgmap v99: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:32.490 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:32 smithi175 bash[29284]: cluster 2024-03-26T03:48:30.559994+0000 mgr.y (mgr.24431) 153 : cluster [DBG] pgmap v99: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:32.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:32 smithi175 bash[21527]: cluster 2024-03-26T03:48:30.559994+0000 mgr.y (mgr.24431) 153 : cluster [DBG] pgmap v99: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:34.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:34 smithi190 bash[23392]: cluster 2024-03-26T03:48:32.560399+0000 mgr.y (mgr.24431) 154 : cluster [DBG] pgmap v100: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:34.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:34 smithi190 bash[23392]: audit 2024-03-26T03:48:33.947869+0000 mon.c (mon.2) 88 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:48:34.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:34 smithi175 bash[29284]: cluster 2024-03-26T03:48:32.560399+0000 mgr.y (mgr.24431) 154 : cluster [DBG] pgmap v100: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:34.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:34 smithi175 bash[29284]: audit 2024-03-26T03:48:33.947869+0000 mon.c (mon.2) 88 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:48:34.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:34 smithi175 bash[21527]: cluster 2024-03-26T03:48:32.560399+0000 mgr.y (mgr.24431) 154 : cluster [DBG] pgmap v100: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:34.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:34 smithi175 bash[21527]: audit 2024-03-26T03:48:33.947869+0000 mon.c (mon.2) 88 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:48:36.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:36 smithi190 bash[23392]: cluster 2024-03-26T03:48:34.561185+0000 mgr.y (mgr.24431) 155 : cluster [DBG] pgmap v101: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:36.490 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:36 smithi175 bash[29284]: cluster 2024-03-26T03:48:34.561185+0000 mgr.y (mgr.24431) 155 : cluster [DBG] pgmap v101: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:36.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:36 smithi175 bash[21527]: cluster 2024-03-26T03:48:34.561185+0000 mgr.y (mgr.24431) 155 : cluster [DBG] pgmap v101: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:38.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:38 smithi190 bash[23392]: cluster 2024-03-26T03:48:36.561598+0000 mgr.y (mgr.24431) 156 : cluster [DBG] pgmap v102: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-26T03:48:38.490 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:38 smithi175 bash[29284]: cluster 2024-03-26T03:48:36.561598+0000 mgr.y (mgr.24431) 156 : cluster [DBG] pgmap v102: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-26T03:48:38.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:38 smithi175 bash[21527]: cluster 2024-03-26T03:48:36.561598+0000 mgr.y (mgr.24431) 156 : cluster [DBG] pgmap v102: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-26T03:48:39.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:48:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:48:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:48:39.990 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:48:39 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:48:40.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:40 smithi190 bash[23392]: cluster 2024-03-26T03:48:38.562023+0000 mgr.y (mgr.24431) 157 : cluster [DBG] pgmap v103: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-26T03:48:40.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:40 smithi175 bash[29284]: cluster 2024-03-26T03:48:38.562023+0000 mgr.y (mgr.24431) 157 : cluster [DBG] pgmap v103: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-26T03:48:40.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:40 smithi175 bash[21527]: cluster 2024-03-26T03:48:38.562023+0000 mgr.y (mgr.24431) 157 : cluster [DBG] pgmap v103: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-26T03:48:41.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:41 smithi190 bash[23392]: audit 2024-03-26T03:48:39.671007+0000 mgr.y (mgr.24431) 158 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:48:41.490 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:41 smithi175 bash[29284]: audit 2024-03-26T03:48:39.671007+0000 mgr.y (mgr.24431) 158 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:48:41.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:41 smithi175 bash[21527]: audit 2024-03-26T03:48:39.671007+0000 mgr.y (mgr.24431) 158 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:48:42.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:42 smithi190 bash[23392]: cluster 2024-03-26T03:48:40.562805+0000 mgr.y (mgr.24431) 159 : cluster [DBG] pgmap v104: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:42.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:42 smithi175 bash[29284]: cluster 2024-03-26T03:48:40.562805+0000 mgr.y (mgr.24431) 159 : cluster [DBG] pgmap v104: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:42.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:42 smithi175 bash[21527]: cluster 2024-03-26T03:48:40.562805+0000 mgr.y (mgr.24431) 159 : cluster [DBG] pgmap v104: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:44.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:44 smithi190 bash[23392]: cluster 2024-03-26T03:48:42.563229+0000 mgr.y (mgr.24431) 160 : cluster [DBG] pgmap v105: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-26T03:48:44.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:44 smithi175 bash[21527]: cluster 2024-03-26T03:48:42.563229+0000 mgr.y (mgr.24431) 160 : cluster [DBG] pgmap v105: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-26T03:48:44.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:44 smithi175 bash[29284]: cluster 2024-03-26T03:48:42.563229+0000 mgr.y (mgr.24431) 160 : cluster [DBG] pgmap v105: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-26T03:48:46.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:46 smithi190 bash[23392]: cluster 2024-03-26T03:48:44.564039+0000 mgr.y (mgr.24431) 161 : cluster [DBG] pgmap v106: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:46.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:46 smithi175 bash[21527]: cluster 2024-03-26T03:48:44.564039+0000 mgr.y (mgr.24431) 161 : cluster [DBG] pgmap v106: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:46.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:46 smithi175 bash[29284]: cluster 2024-03-26T03:48:44.564039+0000 mgr.y (mgr.24431) 161 : cluster [DBG] pgmap v106: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:48.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:48 smithi190 bash[23392]: cluster 2024-03-26T03:48:46.564461+0000 mgr.y (mgr.24431) 162 : cluster [DBG] pgmap v107: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-26T03:48:48.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:48 smithi175 bash[21527]: cluster 2024-03-26T03:48:46.564461+0000 mgr.y (mgr.24431) 162 : cluster [DBG] pgmap v107: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-26T03:48:48.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:48 smithi175 bash[29284]: cluster 2024-03-26T03:48:46.564461+0000 mgr.y (mgr.24431) 162 : cluster [DBG] pgmap v107: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-26T03:48:49.227 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:48:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:48:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:48:49.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:49 smithi190 bash[23392]: audit 2024-03-26T03:48:48.965068+0000 mon.c (mon.2) 89 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:48:49.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:49 smithi175 bash[21527]: audit 2024-03-26T03:48:48.965068+0000 mon.c (mon.2) 89 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:48:49.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:49 smithi175 bash[29284]: audit 2024-03-26T03:48:48.965068+0000 mon.c (mon.2) 89 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:48:49.990 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:48:49 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:48:50.444 INFO:tasks.workunit.client.0.smithi175.stderr:Updating files: 95% (11686/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-26T03:48:50.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:50 smithi190 bash[23392]: cluster 2024-03-26T03:48:48.565091+0000 mgr.y (mgr.24431) 163 : cluster [DBG] pgmap v108: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-26T03:48:50.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:50 smithi190 bash[23392]: audit 2024-03-26T03:48:49.434122+0000 mon.c (mon.2) 90 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:48:50.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:50 smithi190 bash[23392]: audit 2024-03-26T03:48:50.169165+0000 mon.c (mon.2) 91 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:48:50.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:50 smithi190 bash[23392]: audit 2024-03-26T03:48:50.170917+0000 mon.c (mon.2) 92 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:48:50.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:50 smithi190 bash[23392]: audit 2024-03-26T03:48:50.180995+0000 mon.a (mon.0) 863 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:48:50.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:50 smithi175 bash[21527]: cluster 2024-03-26T03:48:48.565091+0000 mgr.y (mgr.24431) 163 : cluster [DBG] pgmap v108: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-26T03:48:50.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:50 smithi175 bash[21527]: audit 2024-03-26T03:48:49.434122+0000 mon.c (mon.2) 90 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:48:50.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:50 smithi175 bash[21527]: audit 2024-03-26T03:48:50.169165+0000 mon.c (mon.2) 91 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:48:50.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:50 smithi175 bash[21527]: audit 2024-03-26T03:48:50.170917+0000 mon.c (mon.2) 92 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:48:50.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:50 smithi175 bash[21527]: audit 2024-03-26T03:48:50.180995+0000 mon.a (mon.0) 863 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:48:50.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:50 smithi175 bash[29284]: cluster 2024-03-26T03:48:48.565091+0000 mgr.y (mgr.24431) 163 : cluster [DBG] pgmap v108: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-26T03:48:50.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:50 smithi175 bash[29284]: audit 2024-03-26T03:48:49.434122+0000 mon.c (mon.2) 90 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:48:50.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:50 smithi175 bash[29284]: audit 2024-03-26T03:48:50.169165+0000 mon.c (mon.2) 91 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:48:50.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:50 smithi175 bash[29284]: audit 2024-03-26T03:48:50.170917+0000 mon.c (mon.2) 92 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:48:50.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:50 smithi175 bash[29284]: audit 2024-03-26T03:48:50.180995+0000 mon.a (mon.0) 863 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:48:51.048 INFO:tasks.workunit.client.0.smithi175.stderr:Note: switching to 'e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003'. 2024-03-26T03:48:51.048 INFO:tasks.workunit.client.0.smithi175.stderr: 2024-03-26T03:48:51.048 INFO:tasks.workunit.client.0.smithi175.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-26T03:48:51.048 INFO:tasks.workunit.client.0.smithi175.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-26T03:48:51.048 INFO:tasks.workunit.client.0.smithi175.stderr:state without impacting any branches by switching back to a branch. 2024-03-26T03:48:51.048 INFO:tasks.workunit.client.0.smithi175.stderr: 2024-03-26T03:48:51.048 INFO:tasks.workunit.client.0.smithi175.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-26T03:48:51.049 INFO:tasks.workunit.client.0.smithi175.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-26T03:48:51.049 INFO:tasks.workunit.client.0.smithi175.stderr: 2024-03-26T03:48:51.049 INFO:tasks.workunit.client.0.smithi175.stderr: git switch -c 2024-03-26T03:48:51.049 INFO:tasks.workunit.client.0.smithi175.stderr: 2024-03-26T03:48:51.049 INFO:tasks.workunit.client.0.smithi175.stderr:Or undo this operation with: 2024-03-26T03:48:51.049 INFO:tasks.workunit.client.0.smithi175.stderr: 2024-03-26T03:48:51.049 INFO:tasks.workunit.client.0.smithi175.stderr: git switch - 2024-03-26T03:48:51.049 INFO:tasks.workunit.client.0.smithi175.stderr: 2024-03-26T03:48:51.049 INFO:tasks.workunit.client.0.smithi175.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-26T03:48:51.049 INFO:tasks.workunit.client.0.smithi175.stderr: 2024-03-26T03:48:51.049 INFO:tasks.workunit.client.0.smithi175.stderr:HEAD is now at e42b8ea316e Merge branch 'rgw-bootstrap-hostnames' of https://github.com/adk3798/ceph into wip-adk-testing-2024-03-25-1628 2024-03-26T03:48:51.055 DEBUG:teuthology.orchestra.run.smithi175:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-26T03:48:51.103 INFO:tasks.workunit.client.0.smithi175.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-26T03:48:51.107 INFO:tasks.workunit.client.0.smithi175.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-26T03:48:51.107 INFO:tasks.workunit.client.0.smithi175.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-26T03:48:51.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:51 smithi190 bash[23392]: audit 2024-03-26T03:48:49.682361+0000 mgr.y (mgr.24431) 164 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:48:51.490 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:51 smithi175 bash[29284]: audit 2024-03-26T03:48:49.682361+0000 mgr.y (mgr.24431) 164 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:48:51.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:51 smithi175 bash[21527]: audit 2024-03-26T03:48:49.682361+0000 mgr.y (mgr.24431) 164 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:48:51.885 INFO:tasks.workunit.client.0.smithi175.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-26T03:48:51.968 INFO:tasks.workunit.client.0.smithi175.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-26T03:48:52.020 INFO:tasks.workunit.client.0.smithi175.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-26T03:48:52.020 INFO:tasks.workunit.client.0.smithi175.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-26T03:48:52.020 INFO:tasks.workunit.client.0.smithi175.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-26T03:48:52.070 INFO:tasks.workunit.client.0.smithi175.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-26T03:48:52.074 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-03-26T03:48:52.074 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-26T03:48:52.119 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-03-26T03:48:52.120 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-03-26T03:48:52.121 DEBUG:teuthology.orchestra.run.smithi175:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2024-03-26T03:48:52.172 INFO:tasks.workunit.client.0.smithi175.stderr:+ parallel=1 2024-03-26T03:48:52.172 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' '' = --serial ']' 2024-03-26T03:48:52.173 INFO:tasks.workunit.client.0.smithi175.stderr:+ crimson=0 2024-03-26T03:48:52.173 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' '' = --crimson ']' 2024-03-26T03:48:52.173 INFO:tasks.workunit.client.0.smithi175.stderr:+ color= 2024-03-26T03:48:52.173 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -t 1 ']' 2024-03-26T03:48:52.173 INFO:tasks.workunit.client.0.smithi175.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-03-26T03:48:52.173 INFO:tasks.workunit.client.0.smithi175.stderr:+ GTEST_OUTPUT_DIR=/home/ubuntu/cephtest/archive/unit_test_xml_report 2024-03-26T03:48:52.173 INFO:tasks.workunit.client.0.smithi175.stderr:+ mkdir -p /home/ubuntu/cephtest/archive/unit_test_xml_report 2024-03-26T03:48:52.174 INFO:tasks.workunit.client.0.smithi175.stderr:+ declare -A pids 2024-03-26T03:48:52.174 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-26T03:48:52.174 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.174 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s api_aio 2024-03-26T03:48:52.175 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' api_aio' 2024-03-26T03:48:52.175 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.175 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo api_aio 2024-03-26T03:48:52.177 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=api_aio 2024-03-26T03:48:52.177 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64393 2024-03-26T03:48:52.178 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test api_aio on pid 64393' 2024-03-26T03:48:52.178 INFO:tasks.workunit.client.0.smithi175.stdout:test api_aio on pid 64393 2024-03-26T03:48:52.179 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=64393 2024-03-26T03:48:52.179 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-26T03:48:52.179 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.179 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio.xml 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2024-03-26T03:48:52.179 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s api_aio_pp 2024-03-26T03:48:52.179 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' api_aio_pp' 2024-03-26T03:48:52.180 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.180 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.180 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.180 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.180 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo api_aio_pp 2024-03-26T03:48:52.180 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.180 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio.xml 2024-03-26T03:48:52.180 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_rados_api_aio.log 2024-03-26T03:48:52.180 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ api_aio: /' 2024-03-26T03:48:52.180 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=api_aio_pp 2024-03-26T03:48:52.181 INFO:tasks.workunit.client.0.smithi175.stdout:test api_aio_pp on pid 64401 2024-03-26T03:48:52.181 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64401 2024-03-26T03:48:52.182 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test api_aio_pp on pid 64401' 2024-03-26T03:48:52.182 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=64401 2024-03-26T03:48:52.182 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-26T03:48:52.182 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.182 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio_pp.xml 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2024-03-26T03:48:52.182 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s api_io 2024-03-26T03:48:52.182 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' api_io' 2024-03-26T03:48:52.182 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo api_io 2024-03-26T03:48:52.182 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.183 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.183 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.183 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.183 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.184 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio_pp.xml 2024-03-26T03:48:52.184 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_rados_api_aio_pp.log 2024-03-26T03:48:52.184 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ api_aio_pp: /' 2024-03-26T03:48:52.184 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=api_io 2024-03-26T03:48:52.184 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64409 2024-03-26T03:48:52.184 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test api_io on pid 64409' 2024-03-26T03:48:52.185 INFO:tasks.workunit.client.0.smithi175.stdout:test api_io on pid 64409 2024-03-26T03:48:52.186 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=64409 2024-03-26T03:48:52.186 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io.xml 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2024-03-26T03:48:52.186 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-26T03:48:52.186 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.186 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s api_io_pp 2024-03-26T03:48:52.186 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' api_io_pp' 2024-03-26T03:48:52.186 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo api_io_pp 2024-03-26T03:48:52.186 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.186 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.186 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.186 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.186 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.187 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io.xml 2024-03-26T03:48:52.187 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_rados_api_io.log 2024-03-26T03:48:52.187 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ api_io: /' 2024-03-26T03:48:52.188 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=api_io_pp 2024-03-26T03:48:52.188 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64417 2024-03-26T03:48:52.188 INFO:tasks.workunit.client.0.smithi175.stdout:test api_io_pp on pid 64417 2024-03-26T03:48:52.189 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test api_io_pp on pid 64417' 2024-03-26T03:48:52.190 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=64417 2024-03-26T03:48:52.190 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-26T03:48:52.190 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.190 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io_pp.xml 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2024-03-26T03:48:52.190 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s api_asio 2024-03-26T03:48:52.190 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' api_asio' 2024-03-26T03:48:52.190 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.190 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.190 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.190 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.190 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo api_asio 2024-03-26T03:48:52.190 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.190 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io_pp.xml 2024-03-26T03:48:52.191 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_rados_api_io_pp.log 2024-03-26T03:48:52.191 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ api_io_pp: /' 2024-03-26T03:48:52.191 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=api_asio 2024-03-26T03:48:52.192 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64425 2024-03-26T03:48:52.192 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test api_asio on pid 64425' 2024-03-26T03:48:52.192 INFO:tasks.workunit.client.0.smithi175.stdout:test api_asio on pid 64425 2024-03-26T03:48:52.193 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=64425 2024-03-26T03:48:52.193 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-26T03:48:52.193 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_asio.xml 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2024-03-26T03:48:52.193 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.193 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s api_list 2024-03-26T03:48:52.193 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' api_list' 2024-03-26T03:48:52.194 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo api_list 2024-03-26T03:48:52.194 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.194 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.194 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.194 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.194 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.194 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_asio.xml 2024-03-26T03:48:52.194 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_rados_api_asio.log 2024-03-26T03:48:52.195 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ api_asio: /' 2024-03-26T03:48:52.196 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=api_list 2024-03-26T03:48:52.196 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64433 2024-03-26T03:48:52.196 INFO:tasks.workunit.client.0.smithi175.stdout:test api_list on pid 64433 2024-03-26T03:48:52.197 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test api_list on pid 64433' 2024-03-26T03:48:52.197 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=64433 2024-03-26T03:48:52.197 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-26T03:48:52.197 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.197 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_list.xml 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2024-03-26T03:48:52.197 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s api_lock 2024-03-26T03:48:52.197 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' api_lock' 2024-03-26T03:48:52.197 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo api_lock 2024-03-26T03:48:52.198 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.198 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.198 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.198 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.198 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_list.xml 2024-03-26T03:48:52.198 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_rados_api_list.log 2024-03-26T03:48:52.202 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ api_list: /' 2024-03-26T03:48:52.214 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.217 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=api_lock 2024-03-26T03:48:52.217 INFO:tasks.workunit.client.0.smithi175.stdout:test api_lock on pid 64456 2024-03-26T03:48:52.217 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64456 2024-03-26T03:48:52.217 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test api_lock on pid 64456' 2024-03-26T03:48:52.218 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=64456 2024-03-26T03:48:52.218 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-26T03:48:52.218 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.218 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock.xml 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2024-03-26T03:48:52.218 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s api_lock_pp 2024-03-26T03:48:52.218 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' api_lock_pp' 2024-03-26T03:48:52.218 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.219 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.219 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.219 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.219 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock.xml 2024-03-26T03:48:52.219 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo api_lock_pp 2024-03-26T03:48:52.219 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_rados_api_lock.log 2024-03-26T03:48:52.221 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.222 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ api_lock: /' 2024-03-26T03:48:52.226 INFO:tasks.workunit.client.0.smithi175.stdout:test api_lock_pp on pid 64495 2024-03-26T03:48:52.226 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=api_lock_pp 2024-03-26T03:48:52.226 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64495 2024-03-26T03:48:52.226 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test api_lock_pp on pid 64495' 2024-03-26T03:48:52.226 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=64495 2024-03-26T03:48:52.226 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-26T03:48:52.226 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock_pp.xml 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2024-03-26T03:48:52.227 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.229 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.229 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.229 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.229 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.229 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock_pp.xml 2024-03-26T03:48:52.229 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s api_misc 2024-03-26T03:48:52.230 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' api_misc' 2024-03-26T03:48:52.230 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.232 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_rados_api_lock_pp.log 2024-03-26T03:48:52.234 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo api_misc 2024-03-26T03:48:52.236 INFO:tasks.workunit.client.0.smithi175.stdout:test api_misc on pid 64520 2024-03-26T03:48:52.236 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=api_misc 2024-03-26T03:48:52.236 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64520 2024-03-26T03:48:52.236 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test api_misc on pid 64520' 2024-03-26T03:48:52.236 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=64520 2024-03-26T03:48:52.236 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-26T03:48:52.236 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.236 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s api_misc_pp 2024-03-26T03:48:52.237 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' api_misc_pp' 2024-03-26T03:48:52.237 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ api_lock_pp: /' 2024-03-26T03:48:52.238 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc.xml 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2024-03-26T03:48:52.239 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo api_misc_pp 2024-03-26T03:48:52.239 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.241 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.242 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.242 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.242 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.242 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=api_misc_pp 2024-03-26T03:48:52.242 INFO:tasks.workunit.client.0.smithi175.stdout:test api_misc_pp on pid 64541 2024-03-26T03:48:52.243 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64541 2024-03-26T03:48:52.243 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test api_misc_pp on pid 64541' 2024-03-26T03:48:52.243 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=64541 2024-03-26T03:48:52.243 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-26T03:48:52.243 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.243 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc.xml 2024-03-26T03:48:52.244 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc_pp.xml 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2024-03-26T03:48:52.244 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_rados_api_misc.log 2024-03-26T03:48:52.246 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s api_tier_pp 2024-03-26T03:48:52.246 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.246 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.246 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.246 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.246 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' api_tier_pp' 2024-03-26T03:48:52.246 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_rados_api_misc_pp.log 2024-03-26T03:48:52.247 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ api_misc: /' 2024-03-26T03:48:52.247 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc_pp.xml 2024-03-26T03:48:52.247 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ api_misc_pp: /' 2024-03-26T03:48:52.247 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.248 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo api_tier_pp 2024-03-26T03:48:52.250 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=api_tier_pp 2024-03-26T03:48:52.250 INFO:tasks.workunit.client.0.smithi175.stdout:test api_tier_pp on pid 64550 2024-03-26T03:48:52.250 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64550 2024-03-26T03:48:52.250 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test api_tier_pp on pid 64550' 2024-03-26T03:48:52.251 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=64550 2024-03-26T03:48:52.251 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-26T03:48:52.251 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.255 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s api_pool 2024-03-26T03:48:52.256 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' api_pool' 2024-03-26T03:48:52.271 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.275 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_tier_pp.xml 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2024-03-26T03:48:52.275 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo api_pool 2024-03-26T03:48:52.277 INFO:tasks.workunit.client.0.smithi175.stdout:test api_pool on pid 64588 2024-03-26T03:48:52.277 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=api_pool 2024-03-26T03:48:52.277 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64588 2024-03-26T03:48:52.277 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test api_pool on pid 64588' 2024-03-26T03:48:52.277 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=64588 2024-03-26T03:48:52.277 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-26T03:48:52.277 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.282 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_pool.xml 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2024-03-26T03:48:52.282 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.282 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.282 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.283 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.283 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.283 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.283 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.283 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.283 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_pool.xml 2024-03-26T03:48:52.283 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_tier_pp.xml 2024-03-26T03:48:52.283 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s api_snapshots 2024-03-26T03:48:52.284 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' api_snapshots' 2024-03-26T03:48:52.286 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ api_tier_pp: /' 2024-03-26T03:48:52.286 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo api_snapshots 2024-03-26T03:48:52.287 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_rados_api_tier_pp.log 2024-03-26T03:48:52.287 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.288 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ api_pool: /' 2024-03-26T03:48:52.289 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_rados_api_pool.log 2024-03-26T03:48:52.290 INFO:tasks.workunit.client.0.smithi175.stdout:test api_snapshots on pid 64617 2024-03-26T03:48:52.290 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=api_snapshots 2024-03-26T03:48:52.290 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64617 2024-03-26T03:48:52.290 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test api_snapshots on pid 64617' 2024-03-26T03:48:52.290 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=64617 2024-03-26T03:48:52.290 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-26T03:48:52.290 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.291 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s api_snapshots_pp 2024-03-26T03:48:52.291 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' api_snapshots_pp' 2024-03-26T03:48:52.291 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo api_snapshots_pp 2024-03-26T03:48:52.291 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.295 INFO:tasks.workunit.client.0.smithi175.stdout:test api_snapshots_pp on pid 64630 2024-03-26T03:48:52.295 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=api_snapshots_pp 2024-03-26T03:48:52.295 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64630 2024-03-26T03:48:52.295 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test api_snapshots_pp on pid 64630' 2024-03-26T03:48:52.295 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=64630 2024-03-26T03:48:52.295 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-26T03:48:52.295 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.298 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s api_stat 2024-03-26T03:48:52.298 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' api_stat' 2024-03-26T03:48:52.298 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots_pp.xml 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2024-03-26T03:48:52.298 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo api_stat 2024-03-26T03:48:52.298 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots.xml 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2024-03-26T03:48:52.300 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.300 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.300 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.300 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.300 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots_pp.xml 2024-03-26T03:48:52.300 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ api_snapshots_pp: /' 2024-03-26T03:48:52.300 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.300 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.300 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.300 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.301 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.301 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots.xml 2024-03-26T03:48:52.301 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ api_snapshots: /' 2024-03-26T03:48:52.301 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_rados_api_snapshots.log 2024-03-26T03:48:52.307 INFO:tasks.workunit.client.0.smithi175.stdout:test api_stat on pid 64647 2024-03-26T03:48:52.308 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=api_stat 2024-03-26T03:48:52.308 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64647 2024-03-26T03:48:52.308 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test api_stat on pid 64647' 2024-03-26T03:48:52.308 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=64647 2024-03-26T03:48:52.308 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-26T03:48:52.308 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.308 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat.xml 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2024-03-26T03:48:52.308 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s api_stat_pp 2024-03-26T03:48:52.308 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' api_stat_pp' 2024-03-26T03:48:52.309 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.309 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.309 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.309 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.310 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_rados_api_stat.log 2024-03-26T03:48:52.311 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2024-03-26T03:48:52.311 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ api_stat: /' 2024-03-26T03:48:52.315 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.315 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat.xml 2024-03-26T03:48:52.319 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo api_stat_pp 2024-03-26T03:48:52.320 INFO:tasks.workunit.client.0.smithi175.stdout:test api_stat_pp on pid 64661 2024-03-26T03:48:52.321 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=api_stat_pp 2024-03-26T03:48:52.321 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64661 2024-03-26T03:48:52.321 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test api_stat_pp on pid 64661' 2024-03-26T03:48:52.321 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=64661 2024-03-26T03:48:52.321 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-26T03:48:52.321 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.321 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s api_watch_notify 2024-03-26T03:48:52.321 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' api_watch_notify' 2024-03-26T03:48:52.322 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat_pp.xml 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2024-03-26T03:48:52.324 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.324 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.324 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.324 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.324 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo api_watch_notify 2024-03-26T03:48:52.324 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ api_stat_pp: /' 2024-03-26T03:48:52.326 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_rados_api_stat_pp.log 2024-03-26T03:48:52.327 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat_pp.xml 2024-03-26T03:48:52.334 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.337 INFO:tasks.workunit.client.0.smithi175.stdout:test api_watch_notify on pid 64713 2024-03-26T03:48:52.338 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=api_watch_notify 2024-03-26T03:48:52.338 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64713 2024-03-26T03:48:52.338 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test api_watch_notify on pid 64713' 2024-03-26T03:48:52.338 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=64713 2024-03-26T03:48:52.339 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-26T03:48:52.339 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.339 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify.xml 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2024-03-26T03:48:52.340 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.341 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.341 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.341 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.341 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ api_watch_notify: /' 2024-03-26T03:48:52.342 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_rados_api_watch_notify.log 2024-03-26T03:48:52.342 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s api_watch_notify_pp 2024-03-26T03:48:52.343 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify.xml 2024-03-26T03:48:52.343 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' api_watch_notify_pp' 2024-03-26T03:48:52.345 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.345 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo api_watch_notify_pp 2024-03-26T03:48:52.353 INFO:tasks.workunit.client.0.smithi175.stdout:test api_watch_notify_pp on pid 64767 2024-03-26T03:48:52.354 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=api_watch_notify_pp 2024-03-26T03:48:52.354 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64767 2024-03-26T03:48:52.354 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test api_watch_notify_pp on pid 64767' 2024-03-26T03:48:52.354 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=64767 2024-03-26T03:48:52.354 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-26T03:48:52.354 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.354 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify_pp.xml 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2024-03-26T03:48:52.355 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.355 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.355 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.355 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.355 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify_pp.xml 2024-03-26T03:48:52.358 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2024-03-26T03:48:52.359 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2024-03-26T03:48:52.360 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s api_cmd 2024-03-26T03:48:52.360 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' api_cmd' 2024-03-26T03:48:52.360 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.361 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo api_cmd 2024-03-26T03:48:52.362 INFO:tasks.workunit.client.0.smithi175.stdout:test api_cmd on pid 64779 2024-03-26T03:48:52.363 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=api_cmd 2024-03-26T03:48:52.363 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64779 2024-03-26T03:48:52.363 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test api_cmd on pid 64779' 2024-03-26T03:48:52.363 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=64779 2024-03-26T03:48:52.363 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-26T03:48:52.363 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.363 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd.xml 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2024-03-26T03:48:52.364 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.364 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.365 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.365 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.366 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd.xml 2024-03-26T03:48:52.366 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_rados_api_cmd.log 2024-03-26T03:48:52.370 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ api_cmd: /' 2024-03-26T03:48:52.376 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s api_cmd_pp 2024-03-26T03:48:52.377 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' api_cmd_pp' 2024-03-26T03:48:52.378 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.379 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo api_cmd_pp 2024-03-26T03:48:52.380 INFO:tasks.workunit.client.0.smithi175.stdout:test api_cmd_pp on pid 64829 2024-03-26T03:48:52.381 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=api_cmd_pp 2024-03-26T03:48:52.381 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64829 2024-03-26T03:48:52.381 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test api_cmd_pp on pid 64829' 2024-03-26T03:48:52.381 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=64829 2024-03-26T03:48:52.381 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-26T03:48:52.381 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.382 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd_pp.xml 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2024-03-26T03:48:52.382 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s api_service 2024-03-26T03:48:52.382 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' api_service' 2024-03-26T03:48:52.383 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo api_service 2024-03-26T03:48:52.383 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.385 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.385 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.385 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.385 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.385 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd_pp.xml 2024-03-26T03:48:52.385 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2024-03-26T03:48:52.386 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ api_cmd_pp: /' 2024-03-26T03:48:52.388 INFO:tasks.workunit.client.0.smithi175.stdout:test api_service on pid 64855 2024-03-26T03:48:52.388 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=api_service 2024-03-26T03:48:52.388 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64855 2024-03-26T03:48:52.388 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test api_service on pid 64855' 2024-03-26T03:48:52.388 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=64855 2024-03-26T03:48:52.389 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-26T03:48:52.389 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.389 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service.xml 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2024-03-26T03:48:52.390 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.390 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.390 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.391 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.391 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service.xml 2024-03-26T03:48:52.391 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s api_service_pp 2024-03-26T03:48:52.391 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' api_service_pp' 2024-03-26T03:48:52.391 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ api_service: /' 2024-03-26T03:48:52.391 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.391 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo api_service_pp 2024-03-26T03:48:52.392 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_rados_api_service.log 2024-03-26T03:48:52.392 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=api_service_pp 2024-03-26T03:48:52.393 INFO:tasks.workunit.client.0.smithi175.stdout:test api_service_pp on pid 64865 2024-03-26T03:48:52.393 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64865 2024-03-26T03:48:52.393 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test api_service_pp on pid 64865' 2024-03-26T03:48:52.393 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=64865 2024-03-26T03:48:52.393 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-26T03:48:52.393 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.394 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s api_c_write_operations 2024-03-26T03:48:52.394 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' api_c_write_operations' 2024-03-26T03:48:52.395 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo api_c_write_operations 2024-03-26T03:48:52.395 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.397 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service_pp.xml 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2024-03-26T03:48:52.398 INFO:tasks.workunit.client.0.smithi175.stdout:test api_c_write_operations on pid 64878 2024-03-26T03:48:52.399 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=api_c_write_operations 2024-03-26T03:48:52.399 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.399 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.399 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.399 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64878 2024-03-26T03:48:52.399 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.399 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test api_c_write_operations on pid 64878' 2024-03-26T03:48:52.399 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=64878 2024-03-26T03:48:52.399 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-26T03:48:52.399 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.399 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_write_operations.xml 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2024-03-26T03:48:52.400 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ api_service_pp: /' 2024-03-26T03:48:52.401 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.401 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.401 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.401 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.401 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_write_operations.xml 2024-03-26T03:48:52.401 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2024-03-26T03:48:52.401 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ api_c_write_operations: /' 2024-03-26T03:48:52.401 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service_pp.xml 2024-03-26T03:48:52.402 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s api_c_read_operations 2024-03-26T03:48:52.402 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' api_c_read_operations' 2024-03-26T03:48:52.403 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo api_c_read_operations 2024-03-26T03:48:52.404 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_rados_api_service_pp.log 2024-03-26T03:48:52.404 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.406 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=api_c_read_operations 2024-03-26T03:48:52.407 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64906 2024-03-26T03:48:52.407 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test api_c_read_operations on pid 64906' 2024-03-26T03:48:52.407 INFO:tasks.workunit.client.0.smithi175.stdout:test api_c_read_operations on pid 64906 2024-03-26T03:48:52.408 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=64906 2024-03-26T03:48:52.408 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-26T03:48:52.408 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.408 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_read_operations.xml 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2024-03-26T03:48:52.408 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s api_cls_remote_reads 2024-03-26T03:48:52.408 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' api_cls_remote_reads' 2024-03-26T03:48:52.408 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo api_cls_remote_reads 2024-03-26T03:48:52.408 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.410 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.410 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.410 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.410 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.411 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_read_operations.xml 2024-03-26T03:48:52.411 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=api_cls_remote_reads 2024-03-26T03:48:52.411 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64914 2024-03-26T03:48:52.411 INFO:tasks.workunit.client.0.smithi175.stdout:test api_cls_remote_reads on pid 64914 2024-03-26T03:48:52.412 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test api_cls_remote_reads on pid 64914' 2024-03-26T03:48:52.412 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=64914 2024-03-26T03:48:52.412 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-26T03:48:52.412 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.412 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2024-03-26T03:48:52.412 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cls_remote_reads --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cls_remote_reads.xml 2>&1 | tee ceph_test_rados_api_cls_remote_reads.log | sed "s/^/ api_cls_remote_reads: /"' 2024-03-26T03:48:52.413 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.413 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.413 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.413 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.413 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_rados_api_cls_remote_reads --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cls_remote_reads.xml 2024-03-26T03:48:52.413 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_rados_api_cls_remote_reads.log 2024-03-26T03:48:52.413 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ api_cls_remote_reads: /' 2024-03-26T03:48:52.419 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ api_c_read_operations: /' 2024-03-26T03:48:52.419 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s list_parallel 2024-03-26T03:48:52.422 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' list_parallel' 2024-03-26T03:48:52.425 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.428 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo list_parallel 2024-03-26T03:48:52.429 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=list_parallel 2024-03-26T03:48:52.429 INFO:tasks.workunit.client.0.smithi175.stdout:test list_parallel on pid 64943 2024-03-26T03:48:52.429 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64943 2024-03-26T03:48:52.429 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test list_parallel on pid 64943' 2024-03-26T03:48:52.429 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=64943 2024-03-26T03:48:52.429 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-26T03:48:52.429 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.430 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/list_parallel.xml 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2024-03-26T03:48:52.430 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s open_pools_parallel 2024-03-26T03:48:52.431 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' open_pools_parallel' 2024-03-26T03:48:52.431 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo open_pools_parallel 2024-03-26T03:48:52.432 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.433 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=open_pools_parallel 2024-03-26T03:48:52.434 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64953 2024-03-26T03:48:52.434 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test open_pools_parallel on pid 64953' 2024-03-26T03:48:52.434 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/open_pools_parallel.xml 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2024-03-26T03:48:52.434 INFO:tasks.workunit.client.0.smithi175.stdout:test open_pools_parallel on pid 64953 2024-03-26T03:48:52.435 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=64953 2024-03-26T03:48:52.435 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-26T03:48:52.436 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.436 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s delete_pools_parallel 2024-03-26T03:48:52.436 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' delete_pools_parallel' 2024-03-26T03:48:52.436 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.436 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.436 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.436 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.436 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.436 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/open_pools_parallel.xml 2024-03-26T03:48:52.436 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2024-03-26T03:48:52.439 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo delete_pools_parallel 2024-03-26T03:48:52.439 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.439 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.439 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.439 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.439 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_rados_list_parallel.log 2024-03-26T03:48:52.440 INFO:tasks.workunit.client.0.smithi175.stdout:test delete_pools_parallel on pid 64964 2024-03-26T03:48:52.441 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=delete_pools_parallel 2024-03-26T03:48:52.441 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ list_parallel: /' 2024-03-26T03:48:52.441 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64964 2024-03-26T03:48:52.441 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test delete_pools_parallel on pid 64964' 2024-03-26T03:48:52.442 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=64964 2024-03-26T03:48:52.442 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-26T03:48:52.442 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.442 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s cls 2024-03-26T03:48:52.442 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' cls' 2024-03-26T03:48:52.442 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.442 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo cls 2024-03-26T03:48:52.443 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=cls 2024-03-26T03:48:52.444 INFO:tasks.workunit.client.0.smithi175.stdout:test cls on pid 64976 2024-03-26T03:48:52.444 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64976 2024-03-26T03:48:52.444 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test cls on pid 64976' 2024-03-26T03:48:52.444 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=64976 2024-03-26T03:48:52.444 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-26T03:48:52.444 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cls 2>&1 | tee ceph_test_neorados_cls.log | sed "s/^/ cls: /"' 2024-03-26T03:48:52.444 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.446 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/list_parallel.xml 2024-03-26T03:48:52.447 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.447 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.447 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.447 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.447 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_neorados_cls 2024-03-26T03:48:52.449 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ cls: /' 2024-03-26T03:48:52.450 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_neorados_cls.log 2024-03-26T03:48:52.453 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s cmd 2024-03-26T03:48:52.453 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' cmd' 2024-03-26T03:48:52.454 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo cmd 2024-03-26T03:48:52.459 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ open_pools_parallel: /' 2024-03-26T03:48:52.459 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/delete_pools_parallel.xml 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2024-03-26T03:48:52.462 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.462 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.462 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.462 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.464 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.465 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/delete_pools_parallel.xml 2024-03-26T03:48:52.465 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2024-03-26T03:48:52.471 INFO:tasks.workunit.client.0.smithi175.stdout:test cmd on pid 65030 2024-03-26T03:48:52.471 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=cmd 2024-03-26T03:48:52.471 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=65030 2024-03-26T03:48:52.471 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test cmd on pid 65030' 2024-03-26T03:48:52.471 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=65030 2024-03-26T03:48:52.472 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-26T03:48:52.472 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.474 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ delete_pools_parallel: /' 2024-03-26T03:48:52.480 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s handler_error 2024-03-26T03:48:52.481 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' handler_error' 2024-03-26T03:48:52.483 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cmd 2>&1 | tee ceph_test_neorados_cmd.log | sed "s/^/ cmd: /"' 2024-03-26T03:48:52.484 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.484 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.484 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.484 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.484 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_neorados_cmd.log 2024-03-26T03:48:52.487 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ cmd: /' 2024-03-26T03:48:52.488 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.488 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_neorados_cmd 2024-03-26T03:48:52.490 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo handler_error 2024-03-26T03:48:52.493 INFO:tasks.workunit.client.0.smithi175.stdout:test handler_error on pid 65087 2024-03-26T03:48:52.494 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=handler_error 2024-03-26T03:48:52.494 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=65087 2024-03-26T03:48:52.495 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test handler_error on pid 65087' 2024-03-26T03:48:52.495 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=65087 2024-03-26T03:48:52.495 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-26T03:48:52.495 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.498 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_handler_error 2>&1 | tee ceph_test_neorados_handler_error.log | sed "s/^/ handler_error: /"' 2024-03-26T03:48:52.501 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s io 2024-03-26T03:48:52.501 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' io' 2024-03-26T03:48:52.504 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.504 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.505 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.505 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.510 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_neorados_handler_error 2024-03-26T03:48:52.510 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_neorados_handler_error.log 2024-03-26T03:48:52.511 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ handler_error: /' 2024-03-26T03:48:52.515 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo io 2024-03-26T03:48:52.518 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.521 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=io 2024-03-26T03:48:52.521 INFO:tasks.workunit.client.0.smithi175.stdout:test io on pid 65162 2024-03-26T03:48:52.521 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=65162 2024-03-26T03:48:52.521 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test io on pid 65162' 2024-03-26T03:48:52.522 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=65162 2024-03-26T03:48:52.522 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-26T03:48:52.522 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.522 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_io 2>&1 | tee ceph_test_neorados_io.log | sed "s/^/ io: /"' 2024-03-26T03:48:52.523 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.523 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.523 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.523 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.523 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s ec_io 2024-03-26T03:48:52.523 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ io: /' 2024-03-26T03:48:52.524 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' ec_io' 2024-03-26T03:48:52.524 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_neorados_io.log 2024-03-26T03:48:52.526 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_neorados_io 2024-03-26T03:48:52.526 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.533 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo ec_io 2024-03-26T03:48:52.537 INFO:tasks.workunit.client.0.smithi175.stdout:test ec_io on pid 65213 2024-03-26T03:48:52.537 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=ec_io 2024-03-26T03:48:52.537 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=65213 2024-03-26T03:48:52.537 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test ec_io on pid 65213' 2024-03-26T03:48:52.537 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=65213 2024-03-26T03:48:52.537 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-26T03:48:52.537 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.537 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_io 2>&1 | tee ceph_test_neorados_ec_io.log | sed "s/^/ ec_io: /"' 2024-03-26T03:48:52.539 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s list 2024-03-26T03:48:52.539 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' list' 2024-03-26T03:48:52.540 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.540 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.540 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.540 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.540 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_neorados_ec_io 2024-03-26T03:48:52.541 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ ec_io: /' 2024-03-26T03:48:52.543 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo list 2024-03-26T03:48:52.543 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.551 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=list 2024-03-26T03:48:52.551 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_neorados_ec_io.log 2024-03-26T03:48:52.551 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=65268 2024-03-26T03:48:52.551 INFO:tasks.workunit.client.0.smithi175.stdout:test list on pid 65268 2024-03-26T03:48:52.552 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test list on pid 65268' 2024-03-26T03:48:52.552 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=65268 2024-03-26T03:48:52.552 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-26T03:48:52.552 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.552 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_list 2>&1 | tee ceph_test_neorados_list.log | sed "s/^/ list: /"' 2024-03-26T03:48:52.554 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.554 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.554 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.554 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.554 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_neorados_list 2024-03-26T03:48:52.555 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ list: /' 2024-03-26T03:48:52.555 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_neorados_list.log 2024-03-26T03:48:52.555 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s ec_list 2024-03-26T03:48:52.555 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' ec_list' 2024-03-26T03:48:52.555 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo ec_list 2024-03-26T03:48:52.555 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.557 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=ec_list 2024-03-26T03:48:52.558 INFO:tasks.workunit.client.0.smithi175.stdout:test ec_list on pid 65291 2024-03-26T03:48:52.558 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=65291 2024-03-26T03:48:52.558 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test ec_list on pid 65291' 2024-03-26T03:48:52.558 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=65291 2024-03-26T03:48:52.559 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-26T03:48:52.559 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.559 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_list 2>&1 | tee ceph_test_neorados_ec_list.log | sed "s/^/ ec_list: /"' 2024-03-26T03:48:52.559 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s misc 2024-03-26T03:48:52.560 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' misc' 2024-03-26T03:48:52.560 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.560 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.561 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.561 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.561 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_neorados_ec_list 2024-03-26T03:48:52.561 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_neorados_ec_list.log 2024-03-26T03:48:52.561 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ ec_list: /' 2024-03-26T03:48:52.563 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo misc 2024-03-26T03:48:52.566 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.568 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=misc 2024-03-26T03:48:52.568 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=65320 2024-03-26T03:48:52.569 INFO:tasks.workunit.client.0.smithi175.stdout:test misc on pid 65320 2024-03-26T03:48:52.569 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test misc on pid 65320' 2024-03-26T03:48:52.569 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=65320 2024-03-26T03:48:52.570 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-26T03:48:52.570 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.570 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_misc 2>&1 | tee ceph_test_neorados_misc.log | sed "s/^/ misc: /"' 2024-03-26T03:48:52.570 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s pool 2024-03-26T03:48:52.570 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' pool' 2024-03-26T03:48:52.571 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo pool 2024-03-26T03:48:52.571 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.571 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.571 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.571 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.571 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.571 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_neorados_misc 2024-03-26T03:48:52.571 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_neorados_misc.log 2024-03-26T03:48:52.572 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ misc: /' 2024-03-26T03:48:52.575 INFO:tasks.workunit.client.0.smithi175.stdout:test pool on pid 65336 2024-03-26T03:48:52.576 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=pool 2024-03-26T03:48:52.576 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=65336 2024-03-26T03:48:52.576 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test pool on pid 65336' 2024-03-26T03:48:52.576 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=65336 2024-03-26T03:48:52.576 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-26T03:48:52.576 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.576 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s read_operations 2024-03-26T03:48:52.576 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' read_operations' 2024-03-26T03:48:52.577 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.577 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo read_operations 2024-03-26T03:48:52.577 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_pool 2>&1 | tee ceph_test_neorados_pool.log | sed "s/^/ pool: /"' 2024-03-26T03:48:52.578 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=read_operations 2024-03-26T03:48:52.578 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=65343 2024-03-26T03:48:52.579 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test read_operations on pid 65343' 2024-03-26T03:48:52.579 INFO:tasks.workunit.client.0.smithi175.stdout:test read_operations on pid 65343 2024-03-26T03:48:52.580 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=65343 2024-03-26T03:48:52.581 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-26T03:48:52.581 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.581 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_read_operations 2>&1 | tee ceph_test_neorados_read_operations.log | sed "s/^/ read_operations: /"' 2024-03-26T03:48:52.581 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s snapshots 2024-03-26T03:48:52.581 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' snapshots' 2024-03-26T03:48:52.581 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.581 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.581 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.581 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.581 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_neorados_pool 2024-03-26T03:48:52.581 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.581 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo snapshots 2024-03-26T03:48:52.581 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.581 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.582 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.582 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ pool: /' 2024-03-26T03:48:52.582 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.582 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ read_operations: /' 2024-03-26T03:48:52.582 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_neorados_pool.log 2024-03-26T03:48:52.582 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_neorados_read_operations 2024-03-26T03:48:52.583 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_neorados_read_operations.log 2024-03-26T03:48:52.584 INFO:tasks.workunit.client.0.smithi175.stdout:test snapshots on pid 65367 2024-03-26T03:48:52.584 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=snapshots 2024-03-26T03:48:52.584 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=65367 2024-03-26T03:48:52.584 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test snapshots on pid 65367' 2024-03-26T03:48:52.584 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=65367 2024-03-26T03:48:52.585 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-26T03:48:52.585 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.585 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s watch_notify 2024-03-26T03:48:52.585 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' watch_notify' 2024-03-26T03:48:52.589 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_snapshots 2>&1 | tee ceph_test_neorados_snapshots.log | sed "s/^/ snapshots: /"' 2024-03-26T03:48:52.591 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.591 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.591 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.591 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.591 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_neorados_snapshots 2024-03-26T03:48:52.591 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_neorados_snapshots.log 2024-03-26T03:48:52.592 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ snapshots: /' 2024-03-26T03:48:52.606 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.608 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo watch_notify 2024-03-26T03:48:52.612 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=watch_notify 2024-03-26T03:48:52.612 INFO:tasks.workunit.client.0.smithi175.stdout:test watch_notify on pid 65427 2024-03-26T03:48:52.613 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=65427 2024-03-26T03:48:52.613 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test watch_notify on pid 65427' 2024-03-26T03:48:52.613 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=65427 2024-03-26T03:48:52.613 INFO:tasks.workunit.client.0.smithi175.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-26T03:48:52.614 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.614 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_watch_notify 2>&1 | tee ceph_test_neorados_watch_notify.log | sed "s/^/ watch_notify: /"' 2024-03-26T03:48:52.614 INFO:tasks.workunit.client.0.smithi175.stderr:++ printf %25s write_operations 2024-03-26T03:48:52.614 INFO:tasks.workunit.client.0.smithi175.stderr:+ r=' write_operations' 2024-03-26T03:48:52.614 INFO:tasks.workunit.client.0.smithi175.stderr:++ echo write_operations 2024-03-26T03:48:52.614 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.615 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.615 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.615 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.616 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_neorados_watch_notify 2024-03-26T03:48:52.616 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ watch_notify: /' 2024-03-26T03:48:52.616 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_neorados_watch_notify.log 2024-03-26T03:48:52.616 INFO:tasks.workunit.client.0.smithi175.stderr:++ awk '{print $1}' 2024-03-26T03:48:52.618 INFO:tasks.workunit.client.0.smithi175.stdout:test write_operations on pid 65456 2024-03-26T03:48:52.618 INFO:tasks.workunit.client.0.smithi175.stderr:+ ff=write_operations 2024-03-26T03:48:52.618 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=65456 2024-03-26T03:48:52.618 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo 'test write_operations on pid 65456' 2024-03-26T03:48:52.619 INFO:tasks.workunit.client.0.smithi175.stderr:+ pids[$f]=65456 2024-03-26T03:48:52.619 INFO:tasks.workunit.client.0.smithi175.stderr:+ ret=0 2024-03-26T03:48:52.619 INFO:tasks.workunit.client.0.smithi175.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_write_operations 2>&1 | tee ceph_test_neorados_write_operations.log | sed "s/^/ write_operations: /"' 2024-03-26T03:48:52.619 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' 1 -eq 1 ']' 2024-03-26T03:48:52.619 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T03:48:52.619 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64550 2024-03-26T03:48:52.619 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 64550 2024-03-26T03:48:52.620 INFO:tasks.workunit.client.0.smithi175.stderr:+ '[' -z '' ']' 2024-03-26T03:48:52.620 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.620 INFO:tasks.workunit.client.0.smithi175.stderr:+ case $- in 2024-03-26T03:48:52.620 INFO:tasks.workunit.client.0.smithi175.stderr:+ return 2024-03-26T03:48:52.620 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph_test_neorados_write_operations 2024-03-26T03:48:52.620 INFO:tasks.workunit.client.0.smithi175.stderr:+ sed 's/^/ write_operations: /' 2024-03-26T03:48:52.620 INFO:tasks.workunit.client.0.smithi175.stderr:+ tee ceph_test_neorados_write_operations.log 2024-03-26T03:48:52.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:52 smithi190 bash[23392]: cluster 2024-03-26T03:48:50.566497+0000 mgr.y (mgr.24431) 165 : cluster [DBG] pgmap v109: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:52.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:52 smithi175 bash[21527]: cluster 2024-03-26T03:48:50.566497+0000 mgr.y (mgr.24431) 165 : cluster [DBG] pgmap v109: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:52.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:52 smithi175 bash[29284]: cluster 2024-03-26T03:48:50.566497+0000 mgr.y (mgr.24431) 165 : cluster [DBG] pgmap v109: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:48:53.486 INFO:tasks.workunit.client.0.smithi175.stdout: api_asio: [==========] Running 12 tests from 1 test suite. 2024-03-26T03:48:53.486 INFO:tasks.workunit.client.0.smithi175.stdout: api_asio: [----------] Global test environment set-up. 2024-03-26T03:48:53.487 INFO:tasks.workunit.client.0.smithi175.stdout: api_asio: [----------] 12 tests from AsioRados 2024-03-26T03:48:53.487 INFO:tasks.workunit.client.0.smithi175.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2024-03-26T03:48:53.487 INFO:tasks.workunit.client.0.smithi175.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (3 ms) 2024-03-26T03:48:53.487 INFO:tasks.workunit.client.0.smithi175.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2024-03-26T03:48:53.487 INFO:tasks.workunit.client.0.smithi175.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (2 ms) 2024-03-26T03:48:53.487 INFO:tasks.workunit.client.0.smithi175.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2024-03-26T03:48:53.487 INFO:tasks.workunit.client.0.smithi175.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (0 ms) 2024-03-26T03:48:53.487 INFO:tasks.workunit.client.0.smithi175.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2024-03-26T03:48:53.487 INFO:tasks.workunit.client.0.smithi175.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (1 ms) 2024-03-26T03:48:53.487 INFO:tasks.workunit.client.0.smithi175.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2024-03-26T03:48:53.487 INFO:tasks.workunit.client.0.smithi175.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (2 ms) 2024-03-26T03:48:53.487 INFO:tasks.workunit.client.0.smithi175.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2024-03-26T03:48:53.487 INFO:tasks.workunit.client.0.smithi175.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (1 ms) 2024-03-26T03:48:53.487 INFO:tasks.workunit.client.0.smithi175.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2024-03-26T03:48:53.487 INFO:tasks.workunit.client.0.smithi175.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (1 ms) 2024-03-26T03:48:53.488 INFO:tasks.workunit.client.0.smithi175.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2024-03-26T03:48:53.488 INFO:tasks.workunit.client.0.smithi175.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (0 ms) 2024-03-26T03:48:53.488 INFO:tasks.workunit.client.0.smithi175.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2024-03-26T03:48:53.488 INFO:tasks.workunit.client.0.smithi175.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (1 ms) 2024-03-26T03:48:53.488 INFO:tasks.workunit.client.0.smithi175.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2024-03-26T03:48:53.488 INFO:tasks.workunit.client.0.smithi175.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (1 ms) 2024-03-26T03:48:53.488 INFO:tasks.workunit.client.0.smithi175.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2024-03-26T03:48:53.488 INFO:tasks.workunit.client.0.smithi175.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (1 ms) 2024-03-26T03:48:53.488 INFO:tasks.workunit.client.0.smithi175.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2024-03-26T03:48:53.488 INFO:tasks.workunit.client.0.smithi175.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (1 ms) 2024-03-26T03:48:53.488 INFO:tasks.workunit.client.0.smithi175.stdout: api_asio: [----------] 12 tests from AsioRados (14 ms total) 2024-03-26T03:48:53.488 INFO:tasks.workunit.client.0.smithi175.stdout: api_asio: 2024-03-26T03:48:53.488 INFO:tasks.workunit.client.0.smithi175.stdout: api_asio: [----------] Global test environment tear-down 2024-03-26T03:48:53.488 INFO:tasks.workunit.client.0.smithi175.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (1256 ms total) 2024-03-26T03:48:53.489 INFO:tasks.workunit.client.0.smithi175.stdout: api_asio: [ PASSED ] 12 tests. 2024-03-26T03:48:53.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:53 smithi190 bash[23392]: cluster 2024-03-26T03:48:52.268482+0000 mon.a (mon.0) 864 : cluster [DBG] osdmap e76: 8 total, 8 up, 8 in 2024-03-26T03:48:53.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:53 smithi190 bash[23392]: audit 2024-03-26T03:48:52.280869+0000 mon.a (mon.0) 865 : audit [INF] from='client.? 172.21.15.175:0/4069691659' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi175-64406-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:53.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:53 smithi190 bash[23392]: audit 2024-03-26T03:48:52.410718+0000 mon.c (mon.2) 93 : audit [INF] from='client.? 172.21.15.175:0/1073317595' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1"}]: dispatch 2024-03-26T03:48:53.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:53 smithi190 bash[23392]: audit 2024-03-26T03:48:52.417627+0000 mon.a (mon.0) 866 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1"}]: dispatch 2024-03-26T03:48:53.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:53 smithi190 bash[23392]: audit 2024-03-26T03:48:52.418649+0000 mon.c (mon.2) 94 : audit [INF] from='client.? 172.21.15.175:0/1073317595' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi175-64770-1"}]: dispatch 2024-03-26T03:48:53.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:53 smithi190 bash[23392]: audit 2024-03-26T03:48:52.419052+0000 mon.a (mon.0) 867 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi175-64770-1"}]: dispatch 2024-03-26T03:48:53.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:53 smithi190 bash[23392]: audit 2024-03-26T03:48:52.419917+0000 mon.c (mon.2) 95 : audit [INF] from='client.? 172.21.15.175:0/1073317595' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:53.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:53 smithi190 bash[23392]: audit 2024-03-26T03:48:52.420590+0000 mon.a (mon.0) 868 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:53.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:53 smithi190 bash[23392]: audit 2024-03-26T03:48:52.439032+0000 mon.c (mon.2) 96 : audit [DBG] from='client.? 172.21.15.175:0/3645527812' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-03-26T03:48:53.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:53 smithi190 bash[23392]: audit 2024-03-26T03:48:52.607185+0000 mon.a (mon.0) 869 : audit [INF] from='client.? 172.21.15.175:0/11262377' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi175-65233-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:53.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:53 smithi190 bash[23392]: audit 2024-03-26T03:48:52.607210+0000 mon.c (mon.2) 97 : audit [INF] from='client.? 172.21.15.175:0/1577208989' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi175-65296-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:53.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:53 smithi190 bash[23392]: audit 2024-03-26T03:48:52.607629+0000 mon.a (mon.0) 870 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi175-65296-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:53.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:53 smithi175 bash[21527]: cluster 2024-03-26T03:48:52.268482+0000 mon.a (mon.0) 864 : cluster [DBG] osdmap e76: 8 total, 8 up, 8 in 2024-03-26T03:48:53.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:53 smithi175 bash[21527]: audit 2024-03-26T03:48:52.280869+0000 mon.a (mon.0) 865 : audit [INF] from='client.? 172.21.15.175:0/4069691659' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi175-64406-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:53.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:53 smithi175 bash[21527]: audit 2024-03-26T03:48:52.410718+0000 mon.c (mon.2) 93 : audit [INF] from='client.? 172.21.15.175:0/1073317595' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1"}]: dispatch 2024-03-26T03:48:53.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:53 smithi175 bash[21527]: audit 2024-03-26T03:48:52.417627+0000 mon.a (mon.0) 866 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1"}]: dispatch 2024-03-26T03:48:53.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:53 smithi175 bash[21527]: audit 2024-03-26T03:48:52.418649+0000 mon.c (mon.2) 94 : audit [INF] from='client.? 172.21.15.175:0/1073317595' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi175-64770-1"}]: dispatch 2024-03-26T03:48:53.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:53 smithi175 bash[21527]: audit 2024-03-26T03:48:52.419052+0000 mon.a (mon.0) 867 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi175-64770-1"}]: dispatch 2024-03-26T03:48:53.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:53 smithi175 bash[21527]: audit 2024-03-26T03:48:52.419917+0000 mon.c (mon.2) 95 : audit [INF] from='client.? 172.21.15.175:0/1073317595' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:53.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:53 smithi175 bash[21527]: audit 2024-03-26T03:48:52.420590+0000 mon.a (mon.0) 868 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:53.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:53 smithi175 bash[21527]: audit 2024-03-26T03:48:52.439032+0000 mon.c (mon.2) 96 : audit [DBG] from='client.? 172.21.15.175:0/3645527812' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-03-26T03:48:53.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:53 smithi175 bash[21527]: audit 2024-03-26T03:48:52.607185+0000 mon.a (mon.0) 869 : audit [INF] from='client.? 172.21.15.175:0/11262377' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi175-65233-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:53.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:53 smithi175 bash[21527]: audit 2024-03-26T03:48:52.607210+0000 mon.c (mon.2) 97 : audit [INF] from='client.? 172.21.15.175:0/1577208989' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi175-65296-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:53.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:53 smithi175 bash[21527]: audit 2024-03-26T03:48:52.607629+0000 mon.a (mon.0) 870 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi175-65296-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:53.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:53 smithi175 bash[29284]: cluster 2024-03-26T03:48:52.268482+0000 mon.a (mon.0) 864 : cluster [DBG] osdmap e76: 8 total, 8 up, 8 in 2024-03-26T03:48:53.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:53 smithi175 bash[29284]: audit 2024-03-26T03:48:52.280869+0000 mon.a (mon.0) 865 : audit [INF] from='client.? 172.21.15.175:0/4069691659' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi175-64406-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:53.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:53 smithi175 bash[29284]: audit 2024-03-26T03:48:52.410718+0000 mon.c (mon.2) 93 : audit [INF] from='client.? 172.21.15.175:0/1073317595' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1"}]: dispatch 2024-03-26T03:48:53.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:53 smithi175 bash[29284]: audit 2024-03-26T03:48:52.417627+0000 mon.a (mon.0) 866 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1"}]: dispatch 2024-03-26T03:48:53.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:53 smithi175 bash[29284]: audit 2024-03-26T03:48:52.418649+0000 mon.c (mon.2) 94 : audit [INF] from='client.? 172.21.15.175:0/1073317595' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi175-64770-1"}]: dispatch 2024-03-26T03:48:53.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:53 smithi175 bash[29284]: audit 2024-03-26T03:48:52.419052+0000 mon.a (mon.0) 867 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi175-64770-1"}]: dispatch 2024-03-26T03:48:53.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:53 smithi175 bash[29284]: audit 2024-03-26T03:48:52.419917+0000 mon.c (mon.2) 95 : audit [INF] from='client.? 172.21.15.175:0/1073317595' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:53.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:53 smithi175 bash[29284]: audit 2024-03-26T03:48:52.420590+0000 mon.a (mon.0) 868 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:53.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:53 smithi175 bash[29284]: audit 2024-03-26T03:48:52.439032+0000 mon.c (mon.2) 96 : audit [DBG] from='client.? 172.21.15.175:0/3645527812' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-03-26T03:48:53.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:53 smithi175 bash[29284]: audit 2024-03-26T03:48:52.607185+0000 mon.a (mon.0) 869 : audit [INF] from='client.? 172.21.15.175:0/11262377' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi175-65233-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:53.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:53 smithi175 bash[29284]: audit 2024-03-26T03:48:52.607210+0000 mon.c (mon.2) 97 : audit [INF] from='client.? 172.21.15.175:0/1577208989' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi175-65296-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:53.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:53 smithi175 bash[29284]: audit 2024-03-26T03:48:52.607629+0000 mon.a (mon.0) 870 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi175-65296-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:54.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: cluster 2024-03-26T03:48:52.567088+0000 mgr.y (mgr.24431) 166 : cluster [DBG] pgmap v111: 196 pgs: 64 unknown, 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T03:48:54.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: cluster 2024-03-26T03:48:53.256392+0000 mon.a (mon.0) 871 : cluster [WRN] Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:48:54.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.288570+0000 mon.a (mon.0) 872 : audit [INF] from='client.? 172.21.15.175:0/4069691659' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi175-64406-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:54.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.288835+0000 mon.a (mon.0) 873 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:54.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.289084+0000 mon.a (mon.0) 874 : audit [INF] from='client.? 172.21.15.175:0/11262377' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi175-65233-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:54.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.289354+0000 mon.a (mon.0) 875 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi175-65296-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:54.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: cluster 2024-03-26T03:48:53.307033+0000 mon.a (mon.0) 876 : cluster [DBG] osdmap e77: 8 total, 8 up, 8 in 2024-03-26T03:48:54.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.328035+0000 mon.c (mon.2) 98 : audit [INF] from='client.? 172.21.15.175:0/1073317595' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi175-64770-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1"}]: dispatch 2024-03-26T03:48:54.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.330278+0000 mon.b (mon.1) 34 : audit [INF] from='client.? 172.21.15.175:0/3706033575' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi175-64668-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.331407+0000 mon.b (mon.1) 35 : audit [INF] from='client.? 172.21.15.175:0/3534209396' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi175-64501-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.332089+0000 mon.b (mon.1) 36 : audit [INF] from='client.? 172.21.15.175:0/3909116745' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.332734+0000 mon.b (mon.1) 37 : audit [INF] from='client.? 172.21.15.175:0/2148657986' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi175-64650-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.333068+0000 mon.c (mon.2) 99 : audit [INF] from='client.? 172.21.15.175:0/1577208989' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi175-65296-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi175-65296-1"}]: dispatch 2024-03-26T03:48:54.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.333431+0000 mon.b (mon.1) 38 : audit [INF] from='client.? 172.21.15.175:0/2354819476' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi175-64470-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.335230+0000 mon.b (mon.1) 39 : audit [INF] from='client.? 172.21.15.175:0/1644353545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64788-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.335888+0000 mon.b (mon.1) 40 : audit [INF] from='client.? 172.21.15.175:0/405093978' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi175-64398-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.340111+0000 mon.c (mon.2) 100 : audit [INF] from='client.? 172.21.15.175:0/2605491901' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi175-64414-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.340973+0000 mon.a (mon.0) 877 : audit [INF] from='client.? 172.21.15.175:0/11262377' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi175-65233-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi175-65233-1"}]: dispatch 2024-03-26T03:48:54.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.341004+0000 mon.c (mon.2) 101 : audit [INF] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi175-64438-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.341489+0000 mon.c (mon.2) 102 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi175-64544-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.341831+0000 mon.c (mon.2) 103 : audit [INF] from='client.? 172.21.15.175:0/905934026' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi175-64594-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.342021+0000 mon.a (mon.0) 878 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi175-64668-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.342182+0000 mon.c (mon.2) 104 : audit [INF] from='client.? 172.21.15.175:0/402874734' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi175-64636-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.342314+0000 mon.a (mon.0) 879 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi175-64770-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1"}]: dispatch 2024-03-26T03:48:54.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.342432+0000 mon.c (mon.2) 105 : audit [INF] from='client.? 172.21.15.175:0/1323749373' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi175-64639-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.342622+0000 mon.a (mon.0) 880 : audit [INF] from='client.? 172.21.15.175:0/1276082353' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi175-64422-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.342850+0000 mon.c (mon.2) 106 : audit [INF] from='client.? 172.21.15.175:0/3904544232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi175-64735-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.342880+0000 mon.a (mon.0) 881 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi175-64501-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.343090+0000 mon.c (mon.2) 107 : audit [INF] from='client.? 172.21.15.175:0/1851694241' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi175-64911-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.343145+0000 mon.a (mon.0) 882 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi175-65296-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi175-65296-1"}]: dispatch 2024-03-26T03:48:54.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.343501+0000 mon.c (mon.2) 108 : audit [INF] from='client.? 172.21.15.175:0/658695337' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64916-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.343827+0000 mon.c (mon.2) 109 : audit [INF] from='client.? 172.21.15.175:0/3947520864' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64843-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.345022+0000 mon.a (mon.0) 883 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.345923+0000 mon.a (mon.0) 884 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi175-64650-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.346175+0000 mon.a (mon.0) 885 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi175-64414-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.346448+0000 mon.a (mon.0) 886 : audit [INF] from='client.? 172.21.15.175:0/1352710026' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.346800+0000 mon.a (mon.0) 887 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi175-64470-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.350272+0000 mon.a (mon.0) 888 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi175-64438-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.350694+0000 mon.a (mon.0) 889 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64788-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.351104+0000 mon.a (mon.0) 890 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi175-64544-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.351454+0000 mon.a (mon.0) 891 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi175-64398-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.351774+0000 mon.a (mon.0) 892 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi175-64594-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.352106+0000 mon.a (mon.0) 893 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi175-64636-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.352388+0000 mon.a (mon.0) 894 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi175-64639-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.352675+0000 mon.a (mon.0) 895 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi175-64735-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.352968+0000 mon.a (mon.0) 896 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi175-64911-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.353307+0000 mon.a (mon.0) 897 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64916-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:54 smithi190 bash[23392]: audit 2024-03-26T03:48:53.353597+0000 mon.a (mon.0) 898 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64843-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: cluster 2024-03-26T03:48:52.567088+0000 mgr.y (mgr.24431) 166 : cluster [DBG] pgmap v111: 196 pgs: 64 unknown, 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T03:48:54.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: cluster 2024-03-26T03:48:53.256392+0000 mon.a (mon.0) 871 : cluster [WRN] Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:48:54.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.288570+0000 mon.a (mon.0) 872 : audit [INF] from='client.? 172.21.15.175:0/4069691659' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi175-64406-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:54.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.288835+0000 mon.a (mon.0) 873 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:54.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.289084+0000 mon.a (mon.0) 874 : audit [INF] from='client.? 172.21.15.175:0/11262377' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi175-65233-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:54.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.289354+0000 mon.a (mon.0) 875 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi175-65296-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:54.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: cluster 2024-03-26T03:48:53.307033+0000 mon.a (mon.0) 876 : cluster [DBG] osdmap e77: 8 total, 8 up, 8 in 2024-03-26T03:48:54.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.328035+0000 mon.c (mon.2) 98 : audit [INF] from='client.? 172.21.15.175:0/1073317595' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi175-64770-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1"}]: dispatch 2024-03-26T03:48:54.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.330278+0000 mon.b (mon.1) 34 : audit [INF] from='client.? 172.21.15.175:0/3706033575' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi175-64668-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.331407+0000 mon.b (mon.1) 35 : audit [INF] from='client.? 172.21.15.175:0/3534209396' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi175-64501-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.332089+0000 mon.b (mon.1) 36 : audit [INF] from='client.? 172.21.15.175:0/3909116745' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.332734+0000 mon.b (mon.1) 37 : audit [INF] from='client.? 172.21.15.175:0/2148657986' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi175-64650-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.333068+0000 mon.c (mon.2) 99 : audit [INF] from='client.? 172.21.15.175:0/1577208989' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi175-65296-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi175-65296-1"}]: dispatch 2024-03-26T03:48:54.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.333431+0000 mon.b (mon.1) 38 : audit [INF] from='client.? 172.21.15.175:0/2354819476' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi175-64470-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.335230+0000 mon.b (mon.1) 39 : audit [INF] from='client.? 172.21.15.175:0/1644353545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64788-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.335888+0000 mon.b (mon.1) 40 : audit [INF] from='client.? 172.21.15.175:0/405093978' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi175-64398-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.340111+0000 mon.c (mon.2) 100 : audit [INF] from='client.? 172.21.15.175:0/2605491901' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi175-64414-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.340973+0000 mon.a (mon.0) 877 : audit [INF] from='client.? 172.21.15.175:0/11262377' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi175-65233-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi175-65233-1"}]: dispatch 2024-03-26T03:48:54.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.341004+0000 mon.c (mon.2) 101 : audit [INF] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi175-64438-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.341489+0000 mon.c (mon.2) 102 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi175-64544-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.341831+0000 mon.c (mon.2) 103 : audit [INF] from='client.? 172.21.15.175:0/905934026' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi175-64594-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.342021+0000 mon.a (mon.0) 878 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi175-64668-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.342182+0000 mon.c (mon.2) 104 : audit [INF] from='client.? 172.21.15.175:0/402874734' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi175-64636-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.342314+0000 mon.a (mon.0) 879 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi175-64770-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1"}]: dispatch 2024-03-26T03:48:54.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.342432+0000 mon.c (mon.2) 105 : audit [INF] from='client.? 172.21.15.175:0/1323749373' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi175-64639-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.342622+0000 mon.a (mon.0) 880 : audit [INF] from='client.? 172.21.15.175:0/1276082353' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi175-64422-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.342850+0000 mon.c (mon.2) 106 : audit [INF] from='client.? 172.21.15.175:0/3904544232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi175-64735-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.342880+0000 mon.a (mon.0) 881 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi175-64501-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.343090+0000 mon.c (mon.2) 107 : audit [INF] from='client.? 172.21.15.175:0/1851694241' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi175-64911-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.343145+0000 mon.a (mon.0) 882 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi175-65296-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi175-65296-1"}]: dispatch 2024-03-26T03:48:54.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.343501+0000 mon.c (mon.2) 108 : audit [INF] from='client.? 172.21.15.175:0/658695337' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64916-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.343827+0000 mon.c (mon.2) 109 : audit [INF] from='client.? 172.21.15.175:0/3947520864' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64843-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.345022+0000 mon.a (mon.0) 883 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.345923+0000 mon.a (mon.0) 884 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi175-64650-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.346175+0000 mon.a (mon.0) 885 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi175-64414-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.346448+0000 mon.a (mon.0) 886 : audit [INF] from='client.? 172.21.15.175:0/1352710026' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.346800+0000 mon.a (mon.0) 887 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi175-64470-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.350272+0000 mon.a (mon.0) 888 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi175-64438-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.350694+0000 mon.a (mon.0) 889 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64788-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.351104+0000 mon.a (mon.0) 890 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi175-64544-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.351454+0000 mon.a (mon.0) 891 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi175-64398-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.351774+0000 mon.a (mon.0) 892 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi175-64594-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.352106+0000 mon.a (mon.0) 893 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi175-64636-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.352388+0000 mon.a (mon.0) 894 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi175-64639-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.352675+0000 mon.a (mon.0) 895 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi175-64735-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.352968+0000 mon.a (mon.0) 896 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi175-64911-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.353307+0000 mon.a (mon.0) 897 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64916-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[21527]: audit 2024-03-26T03:48:53.353597+0000 mon.a (mon.0) 898 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64843-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: cluster 2024-03-26T03:48:52.567088+0000 mgr.y (mgr.24431) 166 : cluster [DBG] pgmap v111: 196 pgs: 64 unknown, 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T03:48:54.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: cluster 2024-03-26T03:48:53.256392+0000 mon.a (mon.0) 871 : cluster [WRN] Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:48:54.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.288570+0000 mon.a (mon.0) 872 : audit [INF] from='client.? 172.21.15.175:0/4069691659' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi175-64406-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:54.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.288835+0000 mon.a (mon.0) 873 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:54.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.289084+0000 mon.a (mon.0) 874 : audit [INF] from='client.? 172.21.15.175:0/11262377' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi175-65233-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:54.750 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.289354+0000 mon.a (mon.0) 875 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi175-65296-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:54.750 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: cluster 2024-03-26T03:48:53.307033+0000 mon.a (mon.0) 876 : cluster [DBG] osdmap e77: 8 total, 8 up, 8 in 2024-03-26T03:48:54.750 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.328035+0000 mon.c (mon.2) 98 : audit [INF] from='client.? 172.21.15.175:0/1073317595' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi175-64770-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1"}]: dispatch 2024-03-26T03:48:54.750 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.330278+0000 mon.b (mon.1) 34 : audit [INF] from='client.? 172.21.15.175:0/3706033575' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi175-64668-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.750 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.331407+0000 mon.b (mon.1) 35 : audit [INF] from='client.? 172.21.15.175:0/3534209396' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi175-64501-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.750 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.332089+0000 mon.b (mon.1) 36 : audit [INF] from='client.? 172.21.15.175:0/3909116745' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.750 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.332734+0000 mon.b (mon.1) 37 : audit [INF] from='client.? 172.21.15.175:0/2148657986' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi175-64650-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.750 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.333068+0000 mon.c (mon.2) 99 : audit [INF] from='client.? 172.21.15.175:0/1577208989' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi175-65296-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi175-65296-1"}]: dispatch 2024-03-26T03:48:54.750 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.333431+0000 mon.b (mon.1) 38 : audit [INF] from='client.? 172.21.15.175:0/2354819476' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi175-64470-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.750 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.335230+0000 mon.b (mon.1) 39 : audit [INF] from='client.? 172.21.15.175:0/1644353545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64788-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.750 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.335888+0000 mon.b (mon.1) 40 : audit [INF] from='client.? 172.21.15.175:0/405093978' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi175-64398-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.750 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.340111+0000 mon.c (mon.2) 100 : audit [INF] from='client.? 172.21.15.175:0/2605491901' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi175-64414-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.750 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.340973+0000 mon.a (mon.0) 877 : audit [INF] from='client.? 172.21.15.175:0/11262377' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi175-65233-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi175-65233-1"}]: dispatch 2024-03-26T03:48:54.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.341004+0000 mon.c (mon.2) 101 : audit [INF] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi175-64438-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.341489+0000 mon.c (mon.2) 102 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi175-64544-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.341831+0000 mon.c (mon.2) 103 : audit [INF] from='client.? 172.21.15.175:0/905934026' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi175-64594-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.342021+0000 mon.a (mon.0) 878 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi175-64668-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.342182+0000 mon.c (mon.2) 104 : audit [INF] from='client.? 172.21.15.175:0/402874734' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi175-64636-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.342314+0000 mon.a (mon.0) 879 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi175-64770-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1"}]: dispatch 2024-03-26T03:48:54.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.342432+0000 mon.c (mon.2) 105 : audit [INF] from='client.? 172.21.15.175:0/1323749373' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi175-64639-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.342622+0000 mon.a (mon.0) 880 : audit [INF] from='client.? 172.21.15.175:0/1276082353' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi175-64422-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.342850+0000 mon.c (mon.2) 106 : audit [INF] from='client.? 172.21.15.175:0/3904544232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi175-64735-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.342880+0000 mon.a (mon.0) 881 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi175-64501-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.343090+0000 mon.c (mon.2) 107 : audit [INF] from='client.? 172.21.15.175:0/1851694241' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi175-64911-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.343145+0000 mon.a (mon.0) 882 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi175-65296-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi175-65296-1"}]: dispatch 2024-03-26T03:48:54.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.343501+0000 mon.c (mon.2) 108 : audit [INF] from='client.? 172.21.15.175:0/658695337' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64916-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.752 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.343827+0000 mon.c (mon.2) 109 : audit [INF] from='client.? 172.21.15.175:0/3947520864' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64843-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.752 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.345022+0000 mon.a (mon.0) 883 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.752 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.345923+0000 mon.a (mon.0) 884 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi175-64650-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.752 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.346175+0000 mon.a (mon.0) 885 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi175-64414-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.752 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.346448+0000 mon.a (mon.0) 886 : audit [INF] from='client.? 172.21.15.175:0/1352710026' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.752 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.346800+0000 mon.a (mon.0) 887 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi175-64470-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.752 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.350272+0000 mon.a (mon.0) 888 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi175-64438-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.752 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.350694+0000 mon.a (mon.0) 889 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64788-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.752 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.351104+0000 mon.a (mon.0) 890 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi175-64544-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.752 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.351454+0000 mon.a (mon.0) 891 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi175-64398-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.752 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.351774+0000 mon.a (mon.0) 892 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi175-64594-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.752 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.352106+0000 mon.a (mon.0) 893 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi175-64636-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.752 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.352388+0000 mon.a (mon.0) 894 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi175-64639-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.752 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.352675+0000 mon.a (mon.0) 895 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi175-64735-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.753 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.352968+0000 mon.a (mon.0) 896 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi175-64911-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.753 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.353307+0000 mon.a (mon.0) 897 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64916-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:54.753 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:54 smithi175 bash[29284]: audit 2024-03-26T03:48:53.353597+0000 mon.a (mon.0) 898 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64843-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:55.149 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [==========] Running 11 tests from 3 test suites. 2024-03-26T03:48:55.149 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [----------] Global test environment set-up. 2024-03-26T03:48:55.149 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [----------] 7 tests from LibRadosList 2024-03-26T03:48:55.149 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2024-03-26T03:48:55.150 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [ OK ] LibRadosList.ListObjects (517 ms) 2024-03-26T03:48:55.150 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2024-03-26T03:48:55.150 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (46 ms) 2024-03-26T03:48:55.150 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2024-03-26T03:48:55.150 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: myset foo1,foo2,foo3 2024-03-26T03:48:55.150 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: foo1 2024-03-26T03:48:55.150 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: foo2 2024-03-26T03:48:55.150 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: foo3 2024-03-26T03:48:55.150 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: myset foo1,foo4,foo5 2024-03-26T03:48:55.151 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: foo4 2024-03-26T03:48:55.151 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: foo5 2024-03-26T03:48:55.151 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: foo1 2024-03-26T03:48:55.151 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: myset foo6,foo7 2024-03-26T03:48:55.151 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: foo7 2024-03-26T03:48:55.151 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: foo6 2024-03-26T03:48:55.151 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-03-26T03:48:55.152 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: ns1:foo4 2024-03-26T03:48:55.152 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: ns1:foo5 2024-03-26T03:48:55.152 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: ns2:foo7 2024-03-26T03:48:55.152 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: ns2:foo6 2024-03-26T03:48:55.152 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: ns1:foo1 2024-03-26T03:48:55.152 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: :foo1 2024-03-26T03:48:55.152 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: :foo2 2024-03-26T03:48:55.152 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: :foo3 2024-03-26T03:48:55.153 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (93 ms) 2024-03-26T03:48:55.153 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2024-03-26T03:48:55.153 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 1 0 2024-03-26T03:48:55.153 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 10 0 2024-03-26T03:48:55.153 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 13 0 2024-03-26T03:48:55.153 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 7 0 2024-03-26T03:48:55.153 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 14 0 2024-03-26T03:48:55.153 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 0 0 2024-03-26T03:48:55.154 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 15 0 2024-03-26T03:48:55.154 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 11 0 2024-03-26T03:48:55.154 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 5 0 2024-03-26T03:48:55.154 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 8 0 2024-03-26T03:48:55.154 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 6 0 2024-03-26T03:48:55.154 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 3 0 2024-03-26T03:48:55.154 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 4 0 2024-03-26T03:48:55.155 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 12 0 2024-03-26T03:48:55.155 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 9 0 2024-03-26T03:48:55.155 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 2 0 2024-03-26T03:48:55.155 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-03-26T03:48:55.155 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (77 ms) 2024-03-26T03:48:55.155 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2024-03-26T03:48:55.155 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: x cursor=MIN 2024-03-26T03:48:55.155 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > oid=1 cursor=13:02547ec2:::1:head 2024-03-26T03:48:55.155 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > oid=10 cursor=13:52ea6a34:::10:head 2024-03-26T03:48:55.156 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > oid=13 cursor=13:566253c9:::13:head 2024-03-26T03:48:55.156 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > oid=7 cursor=13:5c6b0b28:::7:head 2024-03-26T03:48:55.156 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > oid=14 cursor=13:62a1935d:::14:head 2024-03-26T03:48:55.156 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > oid=0 cursor=13:6cac518f:::0:head 2024-03-26T03:48:55.156 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > oid=15 cursor=13:863748b0:::15:head 2024-03-26T03:48:55.156 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > oid=11 cursor=13:89d3ae78:::11:head 2024-03-26T03:48:55.156 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > oid=5 cursor=13:b29083e3:::5:head 2024-03-26T03:48:55.156 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > oid=8 cursor=13:bd63b0f1:::8:head 2024-03-26T03:48:55.156 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > oid=6 cursor=13:c4fdafeb:::6:head 2024-03-26T03:48:55.156 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > oid=3 cursor=13:cfc208b3:::3:head 2024-03-26T03:48:55.156 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > oid=4 cursor=13:d83876eb:::4:head 2024-03-26T03:48:55.156 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > oid=12 cursor=13:de5d7c5f:::12:head 2024-03-26T03:48:55.156 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > oid=9 cursor=13:e960b815:::9:head 2024-03-26T03:48:55.157 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > oid=2 cursor=13:f905c69b:::2:head 2024-03-26T03:48:55.157 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: FIRST> seek to MIN oid=1 2024-03-26T03:48:55.157 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : oid=1 cursor=13:02547ec2:::1:head 2024-03-26T03:48:55.157 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : seek to 13:02547ec2:::1:head 2024-03-26T03:48:55.157 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > 13:02547ec2:::1:head -> 1 2024-03-26T03:48:55.157 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : oid=10 cursor=13:52ea6a34:::10:head 2024-03-26T03:48:55.157 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : seek to 13:52ea6a34:::10:head 2024-03-26T03:48:55.157 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > 13:52ea6a34:::10:head -> 10 2024-03-26T03:48:55.157 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : oid=13 cursor=13:566253c9:::13:head 2024-03-26T03:48:55.157 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : seek to 13:566253c9:::13:head 2024-03-26T03:48:55.157 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > 13:566253c9:::13:head -> 13 2024-03-26T03:48:55.157 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : oid=7 cursor=13:5c6b0b28:::7:head 2024-03-26T03:48:55.158 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : seek to 13:5c6b0b28:::7:head 2024-03-26T03:48:55.158 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > 13:5c6b0b28:::7:head -> 7 2024-03-26T03:48:55.214 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : oid=14 cursor=13:62a1935d:::14:head 2024-03-26T03:48:55.214 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : seek to 13:62a1935d:::14:head 2024-03-26T03:48:55.214 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > 13:62a1935d:::14:head -> 14 2024-03-26T03:48:55.214 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : oid=0 cursor=13:6cac518f:::0:head 2024-03-26T03:48:55.214 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : seek to 13:6cac518f:::0:head 2024-03-26T03:48:55.214 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > 13:6cac518f:::0:head -> 0 2024-03-26T03:48:55.215 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : oid=15 cursor=13:863748b0:::15:head 2024-03-26T03:48:55.215 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : seek to 13:863748b0:::15:head 2024-03-26T03:48:55.215 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > 13:863748b0:::15:head -> 15 2024-03-26T03:48:55.215 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : oid=11 cursor=13:89d3ae78:::11:head 2024-03-26T03:48:55.215 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : seek to 13:89d3ae78:::11:head 2024-03-26T03:48:55.215 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > 13:89d3ae78:::11:head -> 11 2024-03-26T03:48:55.215 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : oid=5 cursor=13:b29083e3:::5:head 2024-03-26T03:48:55.215 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : seek to 13:b29083e3:::5:head 2024-03-26T03:48:55.215 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > 13:b29083e3:::5:head -> 5 2024-03-26T03:48:55.215 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : oid=8 cursor=13:bd63b0f1:::8:head 2024-03-26T03:48:55.215 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : seek to 13:bd63b0f1:::8:head 2024-03-26T03:48:55.215 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > 13:bd63b0f1:::8:head -> 8 2024-03-26T03:48:55.215 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : oid=6 cursor=13:c4fdafeb:::6:head 2024-03-26T03:48:55.215 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : seek to 13:c4fdafeb:::6:head 2024-03-26T03:48:55.215 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > 13:c4fdafeb:::6:head -> 6 2024-03-26T03:48:55.216 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : oid=3 cursor=13:cfc208b3:::3:head 2024-03-26T03:48:55.216 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : seek to 13:cfc208b3:::3:head 2024-03-26T03:48:55.216 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > 13:cfc208b3:::3:head -> 3 2024-03-26T03:48:55.216 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : oid=4 cursor=13:d83876eb:::4:head 2024-03-26T03:48:55.216 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : seek to 13:d83876eb:::4:head 2024-03-26T03:48:55.216 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > 13:d83876eb:::4:head -> 4 2024-03-26T03:48:55.216 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : oid=12 cursor=13:de5d7c5f:::12:head 2024-03-26T03:48:55.216 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : seek to 13:de5d7c5f:::12:head 2024-03-26T03:48:55.216 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > 13:de5d7c5f:::12:head -> 12 2024-03-26T03:48:55.216 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : oid=9 cursor=13:e960b815:::9:head 2024-03-26T03:48:55.216 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : seek to 13:e960b815:::9:head 2024-03-26T03:48:55.216 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > 13:e960b815:::9:head -> 9 2024-03-26T03:48:55.216 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : oid=2 cursor=13:f905c69b:::2:head 2024-03-26T03:48:55.216 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : seek to 13:f905c69b:::2:head 2024-03-26T03:48:55.216 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > 13:f905c69b:::2:head -> 2 2024-03-26T03:48:55.217 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : seek to 13:e960b815:::9:head 2024-03-26T03:48:55.217 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : cursor()=13:e960b815:::9:head expected=13:e960b815:::9:head 2024-03-26T03:48:55.217 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > 13:e960b815:::9:head -> 9 2024-03-26T03:48:55.217 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : entry=9 expected=9 2024-03-26T03:48:55.217 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : seek to 13:de5d7c5f:::12:head 2024-03-26T03:48:55.217 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : cursor()=13:de5d7c5f:::12:head expected=13:de5d7c5f:::12:head 2024-03-26T03:48:55.217 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > 13:de5d7c5f:::12:head -> 12 2024-03-26T03:48:55.217 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : entry=12 expected=12 2024-03-26T03:48:55.217 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : seek to 13:d83876eb:::4:head 2024-03-26T03:48:55.217 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : cursor()=13:d83876eb:::4:head expected=13:d83876eb:::4:head 2024-03-26T03:48:55.217 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > 13:d83876eb:::4:head -> 4 2024-03-26T03:48:55.217 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : entry=4 expected=4 2024-03-26T03:48:55.217 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : seek to 13:863748b0:::15:head 2024-03-26T03:48:55.217 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : cursor()=13:863748b0:::15:head expected=13:863748b0:::15:head 2024-03-26T03:48:55.218 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > 13:863748b0:::15:head -> 15 2024-03-26T03:48:55.218 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : entry=15 expected=15 2024-03-26T03:48:55.218 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : seek to 13:6cac518f:::0:head 2024-03-26T03:48:55.218 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : cursor()=13:6cac518f:::0:head expected=13:6cac518f:::0:head 2024-03-26T03:48:55.218 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > 13:6cac518f:::0:head -> 0 2024-03-26T03:48:55.218 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : entry=0 expected=0 2024-03-26T03:48:55.218 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : seek to 13:cfc208b3:::3:head 2024-03-26T03:48:55.218 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : cursor()=13:cfc208b3:::3:head expected=13:cfc208b3:::3:head 2024-03-26T03:48:55.218 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > 13:cfc208b3:::3:head -> 3 2024-03-26T03:48:55.218 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : entry=3 expected=3 2024-03-26T03:48:55.218 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : seek to 13:c4fdafeb:::6:head 2024-03-26T03:48:55.218 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : cursor()=13:c4fdafeb:::6:head expected=13:c4fdafeb:::6:head 2024-03-26T03:48:55.218 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > 13:c4fdafeb:::6:head -> 6 2024-03-26T03:48:55.218 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : entry=6 expected=6 2024-03-26T03:48:55.218 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : seek to 13:bd63b0f1:::8:head 2024-03-26T03:48:55.219 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : cursor()=13:bd63b0f1:::8:head expected=13:bd63b0f1:::8:head 2024-03-26T03:48:55.219 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > 13:bd63b0f1:::8:head -> 8 2024-03-26T03:48:55.319 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : entry=8 handler_error: Running main() from gmock_main.cc 2024-03-26T03:48:55.319 INFO:tasks.workunit.client.0.smithi175.stdout: handler_error: [==========] Running 1 test from 1 test suite. 2024-03-26T03:48:55.319 INFO:tasks.workunit.client.0.smithi175.stdout: handler_error: [----------] Global test environment set-up. 2024-03-26T03:48:55.319 INFO:tasks.workunit.client.0.smithi175.stdout: handler_error: [----------] 1 test from neocls_handler_error 2024-03-26T03:48:55.319 INFO:tasks.workunit.client.0.smithi175.stdout: handler_error: [ RUN ] neocls_handler_error.test_handler_error 2024-03-26T03:48:55.319 INFO:tasks.workunit.client.0.smithi175.stdout: handler_error: [ OK ] neocls_handler_error.test_handler_error (2778 ms) 2024-03-26T03:48:55.319 INFO:tasks.workunit.client.0.smithi175.stdout: handler_error: [----------] 1 test from neocls_handler_error (2778 ms total) 2024-03-26T03:48:55.319 INFO:tasks.workunit.client.0.smithi175.stdout: handler_error: 2024-03-26T03:48:55.319 INFO:tasks.workunit.client.0.smithi175.stdout: handler_error: [----------] Global test environment tear-down 2024-03-26T03:48:55.320 INFO:tasks.workunit.client.0.smithi175.stdout: handler_error: [==========] 1 test from 1 test suite ran. (2778 ms total) 2024-03-26T03:48:55.320 INFO:tasks.workunit.client.0.smithi175.stdout: handler_error: [ PASSED ] 1 test. 2024-03-26T03:48:55.333 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd_pp: Running main() from gmock_main.cc 2024-03-26T03:48:55.333 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2024-03-26T03:48:55.333 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd_pp: [----------] Global test environment set-up. 2024-03-26T03:48:55.333 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2024-03-26T03:48:55.333 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2024-03-26T03:48:55.333 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (62 ms) 2024-03-26T03:48:55.333 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2024-03-26T03:48:55.334 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (56 ms) 2024-03-26T03:48:55.334 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2024-03-26T03:48:55.334 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (2811 ms) 2024-03-26T03:48:55.334 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (2929 ms total) 2024-03-26T03:48:55.334 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd_pp: 2024-03-26T03:48:55.334 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd_pp: [----------] Global test environment tear-down 2024-03-26T03:48:55.334 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (2929 ms total) 2024-03-26T03:48:55.334 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2024-03-26T03:48:55.339 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: Running main() from gmock_main.cc 2024-03-26T03:48:55.339 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [==========] Running 17 tests from 1 test suite. 2024-03-26T03:48:55.340 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [----------] Global test environment set-up. 2024-03-26T03:48:55.340 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest 2024-03-26T03:48:55.340 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2024-03-26T03:48:55.340 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2024-03-26T03:48:55.340 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2024-03-26T03:48:55.340 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (393 ms) 2024-03-26T03:48:55.340 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2024-03-26T03:48:55.340 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (31 ms) 2024-03-26T03:48:55.340 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2024-03-26T03:48:55.340 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (50 ms) 2024-03-26T03:48:55.340 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2024-03-26T03:48:55.340 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (43 ms) 2024-03-26T03:48:55.340 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2024-03-26T03:48:55.340 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (11 ms) 2024-03-26T03:48:55.341 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2024-03-26T03:48:55.341 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (9 ms) 2024-03-26T03:48:55.341 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2024-03-26T03:48:55.341 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (9 ms) 2024-03-26T03:48:55.341 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2024-03-26T03:48:55.341 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (9 ms) 2024-03-26T03:48:55.341 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2024-03-26T03:48:55.341 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (4 ms) 2024-03-26T03:48:55.341 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2024-03-26T03:48:55.341 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (7 ms) 2024-03-26T03:48:55.341 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2024-03-26T03:48:55.341 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (6 ms) 2024-03-26T03:48:55.341 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat2 2024-03-26T03:48:55.341 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat2 (7 ms) 2024-03-26T03:48:55.341 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2024-03-26T03:48:55.342 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (15 ms) 2024-03-26T03:48:55.342 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2024-03-26T03:48:55.342 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (17 ms) 2024-03-26T03:48:55.342 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2024-03-26T03:48:55.342 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (9 ms) 2024-03-26T03:48:55.342 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2024-03-26T03:48:55.342 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (14 ms) 2024-03-26T03:48:55.342 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest (634 ms total) 2024-03-26T03:48:55.342 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: 2024-03-26T03:48:55.342 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [----------] Global test environment tear-down 2024-03-26T03:48:55.342 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [==========] 17 tests from 1 test suite ran. (2908 ms total) 2024-03-26T03:48:55.342 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_read_operations: [ PASSED ] 17 tests. 2024-03-26T03:48:55.351 INFO:tasks.workunit.client.0.smithi175.stdout: api_cls_remote_reads: [==========] Running 1 test from 1 test suite. 2024-03-26T03:48:55.351 INFO:tasks.workunit.client.0.smithi175.stdout: api_cls_remote_reads: [----------] Global test environment set-up. 2024-03-26T03:48:55.351 INFO:tasks.workunit.client.0.smithi175.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads 2024-03-26T03:48:55.351 INFO:tasks.workunit.client.0.smithi175.stdout: api_cls_remote_reads: [ RUN ] ClsTestRemoteReads.TestGather 2024-03-26T03:48:55.351 INFO:tasks.workunit.client.0.smithi175.stdout: api_cls_remote_reads: [ OK ] ClsTestRemoteReads.TestGather (2871 ms) 2024-03-26T03:48:55.351 INFO:tasks.workunit.client.0.smithi175.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads (2871 ms total) 2024-03-26T03:48:55.351 INFO:tasks.workunit.client.0.smithi175.stdout: api_cls_remote_reads: 2024-03-26T03:48:55.351 INFO:tasks.workunit.client.0.smithi175.stdout: api_cls_remote_reads: [----------] Global test environment tear-down 2024-03-26T03:48:55.351 INFO:tasks.workunit.client.0.smithi175.stdout: api_cls_remote_reads: [==========] 1 test from 1 test suite ran. (2871 ms total) 2024-03-26T03:48:55.351 INFO:tasks.workunit.client.0.smithi175.stdout: api_cls_remote_reads: [ PASSED ] 1 test. 2024-03-26T03:48:55.435 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: Running main() from gmock_main.cc 2024-03-26T03:48:55.435 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2024-03-26T03:48:55.435 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: [----------] Global test environment set-up. 2024-03-26T03:48:55.436 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2024-03-26T03:48:55.436 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2024-03-26T03:48:55.436 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (56 ms) 2024-03-26T03:48:55.436 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2024-03-26T03:48:55.436 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (61 ms) 2024-03-26T03:48:55.436 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2024-03-26T03:48:55.436 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (2842 ms) 2024-03-26T03:48:55.436 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2024-03-26T03:48:55.436 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:54.304339+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi175-64668-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.436 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:54.304574+0000 mon.a [INF] from='client.? 172.21.15.175:0/1276082353' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi175-64422-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.436 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:54.304705+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi175-64501-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.436 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:54.304881+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.436 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:54.305087+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi175-64650-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.436 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:54.305256+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi175-64414-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.437 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:54.305437+0000 mon.a [INF] from='client.? 172.21.15.175:0/1352710026' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.437 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:54.305599+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi175-64470-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.437 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:54.305789+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi175-64438-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.437 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:54.306004+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64788-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.437 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:54.306152+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi175-64544-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.437 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:54.306274+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi175-64398-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.740 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[21527]: audit 2024-03-26T03:48:54.304339+0000 mon.a (mon.0) 899 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi175-64668-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[21527]: audit 2024-03-26T03:48:54.304574+0000 mon.a (mon.0) 900 : audit [INF] from='client.? 172.21.15.175:0/1276082353' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi175-64422-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[21527]: audit 2024-03-26T03:48:54.304705+0000 mon.a (mon.0) 901 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi175-64501-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[21527]: audit 2024-03-26T03:48:54.304881+0000 mon.a (mon.0) 902 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[21527]: audit 2024-03-26T03:48:54.305087+0000 mon.a (mon.0) 903 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi175-64650-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[21527]: audit 2024-03-26T03:48:54.305256+0000 mon.a (mon.0) 904 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi175-64414-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[21527]: audit 2024-03-26T03:48:54.305437+0000 mon.a (mon.0) 905 : audit [INF] from='client.? 172.21.15.175:0/1352710026' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[21527]: audit 2024-03-26T03:48:54.305599+0000 mon.a (mon.0) 906 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi175-64470-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[21527]: audit 2024-03-26T03:48:54.305789+0000 mon.a (mon.0) 907 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi175-64438-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[21527]: audit 2024-03-26T03:48:54.306004+0000 mon.a (mon.0) 908 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64788-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[21527]: audit 2024-03-26T03:48:54.306152+0000 mon.a (mon.0) 909 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi175-64544-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[21527]: audit 2024-03-26T03:48:54.306274+0000 mon.a (mon.0) 910 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi175-64398-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[21527]: audit 2024-03-26T03:48:54.306462+0000 mon.a (mon.0) 911 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi175-64594-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[29284]: audit 2024-03-26T03:48:54.304339+0000 mon.a (mon.0) 899 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi175-64668-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[29284]: audit 2024-03-26T03:48:54.304574+0000 mon.a (mon.0) 900 : audit [INF] from='client.? 172.21.15.175:0/1276082353' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi175-64422-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[29284]: audit 2024-03-26T03:48:54.304705+0000 mon.a (mon.0) 901 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi175-64501-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[29284]: audit 2024-03-26T03:48:54.304881+0000 mon.a (mon.0) 902 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[29284]: audit 2024-03-26T03:48:54.305087+0000 mon.a (mon.0) 903 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi175-64650-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[29284]: audit 2024-03-26T03:48:54.305256+0000 mon.a (mon.0) 904 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi175-64414-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[29284]: audit 2024-03-26T03:48:54.305437+0000 mon.a (mon.0) 905 : audit [INF] from='client.? 172.21.15.175:0/1352710026' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[29284]: audit 2024-03-26T03:48:54.305599+0000 mon.a (mon.0) 906 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi175-64470-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[29284]: audit 2024-03-26T03:48:54.305789+0000 mon.a (mon.0) 907 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi175-64438-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[29284]: audit 2024-03-26T03:48:54.306004+0000 mon.a (mon.0) 908 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64788-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[29284]: audit 2024-03-26T03:48:54.306152+0000 mon.a (mon.0) 909 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi175-64544-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[29284]: audit 2024-03-26T03:48:54.306274+0000 mon.a (mon.0) 910 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi175-64398-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[29284]: audit 2024-03-26T03:48:54.306462+0000 mon.a (mon.0) 911 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi175-64594-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[21527]: audit 2024-03-26T03:48:54.306653+0000 mon.a (mon.0) 912 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi175-64636-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[21527]: audit 2024-03-26T03:48:54.306781+0000 mon.a (mon.0) 913 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi175-64639-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[21527]: audit 2024-03-26T03:48:54.306895+0000 mon.a (mon.0) 914 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi175-64735-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[21527]: audit 2024-03-26T03:48:54.307009+0000 mon.a (mon.0) 915 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi175-64911-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[21527]: audit 2024-03-26T03:48:54.307117+0000 mon.a (mon.0) 916 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64916-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[21527]: audit 2024-03-26T03:48:54.307238+0000 mon.a (mon.0) 917 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64843-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[21527]: cluster 2024-03-26T03:48:54.348512+0000 mon.a (mon.0) 918 : cluster [DBG] osdmap e78: 8 total, 8 up, 8 in 2024-03-26T03:48:55.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[21527]: audit 2024-03-26T03:48:55.309751+0000 mon.a (mon.0) 919 : audit [INF] from='client.? 172.21.15.175:0/11262377' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi175-65233-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi175-65233-1"}]': finished 2024-03-26T03:48:55.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[21527]: audit 2024-03-26T03:48:55.309864+0000 mon.a (mon.0) 920 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi175-64770-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1"}]': finished 2024-03-26T03:48:55.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[21527]: audit 2024-03-26T03:48:55.309953+0000 mon.a (mon.0) 921 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi175-65296-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi175-65296-1"}]': finished 2024-03-26T03:48:55.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[21527]: cluster 2024-03-26T03:48:55.314944+0000 mon.a (mon.0) 922 : cluster [DBG] osdmap e79: 8 total, 8 up, 8 in 2024-03-26T03:48:55.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[21527]: audit 2024-03-26T03:48:55.324677+0000 mon.b (mon.1) 41 : audit [INF] from='client.? 172.21.15.175:0/1297034116' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi175-64406-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:55.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[21527]: audit 2024-03-26T03:48:55.335556+0000 mon.c (mon.2) 110 : audit [INF] from='client.? 172.21.15.175:0/3783807506' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:55.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[21527]: audit 2024-03-26T03:48:55.357737+0000 mon.a (mon.0) 923 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi175-64406-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:55.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[21527]: audit 2024-03-26T03:48:55.357958+0000 mon.a (mon.0) 924 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:55.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[29284]: audit 2024-03-26T03:48:54.306653+0000 mon.a (mon.0) 912 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi175-64636-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[29284]: audit 2024-03-26T03:48:54.306781+0000 mon.a (mon.0) 913 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi175-64639-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[29284]: audit 2024-03-26T03:48:54.306895+0000 mon.a (mon.0) 914 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi175-64735-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[29284]: audit 2024-03-26T03:48:54.307009+0000 mon.a (mon.0) 915 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi175-64911-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[29284]: audit 2024-03-26T03:48:54.307117+0000 mon.a (mon.0) 916 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64916-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[29284]: audit 2024-03-26T03:48:54.307238+0000 mon.a (mon.0) 917 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64843-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[29284]: cluster 2024-03-26T03:48:54.348512+0000 mon.a (mon.0) 918 : cluster [DBG] osdmap e78: 8 total, 8 up, 8 in 2024-03-26T03:48:55.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[29284]: audit 2024-03-26T03:48:55.309751+0000 mon.a (mon.0) 919 : audit [INF] from='client.? 172.21.15.175:0/11262377' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi175-65233-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi175-65233-1"}]': finished 2024-03-26T03:48:55.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[29284]: audit 2024-03-26T03:48:55.309864+0000 mon.a (mon.0) 920 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi175-64770-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1"}]': finished 2024-03-26T03:48:55.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[29284]: audit 2024-03-26T03:48:55.309953+0000 mon.a (mon.0) 921 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi175-65296-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi175-65296-1"}]': finished 2024-03-26T03:48:55.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[29284]: cluster 2024-03-26T03:48:55.314944+0000 mon.a (mon.0) 922 : cluster [DBG] osdmap e79: 8 total, 8 up, 8 in 2024-03-26T03:48:55.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[29284]: audit 2024-03-26T03:48:55.324677+0000 mon.b (mon.1) 41 : audit [INF] from='client.? 172.21.15.175:0/1297034116' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi175-64406-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:55.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[29284]: audit 2024-03-26T03:48:55.335556+0000 mon.c (mon.2) 110 : audit [INF] from='client.? 172.21.15.175:0/3783807506' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:55.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[29284]: audit 2024-03-26T03:48:55.357737+0000 mon.a (mon.0) 923 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi175-64406-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:55.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:55 smithi175 bash[29284]: audit 2024-03-26T03:48:55.357958+0000 mon.a (mon.0) 924 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:55.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:55 smithi190 bash[23392]: audit 2024-03-26T03:48:54.304339+0000 mon.a (mon.0) 899 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi175-64668-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:55 smithi190 bash[23392]: audit 2024-03-26T03:48:54.304574+0000 mon.a (mon.0) 900 : audit [INF] from='client.? 172.21.15.175:0/1276082353' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi175-64422-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:55 smithi190 bash[23392]: audit 2024-03-26T03:48:54.304705+0000 mon.a (mon.0) 901 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi175-64501-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:55 smithi190 bash[23392]: audit 2024-03-26T03:48:54.304881+0000 mon.a (mon.0) 902 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:55 smithi190 bash[23392]: audit 2024-03-26T03:48:54.305087+0000 mon.a (mon.0) 903 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi175-64650-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:55 smithi190 bash[23392]: audit 2024-03-26T03:48:54.305256+0000 mon.a (mon.0) 904 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi175-64414-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:55 smithi190 bash[23392]: audit 2024-03-26T03:48:54.305437+0000 mon.a (mon.0) 905 : audit [INF] from='client.? 172.21.15.175:0/1352710026' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:55 smithi190 bash[23392]: audit 2024-03-26T03:48:54.305599+0000 mon.a (mon.0) 906 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi175-64470-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:55 smithi190 bash[23392]: audit 2024-03-26T03:48:54.305789+0000 mon.a (mon.0) 907 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi175-64438-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:55 smithi190 bash[23392]: audit 2024-03-26T03:48:54.306004+0000 mon.a (mon.0) 908 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64788-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:55 smithi190 bash[23392]: audit 2024-03-26T03:48:54.306152+0000 mon.a (mon.0) 909 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi175-64544-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:55 smithi190 bash[23392]: audit 2024-03-26T03:48:54.306274+0000 mon.a (mon.0) 910 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi175-64398-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:55 smithi190 bash[23392]: audit 2024-03-26T03:48:54.306462+0000 mon.a (mon.0) 911 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi175-64594-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:55 smithi190 bash[23392]: audit 2024-03-26T03:48:54.306653+0000 mon.a (mon.0) 912 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi175-64636-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:55 smithi190 bash[23392]: audit 2024-03-26T03:48:54.306781+0000 mon.a (mon.0) 913 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi175-64639-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:55 smithi190 bash[23392]: audit 2024-03-26T03:48:54.306895+0000 mon.a (mon.0) 914 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi175-64735-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:55 smithi190 bash[23392]: audit 2024-03-26T03:48:54.307009+0000 mon.a (mon.0) 915 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi175-64911-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:55 smithi190 bash[23392]: audit 2024-03-26T03:48:54.307117+0000 mon.a (mon.0) 916 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64916-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:55 smithi190 bash[23392]: audit 2024-03-26T03:48:54.307238+0000 mon.a (mon.0) 917 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64843-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:55.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:55 smithi190 bash[23392]: cluster 2024-03-26T03:48:54.348512+0000 mon.a (mon.0) 918 : cluster [DBG] osdmap e78: 8 total, 8 up, 8 in 2024-03-26T03:48:55.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:55 smithi190 bash[23392]: audit 2024-03-26T03:48:55.309751+0000 mon.a (mon.0) 919 : audit [INF] from='client.? 172.21.15.175:0/11262377' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi175-65233-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi175-65233-1"}]': finished 2024-03-26T03:48:55.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:55 smithi190 bash[23392]: audit 2024-03-26T03:48:55.309864+0000 mon.a (mon.0) 920 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi175-64770-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1"}]': finished 2024-03-26T03:48:55.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:55 smithi190 bash[23392]: audit 2024-03-26T03:48:55.309953+0000 mon.a (mon.0) 921 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi175-65296-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi175-65296-1"}]': finished 2024-03-26T03:48:55.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:55 smithi190 bash[23392]: cluster 2024-03-26T03:48:55.314944+0000 mon.a (mon.0) 922 : cluster [DBG] osdmap e79: 8 total, 8 up, 8 in 2024-03-26T03:48:55.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:55 smithi190 bash[23392]: audit 2024-03-26T03:48:55.324677+0000 mon.b (mon.1) 41 : audit [INF] from='client.? 172.21.15.175:0/1297034116' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi175-64406-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:55.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:55 smithi190 bash[23392]: audit 2024-03-26T03:48:55.335556+0000 mon.c (mon.2) 110 : audit [INF] from='client.? 172.21.15.175:0/3783807506' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:55.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:55 smithi190 bash[23392]: audit 2024-03-26T03:48:55.357737+0000 mon.a (mon.0) 923 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi175-64406-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:55.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:55 smithi190 bash[23392]: audit 2024-03-26T03:48:55.357958+0000 mon.a (mon.0) 924 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:55.753 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:54.306462+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi175-64 list_parallel: process_1_[65021]: starting. 2024-03-26T03:48:55.753 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_1_[65021]: creating pool ceph_test_rados_list_parallel.smithi175-64961 2024-03-26T03:48:55.753 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_1_[65021]: created object 0... 2024-03-26T03:48:55.753 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_1_[65021]: created object 25... 2024-03-26T03:48:55.753 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_1_[65021]: created object 49... 2024-03-26T03:48:55.753 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_1_[65021]: finishing. 2024-03-26T03:48:55.753 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_1_[65021]: shutting down. 2024-03-26T03:48:55.753 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_2_[65022]: starting. 2024-03-26T03:48:55.753 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_2_[65022]: listing objects. 2024-03-26T03:48:55.753 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_2_[65022]: listed object 0... 2024-03-26T03:48:55.754 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_2_[65022]: listed object 25... 2024-03-26T03:48:55.754 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_2_[65022]: saw 50 objects 2024-03-26T03:48:55.754 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_2_[65022]: shutting down. 2024-03-26T03:48:55.754 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:55.754 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_3_[65661]: starting. 2024-03-26T03:48:55.754 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_3_[65661]: creating pool ceph_test_rados_list_parallel.smithi175-64961 2024-03-26T03:48:55.754 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_3_[65661]: created object 0... 2024-03-26T03:48:55.754 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_3_[65661]: created object 25... 2024-03-26T03:48:55.754 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_3_[65661]: created object 49... 2024-03-26T03:48:55.754 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_3_[65661]: finishing. 2024-03-26T03:48:55.754 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_3_[65661]: shutting down. 2024-03-26T03:48:55.754 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:55.754 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_4_[65662]: starting. 2024-03-26T03:48:55.754 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_4_[65662]: listing objects. 2024-03-26T03:48:55.755 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_4_[65662]: listed object 0... 2024-03-26T03:48:55.755 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_4_[65662]: listed object 25... 2024-03-26T03:48:55.755 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_4_[65662]: saw 44 objects 2024-03-26T03:48:55.755 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_4_[65662]: shutting down. 2024-03-26T03:48:55.755 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:55.755 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_5_[65663]: starting. 2024-03-26T03:48:55.755 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_5_[65663]: removed 25 objects... 2024-03-26T03:48:55.755 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_5_[65663]: removed half of the objects 2024-03-26T03:48:55.755 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_5_[65663]: removed 50 objects... 2024-03-26T03:48:55.755 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_5_[65663]: removed 50 objects 2024-03-26T03:48:55.755 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_5_[65663]: shutting down. 2024-03-26T03:48:55.755 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:55.755 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:55.755 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_6_[65709]: starting. 2024-03-26T03:48:55.756 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_6_[65709]: creating pool ceph_test_rados_list_parallel.smithi175-64961 2024-03-26T03:48:55.756 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_6_[65709]: created object 0... 2024-03-26T03:48:55.756 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_6_[65709]: created object 25... 2024-03-26T03:48:55.756 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_6_[65709]: created object 49... 2024-03-26T03:48:55.756 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_6_[65709]: finishing. 2024-03-26T03:48:55.756 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_6_[65709]: shutting down. 2024-03-26T03:48:55.756 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:55.756 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:55.756 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_7_[65710]: starting. 2024-03-26T03:48:55.756 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_7_[65710]: listing objects. 2024-03-26T03:48:55.756 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_7_[65710]: listed object 0... 2024-03-26T03:48:55.756 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_7_[65710]: listed object 25... 2024-03-26T03:48:55.756 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_7_[65710]: listed object 50... 2024-03-26T03:48:55.756 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_7_[65710]: saw 56 objects 2024-03-26T03:48:55.756 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_7_[65710]: shutting down. 2024-03-26T03:48:55.757 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:55.757 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:55.757 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_8_[65711]: starting. 2024-03-26T03:48:55.757 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_8_[65711]: added 25 objects... 2024-03-26T03:48:55.757 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_8_[65711]: added half of the objects 2024-03-26T03:48:55.757 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_8_[65711]: added 50 objects... 2024-03-26T03:48:55.757 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_8_[65711]: added 50 objects 2024-03-26T03:48:55.757 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_8_[65711]: shutting down. 2024-03-26T03:48:55.757 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:55.757 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:55.986 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:55.986 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_9_[66029]: starting. 2024-03-26T03:48:55.986 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_9_[66029]: creating pool ceph_test_rados_list_parallel.smithi175-64961 2024-03-26T03:48:55.986 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_9_[66029]: created object 0... 2024-03-26T03:48:55.986 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_9_[66029]: created object 25... 2024-03-26T03:48:55.986 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_9_[66029]: created object 49... 2024-03-26T03:48:55.986 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_9_[66029]: finishing. 2024-03-26T03:48:55.986 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_9_[66029]: shutting down. 2024-03-26T03:48:55.987 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:55.987 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:55.987 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:55.987 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_10_[66030]: starting. 2024-03-26T03:48:55.987 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_10_[66030]: listing objects. 2024-03-26T03:48:55.987 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_10_[66030]: listed object 0... 2024-03-26T03:48:55.987 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_10_[66030]: listed object 25... 2024-03-26T03:48:55.987 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_10_[66030]: listed object 50... 2024-03-26T03:48:55.987 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_10_[66030]: listed object 75... 2024-03-26T03:48:55.987 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_10_[66030]: saw 97 objects 2024-03-26T03:48:55.987 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_10_[66030]: shutting down. 2024-03-26T03:48:55.987 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:55.987 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:55.987 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:55.987 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_11_[66031]: starting. 2024-03-26T03:48:55.988 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_11_[66031]: added 25 objects... 2024-03-26T03:48:55.988 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_11_[66031]: added half of the objects 2024-03-26T03:48:55.988 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_11_[66031]: added 50 objects... 2024-03-26T03:48:55.988 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_11_[66031]: added 50 objects 2024-03-26T03:48:55.988 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_11_[66031]: shutting down. 2024-03-26T03:48:55.988 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:55.988 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:55.988 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:55.988 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_13_[66033]: starting. 2024-03-26T03:48:55.988 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_13_[66033]: removed 25 objects... 2024-03-26T03:48:55.988 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_13_[66033]: removed half of the objects 2024-03-26T03:48:55.988 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_13_[66033]: removed 50 objects... 2024-03-26T03:48:55.988 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_13_[66033]: removed 50 objects 2024-03-26T03:48:55.988 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_13_[66033]: shutting down. 2024-03-26T03:48:55.988 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:55.989 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:55.989 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:55.989 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_12_[66032]: starting. 2024-03-26T03:48:55.989 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_12_[66032]: added 25 objects... 2024-03-26T03:48:55.989 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_12_[66032]: added half of the objects 2024-03-26T03:48:55.989 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_12_[66032]: added 50 objects... 2024-03-26T03:48:55.989 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_12_[66032]: added 50 objects 2024-03-26T03:48:55.989 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_12_[66032]: shutting down. 2024-03-26T03:48:55.989 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:55.989 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:55.989 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:55.989 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:55.989 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_14_[66109]: starting. 2024-03-26T03:48:55.989 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_14_[66109]: creating pool ceph_test_rados_list_parallel.smithi175-64961 2024-03-26T03:48:55.989 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_14_[66109]: created object 0... 2024-03-26T03:48:55.990 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_14_[66109]: created object 25... 2024-03-26T03:48:55.990 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_14_[66109]: created object 49... 2024-03-26T03:48:55.990 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_14_[66109]: finishing. 2024-03-26T03:48:55.990 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_14_[66109]: shutting down. 2024-03-26T03:48:55.990 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:55.990 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:55.990 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:55.990 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:55.990 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_15_[66110]: starting. 2024-03-26T03:48:55.990 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_15_[66110]: listing objects. 2024-03-26T03:48:55.990 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_15_[66110]: listed object 0... 2024-03-26T03:48:56.358 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_15_[66110]: listed object 25... 2024-03-26T03:48:56.359 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_15_[66110]: listed object 50... 2024-03-26T03:48:56.359 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_15_[66110]: listed object 75... 2024-03-26T03:48:56.359 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_15_[66110]: listed object 100... 2024-03-26T03:48:56.359 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_15_[66110]: listed object 125... 2024-03-26T03:48:56.359 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_15_[66110]: saw 150 objects 2024-03-26T03:48:56.359 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_15_[66110]: shutting down. 2024-03-26T03:48:56.359 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:56.359 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:56.359 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:56.359 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:56.359 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_16_[66111]: starting. 2024-03-26T03:48:56.359 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_16_[66111]: added 25 objects... 2024-03-26T03:48:56.359 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_16_[66111]: added half of the objects 2024-03-26T03:48:56.360 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_16_[66111]: added 50 objects... 2024-03-26T03:48:56.360 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_16_[66111]: added 50 objects 2024-03-26T03:48:56.360 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: process_16_[66111]: shutting down. 2024-03-26T03:48:56.360 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:56.360 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:56.360 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:56.360 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:56.360 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******************************* 2024-03-26T03:48:56.360 INFO:tasks.workunit.client.0.smithi175.stdout: list_parallel: ******* SUCCESS ********** 2024-03-26T03:48:56.407 INFO:tasks.workunit.client.0.smithi175.stdout:594-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:56.407 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:54.306653+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi175-64636-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:56.408 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:54.306781+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi175-64639-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:56.408 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:54.306895+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi175-64735-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:56.408 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:54.307009+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi175-64911-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:56.408 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:54.307117+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64916-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:56.408 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:54.307238+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64843-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:56.408 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:55.309751+0000 mon.a [INF] from='client.? 172.21.15.175:0/11262377' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi175-65233-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi175-65233-1"}]': finished 2024-03-26T03:48:56.408 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:55.309864+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi175-64770-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1"}]': finished 2024-03-26T03:48:56.408 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:55.309953+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi175-65296-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi175-65296-1"}]': finished 2024-03-26T03:48:56.408 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:55.324677+0000 mon.b [INF] from='client.? 172.21.15.175:0/1297034116' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi175-64406-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:56.408 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:55.335556+0000 mon.c [INF] from='client.? 172.21.15.175:0/3783807506' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:56.408 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:55.357737+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi175-64406-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:56.408 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:55.357958+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:56.408 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:55.413156+0000 mon.b [INF] from='client.? 172.21.15.175:0/2115269020' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:48:56.409 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:55.422533+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:48:56.409 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:55.431223+0000 mon.b [INF] from='client.? 172.21.15.175:0/2115269020' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:48:56.409 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:55.435441+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:48:56.409 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:55.441147+0000 client.admin [INF] onexx 2024-03-26T03:48:56.409 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:55.443839+0000 mon.b [INF] from='client.? 172.21.15.175:0/1244868480' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-26T03:48:56.409 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:55.447520+0000 mon.b [INF] from='client.? 172.21.15.175:0/2115269020' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi175-64414-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:56.409 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:55.454413+0000 mon.c [INF] from='client.? 172.21.15.175:0/4087439416' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:48:56.409 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:55.473411+0000 mon.b [INF] from='client.? 172.21.15.175:0/464642862' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:48:56.409 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:55.475507+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-26T03:48:56.409 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:55.483950+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi175-64414-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:56.409 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:55.484375+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:48:56.409 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:55.491649+0000 mon.c [INF] from='client.? 172.21.15.175:0/4087439416' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:48:56.409 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:55.502424+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:48:56.409 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:55.503103+0000 mon.a [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi175-64668-7"}]: dispatch 2024-03-26T03:48:56.410 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:55.503566+0000 mon.c [INF] from='client.? 172.21.15.175:0/4087439416' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi175-64650-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:56.410 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:55.517869+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi175-64650-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:56.410 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:55.518220+0000 mon.a [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi175-64668-7"}]: dispatch 2024-03-26T03:48:56.410 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:55.519098+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:48:56.410 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:55.530358+0000 mon.b [INF] from='client.? 172.21.15.175:0/464642862' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:48:56.410 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:55.538104+0000 mon.a [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi175-64668-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:56.410 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:55.540883+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:48:56.410 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:55.542903+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi175-64422-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:56.410 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:55.547806+0000 mon.b [INF] from='client.? 172.21.15.175:0/464642862' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi175-64422-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:56.410 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:56.317355+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi175-64406-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:56.410 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:56.317447+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:56.410 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:56.317523+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi175-64414-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:56.410 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:56.317628+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi175-64650-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:56.410 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:56.317734+0000 mon.a [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi175-64668-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:56.411 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:56.317831+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi175-64422-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:56.411 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:56.328426+0000 mon.a [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi175-64668-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi175-64668-7"}]: dispatch 2024-03-26T03:48:56.411 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:56.332552+0000 mon.a [INF] from='client.? 172.21.15.175:0/11262377' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi175-65233-1"}]: dispatch 2024-03-26T03:48:56.411 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:56.334369+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi175-64414-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:48:56.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: cluster 2024-03-26T03:48:54.569156+0000 mgr.y (mgr.24431) 167 : cluster [DBG] pgmap v114: 1124 pgs: 12 creating+peering, 212 creating+activating, 149 active+clean, 751 unknown; 582 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:56.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:55.413156+0000 mon.b (mon.1) 42 : audit [INF] from='client.? 172.21.15.175:0/2115269020' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:48:56.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:55.422533+0000 mon.a (mon.0) 925 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:48:56.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:55.431223+0000 mon.b (mon.1) 43 : audit [INF] from='client.? 172.21.15.175:0/2115269020' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:48:56.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:55.435441+0000 mon.a (mon.0) 926 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:48:56.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: cluster 2024-03-26T03:48:55.441147+0000 client.admin (client.?) 0 : cluster [INF] onexx 2024-03-26T03:48:56.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:55.443839+0000 mon.b (mon.1) 44 : audit [INF] from='client.? 172.21.15.175:0/1244868480' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-26T03:48:56.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:55.447520+0000 mon.b (mon.1) 45 : audit [INF] from='client.? 172.21.15.175:0/2115269020' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi175-64414-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:56.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:55.454413+0000 mon.c (mon.2) 111 : audit [INF] from='client.? 172.21.15.175:0/4087439416' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:48:56.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:55.473411+0000 mon.b (mon.1) 46 : audit [INF] from='client.? 172.21.15.175:0/464642862' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:48:56.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:55.475507+0000 mon.a (mon.0) 927 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-26T03:48:56.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:55.483950+0000 mon.a (mon.0) 928 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi175-64414-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:56.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:55.484375+0000 mon.a (mon.0) 929 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:48:56.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:55.491649+0000 mon.c (mon.2) 112 : audit [INF] from='client.? 172.21.15.175:0/4087439416' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:48:56.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:55.502424+0000 mon.a (mon.0) 930 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:48:56.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:55.503103+0000 mon.a (mon.0) 931 : audit [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi175-64668-7"}]: dispatch 2024-03-26T03:48:56.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:55.503566+0000 mon.c (mon.2) 113 : audit [INF] from='client.? 172.21.15.175:0/4087439416' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi175-64650-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:56.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:55.517869+0000 mon.a (mon.0) 932 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi175-64650-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:56.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:55.518220+0000 mon.a (mon.0) 933 : audit [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi175-64668-7"}]: dispatch 2024-03-26T03:48:56.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:55.519098+0000 mon.a (mon.0) 934 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:48:56.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:55.530358+0000 mon.b (mon.1) 47 : audit [INF] from='client.? 172.21.15.175:0/464642862' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:48:56.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:55.538104+0000 mon.a (mon.0) 935 : audit [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi175-64668-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:56.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:55.540883+0000 mon.a (mon.0) 936 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:48:56.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:55.542903+0000 mon.a (mon.0) 937 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi175-64422-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:56.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:55.547806+0000 mon.b (mon.1) 48 : audit [INF] from='client.? 172.21.15.175:0/464642862' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi175-64422-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:56.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:55.665793+0000 mon.c (mon.2) 114 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:48:56.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:56.317355+0000 mon.a (mon.0) 938 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi175-64406-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:56.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:56.317447+0000 mon.a (mon.0) 939 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:56.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:56.317523+0000 mon.a (mon.0) 940 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi175-64414-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:56.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:56.317628+0000 mon.a (mon.0) 941 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi175-64650-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:56.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:56.317734+0000 mon.a (mon.0) 942 : audit [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi175-64668-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:56.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:56.317831+0000 mon.a (mon.0) 943 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi175-64422-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:56.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: cluster 2024-03-26T03:48:56.327756+0000 mon.a (mon.0) 944 : cluster [DBG] osdmap e80: 8 total, 8 up, 8 in 2024-03-26T03:48:56.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:56.328426+0000 mon.a (mon.0) 945 : audit [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi175-64668-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi175-64668-7"}]: dispatch 2024-03-26T03:48:56.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:56.332552+0000 mon.a (mon.0) 946 : audit [INF] from='client.? 172.21.15.175:0/11262377' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi175-65233-1"}]: dispatch 2024-03-26T03:48:56.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:56.334369+0000 mon.a (mon.0) 947 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi175-64414-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:48:56.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:56.336063+0000 mon.b (mon.1) 49 : audit [INF] from='client.? 172.21.15.175:0/2115269020' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi175-64414-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:48:56.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:56.336179+0000 mon.c (mon.2) 115 : audit [INF] from='client.? 172.21.15.175:0/4087439416' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi175-64650-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:48:56.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:56.337555+0000 mon.a (mon.0) 948 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi175-64422-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:48:56.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:56.339628+0000 mon.b (mon.1) 50 : audit [INF] from='client.? 172.21.15.175:0/464642862' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi175-64422-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:48:56.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:56.340697+0000 mon.b (mon.1) 51 : audit [INF] from='client.? 172.21.15.175:0/3798538274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:56.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: cluster 2024-03-26T03:48:54.569156+0000 mgr.y (mgr.24431) 167 : cluster [DBG] pgmap v114: 1124 pgs: 12 creating+peering, 212 creating+activating, 149 active+clean, 751 unknown; 582 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:56.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:55.413156+0000 mon.b (mon.1) 42 : audit [INF] from='client.? 172.21.15.175:0/2115269020' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:48:56.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:55.422533+0000 mon.a (mon.0) 925 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:48:56.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:55.431223+0000 mon.b (mon.1) 43 : audit [INF] from='client.? 172.21.15.175:0/2115269020' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:48:56.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:55.435441+0000 mon.a (mon.0) 926 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:48:56.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: cluster 2024-03-26T03:48:55.441147+0000 client.admin (client.?) 0 : cluster [INF] onexx 2024-03-26T03:48:56.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:55.443839+0000 mon.b (mon.1) 44 : audit [INF] from='client.? 172.21.15.175:0/1244868480' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-26T03:48:56.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:55.447520+0000 mon.b (mon.1) 45 : audit [INF] from='client.? 172.21.15.175:0/2115269020' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi175-64414-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:56.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:55.454413+0000 mon.c (mon.2) 111 : audit [INF] from='client.? 172.21.15.175:0/4087439416' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:48:56.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:55.473411+0000 mon.b (mon.1) 46 : audit [INF] from='client.? 172.21.15.175:0/464642862' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:48:56.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:55.475507+0000 mon.a (mon.0) 927 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-26T03:48:56.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:55.483950+0000 mon.a (mon.0) 928 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi175-64414-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:56.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:55.484375+0000 mon.a (mon.0) 929 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:48:56.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:55.491649+0000 mon.c (mon.2) 112 : audit [INF] from='client.? 172.21.15.175:0/4087439416' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:48:56.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:55.502424+0000 mon.a (mon.0) 930 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:48:56.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:56.341162+0000 mon.a (mon.0) 949 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:56.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:56.341485+0000 mon.b (mon.1) 52 : audit [INF] from='client.? 172.21.15.175:0/1122404867' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:56.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:56.341635+0000 mon.a (mon.0) 950 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:56.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:56.341990+0000 mon.a (mon.0) 951 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi175-64650-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:48:56.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:56.343908+0000 mon.c (mon.2) 116 : audit [INF] from='client.? 172.21.15.175:0/1064151631' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi175-64398-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:56.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:56 smithi190 bash[23392]: audit 2024-03-26T03:48:56.366923+0000 mon.a (mon.0) 952 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi175-64398-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:56.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: cluster 2024-03-26T03:48:54.569156+0000 mgr.y (mgr.24431) 167 : cluster [DBG] pgmap v114: 1124 pgs: 12 creating+peering, 212 creating+activating, 149 active+clean, 751 unknown; 582 KiB data, 232 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:48:56.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:55.413156+0000 mon.b (mon.1) 42 : audit [INF] from='client.? 172.21.15.175:0/2115269020' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:48:56.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:55.422533+0000 mon.a (mon.0) 925 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:48:56.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:55.431223+0000 mon.b (mon.1) 43 : audit [INF] from='client.? 172.21.15.175:0/2115269020' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:48:56.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:55.435441+0000 mon.a (mon.0) 926 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:48:56.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: cluster 2024-03-26T03:48:55.441147+0000 client.admin (client.?) 0 : cluster [INF] onexx 2024-03-26T03:48:56.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:55.443839+0000 mon.b (mon.1) 44 : audit [INF] from='client.? 172.21.15.175:0/1244868480' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-26T03:48:56.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:55.447520+0000 mon.b (mon.1) 45 : audit [INF] from='client.? 172.21.15.175:0/2115269020' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi175-64414-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:56.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:55.454413+0000 mon.c (mon.2) 111 : audit [INF] from='client.? 172.21.15.175:0/4087439416' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:48:56.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:55.473411+0000 mon.b (mon.1) 46 : audit [INF] from='client.? 172.21.15.175:0/464642862' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:48:56.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:55.475507+0000 mon.a (mon.0) 927 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-26T03:48:56.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:55.483950+0000 mon.a (mon.0) 928 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi175-64414-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:56.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:55.484375+0000 mon.a (mon.0) 929 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:48:56.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:55.491649+0000 mon.c (mon.2) 112 : audit [INF] from='client.? 172.21.15.175:0/4087439416' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:48:56.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:55.502424+0000 mon.a (mon.0) 930 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:48:56.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:55.503103+0000 mon.a (mon.0) 931 : audit [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi175-64668-7"}]: dispatch 2024-03-26T03:48:56.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:55.503566+0000 mon.c (mon.2) 113 : audit [INF] from='client.? 172.21.15.175:0/4087439416' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi175-64650-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:56.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:55.517869+0000 mon.a (mon.0) 932 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi175-64650-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:56.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:55.518220+0000 mon.a (mon.0) 933 : audit [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi175-64668-7"}]: dispatch 2024-03-26T03:48:56.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:55.519098+0000 mon.a (mon.0) 934 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:48:56.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:55.530358+0000 mon.b (mon.1) 47 : audit [INF] from='client.? 172.21.15.175:0/464642862' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:48:56.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:55.538104+0000 mon.a (mon.0) 935 : audit [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi175-64668-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:56.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:55.540883+0000 mon.a (mon.0) 936 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:48:56.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:55.542903+0000 mon.a (mon.0) 937 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi175-64422-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:56.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:55.547806+0000 mon.b (mon.1) 48 : audit [INF] from='client.? 172.21.15.175:0/464642862' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi175-64422-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:56.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:55.665793+0000 mon.c (mon.2) 114 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:48:56.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:56.317355+0000 mon.a (mon.0) 938 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi175-64406-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:56.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:56.317447+0000 mon.a (mon.0) 939 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:56.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:56.317523+0000 mon.a (mon.0) 940 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi175-64414-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:56.749 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:55.503103+0000 mon.a (mon.0) 931 : audit [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi175-64668-7"}]: dispatch 2024-03-26T03:48:56.749 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:55.503566+0000 mon.c (mon.2) 113 : audit [INF] from='client.? 172.21.15.175:0/4087439416' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi175-64650-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:56.749 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:55.517869+0000 mon.a (mon.0) 932 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi175-64650-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:56.749 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:55.518220+0000 mon.a (mon.0) 933 : audit [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi175-64668-7"}]: dispatch 2024-03-26T03:48:56.749 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:55.519098+0000 mon.a (mon.0) 934 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:48:56.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:55.530358+0000 mon.b (mon.1) 47 : audit [INF] from='client.? 172.21.15.175:0/464642862' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:48:56.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:55.538104+0000 mon.a (mon.0) 935 : audit [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi175-64668-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:56.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:55.540883+0000 mon.a (mon.0) 936 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:48:56.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:55.542903+0000 mon.a (mon.0) 937 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi175-64422-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:56.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:55.547806+0000 mon.b (mon.1) 48 : audit [INF] from='client.? 172.21.15.175:0/464642862' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi175-64422-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:56.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:55.665793+0000 mon.c (mon.2) 114 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:48:56.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:56.317355+0000 mon.a (mon.0) 938 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi175-64406-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:56.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:56.317447+0000 mon.a (mon.0) 939 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:56.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:56.317523+0000 mon.a (mon.0) 940 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi175-64414-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:56.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:56.317628+0000 mon.a (mon.0) 941 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi175-64650-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:56.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:56.317734+0000 mon.a (mon.0) 942 : audit [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi175-64668-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:56.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:56.317831+0000 mon.a (mon.0) 943 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi175-64422-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:56.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: cluster 2024-03-26T03:48:56.327756+0000 mon.a (mon.0) 944 : cluster [DBG] osdmap e80: 8 total, 8 up, 8 in 2024-03-26T03:48:56.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:56.328426+0000 mon.a (mon.0) 945 : audit [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi175-64668-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi175-64668-7"}]: dispatch 2024-03-26T03:48:56.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:56.332552+0000 mon.a (mon.0) 946 : audit [INF] from='client.? 172.21.15.175:0/11262377' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi175-65233-1"}]: dispatch 2024-03-26T03:48:56.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:56.334369+0000 mon.a (mon.0) 947 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi175-64414-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:48:56.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:56.336063+0000 mon.b (mon.1) 49 : audit [INF] from='client.? 172.21.15.175:0/2115269020' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi175-64414-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:48:56.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:56.336179+0000 mon.c (mon.2) 115 : audit [INF] from='client.? 172.21.15.175:0/4087439416' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi175-64650-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:48:56.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:56.337555+0000 mon.a (mon.0) 948 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi175-64422-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:48:56.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:56.339628+0000 mon.b (mon.1) 50 : audit [INF] from='client.? 172.21.15.175:0/464642862' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi175-64422-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:48:56.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:56.340697+0000 mon.b (mon.1) 51 : audit [INF] from='client.? 172.21.15.175:0/3798538274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:56.752 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:56.317628+0000 mon.a (mon.0) 941 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi175-64650-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:56.752 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:56.317734+0000 mon.a (mon.0) 942 : audit [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi175-64668-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:56.752 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:56.317831+0000 mon.a (mon.0) 943 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi175-64422-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:56.752 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: cluster 2024-03-26T03:48:56.327756+0000 mon.a (mon.0) 944 : cluster [DBG] osdmap e80: 8 total, 8 up, 8 in 2024-03-26T03:48:56.752 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:56.328426+0000 mon.a (mon.0) 945 : audit [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi175-64668-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi175-64668-7"}]: dispatch 2024-03-26T03:48:56.752 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:56.332552+0000 mon.a (mon.0) 946 : audit [INF] from='client.? 172.21.15.175:0/11262377' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi175-65233-1"}]: dispatch 2024-03-26T03:48:56.752 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:56.334369+0000 mon.a (mon.0) 947 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi175-64414-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:48:56.752 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:56.336063+0000 mon.b (mon.1) 49 : audit [INF] from='client.? 172.21.15.175:0/2115269020' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi175-64414-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:48:56.752 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:56.336179+0000 mon.c (mon.2) 115 : audit [INF] from='client.? 172.21.15.175:0/4087439416' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi175-64650-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:48:56.753 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:56.337555+0000 mon.a (mon.0) 948 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi175-64422-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:48:56.753 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:56.339628+0000 mon.b (mon.1) 50 : audit [INF] from='client.? 172.21.15.175:0/464642862' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi175-64422-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:48:56.753 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:56.340697+0000 mon.b (mon.1) 51 : audit [INF] from='client.? 172.21.15.175:0/3798538274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:56.753 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:56.341162+0000 mon.a (mon.0) 949 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:56.753 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:56.341485+0000 mon.b (mon.1) 52 : audit [INF] from='client.? 172.21.15.175:0/1122404867' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:56.753 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:56.341635+0000 mon.a (mon.0) 950 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:56.753 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:56.341990+0000 mon.a (mon.0) 951 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi175-64650-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:48:56.753 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:56.343908+0000 mon.c (mon.2) 116 : audit [INF] from='client.? 172.21.15.175:0/1064151631' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi175-64398-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:56.753 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[29284]: audit 2024-03-26T03:48:56.366923+0000 mon.a (mon.0) 952 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi175-64398-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:56.754 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:56.341162+0000 mon.a (mon.0) 949 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:56.754 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:56.341485+0000 mon.b (mon.1) 52 : audit [INF] from='client.? 172.21.15.175:0/1122404867' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:56.754 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:56.341635+0000 mon.a (mon.0) 950 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:56.754 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:56.341990+0000 mon.a (mon.0) 951 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi175-64650-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:48:56.754 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:56.343908+0000 mon.c (mon.2) 116 : audit [INF] from='client.? 172.21.15.175:0/1064151631' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi175-64398-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:56.754 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:56 smithi175 bash[21527]: audit 2024-03-26T03:48:56.366923+0000 mon.a (mon.0) 952 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi175-64398-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:57.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:57 smithi190 bash[23392]: audit 2024-03-26T03:48:56.405546+0000 mon.a (mon.0) 953 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-03-26T03:48:57.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:57 smithi190 bash[23392]: cluster 2024-03-26T03:48:56.420668+0000 client.admin (client.?) 0 : cluster [INF] twoxx 2024-03-26T03:48:57.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:57 smithi190 bash[23392]: audit 2024-03-26T03:48:56.423427+0000 mon.b (mon.1) 53 : audit [INF] from='client.? 172.21.15.175:0/1244868480' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-03-26T03:48:57.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:57 smithi190 bash[23392]: audit 2024-03-26T03:48:56.442095+0000 mon.a (mon.0) 954 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-03-26T03:48:57.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:57 smithi190 bash[23392]: cluster 2024-03-26T03:48:56.570025+0000 mgr.y (mgr.24431) 168 : cluster [DBG] pgmap v117: 948 pgs: 9 active, 64 creating+peering, 180 creating+activating, 263 active+clean, 432 unknown; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-03-26T03:48:57.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:57 smithi190 bash[23392]: audit 2024-03-26T03:48:56.667936+0000 mon.c (mon.2) 117 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:48:57.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:57 smithi190 bash[23392]: audit 2024-03-26T03:48:57.325972+0000 mon.a (mon.0) 955 : audit [INF] from='client.? 172.21.15.175:0/11262377' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi175-65233-1"}]': finished 2024-03-26T03:48:57.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:57 smithi190 bash[23392]: audit 2024-03-26T03:48:57.326063+0000 mon.a (mon.0) 956 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:57.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:57 smithi190 bash[23392]: audit 2024-03-26T03:48:57.326145+0000 mon.a (mon.0) 957 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:57.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:57 smithi190 bash[23392]: audit 2024-03-26T03:48:57.326216+0000 mon.a (mon.0) 958 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi175-64398-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:57.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:57 smithi190 bash[23392]: cluster 2024-03-26T03:48:57.370786+0000 mon.a (mon.0) 959 : cluster [DBG] osdmap e81: 8 total, 8 up, 8 in 2024-03-26T03:48:57.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:57 smithi190 bash[23392]: audit 2024-03-26T03:48:57.372772+0000 mon.c (mon.2) 118 : audit [INF] from='client.? 172.21.15.175:0/3783807506' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:57.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:57 smithi190 bash[23392]: audit 2024-03-26T03:48:57.375515+0000 mon.a (mon.0) 960 : audit [INF] from='client.? 172.21.15.175:0/11262377' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi175-65233-1"}]: dispatch 2024-03-26T03:48:57.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:57 smithi190 bash[23392]: audit 2024-03-26T03:48:57.376730+0000 mon.a (mon.0) 961 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:57.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:57 smithi190 bash[23392]: audit 2024-03-26T03:48:57.377896+0000 mon.b (mon.1) 54 : audit [INF] from='client.? 172.21.15.175:0/1297034116' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi175-64406-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:57.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:57 smithi190 bash[23392]: audit 2024-03-26T03:48:57.377987+0000 mon.a (mon.0) 962 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi175-64406-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:57.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:57 smithi190 bash[23392]: audit 2024-03-26T03:48:57.393093+0000 mon.c (mon.2) 119 : audit [INF] from='client.? 172.21.15.175:0/1073317595' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1"}]: dispatch 2024-03-26T03:48:57.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:57 smithi190 bash[23392]: audit 2024-03-26T03:48:57.393385+0000 mon.c (mon.2) 120 : audit [INF] from='client.? 172.21.15.175:0/1577208989' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi175-65296-1"}]: dispatch 2024-03-26T03:48:57.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:57 smithi190 bash[23392]: audit 2024-03-26T03:48:57.394751+0000 mon.a (mon.0) 963 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1"}]: dispatch 2024-03-26T03:48:57.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:57 smithi190 bash[23392]: audit 2024-03-26T03:48:57.394992+0000 mon.a (mon.0) 964 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi175-65296-1"}]: dispatch 2024-03-26T03:48:57.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[21527]: audit 2024-03-26T03:48:56.405546+0000 mon.a (mon.0) 953 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-03-26T03:48:57.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[21527]: cluster 2024-03-26T03:48:56.420668+0000 client.admin (client.?) 0 : cluster [INF] twoxx 2024-03-26T03:48:57.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[21527]: audit 2024-03-26T03:48:56.423427+0000 mon.b (mon.1) 53 : audit [INF] from='client.? 172.21.15.175:0/1244868480' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-03-26T03:48:57.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[21527]: audit 2024-03-26T03:48:56.442095+0000 mon.a (mon.0) 954 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-03-26T03:48:57.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[21527]: cluster 2024-03-26T03:48:56.570025+0000 mgr.y (mgr.24431) 168 : cluster [DBG] pgmap v117: 948 pgs: 9 active, 64 creating+peering, 180 creating+activating, 263 active+clean, 432 unknown; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-03-26T03:48:57.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[21527]: audit 2024-03-26T03:48:56.667936+0000 mon.c (mon.2) 117 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:48:57.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[21527]: audit 2024-03-26T03:48:57.325972+0000 mon.a (mon.0) 955 : audit [INF] from='client.? 172.21.15.175:0/11262377' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi175-65233-1"}]': finished 2024-03-26T03:48:57.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[21527]: audit 2024-03-26T03:48:57.326063+0000 mon.a (mon.0) 956 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:57.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[21527]: audit 2024-03-26T03:48:57.326145+0000 mon.a (mon.0) 957 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:57.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[21527]: audit 2024-03-26T03:48:57.326216+0000 mon.a (mon.0) 958 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi175-64398-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:57.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[21527]: cluster 2024-03-26T03:48:57.370786+0000 mon.a (mon.0) 959 : cluster [DBG] osdmap e81: 8 total, 8 up, 8 in 2024-03-26T03:48:57.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[21527]: audit 2024-03-26T03:48:57.372772+0000 mon.c (mon.2) 118 : audit [INF] from='client.? 172.21.15.175:0/3783807506' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:57.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[21527]: audit 2024-03-26T03:48:57.375515+0000 mon.a (mon.0) 960 : audit [INF] from='client.? 172.21.15.175:0/11262377' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi175-65233-1"}]: dispatch 2024-03-26T03:48:57.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[21527]: audit 2024-03-26T03:48:57.376730+0000 mon.a (mon.0) 961 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:57.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[21527]: audit 2024-03-26T03:48:57.377896+0000 mon.b (mon.1) 54 : audit [INF] from='client.? 172.21.15.175:0/1297034116' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi175-64406-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:57.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[21527]: audit 2024-03-26T03:48:57.377987+0000 mon.a (mon.0) 962 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi175-64406-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:57.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[21527]: audit 2024-03-26T03:48:57.393093+0000 mon.c (mon.2) 119 : audit [INF] from='client.? 172.21.15.175:0/1073317595' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1"}]: dispatch 2024-03-26T03:48:57.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[21527]: audit 2024-03-26T03:48:57.393385+0000 mon.c (mon.2) 120 : audit [INF] from='client.? 172.21.15.175:0/1577208989' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi175-65296-1"}]: dispatch 2024-03-26T03:48:57.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[21527]: audit 2024-03-26T03:48:57.394751+0000 mon.a (mon.0) 963 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1"}]: dispatch 2024-03-26T03:48:57.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[21527]: audit 2024-03-26T03:48:57.394992+0000 mon.a (mon.0) 964 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi175-65296-1"}]: dispatch 2024-03-26T03:48:57.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[29284]: audit 2024-03-26T03:48:56.405546+0000 mon.a (mon.0) 953 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-03-26T03:48:57.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[29284]: cluster 2024-03-26T03:48:56.420668+0000 client.admin (client.?) 0 : cluster [INF] twoxx 2024-03-26T03:48:57.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[29284]: audit 2024-03-26T03:48:56.423427+0000 mon.b (mon.1) 53 : audit [INF] from='client.? 172.21.15.175:0/1244868480' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-03-26T03:48:57.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[29284]: audit 2024-03-26T03:48:56.442095+0000 mon.a (mon.0) 954 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-03-26T03:48:57.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[29284]: cluster 2024-03-26T03:48:56.570025+0000 mgr.y (mgr.24431) 168 : cluster [DBG] pgmap v117: 948 pgs: 9 active, 64 creating+peering, 180 creating+activating, 263 active+clean, 432 unknown; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-03-26T03:48:57.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[29284]: audit 2024-03-26T03:48:56.667936+0000 mon.c (mon.2) 117 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:48:57.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[29284]: audit 2024-03-26T03:48:57.325972+0000 mon.a (mon.0) 955 : audit [INF] from='client.? 172.21.15.175:0/11262377' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi175-65233-1"}]': finished 2024-03-26T03:48:57.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[29284]: audit 2024-03-26T03:48:57.326063+0000 mon.a (mon.0) 956 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:57.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[29284]: audit 2024-03-26T03:48:57.326145+0000 mon.a (mon.0) 957 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:57.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[29284]: audit 2024-03-26T03:48:57.326216+0000 mon.a (mon.0) 958 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi175-64398-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:57.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[29284]: cluster 2024-03-26T03:48:57.370786+0000 mon.a (mon.0) 959 : cluster [DBG] osdmap e81: 8 total, 8 up, 8 in 2024-03-26T03:48:57.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[29284]: audit 2024-03-26T03:48:57.372772+0000 mon.c (mon.2) 118 : audit [INF] from='client.? 172.21.15.175:0/3783807506' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:57.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[29284]: audit 2024-03-26T03:48:57.375515+0000 mon.a (mon.0) 960 : audit [INF] from='client.? 172.21.15.175:0/11262377' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi175-65233-1"}]: dispatch 2024-03-26T03:48:57.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[29284]: audit 2024-03-26T03:48:57.376730+0000 mon.a (mon.0) 961 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:57.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[29284]: audit 2024-03-26T03:48:57.377896+0000 mon.b (mon.1) 54 : audit [INF] from='client.? 172.21.15.175:0/1297034116' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi175-64406-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:57.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[29284]: audit 2024-03-26T03:48:57.377987+0000 mon.a (mon.0) 962 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi175-64406-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:57.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[29284]: audit 2024-03-26T03:48:57.393093+0000 mon.c (mon.2) 119 : audit [INF] from='client.? 172.21.15.175:0/1073317595' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1"}]: dispatch 2024-03-26T03:48:57.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[29284]: audit 2024-03-26T03:48:57.393385+0000 mon.c (mon.2) 120 : audit [INF] from='client.? 172.21.15.175:0/1577208989' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi175-65296-1"}]: dispatch 2024-03-26T03:48:57.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[29284]: audit 2024-03-26T03:48:57.394751+0000 mon.a (mon.0) 963 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1"}]: dispatch 2024-03-26T03:48:57.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:57 smithi175 bash[29284]: audit 2024-03-26T03:48:57.394992+0000 mon.a (mon.0) 964 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi175-65296-1"}]: dispatch 2024-03-26T03:48:58.388 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:56.336063+0000 mon.b [INF] from='client.? 172.21.15.175:0/211 cls: Running main() from gmock_main.cc 2024-03-26T03:48:58.388 INFO:tasks.workunit.client.0.smithi175.stdout: cls: [==========] Running 2 tests from 1 test suite. 2024-03-26T03:48:58.388 INFO:tasks.workunit.client.0.smithi175.stdout: cls: [----------] Global test environment set-up. 2024-03-26T03:48:58.388 INFO:tasks.workunit.client.0.smithi175.stdout: cls: [----------] 2 tests from NeoRadosCls 2024-03-26T03:48:58.388 INFO:tasks.workunit.client.0.smithi175.stdout: cls: [ RUN ] NeoRadosCls.DNE 2024-03-26T03:48:58.388 INFO:tasks.workunit.client.0.smithi175.stdout: cls: [ OK ] NeoRadosCls.DNE (2899 ms) 2024-03-26T03:48:58.388 INFO:tasks.workunit.client.0.smithi175.stdout: cls: [ RUN ] NeoRadosCls.RemoteReads 2024-03-26T03:48:58.388 INFO:tasks.workunit.client.0.smithi175.stdout: cls: [ OK ] NeoRadosCls.RemoteReads (3019 ms) 2024-03-26T03:48:58.388 INFO:tasks.workunit.client.0.smithi175.stdout: cls: [----------] 2 tests from NeoRadosCls (5918 ms total) 2024-03-26T03:48:58.388 INFO:tasks.workunit.client.0.smithi175.stdout: cls: 2024-03-26T03:48:58.389 INFO:tasks.workunit.client.0.smithi175.stdout: cls: [----------] Global test environment tear-down 2024-03-26T03:48:58.389 INFO:tasks.workunit.client.0.smithi175.stdout: cls: [==========] 2 tests from 1 test suite ran. (5919 ms total) 2024-03-26T03:48:58.389 INFO:tasks.workunit.client.0.smithi175.stdout: cls: [ PASSED ] 2 tests. 2024-03-26T03:48:58.413 INFO:tasks.workunit.client.0.smithi175.stdout: delete_pools_parallel: process_1_[65069]: starting. 2024-03-26T03:48:58.413 INFO:tasks.workunit.client.0.smithi175.stdout: delete_pools_parallel: process_1_[65069]: creating pool ceph_test_rados_delete_pools_parallel.smithi175-65014 2024-03-26T03:48:58.413 INFO:tasks.workunit.client.0.smithi175.stdout: delete_pools_parallel: process_1_[65069]: created object 0... 2024-03-26T03:48:58.413 INFO:tasks.workunit.client.0.smithi175.stdout: delete_pools_parallel: process_1_[65069]: created object 25... 2024-03-26T03:48:58.413 INFO:tasks.workunit.client.0.smithi175.stdout: delete_pools_parallel: process_1_[65069]: created object 49... 2024-03-26T03:48:58.414 INFO:tasks.workunit.client.0.smithi175.stdout: delete_pools_parallel: process_1_[65069]: finishing. 2024-03-26T03:48:58.414 INFO:tasks.workunit.client.0.smithi175.stdout: delete_pools_parallel: process_1_[65069]: shutting down. 2024-03-26T03:48:58.414 INFO:tasks.workunit.client.0.smithi175.stdout: delete_pools_parallel: process_2_[65071]: starting. 2024-03-26T03:48:58.414 INFO:tasks.workunit.client.0.smithi175.stdout: delete_pools_parallel: process_2_[65071]: deleting pool ceph_test_rados_delete_pools_parallel.smithi175-65014 2024-03-26T03:48:58.414 INFO:tasks.workunit.client.0.smithi175.stdout: delete_pools_parallel: process_2_[65071]: shutting down. 2024-03-26T03:48:58.414 INFO:tasks.workunit.client.0.smithi175.stdout: delete_pools_parallel: ******************************* 2024-03-26T03:48:58.414 INFO:tasks.workunit.client.0.smithi175.stdout: delete_pools_parallel: process_3_[65757]: starting. 2024-03-26T03:48:58.414 INFO:tasks.workunit.client.0.smithi175.stdout: delete_pools_parallel: process_3_[65757]: creating pool ceph_test_rados_delete_pools_parallel.smithi175-65014 2024-03-26T03:48:58.414 INFO:tasks.workunit.client.0.smithi175.stdout: delete_pools_parallel: process_3_[65757]: created object 0... 2024-03-26T03:48:58.414 INFO:tasks.workunit.client.0.smithi175.stdout: delete_pools_parallel: process_3_[65757]: created object 25... 2024-03-26T03:48:58.414 INFO:tasks.workunit.client.0.smithi175.stdout: delete_pools_parallel: process_3_[65757]: created object 49... 2024-03-26T03:48:58.414 INFO:tasks.workunit.client.0.smithi175.stdout: delete_pools_parallel: process_3_[65757]: finishing. 2024-03-26T03:48:58.414 INFO:tasks.workunit.client.0.smithi175.stdout: delete_pools_parallel: process_3_[65757]: shutting down. 2024-03-26T03:48:58.414 INFO:tasks.workunit.client.0.smithi175.stdout: delete_pools_parallel: ******************************* 2024-03-26T03:48:58.415 INFO:tasks.workunit.client.0.smithi175.stdout: delete_pools_parallel: process_5_[65759]: starting. 2024-03-26T03:48:58.415 INFO:tasks.workunit.client.0.smithi175.stdout: delete_pools_parallel: process_5_[65759]: listing objects. 2024-03-26T03:48:58.415 INFO:tasks.workunit.client.0.smithi175.stdout: delete_pools_parallel: process_5_[65759]: listed object 0... 2024-03-26T03:48:58.415 INFO:tasks.workunit.client.0.smithi175.stdout: delete_pools_parallel: process_5_[65759]: listed object 25... 2024-03-26T03:48:58.415 INFO:tasks.workunit.client.0.smithi175.stdout: delete_pools_parallel: process_5_[65759]: saw 50 objects 2024-03-26T03:48:58.415 INFO:tasks.workunit.client.0.smithi175.stdout: delete_pools_parallel: process_5_[65759]: shutting down. 2024-03-26T03:48:58.415 INFO:tasks.workunit.client.0.smithi175.stdout: delete_pools_parallel: ******************************* 2024-03-26T03:48:58.415 INFO:tasks.workunit.client.0.smithi175.stdout: delete_pools_parallel: process_4_[65758]: starting. 2024-03-26T03:48:58.415 INFO:tasks.workunit.client.0.smithi175.stdout: delete_pools_parallel: process_4_[65758]: deleting pool ceph_test_rados_delete_pools_parallel.smithi175-65014 2024-03-26T03:48:58.415 INFO:tasks.workunit.client.0.smithi175.stdout: delete_pools_parallel: process_4_[65758]: shutting down. 2024-03-26T03:48:58.415 INFO:tasks.workunit.client.0.smithi175.stdout: delete_pools_parallel: ******************************* 2024-03-26T03:48:58.415 INFO:tasks.workunit.client.0.smithi175.stdout: delete_pools_parallel: ******************************* 2024-03-26T03:48:58.415 INFO:tasks.workunit.client.0.smithi175.stdout: delete_pools_parallel: ******* SUCCESS ********** 2024-03-26T03:48:58.416 INFO:tasks.workunit.client.0.smithi175.stdout: open_pools_parallel: process_1_[64986]: starting. 2024-03-26T03:48:58.416 INFO:tasks.workunit.client.0.smithi175.stdout: open_pools_parallel: process_1_[64986]: creating pool ceph_test_rados_open_pools_parallel.smithi175-64958 2024-03-26T03:48:58.416 INFO:tasks.workunit.client.0.smithi175.stdout: open_pools_parallel: process_1_[64986]: created object 0... 2024-03-26T03:48:58.416 INFO:tasks.workunit.client.0.smithi175.stdout: open_pools_parallel: process_1_[64986]: created object 25... 2024-03-26T03:48:58.416 INFO:tasks.workunit.client.0.smithi175.stdout: open_pools_parallel: process_1_[64986]: created object 49... 2024-03-26T03:48:58.416 INFO:tasks.workunit.client.0.smithi175.stdout: open_pools_parallel: process_1_[64986]: finishing. 2024-03-26T03:48:58.416 INFO:tasks.workunit.client.0.smithi175.stdout: open_pools_parallel: process_1_[64986]: shutting down. 2024-03-26T03:48:58.416 INFO:tasks.workunit.client.0.smithi175.stdout: open_pools_parallel: process_2_[64987]: starting. 2024-03-26T03:48:58.416 INFO:tasks.workunit.client.0.smithi175.stdout: open_pools_parallel: process_2_[64987]: rados_pool_create. 2024-03-26T03:48:58.416 INFO:tasks.workunit.client.0.smithi175.stdout: open_pools_parallel: process_2_[64987]: rados_ioctx_create. 2024-03-26T03:48:58.416 INFO:tasks.workunit.client.0.smithi175.stdout: open_pools_parallel: process_2_[64987]: shutting down. 2024-03-26T03:48:58.416 INFO:tasks.workunit.client.0.smithi175.stdout: open_pools_parallel: ******************************* 2024-03-26T03:48:58.416 INFO:tasks.workunit.client.0.smithi175.stdout: open_pools_parallel: process_3_[65760]: starting. 2024-03-26T03:48:58.416 INFO:tasks.workunit.client.0.smithi175.stdout: open_pools_parallel: process_3_[65760]: creating pool ceph_test_rados_open_pools_parallel.smithi175-64958 2024-03-26T03:48:58.416 INFO:tasks.workunit.client.0.smithi175.stdout: open_pools_parallel: process_3_[65760]: created object 0... 2024-03-26T03:48:58.417 INFO:tasks.workunit.client.0.smithi175.stdout: open_pools_parallel: process_3_[65760]: created object 25... 2024-03-26T03:48:58.417 INFO:tasks.workunit.client.0.smithi175.stdout: open_pools_parallel: process_3_[65760]: created object 49... 2024-03-26T03:48:58.417 INFO:tasks.workunit.client.0.smithi175.stdout: open_pools_parallel: process_3_[65760]: finishing. 2024-03-26T03:48:58.417 INFO:tasks.workunit.client.0.smithi175.stdout: open_pools_parallel: process_3_[65760]: shutting down. 2024-03-26T03:48:58.417 INFO:tasks.workunit.client.0.smithi175.stdout: open_pools_parallel: ******************************* 2024-03-26T03:48:58.417 INFO:tasks.workunit.client.0.smithi175.stdout: open_pools_parallel: process_4_[65761]: starting. 2024-03-26T03:48:58.417 INFO:tasks.workunit.client.0.smithi175.stdout: open_pools_parallel: process_4_[65761]: rados_pool_create. 2024-03-26T03:48:58.417 INFO:tasks.workunit.client.0.smithi175.stdout: open_pools_parallel: process_4_[65761]: rados_ioctx_create. 2024-03-26T03:48:58.417 INFO:tasks.workunit.client.0.smithi175.stdout: open_pools_parallel: process_4_[65761]: shutting down. 2024-03-26T03:48:58.417 INFO:tasks.workunit.client.0.smithi175.stdout: open_pools_parallel: ******************************* 2024-03-26T03:48:58.417 INFO:tasks.workunit.client.0.smithi175.stdout: open_pools_parallel: ******************************* 2024-03-26T03:48:58.417 INFO:tasks.workunit.client.0.smithi175.stdout: open_pools_parallel: ******* SUCCESS ********** 2024-03-26T03:48:58.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:57.419745+0000 mon.a (mon.0) 965 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-03-26T03:48:58.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:57.452672+0000 mon.a (mon.0) 966 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:48:58.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:57.455200+0000 mon.b (mon.1) 55 : audit [INF] from='client.? 172.21.15.175:0/542670412' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:48:58.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:57.458226+0000 mon.a (mon.0) 967 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:48:58.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:57.463068+0000 mon.b (mon.1) 56 : audit [INF] from='client.? 172.21.15.175:0/542670412' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:48:58.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:57.463486+0000 mon.a (mon.0) 968 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi175-64501-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:58.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:57.468559+0000 mon.b (mon.1) 57 : audit [INF] from='client.? 172.21.15.175:0/542670412' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi175-64501-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:58.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:57.473705+0000 mon.a (mon.0) 969 : audit [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi175-64470-10"}]: dispatch 2024-03-26T03:48:58.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:57.480042+0000 mon.a (mon.0) 970 : audit [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi175-64470-10"}]: dispatch 2024-03-26T03:48:58.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:57.482444+0000 mon.a (mon.0) 971 : audit [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi175-64470-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:58.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:57.669288+0000 mon.c (mon.2) 121 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:48:58.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:57.864847+0000 mon.a (mon.0) 972 : audit [DBG] from='client.? 172.21.15.175:0/1820771429' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-26T03:48:58.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:58.335738+0000 mon.a (mon.0) 973 : audit [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi175-64668-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi175-64668-7"}]': finished 2024-03-26T03:48:58.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:58.335926+0000 mon.a (mon.0) 974 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi175-64414-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi175-64414-16"}]': finished 2024-03-26T03:48:58.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:58.336164+0000 mon.a (mon.0) 975 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi175-64422-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi175-64422-22"}]': finished 2024-03-26T03:48:58.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:58.336394+0000 mon.a (mon.0) 976 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi175-64650-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi175-64650-7"}]': finished 2024-03-26T03:48:58.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:58.336650+0000 mon.a (mon.0) 977 : audit [INF] from='client.? 172.21.15.175:0/11262377' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi175-65233-1"}]': finished 2024-03-26T03:48:58.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:58.336786+0000 mon.a (mon.0) 978 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:58.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:58.336960+0000 mon.a (mon.0) 979 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi175-64406-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:58.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:58.337178+0000 mon.a (mon.0) 980 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1"}]': finished 2024-03-26T03:48:58.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:58.337339+0000 mon.a (mon.0) 981 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi175-65296-1"}]': finished 2024-03-26T03:48:58.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:58.337546+0000 mon.a (mon.0) 982 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi175-64501-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:58.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:58.337815+0000 mon.a (mon.0) 983 : audit [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi175-64470-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:58.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:58.352829+0000 mon.b (mon.1) 58 : audit [INF] from='client.? 172.21.15.175:0/542670412' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi175-64501-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:48:58.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:58.369487+0000 mon.c (mon.2) 122 : audit [INF] from='client.? 172.21.15.175:0/1577208989' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi175-65296-1"}]: dispatch 2024-03-26T03:48:58.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:58.370830+0000 mon.c (mon.2) 123 : audit [INF] from='client.? 172.21.15.175:0/1073317595' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi175-64770-1"}]: dispatch 2024-03-26T03:48:58.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:58.380025+0000 mon.c (mon.2) 124 : audit [INF] from='client.? 172.21.15.175:0/3783807506' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-4", "tierpool": "test-rados-api-smithi175-64594-4-cache"}]: dispatch 2024-03-26T03:48:58.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: cluster 2024-03-26T03:48:58.383687+0000 mon.a (mon.0) 984 : cluster [DBG] osdmap e82: 8 total, 8 up, 8 in 2024-03-26T03:48:58.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:58.385980+0000 mon.b (mon.1) 59 : audit [INF] from='client.? 172.21.15.175:0/1297034116' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi175-64406-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-26T03:48:58.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:58.386539+0000 mon.a (mon.0) 985 : audit [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi175-64470-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi175-64470-10"}]: dispatch 2024-03-26T03:48:58.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:58.386948+0000 mon.a (mon.0) 986 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi175-64501-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:48:58.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:58.387213+0000 mon.a (mon.0) 987 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi175-65296-1"}]: dispatch 2024-03-26T03:48:58.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:58.389266+0000 mon.a (mon.0) 988 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi175-64770-1"}]: dispatch 2024-03-26T03:48:58.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:58.390925+0000 mon.a (mon.0) 989 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-4", "tierpool": "test-rados-api-smithi175-64594-4-cache"}]: dispatch 2024-03-26T03:48:58.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:58 smithi190 bash[23392]: audit 2024-03-26T03:48:58.391189+0000 mon.a (mon.0) 990 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi175-64406-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-26T03:48:58.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:57.419745+0000 mon.a (mon.0) 965 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-03-26T03:48:58.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:57.452672+0000 mon.a (mon.0) 966 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:48:58.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:57.455200+0000 mon.b (mon.1) 55 : audit [INF] from='client.? 172.21.15.175:0/542670412' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:48:58.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:57.458226+0000 mon.a (mon.0) 967 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:48:58.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:57.463068+0000 mon.b (mon.1) 56 : audit [INF] from='client.? 172.21.15.175:0/542670412' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:48:58.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:57.463486+0000 mon.a (mon.0) 968 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi175-64501-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:58.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:57.468559+0000 mon.b (mon.1) 57 : audit [INF] from='client.? 172.21.15.175:0/542670412' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi175-64501-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:58.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:57.473705+0000 mon.a (mon.0) 969 : audit [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi175-64470-10"}]: dispatch 2024-03-26T03:48:58.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:57.480042+0000 mon.a (mon.0) 970 : audit [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi175-64470-10"}]: dispatch 2024-03-26T03:48:58.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:57.482444+0000 mon.a (mon.0) 971 : audit [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi175-64470-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:58.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:57.669288+0000 mon.c (mon.2) 121 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:48:58.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:57.864847+0000 mon.a (mon.0) 972 : audit [DBG] from='client.? 172.21.15.175:0/1820771429' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-26T03:48:58.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:58.335738+0000 mon.a (mon.0) 973 : audit [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi175-64668-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi175-64668-7"}]': finished 2024-03-26T03:48:58.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:58.335926+0000 mon.a (mon.0) 974 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi175-64414-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi175-64414-16"}]': finished 2024-03-26T03:48:58.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:57.419745+0000 mon.a (mon.0) 965 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-03-26T03:48:58.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:57.452672+0000 mon.a (mon.0) 966 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:48:58.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:57.455200+0000 mon.b (mon.1) 55 : audit [INF] from='client.? 172.21.15.175:0/542670412' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:48:58.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:57.458226+0000 mon.a (mon.0) 967 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:48:58.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:57.463068+0000 mon.b (mon.1) 56 : audit [INF] from='client.? 172.21.15.175:0/542670412' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:48:58.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:57.463486+0000 mon.a (mon.0) 968 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi175-64501-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:58.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:57.468559+0000 mon.b (mon.1) 57 : audit [INF] from='client.? 172.21.15.175:0/542670412' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi175-64501-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:58.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:57.473705+0000 mon.a (mon.0) 969 : audit [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi175-64470-10"}]: dispatch 2024-03-26T03:48:58.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:57.480042+0000 mon.a (mon.0) 970 : audit [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi175-64470-10"}]: dispatch 2024-03-26T03:48:58.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:57.482444+0000 mon.a (mon.0) 971 : audit [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi175-64470-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:58.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:57.669288+0000 mon.c (mon.2) 121 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:48:58.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:57.864847+0000 mon.a (mon.0) 972 : audit [DBG] from='client.? 172.21.15.175:0/1820771429' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-26T03:48:58.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:58.335738+0000 mon.a (mon.0) 973 : audit [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi175-64668-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi175-64668-7"}]': finished 2024-03-26T03:48:58.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:58.335926+0000 mon.a (mon.0) 974 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi175-64414-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi175-64414-16"}]': finished 2024-03-26T03:48:58.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:58.336164+0000 mon.a (mon.0) 975 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi175-64422-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi175-64422-22"}]': finished 2024-03-26T03:48:58.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:58.336394+0000 mon.a (mon.0) 976 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi175-64650-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi175-64650-7"}]': finished 2024-03-26T03:48:58.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:58.336650+0000 mon.a (mon.0) 977 : audit [INF] from='client.? 172.21.15.175:0/11262377' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi175-65233-1"}]': finished 2024-03-26T03:48:58.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:58.336786+0000 mon.a (mon.0) 978 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:58.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:58.336960+0000 mon.a (mon.0) 979 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi175-64406-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:58.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:58.337178+0000 mon.a (mon.0) 980 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1"}]': finished 2024-03-26T03:48:58.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:58.337339+0000 mon.a (mon.0) 981 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi175-65296-1"}]': finished 2024-03-26T03:48:58.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:58.337546+0000 mon.a (mon.0) 982 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi175-64501-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:58.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:58.337815+0000 mon.a (mon.0) 983 : audit [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi175-64470-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:58.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:58.352829+0000 mon.b (mon.1) 58 : audit [INF] from='client.? 172.21.15.175:0/542670412' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi175-64501-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:48:58.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:58.369487+0000 mon.c (mon.2) 122 : audit [INF] from='client.? 172.21.15.175:0/1577208989' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi175-65296-1"}]: dispatch 2024-03-26T03:48:58.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:58.370830+0000 mon.c (mon.2) 123 : audit [INF] from='client.? 172.21.15.175:0/1073317595' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi175-64770-1"}]: dispatch 2024-03-26T03:48:58.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:58.380025+0000 mon.c (mon.2) 124 : audit [INF] from='client.? 172.21.15.175:0/3783807506' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-4", "tierpool": "test-rados-api-smithi175-64594-4-cache"}]: dispatch 2024-03-26T03:48:58.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: cluster 2024-03-26T03:48:58.383687+0000 mon.a (mon.0) 984 : cluster [DBG] osdmap e82: 8 total, 8 up, 8 in 2024-03-26T03:48:58.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:58.336164+0000 mon.a (mon.0) 975 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi175-64422-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi175-64422-22"}]': finished 2024-03-26T03:48:58.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:58.336394+0000 mon.a (mon.0) 976 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi175-64650-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi175-64650-7"}]': finished 2024-03-26T03:48:58.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:58.336650+0000 mon.a (mon.0) 977 : audit [INF] from='client.? 172.21.15.175:0/11262377' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi175-65233-1"}]': finished 2024-03-26T03:48:58.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:58.336786+0000 mon.a (mon.0) 978 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:58.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:58.336960+0000 mon.a (mon.0) 979 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi175-64406-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:58.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:58.337178+0000 mon.a (mon.0) 980 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1"}]': finished 2024-03-26T03:48:58.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:58.337339+0000 mon.a (mon.0) 981 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi175-65296-1"}]': finished 2024-03-26T03:48:58.749 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:58.337546+0000 mon.a (mon.0) 982 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi175-64501-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:58.749 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:58.337815+0000 mon.a (mon.0) 983 : audit [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi175-64470-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:58.749 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:58.352829+0000 mon.b (mon.1) 58 : audit [INF] from='client.? 172.21.15.175:0/542670412' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi175-64501-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:48:58.749 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:58.369487+0000 mon.c (mon.2) 122 : audit [INF] from='client.? 172.21.15.175:0/1577208989' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi175-65296-1"}]: dispatch 2024-03-26T03:48:58.749 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:58.370830+0000 mon.c (mon.2) 123 : audit [INF] from='client.? 172.21.15.175:0/1073317595' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi175-64770-1"}]: dispatch 2024-03-26T03:48:58.749 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:58.380025+0000 mon.c (mon.2) 124 : audit [INF] from='client.? 172.21.15.175:0/3783807506' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-4", "tierpool": "test-rados-api-smithi175-64594-4-cache"}]: dispatch 2024-03-26T03:48:58.749 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: cluster 2024-03-26T03:48:58.383687+0000 mon.a (mon.0) 984 : cluster [DBG] osdmap e82: 8 total, 8 up, 8 in 2024-03-26T03:48:58.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:58.385980+0000 mon.b (mon.1) 59 : audit [INF] from='client.? 172.21.15.175:0/1297034116' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi175-64406-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-26T03:48:58.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:58.386539+0000 mon.a (mon.0) 985 : audit [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi175-64470-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi175-64470-10"}]: dispatch 2024-03-26T03:48:58.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:58.386948+0000 mon.a (mon.0) 986 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi175-64501-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:48:58.750 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:58.387213+0000 mon.a (mon.0) 987 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi175-65296-1"}]: dispatch 2024-03-26T03:48:58.750 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:58.389266+0000 mon.a (mon.0) 988 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi175-64770-1"}]: dispatch 2024-03-26T03:48:58.750 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:58.390925+0000 mon.a (mon.0) 989 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-4", "tierpool": "test-rados-api-smithi175-64594-4-cache"}]: dispatch 2024-03-26T03:48:58.750 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[29284]: audit 2024-03-26T03:48:58.391189+0000 mon.a (mon.0) 990 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi175-64406-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-26T03:48:58.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:58.385980+0000 mon.b (mon.1) 59 : audit [INF] from='client.? 172.21.15.175:0/1297034116' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi175-64406-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-26T03:48:58.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:58.386539+0000 mon.a (mon.0) 985 : audit [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi175-64470-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi175-64470-10"}]: dispatch 2024-03-26T03:48:58.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:58.386948+0000 mon.a (mon.0) 986 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi175-64501-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:48:58.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:58.387213+0000 mon.a (mon.0) 987 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi175-65296-1"}]: dispatch 2024-03-26T03:48:58.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:58.389266+0000 mon.a (mon.0) 988 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi175-64770-1"}]: dispatch 2024-03-26T03:48:58.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:58.390925+0000 mon.a (mon.0) 989 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-4", "tierpool": "test-rados-api-smithi175-64594-4-cache"}]: dispatch 2024-03-26T03:48:58.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21527]: audit 2024-03-26T03:48:58.391189+0000 mon.a (mon.0) 990 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi175-64406-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-26T03:48:59.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:48:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:48:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:48:59.408 INFO:tasks.workunit.client.0.smithi175.stdout: cmd: Running main() from gmock_main.cc 2024-03-26T03:48:59.408 INFO:tasks.workunit.client.0.smithi175.stdout: cmd: [==========] Running 3 tests from 1 test suite. 2024-03-26T03:48:59.408 INFO:tasks.workunit.client.0.smithi175.stdout: cmd: [----------] Global test environment set-up. 2024-03-26T03:48:59.408 INFO:tasks.workunit.client.0.smithi175.stdout: cmd: [----------] 3 tests from NeoRadosCmd 2024-03-26T03:48:59.408 INFO:tasks.workunit.client.0.smithi175.stdout: cmd: [ RUN ] NeoRadosCmd.MonDescribe 2024-03-26T03:48:59.409 INFO:tasks.workunit.client.0.smithi175.stdout: cmd: [ OK ] NeoRadosCmd.MonDescribe (1842 ms) 2024-03-26T03:48:59.409 INFO:tasks.workunit.client.0.smithi175.stdout: cmd: [ RUN ] NeoRadosCmd.OSDCmd 2024-03-26T03:48:59.409 INFO:tasks.workunit.client.0.smithi175.stdout: cmd: [ OK ] NeoRadosCmd.OSDCmd (1980 ms) 2024-03-26T03:48:59.409 INFO:tasks.workunit.client.0.smithi175.stdout: cmd: [ RUN ] NeoRadosCmd.PGCmd 2024-03-26T03:48:59.409 INFO:tasks.workunit.client.0.smithi175.stdout: cmd: [ OK ] NeoRadosCmd.PGCmd (3065 ms) 2024-03-26T03:48:59.409 INFO:tasks.workunit.client.0.smithi175.stdout: cmd: [----------] 3 tests from NeoRadosCmd (6887 ms total) 2024-03-26T03:48:59.409 INFO:tasks.workunit.client.0.smithi175.stdout: cmd: 2024-03-26T03:48:59.409 INFO:tasks.workunit.client.0.smithi175.stdout: cmd: [----------] Global test environment tear-down 2024-03-26T03:48:59.409 INFO:tasks.workunit.client.0.smithi175.stdout: cmd: [==========] 3 tests from 1 test suite ran. (6887 ms total) 2024-03-26T03:48:59.409 INFO:tasks.workunit.client.0.smithi175.stdout: cmd: [ PASSED ] 3 tests. 2024-03-26T03:48:59.445 INFO:tasks.workunit.client.0.smithi175.stdout:5269020' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi175-64414-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:48:59.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:56.336179+0000 mon.c [INF] from='client.? 172.21.15.175:0/4087439416' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi175-64650-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:48:59.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:56.337555+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi175-64422-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:48:59.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:56.339628+0000 mon.b [INF] from='client.? 172.21.15.175:0/464642862' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi175-64422-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:48:59.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:56.340697+0000 mon.b [INF] from='client.? 172.21.15.175:0/3798538274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:59.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:56.341162+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:59.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:56.341485+0000 mon.b [INF] from='client.? 172.21.15.175:0/1122404867' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:59.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:56.341635+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:59.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:56.341990+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi175-64650-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:48:59.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:56.343908+0000 mon.c [INF] from='client.? 172.21.15.175:0/1064151631' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi175-64398-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:59.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:56.366923+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi175-64398-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:59.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:57.419745+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-03-26T03:48:59.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:57.452672+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:48:59.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:57.455200+0000 mon.b [INF] from='client.? 172.21.15.175:0/542670412' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:48:59.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:57.458226+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:48:59.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:57.463068+0000 mon.b [INF] from='client.? 172.21.15.175:0/542670412' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:48:59.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:57.463486+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi175-64501-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:59.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:57.468559+0000 mon.b [INF] from='client.? 172.21.15.175:0/542670412' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi175-64501-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:59.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:57.473705+0000 mon.a [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi175-64470-10"}]: dispatch 2024-03-26T03:48:59.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:57.480042+0000 mon.a [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi175-64470-10"}]: dispatch 2024-03-26T03:48:59.447 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:57.482444+0000 mon.a [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi175-64470-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:59.447 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.335738+0000 mon.a [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi175-64668-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi175-64668-7"}]': finished 2024-03-26T03:48:59.447 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.335926+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi175-64414-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi175-64414-16"}]': finished 2024-03-26T03:48:59.447 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.336164+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi175-64422-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi175-64422-22"}]': finished 2024-03-26T03:48:59.447 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.336394+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi175-64650-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi175-64650-7"}]': finished 2024-03-26T03:48:59.447 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.336650+0000 mon.a [INF] from='client.? 172.21.15.175:0/11262377' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi175-65233-1"}]': finished 2024-03-26T03:48:59.447 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.336786+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:59.447 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.336960+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi175-64406-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:48:59.447 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.337178+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi175-64770-1"}]': finished 2024-03-26T03:48:59.453 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.337339+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi175-65296-1"}]': finished 2024-03-26T03:48:59.453 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.337546+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi175-64501-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:59.453 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.337815+0000 mon.a [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi175-64470-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:59.453 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.352829+0000 mon.b [INF] from='client.? 172.21.15.175:0/542670412' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi175-64501-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:48:59.453 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.369487+0000 mon.c [INF] from='client.? 172.21.15.175:0/1577208989' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi175-65296-1"}]: dispatch 2024-03-26T03:48:59.453 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.370830+0000 mon.c [INF] from='client.? 172.21.15.175:0/1073317595' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi175-64770-1"}]: dispatch 2024-03-26T03:48:59.453 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.380025+0000 mon.c [INF] from='client.? 172.21.15.175:0/3783807506' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-4", "tierpool": "test-rados-api-smithi175-64594-4-cache"}]: dispatch 2024-03-26T03:48:59.453 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.385980+0000 mon.b [INF] from='client.? 172.21.15.175:0/1297034116' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi175-64406-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-26T03:48:59.453 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.386539+0000 mon.a [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi175-64470-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi175-64470-10"}]: dispatch 2024-03-26T03:48:59.454 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.386948+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi175-64501-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:48:59.454 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.387213+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi175-65296-1"}]: dispatch 2024-03-26T03:48:59.454 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.389266+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi175-64770-1"}]: dispatch 2024-03-26T03:48:59.454 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.390925+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-4", "tierpool": "test-rados-api-smithi175-64594-4-cache"}]: dispatch 2024-03-26T03:48:59.454 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.391189+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi175-64406-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-26T03:48:59.454 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.443948+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi175-65233-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:59.454 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.446075+0000 mon.b [INF] from='client.? 172.21.15.175:0/1737838842' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi175-65233-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:59.454 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.592993+0000 mon.c [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.0"}]: dispatch 2024-03-26T03:48:59.454 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.593946+0000 mon.c [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.1"}]: dispatch 2024-03-26T03:48:59.454 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.594732+0000 mon.c [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.2"}]: dispatch 2024-03-26T03:48:59.454 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.595557+0000 mon.c [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.3"}]: dispatch 2024-03-26T03:48:59.454 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.596495+0000 mon.c [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.4"}]: dispatch 2024-03-26T03:48:59.454 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.597257+0000 mon.c [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.5"}]: dispatch 2024-03-26T03:48:59.454 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.597982+0000 mon.c [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.6"}]: dispatch 2024-03-26T03:48:59.455 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.598914+0000 mon.c [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.7"}]: dispatch 2024-03-26T03:48:59.455 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.599701+0000 mon.c [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.8"}]: dispatch 2024-03-26T03:48:59.455 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:58.600921+0000 mon.c [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.9"}]: dispatch 2024-03-26T03:48:59.455 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:59.345346+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi175-65296-1"}]': finished 2024-03-26T03:48:59.455 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:59.345526+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi175-64770-1"}]': finished 2024-03-26T03:48:59.456 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:59.345680+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-4", "tierpool": "test-rados-api-smithi175-64594-4-cache"}]': finished 2024-03-26T03:48:59.456 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:59.345918+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi175-64406-3", "field": "max_bytes", "val": "4096"}]': finished 2024-03-26T03:48:59.456 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:59.346169+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi175-65233-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:59.456 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:59.358756+0000 mon.c [INF] from='client.? 172.21.15.175:0/3783807506' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-4", "overlaypool": "test-rados-api-smithi175-64594-4-cache"}]: dispatch 2024-03-26T03:48:59.456 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:59.362458+0000 mon.b [INF] from='client.? 172.21.15.175:0/1737838842' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi175-65233-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi175-65233-2"}]: dispatch 2024-03-26T03:48:59.544 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:59.379367+0000 mon.a [INF] from='client.? ' entity='client api_io_pp: Running main() from gmock_main.cc 2024-03-26T03:48:59.544 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [==========] Running 37 tests from 2 test suites. 2024-03-26T03:48:59.544 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [----------] Global test environment set-up. 2024-03-26T03:48:59.544 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP 2024-03-26T03:48:59.544 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: seed 64422 2024-03-26T03:48:59.544 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2024-03-26T03:48:59.544 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-03-26T03:48:59.544 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2024-03-26T03:48:59.545 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (413 ms) 2024-03-26T03:48:59.545 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2024-03-26T03:48:59.545 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (70 ms) 2024-03-26T03:48:59.545 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2024-03-26T03:48:59.545 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (8 ms) 2024-03-26T03:48:59.545 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2024-03-26T03:48:59.545 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (4 ms) 2024-03-26T03:48:59.545 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2024-03-26T03:48:59.545 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (5 ms) 2024-03-26T03:48:59.545 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2024-03-26T03:48:59.545 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (4 ms) 2024-03-26T03:48:59.545 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-03-26T03:48:59.545 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (4 ms) 2024-03-26T03:48:59.545 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-03-26T03:48:59.545 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (6 ms) 2024-03-26T03:48:59.546 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-03-26T03:48:59.546 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (4 ms) 2024-03-26T03:48:59.546 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-03-26T03:48:59.546 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (4 ms) 2024-03-26T03:48:59.546 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-03-26T03:48:59.546 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (7 ms) 2024-03-26T03:48:59.546 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2024-03-26T03:48:59.546 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (7 ms) 2024-03-26T03:48:59.546 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2024-03-26T03:48:59.546 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (4 ms) 2024-03-26T03:48:59.546 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-03-26T03:48:59.546 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (4 ms) 2024-03-26T03:48:59.546 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2024-03-26T03:48:59.546 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (18 ms) 2024-03-26T03:48:59.547 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-03-26T03:48:59.547 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (7 ms) 2024-03-26T03:48:59.547 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2024-03-26T03:48:59.547 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (4 ms) 2024-03-26T03:48:59.547 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-03-26T03:48:59.547 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (3 ms) 2024-03-26T03:48:59.547 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-03-26T03:48:59.547 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (5 ms) 2024-03-26T03:48:59.547 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP (584 ms total) 2024-03-26T03:48:59.547 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: 2024-03-26T03:48:59.547 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP 2024-03-26T03:48:59.547 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2024-03-26T03:48:59.547 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (1134 ms) 2024-03-26T03:48:59.547 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2024-03-26T03:48:59.547 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (19 ms) 2024-03-26T03:48:59.548 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-03-26T03:48:59.548 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (5 ms) 2024-03-26T03:48:59.548 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2024-03-26T03:48:59.548 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (5 ms) 2024-03-26T03:48:59.548 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2024-03-26T03:48:59.548 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP2 (5 ms) 2024-03-26T03:48:59.740 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:48:59.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:58.443948+0000 mon.a (mon.0) 991 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi175-65233-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:59.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:58.446075+0000 mon.b (mon.1) 60 : audit [INF] from='client.? 172.21.15.175:0/1737838842' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi175-65233-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:59.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: cluster 2024-03-26T03:48:58.571451+0000 mgr.y (mgr.24431) 169 : cluster [DBG] pgmap v120: 740 pgs: 6 active, 64 creating+peering, 122 creating+activating, 228 active+clean, 320 unknown; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-03-26T03:48:59.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:58.592993+0000 mon.c (mon.2) 125 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.0"}]: dispatch 2024-03-26T03:48:59.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:58.593317+0000 mgr.y (mgr.24431) 170 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.0"}]: dispatch 2024-03-26T03:48:59.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:58.593946+0000 mon.c (mon.2) 126 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.1"}]: dispatch 2024-03-26T03:48:59.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:58.594129+0000 mgr.y (mgr.24431) 171 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.1"}]: dispatch 2024-03-26T03:48:59.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:58.594732+0000 mon.c (mon.2) 127 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.2"}]: dispatch 2024-03-26T03:48:59.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:58.595058+0000 mgr.y (mgr.24431) 172 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.2"}]: dispatch 2024-03-26T03:48:59.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:58.595557+0000 mon.c (mon.2) 128 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.3"}]: dispatch 2024-03-26T03:48:59.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:58.595710+0000 mgr.y (mgr.24431) 173 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.3"}]: dispatch 2024-03-26T03:48:59.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:58.596495+0000 mon.c (mon.2) 129 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.4"}]: dispatch 2024-03-26T03:48:59.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:58.596649+0000 mgr.y (mgr.24431) 174 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.4"}]: dispatch 2024-03-26T03:48:59.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:58.597257+0000 mon.c (mon.2) 130 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.5"}]: dispatch 2024-03-26T03:48:59.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:58.597506+0000 mgr.y (mgr.24431) 175 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.5"}]: dispatch 2024-03-26T03:48:59.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:58.597982+0000 mon.c (mon.2) 131 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.6"}]: dispatch 2024-03-26T03:48:59.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:58.598243+0000 mgr.y (mgr.24431) 176 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.6"}]: dispatch 2024-03-26T03:48:59.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:58.443948+0000 mon.a (mon.0) 991 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi175-65233-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:59.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:58.446075+0000 mon.b (mon.1) 60 : audit [INF] from='client.? 172.21.15.175:0/1737838842' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi175-65233-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:59.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: cluster 2024-03-26T03:48:58.571451+0000 mgr.y (mgr.24431) 169 : cluster [DBG] pgmap v120: 740 pgs: 6 active, 64 creating+peering, 122 creating+activating, 228 active+clean, 320 unknown; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-03-26T03:48:59.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:58.592993+0000 mon.c (mon.2) 125 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.0"}]: dispatch 2024-03-26T03:48:59.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:58.593317+0000 mgr.y (mgr.24431) 170 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.0"}]: dispatch 2024-03-26T03:48:59.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:58.593946+0000 mon.c (mon.2) 126 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.1"}]: dispatch 2024-03-26T03:48:59.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:58.594129+0000 mgr.y (mgr.24431) 171 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.1"}]: dispatch 2024-03-26T03:48:59.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:58.594732+0000 mon.c (mon.2) 127 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.2"}]: dispatch 2024-03-26T03:48:59.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:58.595058+0000 mgr.y (mgr.24431) 172 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.2"}]: dispatch 2024-03-26T03:48:59.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:58.595557+0000 mon.c (mon.2) 128 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.3"}]: dispatch 2024-03-26T03:48:59.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:58.595710+0000 mgr.y (mgr.24431) 173 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.3"}]: dispatch 2024-03-26T03:48:59.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:58.596495+0000 mon.c (mon.2) 129 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.4"}]: dispatch 2024-03-26T03:48:59.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:58.596649+0000 mgr.y (mgr.24431) 174 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.4"}]: dispatch 2024-03-26T03:48:59.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:58.597257+0000 mon.c (mon.2) 130 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.5"}]: dispatch 2024-03-26T03:48:59.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:58.597506+0000 mgr.y (mgr.24431) 175 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.5"}]: dispatch 2024-03-26T03:48:59.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:58.597982+0000 mon.c (mon.2) 131 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.6"}]: dispatch 2024-03-26T03:48:59.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:58.598243+0000 mgr.y (mgr.24431) 176 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.6"}]: dispatch 2024-03-26T03:48:59.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:58.598914+0000 mon.c (mon.2) 132 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.7"}]: dispatch 2024-03-26T03:48:59.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:58.599144+0000 mgr.y (mgr.24431) 177 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.7"}]: dispatch 2024-03-26T03:48:59.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:58.599701+0000 mon.c (mon.2) 133 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.8"}]: dispatch 2024-03-26T03:48:59.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:58.599917+0000 mgr.y (mgr.24431) 178 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.8"}]: dispatch 2024-03-26T03:48:59.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:58.600921+0000 mon.c (mon.2) 134 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.9"}]: dispatch 2024-03-26T03:48:59.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:58.601195+0000 mgr.y (mgr.24431) 179 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.9"}]: dispatch 2024-03-26T03:48:59.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:58.670410+0000 mon.c (mon.2) 135 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:48:59.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: cluster 2024-03-26T03:48:58.763725+0000 osd.3 (osd.3) 3 : cluster [DBG] 16.4 deep-scrub starts 2024-03-26T03:48:59.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: cluster 2024-03-26T03:48:58.768517+0000 osd.3 (osd.3) 4 : cluster [DBG] 16.4 deep-scrub ok 2024-03-26T03:48:59.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: cluster 2024-03-26T03:48:58.828363+0000 osd.1 (osd.1) 3 : cluster [DBG] 16.9 deep-scrub starts 2024-03-26T03:48:59.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: cluster 2024-03-26T03:48:58.830741+0000 osd.1 (osd.1) 4 : cluster [DBG] 16.9 deep-scrub ok 2024-03-26T03:48:59.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: cluster 2024-03-26T03:48:58.973283+0000 osd.4 (osd.4) 3 : cluster [DBG] 16.2 deep-scrub starts 2024-03-26T03:48:59.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: cluster 2024-03-26T03:48:58.978567+0000 osd.4 (osd.4) 4 : cluster [DBG] 16.2 deep-scrub ok 2024-03-26T03:48:59.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:59.345346+0000 mon.a (mon.0) 992 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi175-65296-1"}]': finished 2024-03-26T03:48:59.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:59.345526+0000 mon.a (mon.0) 993 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi175-64770-1"}]': finished 2024-03-26T03:48:59.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:59.345680+0000 mon.a (mon.0) 994 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-4", "tierpool": "test-rados-api-smithi175-64594-4-cache"}]': finished 2024-03-26T03:48:59.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:59.345918+0000 mon.a (mon.0) 995 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi175-64406-3", "field": "max_bytes", "val": "4096"}]': finished 2024-03-26T03:48:59.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:59.346169+0000 mon.a (mon.0) 996 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi175-65233-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:59.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:59.358756+0000 mon.c (mon.2) 136 : audit [INF] from='client.? 172.21.15.175:0/3783807506' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-4", "overlaypool": "test-rados-api-smithi175-64594-4-cache"}]: dispatch 2024-03-26T03:48:59.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:59.362458+0000 mon.b (mon.1) 61 : audit [INF] from='client.? 172.21.15.175:0/1737838842' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi175-65233-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi175-65233-2"}]: dispatch 2024-03-26T03:48:59.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: cluster 2024-03-26T03:48:59.374252+0000 mon.a (mon.0) 997 : cluster [DBG] osdmap e83: 8 total, 8 up, 8 in 2024-03-26T03:48:59.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:59.379367+0000 mon.a (mon.0) 998 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi175-65233-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi175-65233-2"}]: dispatch 2024-03-26T03:48:59.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:59.379841+0000 mon.a (mon.0) 999 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-4", "overlaypool": "test-rados-api-smithi175-64594-4-cache"}]: dispatch 2024-03-26T03:48:59.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:59.380190+0000 mon.c (mon.2) 137 : audit [INF] from='client.? 172.21.15.175:0/1208043411' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:59.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:59.389130+0000 mon.a (mon.0) 1000 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:59.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:59.391381+0000 mon.b (mon.1) 62 : audit [INF] from='client.? 172.21.15.175:0/3671149507' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:59.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:59.391434+0000 mon.c (mon.2) 138 : audit [INF] from='client.? 172.21.15.175:0/2754872702' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi175-64398-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:59.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:59.396079+0000 mon.a (mon.0) 1001 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:59.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:59.396429+0000 mon.a (mon.0) 1002 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi175-64398-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:59.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:59.423612+0000 mon.a (mon.0) 1003 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi175-65296-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:59.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:48:59 smithi190 bash[23392]: audit 2024-03-26T03:48:59.427716+0000 mon.b (mon.1) 63 : audit [INF] from='client.? 172.21.15.175:0/2671281638' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi175-65296-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:59.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:58.443948+0000 mon.a (mon.0) 991 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi175-65233-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:59.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:58.446075+0000 mon.b (mon.1) 60 : audit [INF] from='client.? 172.21.15.175:0/1737838842' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi175-65233-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:59.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: cluster 2024-03-26T03:48:58.571451+0000 mgr.y (mgr.24431) 169 : cluster [DBG] pgmap v120: 740 pgs: 6 active, 64 creating+peering, 122 creating+activating, 228 active+clean, 320 unknown; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-03-26T03:48:59.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:58.592993+0000 mon.c (mon.2) 125 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.0"}]: dispatch 2024-03-26T03:48:59.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:58.593317+0000 mgr.y (mgr.24431) 170 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.0"}]: dispatch 2024-03-26T03:48:59.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:58.593946+0000 mon.c (mon.2) 126 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.1"}]: dispatch 2024-03-26T03:48:59.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:58.594129+0000 mgr.y (mgr.24431) 171 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.1"}]: dispatch 2024-03-26T03:48:59.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:58.594732+0000 mon.c (mon.2) 127 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.2"}]: dispatch 2024-03-26T03:48:59.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:58.595058+0000 mgr.y (mgr.24431) 172 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.2"}]: dispatch 2024-03-26T03:48:59.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:58.595557+0000 mon.c (mon.2) 128 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.3"}]: dispatch 2024-03-26T03:48:59.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:58.595710+0000 mgr.y (mgr.24431) 173 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.3"}]: dispatch 2024-03-26T03:48:59.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:58.596495+0000 mon.c (mon.2) 129 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.4"}]: dispatch 2024-03-26T03:48:59.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:58.596649+0000 mgr.y (mgr.24431) 174 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.4"}]: dispatch 2024-03-26T03:48:59.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:58.597257+0000 mon.c (mon.2) 130 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.5"}]: dispatch 2024-03-26T03:48:59.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:58.597506+0000 mgr.y (mgr.24431) 175 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.5"}]: dispatch 2024-03-26T03:48:59.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:58.597982+0000 mon.c (mon.2) 131 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.6"}]: dispatch 2024-03-26T03:48:59.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:58.598243+0000 mgr.y (mgr.24431) 176 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.6"}]: dispatch 2024-03-26T03:48:59.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:58.598914+0000 mon.c (mon.2) 132 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.7"}]: dispatch 2024-03-26T03:48:59.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:58.599144+0000 mgr.y (mgr.24431) 177 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.7"}]: dispatch 2024-03-26T03:48:59.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:58.599701+0000 mon.c (mon.2) 133 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.8"}]: dispatch 2024-03-26T03:48:59.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:58.599917+0000 mgr.y (mgr.24431) 178 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.8"}]: dispatch 2024-03-26T03:48:59.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:58.600921+0000 mon.c (mon.2) 134 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.9"}]: dispatch 2024-03-26T03:48:59.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:58.601195+0000 mgr.y (mgr.24431) 179 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.9"}]: dispatch 2024-03-26T03:48:59.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:58.670410+0000 mon.c (mon.2) 135 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:48:59.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: cluster 2024-03-26T03:48:58.763725+0000 osd.3 (osd.3) 3 : cluster [DBG] 16.4 deep-scrub starts 2024-03-26T03:48:59.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: cluster 2024-03-26T03:48:58.768517+0000 osd.3 (osd.3) 4 : cluster [DBG] 16.4 deep-scrub ok 2024-03-26T03:48:59.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: cluster 2024-03-26T03:48:58.828363+0000 osd.1 (osd.1) 3 : cluster [DBG] 16.9 deep-scrub starts 2024-03-26T03:48:59.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: cluster 2024-03-26T03:48:58.830741+0000 osd.1 (osd.1) 4 : cluster [DBG] 16.9 deep-scrub ok 2024-03-26T03:48:59.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: cluster 2024-03-26T03:48:58.973283+0000 osd.4 (osd.4) 3 : cluster [DBG] 16.2 deep-scrub starts 2024-03-26T03:48:59.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: cluster 2024-03-26T03:48:58.978567+0000 osd.4 (osd.4) 4 : cluster [DBG] 16.2 deep-scrub ok 2024-03-26T03:48:59.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:59.345346+0000 mon.a (mon.0) 992 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi175-65296-1"}]': finished 2024-03-26T03:48:59.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:59.345526+0000 mon.a (mon.0) 993 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi175-64770-1"}]': finished 2024-03-26T03:48:59.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:59.345680+0000 mon.a (mon.0) 994 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-4", "tierpool": "test-rados-api-smithi175-64594-4-cache"}]': finished 2024-03-26T03:48:59.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:59.345918+0000 mon.a (mon.0) 995 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi175-64406-3", "field": "max_bytes", "val": "4096"}]': finished 2024-03-26T03:48:59.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:59.346169+0000 mon.a (mon.0) 996 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi175-65233-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:59.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:59.358756+0000 mon.c (mon.2) 136 : audit [INF] from='client.? 172.21.15.175:0/3783807506' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-4", "overlaypool": "test-rados-api-smithi175-64594-4-cache"}]: dispatch 2024-03-26T03:48:59.749 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:58.598914+0000 mon.c (mon.2) 132 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.7"}]: dispatch 2024-03-26T03:48:59.749 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:58.599144+0000 mgr.y (mgr.24431) 177 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.7"}]: dispatch 2024-03-26T03:48:59.749 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:58.599701+0000 mon.c (mon.2) 133 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.8"}]: dispatch 2024-03-26T03:48:59.749 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:58.599917+0000 mgr.y (mgr.24431) 178 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.8"}]: dispatch 2024-03-26T03:48:59.749 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:58.600921+0000 mon.c (mon.2) 134 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.9"}]: dispatch 2024-03-26T03:48:59.749 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:58.601195+0000 mgr.y (mgr.24431) 179 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.9"}]: dispatch 2024-03-26T03:48:59.749 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:58.670410+0000 mon.c (mon.2) 135 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:48:59.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: cluster 2024-03-26T03:48:58.763725+0000 osd.3 (osd.3) 3 : cluster [DBG] 16.4 deep-scrub starts 2024-03-26T03:48:59.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: cluster 2024-03-26T03:48:58.768517+0000 osd.3 (osd.3) 4 : cluster [DBG] 16.4 deep-scrub ok 2024-03-26T03:48:59.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: cluster 2024-03-26T03:48:58.828363+0000 osd.1 (osd.1) 3 : cluster [DBG] 16.9 deep-scrub starts 2024-03-26T03:48:59.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: cluster 2024-03-26T03:48:58.830741+0000 osd.1 (osd.1) 4 : cluster [DBG] 16.9 deep-scrub ok 2024-03-26T03:48:59.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: cluster 2024-03-26T03:48:58.973283+0000 osd.4 (osd.4) 3 : cluster [DBG] 16.2 deep-scrub starts 2024-03-26T03:48:59.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: cluster 2024-03-26T03:48:58.978567+0000 osd.4 (osd.4) 4 : cluster [DBG] 16.2 deep-scrub ok 2024-03-26T03:48:59.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:59.345346+0000 mon.a (mon.0) 992 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi175-65296-1"}]': finished 2024-03-26T03:48:59.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:59.345526+0000 mon.a (mon.0) 993 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi175-64770-1"}]': finished 2024-03-26T03:48:59.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:59.345680+0000 mon.a (mon.0) 994 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-4", "tierpool": "test-rados-api-smithi175-64594-4-cache"}]': finished 2024-03-26T03:48:59.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:59.345918+0000 mon.a (mon.0) 995 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi175-64406-3", "field": "max_bytes", "val": "4096"}]': finished 2024-03-26T03:48:59.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:59.346169+0000 mon.a (mon.0) 996 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi175-65233-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:48:59.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:59.358756+0000 mon.c (mon.2) 136 : audit [INF] from='client.? 172.21.15.175:0/3783807506' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-4", "overlaypool": "test-rados-api-smithi175-64594-4-cache"}]: dispatch 2024-03-26T03:48:59.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:59.362458+0000 mon.b (mon.1) 61 : audit [INF] from='client.? 172.21.15.175:0/1737838842' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi175-65233-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi175-65233-2"}]: dispatch 2024-03-26T03:48:59.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: cluster 2024-03-26T03:48:59.374252+0000 mon.a (mon.0) 997 : cluster [DBG] osdmap e83: 8 total, 8 up, 8 in 2024-03-26T03:48:59.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:59.379367+0000 mon.a (mon.0) 998 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi175-65233-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi175-65233-2"}]: dispatch 2024-03-26T03:48:59.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:59.379841+0000 mon.a (mon.0) 999 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-4", "overlaypool": "test-rados-api-smithi175-64594-4-cache"}]: dispatch 2024-03-26T03:48:59.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:59.380190+0000 mon.c (mon.2) 137 : audit [INF] from='client.? 172.21.15.175:0/1208043411' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:59.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:59.389130+0000 mon.a (mon.0) 1000 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:59.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:59.391381+0000 mon.b (mon.1) 62 : audit [INF] from='client.? 172.21.15.175:0/3671149507' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:59.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:59.391434+0000 mon.c (mon.2) 138 : audit [INF] from='client.? 172.21.15.175:0/2754872702' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi175-64398-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:59.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:59.396079+0000 mon.a (mon.0) 1001 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:59.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:59.396429+0000 mon.a (mon.0) 1002 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi175-64398-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:59.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:59.423612+0000 mon.a (mon.0) 1003 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi175-65296-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:59.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[29284]: audit 2024-03-26T03:48:59.427716+0000 mon.b (mon.1) 63 : audit [INF] from='client.? 172.21.15.175:0/2671281638' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi175-65296-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:59.752 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:59.362458+0000 mon.b (mon.1) 61 : audit [INF] from='client.? 172.21.15.175:0/1737838842' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi175-65233-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi175-65233-2"}]: dispatch 2024-03-26T03:48:59.752 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: cluster 2024-03-26T03:48:59.374252+0000 mon.a (mon.0) 997 : cluster [DBG] osdmap e83: 8 total, 8 up, 8 in 2024-03-26T03:48:59.752 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:59.379367+0000 mon.a (mon.0) 998 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi175-65233-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi175-65233-2"}]: dispatch 2024-03-26T03:48:59.752 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:59.379841+0000 mon.a (mon.0) 999 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-4", "overlaypool": "test-rados-api-smithi175-64594-4-cache"}]: dispatch 2024-03-26T03:48:59.752 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:59.380190+0000 mon.c (mon.2) 137 : audit [INF] from='client.? 172.21.15.175:0/1208043411' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:59.752 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:59.389130+0000 mon.a (mon.0) 1000 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:59.752 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:59.391381+0000 mon.b (mon.1) 62 : audit [INF] from='client.? 172.21.15.175:0/3671149507' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:59.752 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:59.391434+0000 mon.c (mon.2) 138 : audit [INF] from='client.? 172.21.15.175:0/2754872702' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi175-64398-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:59.752 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:59.396079+0000 mon.a (mon.0) 1001 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:59.752 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:59.396429+0000 mon.a (mon.0) 1002 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi175-64398-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:48:59.753 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:59.423612+0000 mon.a (mon.0) 1003 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi175-65296-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:48:59.753 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:48:59 smithi175 bash[21527]: audit 2024-03-26T03:48:59.427716+0000 mon.b (mon.1) 63 : audit [INF] from='client.? 172.21.15.175:0/2671281638' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi175-65296-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:49:00.469 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ].admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi175-65233-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi175-65233-2"}]: dispatch 2024-03-26T03:49:00.469 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:59.379841+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-4", "overlaypool": "test-rados-api-smithi175-64594-4-cache"}]: dispatch 2024-03-26T03:49:00.469 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:59.380190+0000 mon.c [INF] from='client.? 172.21.15.175:0/1208043411' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:00.469 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:59.389130+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:00.469 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:59.391381+0000 mon.b [INF] from='client.? 172.21.15.175:0/3671149507' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:00.469 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:59.391434+0000 mon.c [INF] from='client.? 172.21.15.175:0/2754872702' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi175-64398-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:00.469 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:59.396079+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:00.470 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:59.396429+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi175-64398-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:00.470 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:59.423612+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi175-65296-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:49:00.470 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:59.427716+0000 mon.b [INF] from='client.? 172.21.15.175:0/2671281638' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi175-65296-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:49:00.470 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:59.424616+0000 mon.a [WRN] Health check update: 12 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:00.470 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:59.451305+0000 mon.b [INF] from='client.? 172.21.15.175:0/1244868480' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-26T03:49:00.470 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:59.462936+0000 client.admin [INF] threexx 2024-03-26T03:49:00.470 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:48:59.469229+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-26T03:49:00.470 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:49:00.355071+0000 mon.a [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi175-64470-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi175-64470-10"}]': finished 2024-03-26T03:49:00.470 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:49:00.355330+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi175-64501-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi175-64501-10"}]': finished 2024-03-26T03:49:00.470 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:49:00.355636+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-4", "overlaypool": "test-rados-api-smithi175-64594-4-cache"}]': finished 2024-03-26T03:49:00.470 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:49:00.355891+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:00.470 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:49:00.356151+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:00.470 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:49:00.356367+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi175-64398-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:00.470 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:49:00.356600+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi175-65296-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:49:00.471 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:49:00.366809+0000 mon.c [INF] from='client.? 172.21.15.175:0/3783807506' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-4-cache", "mode": "writeback"}]: dispatch 2024-03-26T03:49:00.471 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:49:00.373616+0000 mon.b [INF] from='client.? 172.21.15.175:0/2671281638' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi175-65296-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi175-65296-2"}]: dispatch 2024-03-26T03:49:00.471 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:49:00.382908+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi175-65296-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi175-65296-2"}]: dispatch 2024-03-26T03:49:00.471 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:49:00.384806+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-4-cache", "mode": "writeback"}]: dispatch 2024-03-26T03:49:00.471 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:49:00.384845+0000 mon.c [INF] from='client.? 172.21.15.175:0/4207065844' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:00.471 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:49:00.388488+0000 mon.b [INF] from='client.? 172.21.15.175:0/2115269020' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:49:00.471 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:49:00.389172+0000 mon.b [INF] from='client.? 172.21.15.175:0/464642862' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:49:00.471 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:49:00.389215+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:49:00.471 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:49:00.390713+0000 mon.a [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi175-64668-7"}]: dispatch 2024-03-26T03:49:00.471 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:49:00.391039+0000 mon.c [INF] from='client.? 172.21.15.175:0/4087439416' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:49:00.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: cluster 2024-03-26T03:48:58.958929+0000 osd.5 (osd.5) 3 : cluster [DBG] 16.3 deep-scrub starts 2024-03-26T03:49:00.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: cluster 2024-03-26T03:48:58.961365+0000 osd.5 (osd.5) 4 : cluster [DBG] 16.3 deep-scrub ok 2024-03-26T03:49:00.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: cluster 2024-03-26T03:48:59.040711+0000 osd.0 (osd.0) 3 : cluster [DBG] 16.5 deep-scrub starts 2024-03-26T03:49:00.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: cluster 2024-03-26T03:48:59.043044+0000 osd.0 (osd.0) 4 : cluster [DBG] 16.5 deep-scrub ok 2024-03-26T03:49:00.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: cluster 2024-03-26T03:48:59.277734+0000 osd.2 (osd.2) 3 : cluster [DBG] 16.1 deep-scrub starts 2024-03-26T03:49:00.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: cluster 2024-03-26T03:48:59.281820+0000 osd.2 (osd.2) 4 : cluster [DBG] 16.1 deep-scrub ok 2024-03-26T03:49:00.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: cluster 2024-03-26T03:48:59.343902+0000 osd.6 (osd.6) 3 : cluster [DBG] 16.6 deep-scrub starts 2024-03-26T03:49:00.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: cluster 2024-03-26T03:48:59.346280+0000 osd.6 (osd.6) 4 : cluster [DBG] 16.6 deep-scrub ok 2024-03-26T03:49:00.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: cluster 2024-03-26T03:48:59.424616+0000 mon.a (mon.0) 1004 : cluster [WRN] Health check update: 12 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:00.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: audit 2024-03-26T03:48:59.451305+0000 mon.b (mon.1) 64 : audit [INF] from='client.? 172.21.15.175:0/1244868480' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-26T03:49:00.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: cluster 2024-03-26T03:48:59.462936+0000 client.admin (client.?) 0 : cluster [INF] threexx 2024-03-26T03:49:00.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: audit 2024-03-26T03:48:59.469229+0000 mon.a (mon.0) 1005 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-26T03:49:00.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: audit 2024-03-26T03:48:59.672007+0000 mon.c (mon.2) 139 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:00.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: audit 2024-03-26T03:48:59.684245+0000 mgr.y (mgr.24431) 180 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:49:00.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: cluster 2024-03-26T03:48:59.779904+0000 osd.3 (osd.3) 5 : cluster [DBG] 16.7 deep-scrub starts 2024-03-26T03:49:00.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: cluster 2024-03-26T03:48:59.783660+0000 osd.3 (osd.3) 6 : cluster [DBG] 16.7 deep-scrub ok 2024-03-26T03:49:00.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: audit 2024-03-26T03:48:59.899923+0000 mon.a (mon.0) 1006 : audit [DBG] from='client.? 172.21.15.175:0/2992910306' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-26T03:49:00.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: audit 2024-03-26T03:49:00.355071+0000 mon.a (mon.0) 1007 : audit [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi175-64470-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi175-64470-10"}]': finished 2024-03-26T03:49:00.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: audit 2024-03-26T03:49:00.355330+0000 mon.a (mon.0) 1008 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi175-64501-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi175-64501-10"}]': finished 2024-03-26T03:49:00.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: audit 2024-03-26T03:49:00.355636+0000 mon.a (mon.0) 1009 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-4", "overlaypool": "test-rados-api-smithi175-64594-4-cache"}]': finished 2024-03-26T03:49:00.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: audit 2024-03-26T03:49:00.355891+0000 mon.a (mon.0) 1010 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:00.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: audit 2024-03-26T03:49:00.356151+0000 mon.a (mon.0) 1011 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:00.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: audit 2024-03-26T03:49:00.356367+0000 mon.a (mon.0) 1012 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi175-64398-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:00.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: audit 2024-03-26T03:49:00.356600+0000 mon.a (mon.0) 1013 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi175-65296-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:49:00.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: audit 2024-03-26T03:49:00.366809+0000 mon.c (mon.2) 140 : audit [INF] from='client.? 172.21.15.175:0/3783807506' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-4-cache", "mode": "writeback"}]: dispatch 2024-03-26T03:49:00.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: audit 2024-03-26T03:49:00.373616+0000 mon.b (mon.1) 65 : audit [INF] from='client.? 172.21.15.175:0/2671281638' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi175-65296-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi175-65296-2"}]: dispatch 2024-03-26T03:49:00.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: cluster 2024-03-26T03:49:00.378301+0000 mon.a (mon.0) 1014 : cluster [DBG] osdmap e84: 8 total, 8 up, 8 in 2024-03-26T03:49:00.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: audit 2024-03-26T03:49:00.382908+0000 mon.a (mon.0) 1015 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi175-65296-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi175-65296-2"}]: dispatch 2024-03-26T03:49:00.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: audit 2024-03-26T03:49:00.384806+0000 mon.a (mon.0) 1016 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-4-cache", "mode": "writeback"}]: dispatch 2024-03-26T03:49:00.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: audit 2024-03-26T03:49:00.384845+0000 mon.c (mon.2) 141 : audit [INF] from='client.? 172.21.15.175:0/4207065844' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:00.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: audit 2024-03-26T03:49:00.388488+0000 mon.b (mon.1) 66 : audit [INF] from='client.? 172.21.15.175:0/2115269020' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:49:00.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: audit 2024-03-26T03:49:00.389172+0000 mon.b (mon.1) 67 : audit [INF] from='client.? 172.21.15.175:0/464642862' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:49:00.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: audit 2024-03-26T03:49:00.389215+0000 mon.a (mon.0) 1017 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:49:00.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: audit 2024-03-26T03:49:00.390713+0000 mon.a (mon.0) 1018 : audit [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi175-64668-7"}]: dispatch 2024-03-26T03:49:00.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: audit 2024-03-26T03:49:00.391039+0000 mon.c (mon.2) 142 : audit [INF] from='client.? 172.21.15.175:0/4087439416' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:49:00.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: audit 2024-03-26T03:49:00.392734+0000 mon.a (mon.0) 1019 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:49:00.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: audit 2024-03-26T03:49:00.393207+0000 mon.a (mon.0) 1020 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:00.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:00 smithi190 bash[23392]: audit 2024-03-26T03:49:00.393623+0000 mon.a (mon.0) 1021 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:49:00.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: cluster 2024-03-26T03:48:58.958929+0000 osd.5 (osd.5) 3 : cluster [DBG] 16.3 deep-scrub starts 2024-03-26T03:49:00.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: cluster 2024-03-26T03:48:58.961365+0000 osd.5 (osd.5) 4 : cluster [DBG] 16.3 deep-scrub ok 2024-03-26T03:49:00.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: cluster 2024-03-26T03:48:59.040711+0000 osd.0 (osd.0) 3 : cluster [DBG] 16.5 deep-scrub starts 2024-03-26T03:49:00.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: cluster 2024-03-26T03:48:59.043044+0000 osd.0 (osd.0) 4 : cluster [DBG] 16.5 deep-scrub ok 2024-03-26T03:49:00.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: cluster 2024-03-26T03:48:59.277734+0000 osd.2 (osd.2) 3 : cluster [DBG] 16.1 deep-scrub starts 2024-03-26T03:49:00.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: cluster 2024-03-26T03:48:59.281820+0000 osd.2 (osd.2) 4 : cluster [DBG] 16.1 deep-scrub ok 2024-03-26T03:49:00.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: cluster 2024-03-26T03:48:59.343902+0000 osd.6 (osd.6) 3 : cluster [DBG] 16.6 deep-scrub starts 2024-03-26T03:49:00.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: cluster 2024-03-26T03:48:59.346280+0000 osd.6 (osd.6) 4 : cluster [DBG] 16.6 deep-scrub ok 2024-03-26T03:49:00.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: cluster 2024-03-26T03:48:59.424616+0000 mon.a (mon.0) 1004 : cluster [WRN] Health check update: 12 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:00.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: audit 2024-03-26T03:48:59.451305+0000 mon.b (mon.1) 64 : audit [INF] from='client.? 172.21.15.175:0/1244868480' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-26T03:49:00.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: cluster 2024-03-26T03:48:59.462936+0000 client.admin (client.?) 0 : cluster [INF] threexx 2024-03-26T03:49:00.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: audit 2024-03-26T03:48:59.469229+0000 mon.a (mon.0) 1005 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-26T03:49:00.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: audit 2024-03-26T03:48:59.672007+0000 mon.c (mon.2) 139 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:00.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: audit 2024-03-26T03:48:59.684245+0000 mgr.y (mgr.24431) 180 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:49:00.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: cluster 2024-03-26T03:48:59.779904+0000 osd.3 (osd.3) 5 : cluster [DBG] 16.7 deep-scrub starts 2024-03-26T03:49:00.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: cluster 2024-03-26T03:48:59.783660+0000 osd.3 (osd.3) 6 : cluster [DBG] 16.7 deep-scrub ok 2024-03-26T03:49:00.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: audit 2024-03-26T03:48:59.899923+0000 mon.a (mon.0) 1006 : audit [DBG] from='client.? 172.21.15.175:0/2992910306' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-26T03:49:00.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: audit 2024-03-26T03:49:00.355071+0000 mon.a (mon.0) 1007 : audit [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi175-64470-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi175-64470-10"}]': finished 2024-03-26T03:49:00.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: audit 2024-03-26T03:49:00.355330+0000 mon.a (mon.0) 1008 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi175-64501-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi175-64501-10"}]': finished 2024-03-26T03:49:00.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: audit 2024-03-26T03:49:00.355636+0000 mon.a (mon.0) 1009 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-4", "overlaypool": "test-rados-api-smithi175-64594-4-cache"}]': finished 2024-03-26T03:49:00.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: cluster 2024-03-26T03:48:58.958929+0000 osd.5 (osd.5) 3 : cluster [DBG] 16.3 deep-scrub starts 2024-03-26T03:49:00.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: cluster 2024-03-26T03:48:58.961365+0000 osd.5 (osd.5) 4 : cluster [DBG] 16.3 deep-scrub ok 2024-03-26T03:49:00.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: cluster 2024-03-26T03:48:59.040711+0000 osd.0 (osd.0) 3 : cluster [DBG] 16.5 deep-scrub starts 2024-03-26T03:49:00.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: cluster 2024-03-26T03:48:59.043044+0000 osd.0 (osd.0) 4 : cluster [DBG] 16.5 deep-scrub ok 2024-03-26T03:49:00.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: cluster 2024-03-26T03:48:59.277734+0000 osd.2 (osd.2) 3 : cluster [DBG] 16.1 deep-scrub starts 2024-03-26T03:49:00.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: cluster 2024-03-26T03:48:59.281820+0000 osd.2 (osd.2) 4 : cluster [DBG] 16.1 deep-scrub ok 2024-03-26T03:49:00.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: cluster 2024-03-26T03:48:59.343902+0000 osd.6 (osd.6) 3 : cluster [DBG] 16.6 deep-scrub starts 2024-03-26T03:49:00.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: cluster 2024-03-26T03:48:59.346280+0000 osd.6 (osd.6) 4 : cluster [DBG] 16.6 deep-scrub ok 2024-03-26T03:49:00.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: cluster 2024-03-26T03:48:59.424616+0000 mon.a (mon.0) 1004 : cluster [WRN] Health check update: 12 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:00.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: audit 2024-03-26T03:48:59.451305+0000 mon.b (mon.1) 64 : audit [INF] from='client.? 172.21.15.175:0/1244868480' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-26T03:49:00.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: cluster 2024-03-26T03:48:59.462936+0000 client.admin (client.?) 0 : cluster [INF] threexx 2024-03-26T03:49:00.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: audit 2024-03-26T03:48:59.469229+0000 mon.a (mon.0) 1005 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-26T03:49:00.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: audit 2024-03-26T03:48:59.672007+0000 mon.c (mon.2) 139 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:00.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: audit 2024-03-26T03:48:59.684245+0000 mgr.y (mgr.24431) 180 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:49:00.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: cluster 2024-03-26T03:48:59.779904+0000 osd.3 (osd.3) 5 : cluster [DBG] 16.7 deep-scrub starts 2024-03-26T03:49:00.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: cluster 2024-03-26T03:48:59.783660+0000 osd.3 (osd.3) 6 : cluster [DBG] 16.7 deep-scrub ok 2024-03-26T03:49:00.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: audit 2024-03-26T03:48:59.899923+0000 mon.a (mon.0) 1006 : audit [DBG] from='client.? 172.21.15.175:0/2992910306' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-26T03:49:00.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: audit 2024-03-26T03:49:00.355071+0000 mon.a (mon.0) 1007 : audit [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi175-64470-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi175-64470-10"}]': finished 2024-03-26T03:49:00.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: audit 2024-03-26T03:49:00.355330+0000 mon.a (mon.0) 1008 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi175-64501-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi175-64501-10"}]': finished 2024-03-26T03:49:00.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: audit 2024-03-26T03:49:00.355636+0000 mon.a (mon.0) 1009 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-4", "overlaypool": "test-rados-api-smithi175-64594-4-cache"}]': finished 2024-03-26T03:49:00.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: audit 2024-03-26T03:49:00.355891+0000 mon.a (mon.0) 1010 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:00.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: audit 2024-03-26T03:49:00.356151+0000 mon.a (mon.0) 1011 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:00.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: audit 2024-03-26T03:49:00.356367+0000 mon.a (mon.0) 1012 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi175-64398-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:00.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: audit 2024-03-26T03:49:00.356600+0000 mon.a (mon.0) 1013 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi175-65296-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:49:00.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: audit 2024-03-26T03:49:00.366809+0000 mon.c (mon.2) 140 : audit [INF] from='client.? 172.21.15.175:0/3783807506' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-4-cache", "mode": "writeback"}]: dispatch 2024-03-26T03:49:00.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: audit 2024-03-26T03:49:00.373616+0000 mon.b (mon.1) 65 : audit [INF] from='client.? 172.21.15.175:0/2671281638' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi175-65296-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi175-65296-2"}]: dispatch 2024-03-26T03:49:00.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: cluster 2024-03-26T03:49:00.378301+0000 mon.a (mon.0) 1014 : cluster [DBG] osdmap e84: 8 total, 8 up, 8 in 2024-03-26T03:49:00.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: audit 2024-03-26T03:49:00.382908+0000 mon.a (mon.0) 1015 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi175-65296-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi175-65296-2"}]: dispatch 2024-03-26T03:49:00.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: audit 2024-03-26T03:49:00.384806+0000 mon.a (mon.0) 1016 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-4-cache", "mode": "writeback"}]: dispatch 2024-03-26T03:49:00.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: audit 2024-03-26T03:49:00.384845+0000 mon.c (mon.2) 141 : audit [INF] from='client.? 172.21.15.175:0/4207065844' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:00.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: audit 2024-03-26T03:49:00.388488+0000 mon.b (mon.1) 66 : audit [INF] from='client.? 172.21.15.175:0/2115269020' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:49:00.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: audit 2024-03-26T03:49:00.389172+0000 mon.b (mon.1) 67 : audit [INF] from='client.? 172.21.15.175:0/464642862' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:49:00.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: audit 2024-03-26T03:49:00.389215+0000 mon.a (mon.0) 1017 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:49:00.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: audit 2024-03-26T03:49:00.390713+0000 mon.a (mon.0) 1018 : audit [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi175-64668-7"}]: dispatch 2024-03-26T03:49:00.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: audit 2024-03-26T03:49:00.391039+0000 mon.c (mon.2) 142 : audit [INF] from='client.? 172.21.15.175:0/4087439416' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:49:00.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: audit 2024-03-26T03:49:00.392734+0000 mon.a (mon.0) 1019 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:49:00.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: audit 2024-03-26T03:49:00.393207+0000 mon.a (mon.0) 1020 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:00.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[29284]: audit 2024-03-26T03:49:00.393623+0000 mon.a (mon.0) 1021 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:49:00.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: audit 2024-03-26T03:49:00.355891+0000 mon.a (mon.0) 1010 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:00.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: audit 2024-03-26T03:49:00.356151+0000 mon.a (mon.0) 1011 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:00.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: audit 2024-03-26T03:49:00.356367+0000 mon.a (mon.0) 1012 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi175-64398-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:00.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: audit 2024-03-26T03:49:00.356600+0000 mon.a (mon.0) 1013 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi175-65296-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:49:00.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: audit 2024-03-26T03:49:00.366809+0000 mon.c (mon.2) 140 : audit [INF] from='client.? 172.21.15.175:0/3783807506' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-4-cache", "mode": "writeback"}]: dispatch 2024-03-26T03:49:00.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: audit 2024-03-26T03:49:00.373616+0000 mon.b (mon.1) 65 : audit [INF] from='client.? 172.21.15.175:0/2671281638' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi175-65296-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi175-65296-2"}]: dispatch 2024-03-26T03:49:00.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: cluster 2024-03-26T03:49:00.378301+0000 mon.a (mon.0) 1014 : cluster [DBG] osdmap e84: 8 total, 8 up, 8 in 2024-03-26T03:49:00.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: audit 2024-03-26T03:49:00.382908+0000 mon.a (mon.0) 1015 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi175-65296-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi175-65296-2"}]: dispatch 2024-03-26T03:49:00.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: audit 2024-03-26T03:49:00.384806+0000 mon.a (mon.0) 1016 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-4-cache", "mode": "writeback"}]: dispatch 2024-03-26T03:49:00.751 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: audit 2024-03-26T03:49:00.384845+0000 mon.c (mon.2) 141 : audit [INF] from='client.? 172.21.15.175:0/4207065844' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:00.751 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: audit 2024-03-26T03:49:00.388488+0000 mon.b (mon.1) 66 : audit [INF] from='client.? 172.21.15.175:0/2115269020' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:49:00.751 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: audit 2024-03-26T03:49:00.389172+0000 mon.b (mon.1) 67 : audit [INF] from='client.? 172.21.15.175:0/464642862' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:49:00.751 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: audit 2024-03-26T03:49:00.389215+0000 mon.a (mon.0) 1017 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:49:00.751 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: audit 2024-03-26T03:49:00.390713+0000 mon.a (mon.0) 1018 : audit [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi175-64668-7"}]: dispatch 2024-03-26T03:49:00.751 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: audit 2024-03-26T03:49:00.391039+0000 mon.c (mon.2) 142 : audit [INF] from='client.? 172.21.15.175:0/4087439416' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:49:00.751 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: audit 2024-03-26T03:49:00.392734+0000 mon.a (mon.0) 1019 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:49:00.751 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: audit 2024-03-26T03:49:00.393207+0000 mon.a (mon.0) 1020 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:00.751 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:00 smithi175 bash[21527]: audit 2024-03-26T03:49:00.393623+0000 mon.a (mon.0) 1021 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:49:01.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:01 smithi190 bash[23392]: cluster 2024-03-26T03:49:00.059016+0000 osd.0 (osd.0) 5 : cluster [DBG] 16.8 deep-scrub starts 2024-03-26T03:49:01.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:01 smithi190 bash[23392]: cluster 2024-03-26T03:49:00.065407+0000 osd.0 (osd.0) 6 : cluster [DBG] 16.8 deep-scrub ok 2024-03-26T03:49:01.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:01 smithi190 bash[23392]: audit 2024-03-26T03:49:00.462112+0000 mon.a (mon.0) 1022 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-03-26T03:49:01.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:01 smithi190 bash[23392]: cluster 2024-03-26T03:49:00.480964+0000 client.admin (client.?) 0 : cluster [INF] fourxx 2024-03-26T03:49:01.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:01 smithi190 bash[23392]: audit 2024-03-26T03:49:00.481482+0000 mon.a (mon.0) 1023 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-03-26T03:49:01.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:01 smithi190 bash[23392]: audit 2024-03-26T03:49:00.486354+0000 mon.b (mon.1) 68 : audit [INF] from='client.? 172.21.15.175:0/1244868480' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-03-26T03:49:01.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:01 smithi190 bash[23392]: cluster 2024-03-26T03:49:00.572421+0000 mgr.y (mgr.24431) 181 : cluster [DBG] pgmap v123: 852 pgs: 192 creating+peering, 580 active+clean, 80 unknown; 145 MiB data, 728 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 544 op/s 2024-03-26T03:49:01.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:01 smithi190 bash[23392]: audit 2024-03-26T03:49:00.674974+0000 mon.c (mon.2) 143 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:01.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:01 smithi190 bash[23392]: cluster 2024-03-26T03:49:01.359047+0000 mon.a (mon.0) 1024 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:49:01.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:01 smithi190 bash[23392]: audit 2024-03-26T03:49:01.362222+0000 mon.a (mon.0) 1025 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi175-65233-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi175-65233-2"}]': finished 2024-03-26T03:49:01.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:01 smithi190 bash[23392]: audit 2024-03-26T03:49:01.362363+0000 mon.a (mon.0) 1026 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-4-cache", "mode": "writeback"}]': finished 2024-03-26T03:49:01.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:01 smithi190 bash[23392]: audit 2024-03-26T03:49:01.362486+0000 mon.a (mon.0) 1027 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi175-64414-16"}]': finished 2024-03-26T03:49:01.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:01 smithi190 bash[23392]: audit 2024-03-26T03:49:01.362609+0000 mon.a (mon.0) 1028 : audit [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi175-64668-7"}]': finished 2024-03-26T03:49:01.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:01 smithi190 bash[23392]: audit 2024-03-26T03:49:01.362713+0000 mon.a (mon.0) 1029 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi175-64422-22"}]': finished 2024-03-26T03:49:01.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:01 smithi190 bash[23392]: audit 2024-03-26T03:49:01.362804+0000 mon.a (mon.0) 1030 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:01.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:01 smithi190 bash[23392]: audit 2024-03-26T03:49:01.362931+0000 mon.a (mon.0) 1031 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi175-64650-7"}]': finished 2024-03-26T03:49:01.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:01 smithi190 bash[23392]: audit 2024-03-26T03:49:01.371096+0000 mon.c (mon.2) 144 : audit [INF] from='client.? 172.21.15.175:0/4087439416' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:49:01.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:01 smithi190 bash[23392]: audit 2024-03-26T03:49:01.379102+0000 mon.b (mon.1) 69 : audit [INF] from='client.? 172.21.15.175:0/2115269020' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:49:01.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:01 smithi190 bash[23392]: audit 2024-03-26T03:49:01.379975+0000 mon.b (mon.1) 70 : audit [INF] from='client.? 172.21.15.175:0/464642862' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:49:01.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:01 smithi190 bash[23392]: cluster 2024-03-26T03:49:01.382649+0000 mon.a (mon.0) 1032 : cluster [DBG] osdmap e85: 8 total, 8 up, 8 in 2024-03-26T03:49:01.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:01 smithi190 bash[23392]: audit 2024-03-26T03:49:01.400358+0000 mon.a (mon.0) 1033 : audit [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi175-64668-7"}]: dispatch 2024-03-26T03:49:01.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:01 smithi190 bash[23392]: audit 2024-03-26T03:49:01.401675+0000 mon.a (mon.0) 1034 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:49:01.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:01 smithi190 bash[23392]: audit 2024-03-26T03:49:01.402120+0000 mon.a (mon.0) 1035 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:49:01.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:01 smithi190 bash[23392]: audit 2024-03-26T03:49:01.402685+0000 mon.a (mon.0) 1036 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:49:01.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:01 smithi190 bash[23392]: audit 2024-03-26T03:49:01.405252+0000 mon.a (mon.0) 1037 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi175-64538-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:01.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:01 smithi190 bash[23392]: audit 2024-03-26T03:49:01.405785+0000 mon.b (mon.1) 71 : audit [INF] from='client.? 172.21.15.175:0/60974697' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi175-64538-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:01.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[21527]: cluster 2024-03-26T03:49:00.059016+0000 osd.0 (osd.0) 5 : cluster [DBG] 16.8 deep-scrub starts 2024-03-26T03:49:01.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[21527]: cluster 2024-03-26T03:49:00.065407+0000 osd.0 (osd.0) 6 : cluster [DBG] 16.8 deep-scrub ok 2024-03-26T03:49:01.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[21527]: audit 2024-03-26T03:49:00.462112+0000 mon.a (mon.0) 1022 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-03-26T03:49:01.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[21527]: cluster 2024-03-26T03:49:00.480964+0000 client.admin (client.?) 0 : cluster [INF] fourxx 2024-03-26T03:49:01.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[21527]: audit 2024-03-26T03:49:00.481482+0000 mon.a (mon.0) 1023 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-03-26T03:49:01.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[21527]: audit 2024-03-26T03:49:00.486354+0000 mon.b (mon.1) 68 : audit [INF] from='client.? 172.21.15.175:0/1244868480' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-03-26T03:49:01.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[21527]: cluster 2024-03-26T03:49:00.572421+0000 mgr.y (mgr.24431) 181 : cluster [DBG] pgmap v123: 852 pgs: 192 creating+peering, 580 active+clean, 80 unknown; 145 MiB data, 728 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 544 op/s 2024-03-26T03:49:01.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[21527]: audit 2024-03-26T03:49:00.674974+0000 mon.c (mon.2) 143 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:01.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[21527]: cluster 2024-03-26T03:49:01.359047+0000 mon.a (mon.0) 1024 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:49:01.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[21527]: audit 2024-03-26T03:49:01.362222+0000 mon.a (mon.0) 1025 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi175-65233-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi175-65233-2"}]': finished 2024-03-26T03:49:01.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[21527]: audit 2024-03-26T03:49:01.362363+0000 mon.a (mon.0) 1026 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-4-cache", "mode": "writeback"}]': finished 2024-03-26T03:49:01.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[21527]: audit 2024-03-26T03:49:01.362486+0000 mon.a (mon.0) 1027 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi175-64414-16"}]': finished 2024-03-26T03:49:01.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[21527]: audit 2024-03-26T03:49:01.362609+0000 mon.a (mon.0) 1028 : audit [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi175-64668-7"}]': finished 2024-03-26T03:49:01.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[21527]: audit 2024-03-26T03:49:01.362713+0000 mon.a (mon.0) 1029 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi175-64422-22"}]': finished 2024-03-26T03:49:01.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[21527]: audit 2024-03-26T03:49:01.362804+0000 mon.a (mon.0) 1030 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:01.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[21527]: audit 2024-03-26T03:49:01.362931+0000 mon.a (mon.0) 1031 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi175-64650-7"}]': finished 2024-03-26T03:49:01.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[21527]: audit 2024-03-26T03:49:01.371096+0000 mon.c (mon.2) 144 : audit [INF] from='client.? 172.21.15.175:0/4087439416' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:49:01.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[29284]: cluster 2024-03-26T03:49:00.059016+0000 osd.0 (osd.0) 5 : cluster [DBG] 16.8 deep-scrub starts 2024-03-26T03:49:01.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[29284]: cluster 2024-03-26T03:49:00.065407+0000 osd.0 (osd.0) 6 : cluster [DBG] 16.8 deep-scrub ok 2024-03-26T03:49:01.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[29284]: audit 2024-03-26T03:49:00.462112+0000 mon.a (mon.0) 1022 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-03-26T03:49:01.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[29284]: cluster 2024-03-26T03:49:00.480964+0000 client.admin (client.?) 0 : cluster [INF] fourxx 2024-03-26T03:49:01.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[29284]: audit 2024-03-26T03:49:00.481482+0000 mon.a (mon.0) 1023 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-03-26T03:49:01.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[29284]: audit 2024-03-26T03:49:00.486354+0000 mon.b (mon.1) 68 : audit [INF] from='client.? 172.21.15.175:0/1244868480' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-03-26T03:49:01.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[29284]: cluster 2024-03-26T03:49:00.572421+0000 mgr.y (mgr.24431) 181 : cluster [DBG] pgmap v123: 852 pgs: 192 creating+peering, 580 active+clean, 80 unknown; 145 MiB data, 728 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 544 op/s 2024-03-26T03:49:01.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[29284]: audit 2024-03-26T03:49:00.674974+0000 mon.c (mon.2) 143 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:01.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[29284]: cluster 2024-03-26T03:49:01.359047+0000 mon.a (mon.0) 1024 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:49:01.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[29284]: audit 2024-03-26T03:49:01.362222+0000 mon.a (mon.0) 1025 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi175-65233-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi175-65233-2"}]': finished 2024-03-26T03:49:01.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[29284]: audit 2024-03-26T03:49:01.362363+0000 mon.a (mon.0) 1026 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-4-cache", "mode": "writeback"}]': finished 2024-03-26T03:49:01.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[29284]: audit 2024-03-26T03:49:01.362486+0000 mon.a (mon.0) 1027 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi175-64414-16"}]': finished 2024-03-26T03:49:01.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[29284]: audit 2024-03-26T03:49:01.362609+0000 mon.a (mon.0) 1028 : audit [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi175-64668-7"}]': finished 2024-03-26T03:49:01.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[29284]: audit 2024-03-26T03:49:01.362713+0000 mon.a (mon.0) 1029 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi175-64422-22"}]': finished 2024-03-26T03:49:01.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[29284]: audit 2024-03-26T03:49:01.362804+0000 mon.a (mon.0) 1030 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:01.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[29284]: audit 2024-03-26T03:49:01.362931+0000 mon.a (mon.0) 1031 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi175-64650-7"}]': finished 2024-03-26T03:49:01.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[29284]: audit 2024-03-26T03:49:01.371096+0000 mon.c (mon.2) 144 : audit [INF] from='client.? 172.21.15.175:0/4087439416' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:49:01.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[21527]: audit 2024-03-26T03:49:01.379102+0000 mon.b (mon.1) 69 : audit [INF] from='client.? 172.21.15.175:0/2115269020' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:49:01.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[21527]: audit 2024-03-26T03:49:01.379975+0000 mon.b (mon.1) 70 : audit [INF] from='client.? 172.21.15.175:0/464642862' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:49:01.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[21527]: cluster 2024-03-26T03:49:01.382649+0000 mon.a (mon.0) 1032 : cluster [DBG] osdmap e85: 8 total, 8 up, 8 in 2024-03-26T03:49:01.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[21527]: audit 2024-03-26T03:49:01.400358+0000 mon.a (mon.0) 1033 : audit [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi175-64668-7"}]: dispatch 2024-03-26T03:49:01.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[21527]: audit 2024-03-26T03:49:01.401675+0000 mon.a (mon.0) 1034 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:49:01.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[21527]: audit 2024-03-26T03:49:01.402120+0000 mon.a (mon.0) 1035 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:49:01.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[21527]: audit 2024-03-26T03:49:01.402685+0000 mon.a (mon.0) 1036 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:49:01.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[21527]: audit 2024-03-26T03:49:01.405252+0000 mon.a (mon.0) 1037 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi175-64538-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:01.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[21527]: audit 2024-03-26T03:49:01.405785+0000 mon.b (mon.1) 71 : audit [INF] from='client.? 172.21.15.175:0/60974697' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi175-64538-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:01.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[29284]: audit 2024-03-26T03:49:01.379102+0000 mon.b (mon.1) 69 : audit [INF] from='client.? 172.21.15.175:0/2115269020' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:49:01.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[29284]: audit 2024-03-26T03:49:01.379975+0000 mon.b (mon.1) 70 : audit [INF] from='client.? 172.21.15.175:0/464642862' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:49:01.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[29284]: cluster 2024-03-26T03:49:01.382649+0000 mon.a (mon.0) 1032 : cluster [DBG] osdmap e85: 8 total, 8 up, 8 in 2024-03-26T03:49:01.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[29284]: audit 2024-03-26T03:49:01.400358+0000 mon.a (mon.0) 1033 : audit [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi175-64668-7"}]: dispatch 2024-03-26T03:49:01.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[29284]: audit 2024-03-26T03:49:01.401675+0000 mon.a (mon.0) 1034 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi175-64414-16"}]: dispatch 2024-03-26T03:49:01.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[29284]: audit 2024-03-26T03:49:01.402120+0000 mon.a (mon.0) 1035 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:49:01.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[29284]: audit 2024-03-26T03:49:01.402685+0000 mon.a (mon.0) 1036 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:49:01.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[29284]: audit 2024-03-26T03:49:01.405252+0000 mon.a (mon.0) 1037 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi175-64538-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:01.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:01 smithi175 bash[29284]: audit 2024-03-26T03:49:01.405785+0000 mon.b (mon.1) 71 : audit [INF] from='client.? 172.21.15.175:0/60974697' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi175-64538-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:02.386 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:49:00.392734+0000 mon.a [INF] from='client.? ' entity='client.admin' c api_io: Running main() from gmock_main.cc 2024-03-26T03:49:02.386 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [==========] Running 24 tests from 2 test suites. 2024-03-26T03:49:02.386 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [----------] Global test environment set-up. 2024-03-26T03:49:02.386 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [----------] 14 tests from LibRadosIo 2024-03-26T03:49:02.386 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2024-03-26T03:49:02.386 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (434 ms) 2024-03-26T03:49:02.386 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2024-03-26T03:49:02.386 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2024-03-26T03:49:02.386 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2024-03-26T03:49:02.386 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: no timeout :/ 2024-03-26T03:49:02.387 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: no timeout :/ 2024-03-26T03:49:02.387 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: no timeout :/ 2024-03-26T03:49:02.387 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: no timeout :/ 2024-03-26T03:49:02.387 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: no timeout :/ 2024-03-26T03:49:02.387 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (59 ms) 2024-03-26T03:49:02.387 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2024-03-26T03:49:02.387 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (7 ms) 2024-03-26T03:49:02.387 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2024-03-26T03:49:02.387 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ OK ] LibRadosIo.Checksum (5 ms) 2024-03-26T03:49:02.387 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-03-26T03:49:02.387 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (6 ms) 2024-03-26T03:49:02.387 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2024-03-26T03:49:02.387 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (6 ms) 2024-03-26T03:49:02.387 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2024-03-26T03:49:02.387 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (7 ms) 2024-03-26T03:49:02.388 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ RUN ] LibRadosIo.ZeroLenZero 2024-03-26T03:49:02.388 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ OK ] LibRadosIo.ZeroLenZero (2 ms) 2024-03-26T03:49:02.388 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2024-03-26T03:49:02.388 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ OK ] LibRadosIo.TruncTest (5 ms) 2024-03-26T03:49:02.388 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2024-03-26T03:49:02.388 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (5 ms) 2024-03-26T03:49:02.388 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2024-03-26T03:49:02.388 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (6 ms) 2024-03-26T03:49:02.388 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2024-03-26T03:49:02.388 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ OK ] LibRadosIo.RmXattr (13 ms) 2024-03-26T03:49:02.388 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2024-03-26T03:49:02.388 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ OK ] LibRadosIo.XattrIter (7 ms) 2024-03-26T03:49:02.388 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [----------] 14 tests from LibRadosIo (562 ms total) 2024-03-26T03:49:02.388 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: 2024-03-26T03:49:02.389 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2024-03-26T03:49:02.389 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2024-03-26T03:49:02.389 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (1129 ms) 2024-03-26T03:49:02.389 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2024-03-26T03:49:02.389 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (8 ms) 2024-03-26T03:49:02.389 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-03-26T03:49:02.389 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (7 ms) 2024-03-26T03:49:02.389 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-03-26T03:49:02.389 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (6 ms) 2024-03-26T03:49:02.389 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2024-03-26T03:49:02.389 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (9 ms) 2024-03-26T03:49:02.389 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2024-03-26T03:49:02.389 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (6 ms) 2024-03-26T03:49:02.389 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2024-03-26T03:49:02.389 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (4 ms) 2024-03-26T03:49:02.390 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-03-26T03:49:02.390 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (3 ms) 2024-03-26T03:49:02.390 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2024-03-26T03:49:02.390 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (10 ms) 2024-03-26T03:49:02.390 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2024-03-26T03:49:02.390 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (6 ms) 2024-03-26T03:49:02.390 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [----------] 10 tests from LibRadosIoEC (1188 ms total) 2024-03-26T03:49:02.390 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: 2024-03-26T03:49:02.391 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [------ LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-03-26T03:49:02.391 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (4 ms) 2024-03-26T03:49:02.391 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-03-26T03:49:02.391 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (8 ms) 2024-03-26T03:49:02.391 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-03-26T03:49:02.391 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (5 ms) 2024-03-26T03:49:02.391 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-03-26T03:49:02.391 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (4 ms) 2024-03-26T03:49:02.391 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2024-03-26T03:49:02.391 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (5 ms) 2024-03-26T03:49:02.391 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2024-03-26T03:49:02.391 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (2 ms) 2024-03-26T03:49:02.391 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-03-26T03:49:02.392 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (4 ms) 2024-03-26T03:49:02.392 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2024-03-26T03:49:02.392 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (8 ms) 2024-03-26T03:49:02.392 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2024-03-26T03:49:02.392 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (5 ms) 2024-03-26T03:49:02.392 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2024-03-26T03:49:02.392 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (10 ms) 2024-03-26T03:49:02.392 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-03-26T03:49:02.392 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (5 ms) 2024-03-26T03:49:02.392 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-03-26T03:49:02.392 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (7 ms) 2024-03-26T03:49:02.392 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP (1235 ms total) 2024-03-26T03:49:02.392 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: 2024-03-26T03:49:02.392 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [----------] Global test environment tear-down 2024-03-26T03:49:02.392 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [==========] 37 tests from 2 test suites ran. (10178 ms total) 2024-03-26T03:49:02.393 INFO:tasks.workunit.client.0.smithi175.stdout: api_io_pp: [ PASSED ] 37 tests. 2024-03-26T03:49:02.393 INFO:tasks.workunit.client.0.smithi175.stdout:----] Global test environment tear-down 2024-03-26T03:49:02.393 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [==========] 24 tests from 2 test suites ran. (10185 ms total) 2024-03-26T03:49:02.393 INFO:tasks.workunit.client.0.smithi175.stdout: api_io: [ PASSED ] 24 tests. 2024-03-26T03:49:02.408 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat_pp: Running main() from gmock_main.cc 2024-03-26T03:49:02.409 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2024-03-26T03:49:02.409 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat_pp: [----------] Global test environment set-up. 2024-03-26T03:49:02.409 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2024-03-26T03:49:02.409 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat_pp: seed 64668 2024-03-26T03:49:02.409 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2024-03-26T03:49:02.409 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (445 ms) 2024-03-26T03:49:02.409 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-03-26T03:49:02.409 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (7 ms) 2024-03-26T03:49:02.409 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2024-03-26T03:49:02.409 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (0 ms) 2024-03-26T03:49:02.409 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2024-03-26T03:49:02.409 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (3 ms) 2024-03-26T03:49:02.409 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2024-03-26T03:49:02.409 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (60 ms) 2024-03-26T03:49:02.409 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (515 ms total) 2024-03-26T03:49:02.410 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat_pp: 2024-03-26T03:49:02.410 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2024-03-26T03:49:02.410 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2024-03-26T03:49:02.410 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (1096 ms) 2024-03-26T03:49:02.410 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2024-03-26T03:49:02.410 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (0 ms) 2024-03-26T03:49:02.410 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2024-03-26T03:49:02.410 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (3 ms) 2024-03-26T03:49:02.410 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2024-03-26T03:49:02.410 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (11 ms) 2024-03-26T03:49:02.410 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (1110 ms total) 2024-03-26T03:49:02.410 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat_pp: 2024-03-26T03:49:02.410 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat_pp: [----------] Global test environment tear-down 2024-03-26T03:49:02.410 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (10053 ms total) 2024-03-26T03:49:02.410 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat_pp: [ PASSED ] 9 tests. 2024-03-26T03:49:02.418 INFO:tasks.workunit.client.0.smithi175.stdout: list: Running main() from gmock_main.cc 2024-03-26T03:49:02.418 INFO:tasks.workunit.client.0.smithi175.stdout: list: [==========] Running 3 tests from 1 test suite. 2024-03-26T03:49:02.418 INFO:tasks.workunit.client.0.smithi175.stdout: list: [----------] Global test environment set-up. 2024-03-26T03:49:02.418 INFO:tasks.workunit.client.0.smithi175.stdout: list: [----------] 3 tests from NeoradosList 2024-03-26T03:49:02.418 INFO:tasks.workunit.client.0.smithi175.stdout: list: [ RUN ] NeoradosList.ListObjects 2024-03-26T03:49:02.418 INFO:tasks.workunit.client.0.smithi175.stdout: list: [ OK ] NeoradosList.ListObjects (2748 ms) 2024-03-26T03:49:02.418 INFO:tasks.workunit.client.0.smithi175.stdout: list: [ RUN ] NeoradosList.ListObjectsNS 2024-03-26T03:49:02.418 INFO:tasks.workunit.client.0.smithi175.stdout: list: [ OK ] NeoradosList.ListObjectsNS (3079 ms) 2024-03-26T03:49:02.418 INFO:tasks.workunit.client.0.smithi175.stdout: list: [ RUN ] NeoradosList.ListObjectsMany 2024-03-26T03:49:02.418 INFO:tasks.workunit.client.0.smithi175.stdout: list: [ OK ] NeoradosList.ListObjectsMany (4015 ms) 2024-03-26T03:49:02.418 INFO:tasks.workunit.client.0.smithi175.stdout: list: [----------] 3 tests from NeoradosList (9842 ms total) 2024-03-26T03:49:02.418 INFO:tasks.workunit.client.0.smithi175.stdout: list: 2024-03-26T03:49:02.419 INFO:tasks.workunit.client.0.smithi175.stdout: list: [----------] Global test environment tear-down 2024-03-26T03:49:02.419 INFO:tasks.workunit.client.0.smithi175.stdout: list: [==========] 3 tests from 1 test suite ran. (9842 ms total) 2024-03-26T03:49:02.419 INFO:tasks.workunit.client.0.smithi175.stdout: list: [ PASSED ] 3 tests. 2024-03-26T03:49:02.423 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat: Running main() from gmock_main.cc 2024-03-26T03:49:02.423 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat: [==========] Running 9 tests from 2 test suites. 2024-03-26T03:49:02.423 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat: [----------] Global test environment set-up. 2024-03-26T03:49:02.424 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat: [----------] 5 tests from LibRadosStat 2024-03-26T03:49:02.424 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2024-03-26T03:49:02.424 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat: [ OK ] LibRadosStat.Stat (459 ms) 2024-03-26T03:49:02.424 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat: [ RUN ] LibRadosStat.Stat2 2024-03-26T03:49:02.424 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat: [ OK ] LibRadosStat.Stat2 (17 ms) 2024-03-26T03:49:02.424 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2024-03-26T03:49:02.424 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat: [ OK ] LibRadosStat.StatNS (25 ms) 2024-03-26T03:49:02.424 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2024-03-26T03:49:02.424 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (1 ms) 2024-03-26T03:49:02.424 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2024-03-26T03:49:02.424 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (6 ms) 2024-03-26T03:49:02.424 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat: [----------] 5 tests from LibRadosStat (508 ms total) 2024-03-26T03:49:02.424 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat: 2024-03-26T03:49:02.424 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2024-03-26T03:49:02.424 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2024-03-26T03:49:02.424 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (1126 ms) 2024-03-26T03:49:02.425 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2024-03-26T03:49:02.425 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (12 ms) 2024-03-26T03:49:02.425 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2024-03-26T03:49:02.425 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (1 ms) 2024-03-26T03:49:02.425 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2024-03-26T03:49:02.425 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (6 ms) 2024-03-26T03:49:02.425 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (1145 ms total) 2024-03-26T03:49:02.425 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat: 2024-03-26T03:49:02.425 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat: [----------] Global test environment tear-down 2024-03-26T03:49:02.425 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat: [==========] 9 tests from 2 test suites ran. (10079 ms total) 2024-03-26T03:49:02.425 INFO:tasks.workunit.client.0.smithi175.stdout: api_stat: [ PASSED ] 9 tests. 2024-03-26T03:49:02.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:02 smithi190 bash[23392]: cluster 2024-03-26T03:49:01.107717+0000 osd.0 (osd.0) 7 : cluster [DBG] 16.0 deep-scrub starts 2024-03-26T03:49:02.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:02 smithi190 bash[23392]: cluster 2024-03-26T03:49:01.109384+0000 osd.0 (osd.0) 8 : cluster [DBG] 16.0 deep-scrub ok 2024-03-26T03:49:02.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:02 smithi190 bash[23392]: audit 2024-03-26T03:49:01.473822+0000 mon.a (mon.0) 1038 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-03-26T03:49:02.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:02 smithi190 bash[23392]: audit 2024-03-26T03:49:01.545995+0000 mon.c (mon.2) 145 : audit [INF] from='client.? 172.21.15.175:0/4207065844' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64770-6", "pg_num": 4}]: dispatch 2024-03-26T03:49:02.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:02 smithi190 bash[23392]: audit 2024-03-26T03:49:01.546417+0000 mon.a (mon.0) 1039 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64770-6", "pg_num": 4}]: dispatch 2024-03-26T03:49:02.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:02 smithi190 bash[23392]: audit 2024-03-26T03:49:01.550999+0000 mon.c (mon.2) 146 : audit [INF] from='client.? 172.21.15.175:0/3783807506' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-4"}]: dispatch 2024-03-26T03:49:02.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:02 smithi190 bash[23392]: audit 2024-03-26T03:49:01.552405+0000 mon.a (mon.0) 1040 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-4"}]: dispatch 2024-03-26T03:49:02.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:02 smithi190 bash[23392]: audit 2024-03-26T03:49:01.676941+0000 mon.c (mon.2) 147 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:02.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:02 smithi190 bash[23392]: audit 2024-03-26T03:49:02.372026+0000 mon.a (mon.0) 1041 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi175-65296-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi175-65296-2"}]': finished 2024-03-26T03:49:02.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:02 smithi190 bash[23392]: audit 2024-03-26T03:49:02.372385+0000 mon.a (mon.0) 1042 : audit [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi175-64668-7"}]': finished 2024-03-26T03:49:02.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:02 smithi190 bash[23392]: audit 2024-03-26T03:49:02.372542+0000 mon.a (mon.0) 1043 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi175-64414-16"}]': finished 2024-03-26T03:49:02.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:02 smithi190 bash[23392]: audit 2024-03-26T03:49:02.372681+0000 mon.a (mon.0) 1044 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi175-64650-7"}]': finished 2024-03-26T03:49:02.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:02 smithi190 bash[23392]: audit 2024-03-26T03:49:02.372872+0000 mon.a (mon.0) 1045 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi175-64422-22"}]': finished 2024-03-26T03:49:02.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:02 smithi190 bash[23392]: audit 2024-03-26T03:49:02.373089+0000 mon.a (mon.0) 1046 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi175-64538-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:02.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:02 smithi190 bash[23392]: audit 2024-03-26T03:49:02.373227+0000 mon.a (mon.0) 1047 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64770-6", "pg_num": 4}]': finished 2024-03-26T03:49:02.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:02 smithi190 bash[23392]: audit 2024-03-26T03:49:02.373406+0000 mon.a (mon.0) 1048 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-4"}]': finished 2024-03-26T03:49:02.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:02 smithi190 bash[23392]: cluster 2024-03-26T03:49:02.385638+0000 mon.a (mon.0) 1049 : cluster [DBG] osdmap e86: 8 total, 8 up, 8 in 2024-03-26T03:49:02.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:02 smithi190 bash[23392]: audit 2024-03-26T03:49:02.396128+0000 mon.c (mon.2) 148 : audit [INF] from='client.? 172.21.15.175:0/4207065844' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4", "tierpool": "test-rados-api-smithi175-64770-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:49:02.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:02 smithi190 bash[23392]: audit 2024-03-26T03:49:02.400017+0000 mon.a (mon.0) 1050 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4", "tierpool": "test-rados-api-smithi175-64770-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:49:02.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:02 smithi190 bash[23392]: audit 2024-03-26T03:49:02.401752+0000 mon.a (mon.0) 1051 : audit [INF] from='client.? 172.21.15.175:0/1781271865' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi175-64398-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:02.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:02 smithi190 bash[23392]: audit 2024-03-26T03:49:02.413111+0000 mon.c (mon.2) 149 : audit [INF] from='client.? 172.21.15.175:0/3783807506' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-4", "tierpool": "test-rados-api-smithi175-64594-4-cache"}]: dispatch 2024-03-26T03:49:02.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:02 smithi190 bash[23392]: audit 2024-03-26T03:49:02.418606+0000 mon.a (mon.0) 1052 : audit [INF] from='client.? 172.21.15.175:0/1144477741' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:02.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:02 smithi190 bash[23392]: audit 2024-03-26T03:49:02.419388+0000 mon.a (mon.0) 1053 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-4", "tierpool": "test-rados-api-smithi175-64594-4-cache"}]: dispatch 2024-03-26T03:49:02.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:02 smithi190 bash[23392]: audit 2024-03-26T03:49:02.426634+0000 mon.a (mon.0) 1054 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:02.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:02 smithi190 bash[23392]: audit 2024-03-26T03:49:02.430485+0000 mon.b (mon.1) 72 : audit [INF] from='client.? 172.21.15.175:0/3372382017' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:02.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[21527]: cluster 2024-03-26T03:49:01.107717+0000 osd.0 (osd.0) 7 : cluster [DBG] 16.0 deep-scrub starts 2024-03-26T03:49:02.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[21527]: cluster 2024-03-26T03:49:01.109384+0000 osd.0 (osd.0) 8 : cluster [DBG] 16.0 deep-scrub ok 2024-03-26T03:49:02.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[21527]: audit 2024-03-26T03:49:01.473822+0000 mon.a (mon.0) 1038 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-03-26T03:49:02.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[21527]: audit 2024-03-26T03:49:01.545995+0000 mon.c (mon.2) 145 : audit [INF] from='client.? 172.21.15.175:0/4207065844' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64770-6", "pg_num": 4}]: dispatch 2024-03-26T03:49:02.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[21527]: audit 2024-03-26T03:49:01.546417+0000 mon.a (mon.0) 1039 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64770-6", "pg_num": 4}]: dispatch 2024-03-26T03:49:02.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[21527]: audit 2024-03-26T03:49:01.550999+0000 mon.c (mon.2) 146 : audit [INF] from='client.? 172.21.15.175:0/3783807506' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-4"}]: dispatch 2024-03-26T03:49:02.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[21527]: audit 2024-03-26T03:49:01.552405+0000 mon.a (mon.0) 1040 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-4"}]: dispatch 2024-03-26T03:49:02.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[21527]: audit 2024-03-26T03:49:01.676941+0000 mon.c (mon.2) 147 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:02.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[21527]: audit 2024-03-26T03:49:02.372026+0000 mon.a (mon.0) 1041 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi175-65296-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi175-65296-2"}]': finished 2024-03-26T03:49:02.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[21527]: audit 2024-03-26T03:49:02.372385+0000 mon.a (mon.0) 1042 : audit [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi175-64668-7"}]': finished 2024-03-26T03:49:02.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[21527]: audit 2024-03-26T03:49:02.372542+0000 mon.a (mon.0) 1043 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi175-64414-16"}]': finished 2024-03-26T03:49:02.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[21527]: audit 2024-03-26T03:49:02.372681+0000 mon.a (mon.0) 1044 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi175-64650-7"}]': finished 2024-03-26T03:49:02.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[21527]: audit 2024-03-26T03:49:02.372872+0000 mon.a (mon.0) 1045 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi175-64422-22"}]': finished 2024-03-26T03:49:02.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[21527]: audit 2024-03-26T03:49:02.373089+0000 mon.a (mon.0) 1046 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi175-64538-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:02.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[21527]: audit 2024-03-26T03:49:02.373227+0000 mon.a (mon.0) 1047 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64770-6", "pg_num": 4}]': finished 2024-03-26T03:49:02.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[21527]: audit 2024-03-26T03:49:02.373406+0000 mon.a (mon.0) 1048 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-4"}]': finished 2024-03-26T03:49:02.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[21527]: cluster 2024-03-26T03:49:02.385638+0000 mon.a (mon.0) 1049 : cluster [DBG] osdmap e86: 8 total, 8 up, 8 in 2024-03-26T03:49:02.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[21527]: audit 2024-03-26T03:49:02.396128+0000 mon.c (mon.2) 148 : audit [INF] from='client.? 172.21.15.175:0/4207065844' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4", "tierpool": "test-rados-api-smithi175-64770-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:49:02.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[21527]: audit 2024-03-26T03:49:02.400017+0000 mon.a (mon.0) 1050 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4", "tierpool": "test-rados-api-smithi175-64770-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:49:02.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[21527]: audit 2024-03-26T03:49:02.401752+0000 mon.a (mon.0) 1051 : audit [INF] from='client.? 172.21.15.175:0/1781271865' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi175-64398-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:02.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[21527]: audit 2024-03-26T03:49:02.413111+0000 mon.c (mon.2) 149 : audit [INF] from='client.? 172.21.15.175:0/3783807506' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-4", "tierpool": "test-rados-api-smithi175-64594-4-cache"}]: dispatch 2024-03-26T03:49:02.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[21527]: audit 2024-03-26T03:49:02.418606+0000 mon.a (mon.0) 1052 : audit [INF] from='client.? 172.21.15.175:0/1144477741' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:02.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[21527]: audit 2024-03-26T03:49:02.419388+0000 mon.a (mon.0) 1053 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-4", "tierpool": "test-rados-api-smithi175-64594-4-cache"}]: dispatch 2024-03-26T03:49:02.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[21527]: audit 2024-03-26T03:49:02.426634+0000 mon.a (mon.0) 1054 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:02.749 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[21527]: audit 2024-03-26T03:49:02.430485+0000 mon.b (mon.1) 72 : audit [INF] from='client.? 172.21.15.175:0/3372382017' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:02.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[29284]: cluster 2024-03-26T03:49:01.107717+0000 osd.0 (osd.0) 7 : cluster [DBG] 16.0 deep-scrub starts 2024-03-26T03:49:02.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[29284]: cluster 2024-03-26T03:49:01.109384+0000 osd.0 (osd.0) 8 : cluster [DBG] 16.0 deep-scrub ok 2024-03-26T03:49:02.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[29284]: audit 2024-03-26T03:49:01.473822+0000 mon.a (mon.0) 1038 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-03-26T03:49:02.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[29284]: audit 2024-03-26T03:49:01.545995+0000 mon.c (mon.2) 145 : audit [INF] from='client.? 172.21.15.175:0/4207065844' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64770-6", "pg_num": 4}]: dispatch 2024-03-26T03:49:02.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[29284]: audit 2024-03-26T03:49:01.546417+0000 mon.a (mon.0) 1039 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64770-6", "pg_num": 4}]: dispatch 2024-03-26T03:49:02.749 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[29284]: audit 2024-03-26T03:49:01.550999+0000 mon.c (mon.2) 146 : audit [INF] from='client.? 172.21.15.175:0/3783807506' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-4"}]: dispatch 2024-03-26T03:49:02.750 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[29284]: audit 2024-03-26T03:49:01.552405+0000 mon.a (mon.0) 1040 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-4"}]: dispatch 2024-03-26T03:49:02.750 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[29284]: audit 2024-03-26T03:49:01.676941+0000 mon.c (mon.2) 147 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:02.750 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[29284]: audit 2024-03-26T03:49:02.372026+0000 mon.a (mon.0) 1041 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi175-65296-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi175-65296-2"}]': finished 2024-03-26T03:49:02.750 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[29284]: audit 2024-03-26T03:49:02.372385+0000 mon.a (mon.0) 1042 : audit [INF] from='client.? 172.21.15.175:0/1252491531' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi175-64668-7"}]': finished 2024-03-26T03:49:02.750 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[29284]: audit 2024-03-26T03:49:02.372542+0000 mon.a (mon.0) 1043 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi175-64414-16"}]': finished 2024-03-26T03:49:02.750 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[29284]: audit 2024-03-26T03:49:02.372681+0000 mon.a (mon.0) 1044 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi175-64650-7"}]': finished 2024-03-26T03:49:02.750 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[29284]: audit 2024-03-26T03:49:02.372872+0000 mon.a (mon.0) 1045 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi175-64422-22"}]': finished 2024-03-26T03:49:02.750 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[29284]: audit 2024-03-26T03:49:02.373089+0000 mon.a (mon.0) 1046 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi175-64538-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:02.750 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[29284]: audit 2024-03-26T03:49:02.373227+0000 mon.a (mon.0) 1047 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64770-6", "pg_num": 4}]': finished 2024-03-26T03:49:02.750 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[29284]: audit 2024-03-26T03:49:02.373406+0000 mon.a (mon.0) 1048 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-4"}]': finished 2024-03-26T03:49:02.750 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[29284]: cluster 2024-03-26T03:49:02.385638+0000 mon.a (mon.0) 1049 : cluster [DBG] osdmap e86: 8 total, 8 up, 8 in 2024-03-26T03:49:02.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[29284]: audit 2024-03-26T03:49:02.396128+0000 mon.c (mon.2) 148 : audit [INF] from='client.? 172.21.15.175:0/4207065844' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4", "tierpool": "test-rados-api-smithi175-64770-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:49:02.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[29284]: audit 2024-03-26T03:49:02.400017+0000 mon.a (mon.0) 1050 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4", "tierpool": "test-rados-api-smithi175-64770-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:49:02.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[29284]: audit 2024-03-26T03:49:02.401752+0000 mon.a (mon.0) 1051 : audit [INF] from='client.? 172.21.15.175:0/1781271865' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi175-64398-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:02.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[29284]: audit 2024-03-26T03:49:02.413111+0000 mon.c (mon.2) 149 : audit [INF] from='client.? 172.21.15.175:0/3783807506' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-4", "tierpool": "test-rados-api-smithi175-64594-4-cache"}]: dispatch 2024-03-26T03:49:02.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[29284]: audit 2024-03-26T03:49:02.418606+0000 mon.a (mon.0) 1052 : audit [INF] from='client.? 172.21.15.175:0/1144477741' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:02.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[29284]: audit 2024-03-26T03:49:02.419388+0000 mon.a (mon.0) 1053 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-4", "tierpool": "test-rados-api-smithi175-64594-4-cache"}]: dispatch 2024-03-26T03:49:02.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[29284]: audit 2024-03-26T03:49:02.426634+0000 mon.a (mon.0) 1054 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:02.751 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:02 smithi175 bash[29284]: audit 2024-03-26T03:49:02.430485+0000 mon.b (mon.1) 72 : audit [INF] from='client.? 172.21.15.175:0/3372382017' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:03.497 INFO:tasks.workunit.client.0.smithi175.stdout:md=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi175-64422-22"}]: dispatch 2024-03-26T03:49:03.497 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:49:00.393207+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:03.497 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: got: 2024-03-26T03:49:00.393623+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi175-64650-7"}]: dispatch 2024-03-26T03:49:03.497 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (8151 ms) 2024-03-26T03:49:03.497 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (11110 ms total) 2024-03-26T03:49:03.498 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: 2024-03-26T03:49:03.498 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: [----------] Global test environment tear-down 2024-03-26T03:49:03.498 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (11110 ms total) 2024-03-26T03:49:03.498 INFO:tasks.workunit.client.0.smithi175.stdout: api_cmd: [ PASSED ] 4 tests. 2024-03-26T03:49:03.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:03 smithi190 bash[23392]: audit 2024-03-26T03:49:02.550074+0000 mon.a (mon.0) 1055 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi175-64538-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:03.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:03 smithi190 bash[23392]: audit 2024-03-26T03:49:02.550713+0000 mon.b (mon.1) 73 : audit [DBG] from='client.? 172.21.15.175:0/60974697' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-03-26T03:49:03.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:03 smithi190 bash[23392]: audit 2024-03-26T03:49:02.555300+0000 mon.b (mon.1) 74 : audit [INF] from='client.? 172.21.15.175:0/60974697' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi175-64538-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:03.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:03 smithi190 bash[23392]: cluster 2024-03-26T03:49:02.573244+0000 mgr.y (mgr.24431) 182 : cluster [DBG] pgmap v126: 808 pgs: 548 active+clean, 260 unknown; 145 MiB data, 728 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 545 op/s 2024-03-26T03:49:03.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:03 smithi190 bash[23392]: audit 2024-03-26T03:49:02.678576+0000 mon.c (mon.2) 150 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:03.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:03 smithi190 bash[23392]: cluster 2024-03-26T03:49:03.375537+0000 mon.a (mon.0) 1056 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:49:03.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:03 smithi190 bash[23392]: audit 2024-03-26T03:49:03.380168+0000 mon.a (mon.0) 1057 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4", "tierpool": "test-rados-api-smithi175-64770-6", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:49:03.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:03 smithi190 bash[23392]: audit 2024-03-26T03:49:03.380472+0000 mon.a (mon.0) 1058 : audit [INF] from='client.? 172.21.15.175:0/1781271865' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi175-64398-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:03.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:03 smithi190 bash[23392]: audit 2024-03-26T03:49:03.380674+0000 mon.a (mon.0) 1059 : audit [INF] from='client.? 172.21.15.175:0/1144477741' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:03.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:03 smithi190 bash[23392]: audit 2024-03-26T03:49:03.380876+0000 mon.a (mon.0) 1060 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-4", "tierpool": "test-rados-api-smithi175-64594-4-cache"}]': finished 2024-03-26T03:49:03.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:03 smithi190 bash[23392]: audit 2024-03-26T03:49:03.381109+0000 mon.a (mon.0) 1061 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:03.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:03 smithi190 bash[23392]: audit 2024-03-26T03:49:03.381325+0000 mon.a (mon.0) 1062 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi175-64538-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:03.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:03 smithi190 bash[23392]: cluster 2024-03-26T03:49:03.392606+0000 mon.a (mon.0) 1063 : cluster [DBG] osdmap e87: 8 total, 8 up, 8 in 2024-03-26T03:49:03.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:03 smithi190 bash[23392]: audit 2024-03-26T03:49:03.395568+0000 mon.c (mon.2) 151 : audit [INF] from='client.? 172.21.15.175:0/4207065844' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4", "overlaypool": "test-rados-api-smithi175-64770-6"}]: dispatch 2024-03-26T03:49:03.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:03 smithi190 bash[23392]: audit 2024-03-26T03:49:03.402937+0000 mon.b (mon.1) 75 : audit [INF] from='client.? 172.21.15.175:0/60974697' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi175-64538-1","app": "app2"}]: dispatch 2024-03-26T03:49:03.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:03 smithi190 bash[23392]: audit 2024-03-26T03:49:03.403722+0000 mon.b (mon.1) 76 : audit [INF] from='client.? 172.21.15.175:0/1737838842' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi175-65233-2"}]: dispatch 2024-03-26T03:49:03.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:03 smithi190 bash[23392]: audit 2024-03-26T03:49:03.405218+0000 mon.a (mon.0) 1064 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4", "overlaypool": "test-rados-api-smithi175-64770-6"}]: dispatch 2024-03-26T03:49:03.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:03 smithi190 bash[23392]: audit 2024-03-26T03:49:03.406156+0000 mon.a (mon.0) 1065 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi175-65233-2"}]: dispatch 2024-03-26T03:49:03.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:03 smithi190 bash[23392]: audit 2024-03-26T03:49:03.407919+0000 mon.b (mon.1) 77 : audit [INF] from='client.? 172.21.15.175:0/60974697' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi175-64538-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:03.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:03 smithi190 bash[23392]: audit 2024-03-26T03:49:03.408760+0000 mon.a (mon.0) 1066 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi175-64538-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:03.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[21527]: audit 2024-03-26T03:49:02.550074+0000 mon.a (mon.0) 1055 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi175-64538-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:03.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[21527]: audit 2024-03-26T03:49:02.550713+0000 mon.b (mon.1) 73 : audit [DBG] from='client.? 172.21.15.175:0/60974697' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-03-26T03:49:03.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[21527]: audit 2024-03-26T03:49:02.555300+0000 mon.b (mon.1) 74 : audit [INF] from='client.? 172.21.15.175:0/60974697' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi175-64538-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:03.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[21527]: cluster 2024-03-26T03:49:02.573244+0000 mgr.y (mgr.24431) 182 : cluster [DBG] pgmap v126: 808 pgs: 548 active+clean, 260 unknown; 145 MiB data, 728 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 545 op/s 2024-03-26T03:49:03.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[21527]: audit 2024-03-26T03:49:02.678576+0000 mon.c (mon.2) 150 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:03.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[21527]: cluster 2024-03-26T03:49:03.375537+0000 mon.a (mon.0) 1056 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:49:03.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[21527]: audit 2024-03-26T03:49:03.380168+0000 mon.a (mon.0) 1057 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4", "tierpool": "test-rados-api-smithi175-64770-6", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:49:03.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[21527]: audit 2024-03-26T03:49:03.380472+0000 mon.a (mon.0) 1058 : audit [INF] from='client.? 172.21.15.175:0/1781271865' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi175-64398-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:03.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[21527]: audit 2024-03-26T03:49:03.380674+0000 mon.a (mon.0) 1059 : audit [INF] from='client.? 172.21.15.175:0/1144477741' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:03.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[21527]: audit 2024-03-26T03:49:03.380876+0000 mon.a (mon.0) 1060 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-4", "tierpool": "test-rados-api-smithi175-64594-4-cache"}]': finished 2024-03-26T03:49:03.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[21527]: audit 2024-03-26T03:49:03.381109+0000 mon.a (mon.0) 1061 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:03.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[21527]: audit 2024-03-26T03:49:03.381325+0000 mon.a (mon.0) 1062 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi175-64538-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:03.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[21527]: cluster 2024-03-26T03:49:03.392606+0000 mon.a (mon.0) 1063 : cluster [DBG] osdmap e87: 8 total, 8 up, 8 in 2024-03-26T03:49:03.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[21527]: audit 2024-03-26T03:49:03.395568+0000 mon.c (mon.2) 151 : audit [INF] from='client.? 172.21.15.175:0/4207065844' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4", "overlaypool": "test-rados-api-smithi175-64770-6"}]: dispatch 2024-03-26T03:49:03.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[29284]: audit 2024-03-26T03:49:02.550074+0000 mon.a (mon.0) 1055 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi175-64538-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:03.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[29284]: audit 2024-03-26T03:49:02.550713+0000 mon.b (mon.1) 73 : audit [DBG] from='client.? 172.21.15.175:0/60974697' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-03-26T03:49:03.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[29284]: audit 2024-03-26T03:49:02.555300+0000 mon.b (mon.1) 74 : audit [INF] from='client.? 172.21.15.175:0/60974697' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi175-64538-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:03.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[29284]: cluster 2024-03-26T03:49:02.573244+0000 mgr.y (mgr.24431) 182 : cluster [DBG] pgmap v126: 808 pgs: 548 active+clean, 260 unknown; 145 MiB data, 728 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 545 op/s 2024-03-26T03:49:03.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[29284]: audit 2024-03-26T03:49:02.678576+0000 mon.c (mon.2) 150 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:03.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[29284]: cluster 2024-03-26T03:49:03.375537+0000 mon.a (mon.0) 1056 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:49:03.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[29284]: audit 2024-03-26T03:49:03.380168+0000 mon.a (mon.0) 1057 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4", "tierpool": "test-rados-api-smithi175-64770-6", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:49:03.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[29284]: audit 2024-03-26T03:49:03.380472+0000 mon.a (mon.0) 1058 : audit [INF] from='client.? 172.21.15.175:0/1781271865' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi175-64398-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:03.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[29284]: audit 2024-03-26T03:49:03.380674+0000 mon.a (mon.0) 1059 : audit [INF] from='client.? 172.21.15.175:0/1144477741' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:03.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[29284]: audit 2024-03-26T03:49:03.380876+0000 mon.a (mon.0) 1060 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-4", "tierpool": "test-rados-api-smithi175-64594-4-cache"}]': finished 2024-03-26T03:49:03.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[29284]: audit 2024-03-26T03:49:03.381109+0000 mon.a (mon.0) 1061 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:03.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[29284]: audit 2024-03-26T03:49:03.381325+0000 mon.a (mon.0) 1062 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi175-64538-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:03.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[29284]: cluster 2024-03-26T03:49:03.392606+0000 mon.a (mon.0) 1063 : cluster [DBG] osdmap e87: 8 total, 8 up, 8 in 2024-03-26T03:49:03.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[29284]: audit 2024-03-26T03:49:03.395568+0000 mon.c (mon.2) 151 : audit [INF] from='client.? 172.21.15.175:0/4207065844' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4", "overlaypool": "test-rados-api-smithi175-64770-6"}]: dispatch 2024-03-26T03:49:03.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[29284]: audit 2024-03-26T03:49:03.402937+0000 mon.b (mon.1) 75 : audit [INF] from='client.? 172.21.15.175:0/60974697' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi175-64538-1","app": "app2"}]: dispatch 2024-03-26T03:49:03.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[29284]: audit 2024-03-26T03:49:03.403722+0000 mon.b (mon.1) 76 : audit [INF] from='client.? 172.21.15.175:0/1737838842' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi175-65233-2"}]: dispatch 2024-03-26T03:49:03.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[29284]: audit 2024-03-26T03:49:03.405218+0000 mon.a (mon.0) 1064 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4", "overlaypool": "test-rados-api-smithi175-64770-6"}]: dispatch 2024-03-26T03:49:03.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[29284]: audit 2024-03-26T03:49:03.406156+0000 mon.a (mon.0) 1065 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi175-65233-2"}]: dispatch 2024-03-26T03:49:03.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[29284]: audit 2024-03-26T03:49:03.407919+0000 mon.b (mon.1) 77 : audit [INF] from='client.? 172.21.15.175:0/60974697' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi175-64538-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:03.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[29284]: audit 2024-03-26T03:49:03.408760+0000 mon.a (mon.0) 1066 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi175-64538-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:03.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[21527]: audit 2024-03-26T03:49:03.402937+0000 mon.b (mon.1) 75 : audit [INF] from='client.? 172.21.15.175:0/60974697' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi175-64538-1","app": "app2"}]: dispatch 2024-03-26T03:49:03.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[21527]: audit 2024-03-26T03:49:03.403722+0000 mon.b (mon.1) 76 : audit [INF] from='client.? 172.21.15.175:0/1737838842' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi175-65233-2"}]: dispatch 2024-03-26T03:49:03.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[21527]: audit 2024-03-26T03:49:03.405218+0000 mon.a (mon.0) 1064 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4", "overlaypool": "test-rados-api-smithi175-64770-6"}]: dispatch 2024-03-26T03:49:03.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[21527]: audit 2024-03-26T03:49:03.406156+0000 mon.a (mon.0) 1065 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi175-65233-2"}]: dispatch 2024-03-26T03:49:03.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[21527]: audit 2024-03-26T03:49:03.407919+0000 mon.b (mon.1) 77 : audit [INF] from='client.? 172.21.15.175:0/60974697' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi175-64538-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:03.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:03 smithi175 bash[21527]: audit 2024-03-26T03:49:03.408760+0000 mon.a (mon.0) 1066 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi175-64538-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:04.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:04 smithi190 bash[23392]: audit 2024-03-26T03:49:03.679941+0000 mon.c (mon.2) 152 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:04.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:04 smithi190 bash[23392]: audit 2024-03-26T03:49:04.003446+0000 mon.a (mon.0) 1067 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:49:04.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:04 smithi190 bash[23392]: audit 2024-03-26T03:49:04.007471+0000 mon.c (mon.2) 153 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:49:04.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:04 smithi190 bash[23392]: audit 2024-03-26T03:49:04.385008+0000 mon.a (mon.0) 1068 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4", "overlaypool": "test-rados-api-smithi175-64770-6"}]': finished 2024-03-26T03:49:04.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:04 smithi190 bash[23392]: audit 2024-03-26T03:49:04.385175+0000 mon.a (mon.0) 1069 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi175-65233-2"}]': finished 2024-03-26T03:49:04.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:04 smithi190 bash[23392]: audit 2024-03-26T03:49:04.385376+0000 mon.a (mon.0) 1070 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi175-64538-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:04.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:04 smithi190 bash[23392]: cluster 2024-03-26T03:49:04.394891+0000 mon.a (mon.0) 1071 : cluster [DBG] osdmap e88: 8 total, 8 up, 8 in 2024-03-26T03:49:04.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:04 smithi190 bash[23392]: audit 2024-03-26T03:49:04.402311+0000 mon.b (mon.1) 78 : audit [INF] from='client.? 172.21.15.175:0/1737838842' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi175-65233-2"}]: dispatch 2024-03-26T03:49:04.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:04 smithi190 bash[23392]: audit 2024-03-26T03:49:04.403261+0000 mon.b (mon.1) 79 : audit [INF] from='client.? 172.21.15.175:0/542670412' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:49:04.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:04 smithi190 bash[23392]: audit 2024-03-26T03:49:04.405290+0000 mon.b (mon.1) 80 : audit [INF] from='client.? 172.21.15.175:0/60974697' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi175-64538-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-03-26T03:49:04.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:04 smithi190 bash[23392]: audit 2024-03-26T03:49:04.406653+0000 mon.b (mon.1) 81 : audit [INF] from='client.? 172.21.15.175:0/60974697' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi175-64538-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-26T03:49:04.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:04 smithi190 bash[23392]: audit 2024-03-26T03:49:04.407452+0000 mon.b (mon.1) 82 : audit [INF] from='client.? 172.21.15.175:0/2671281638' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi175-65296-2"}]: dispatch 2024-03-26T03:49:04.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:04 smithi190 bash[23392]: audit 2024-03-26T03:49:04.412345+0000 mon.a (mon.0) 1072 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi175-65233-2"}]: dispatch 2024-03-26T03:49:04.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:04 smithi190 bash[23392]: audit 2024-03-26T03:49:04.412804+0000 mon.a (mon.0) 1073 : audit [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi175-64470-10"}]: dispatch 2024-03-26T03:49:04.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:04 smithi190 bash[23392]: audit 2024-03-26T03:49:04.412883+0000 mon.c (mon.2) 154 : audit [INF] from='client.? 172.21.15.175:0/4207065844' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64770-6", "mode": "writeback"}]: dispatch 2024-03-26T03:49:04.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:04 smithi190 bash[23392]: audit 2024-03-26T03:49:04.413099+0000 mon.a (mon.0) 1074 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:49:04.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:04 smithi190 bash[23392]: audit 2024-03-26T03:49:04.413506+0000 mon.a (mon.0) 1075 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi175-64538-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-26T03:49:04.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:04 smithi190 bash[23392]: audit 2024-03-26T03:49:04.414707+0000 mon.a (mon.0) 1076 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi175-65296-2"}]: dispatch 2024-03-26T03:49:04.996 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:04 smithi190 bash[23392]: audit 2024-03-26T03:49:04.415016+0000 mon.a (mon.0) 1077 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64770-6", "mode": "writeback"}]: dispatch 2024-03-26T03:49:04.996 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:04 smithi190 bash[23392]: audit 2024-03-26T03:49:04.424094+0000 mon.a (mon.0) 1078 : audit [INF] from='client.? 172.21.15.175:0/175418209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi175-64639-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:04.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[21527]: audit 2024-03-26T03:49:03.679941+0000 mon.c (mon.2) 152 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:04.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[21527]: audit 2024-03-26T03:49:04.003446+0000 mon.a (mon.0) 1067 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:49:04.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[21527]: audit 2024-03-26T03:49:04.007471+0000 mon.c (mon.2) 153 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:49:04.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[21527]: audit 2024-03-26T03:49:04.385008+0000 mon.a (mon.0) 1068 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4", "overlaypool": "test-rados-api-smithi175-64770-6"}]': finished 2024-03-26T03:49:04.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[21527]: audit 2024-03-26T03:49:04.385175+0000 mon.a (mon.0) 1069 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi175-65233-2"}]': finished 2024-03-26T03:49:04.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[21527]: audit 2024-03-26T03:49:04.385376+0000 mon.a (mon.0) 1070 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi175-64538-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:04.997 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[21527]: cluster 2024-03-26T03:49:04.394891+0000 mon.a (mon.0) 1071 : cluster [DBG] osdmap e88: 8 total, 8 up, 8 in 2024-03-26T03:49:04.997 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[21527]: audit 2024-03-26T03:49:04.402311+0000 mon.b (mon.1) 78 : audit [INF] from='client.? 172.21.15.175:0/1737838842' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi175-65233-2"}]: dispatch 2024-03-26T03:49:04.997 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[21527]: audit 2024-03-26T03:49:04.403261+0000 mon.b (mon.1) 79 : audit [INF] from='client.? 172.21.15.175:0/542670412' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:49:04.997 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[21527]: audit 2024-03-26T03:49:04.405290+0000 mon.b (mon.1) 80 : audit [INF] from='client.? 172.21.15.175:0/60974697' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi175-64538-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-03-26T03:49:04.997 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[21527]: audit 2024-03-26T03:49:04.406653+0000 mon.b (mon.1) 81 : audit [INF] from='client.? 172.21.15.175:0/60974697' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi175-64538-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-26T03:49:04.997 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[21527]: audit 2024-03-26T03:49:04.407452+0000 mon.b (mon.1) 82 : audit [INF] from='client.? 172.21.15.175:0/2671281638' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi175-65296-2"}]: dispatch 2024-03-26T03:49:04.997 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[21527]: audit 2024-03-26T03:49:04.412345+0000 mon.a (mon.0) 1072 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi175-65233-2"}]: dispatch 2024-03-26T03:49:04.997 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[21527]: audit 2024-03-26T03:49:04.412804+0000 mon.a (mon.0) 1073 : audit [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi175-64470-10"}]: dispatch 2024-03-26T03:49:04.997 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[21527]: audit 2024-03-26T03:49:04.412883+0000 mon.c (mon.2) 154 : audit [INF] from='client.? 172.21.15.175:0/4207065844' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64770-6", "mode": "writeback"}]: dispatch 2024-03-26T03:49:04.997 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[29284]: audit 2024-03-26T03:49:03.679941+0000 mon.c (mon.2) 152 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:04.998 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[29284]: audit 2024-03-26T03:49:04.003446+0000 mon.a (mon.0) 1067 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:49:04.998 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[29284]: audit 2024-03-26T03:49:04.007471+0000 mon.c (mon.2) 153 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:49:04.998 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[29284]: audit 2024-03-26T03:49:04.385008+0000 mon.a (mon.0) 1068 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4", "overlaypool": "test-rados-api-smithi175-64770-6"}]': finished 2024-03-26T03:49:04.998 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[29284]: audit 2024-03-26T03:49:04.385175+0000 mon.a (mon.0) 1069 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi175-65233-2"}]': finished 2024-03-26T03:49:04.998 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[29284]: audit 2024-03-26T03:49:04.385376+0000 mon.a (mon.0) 1070 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi175-64538-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:04.998 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[29284]: cluster 2024-03-26T03:49:04.394891+0000 mon.a (mon.0) 1071 : cluster [DBG] osdmap e88: 8 total, 8 up, 8 in 2024-03-26T03:49:04.998 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[29284]: audit 2024-03-26T03:49:04.402311+0000 mon.b (mon.1) 78 : audit [INF] from='client.? 172.21.15.175:0/1737838842' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi175-65233-2"}]: dispatch 2024-03-26T03:49:04.998 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[29284]: audit 2024-03-26T03:49:04.403261+0000 mon.b (mon.1) 79 : audit [INF] from='client.? 172.21.15.175:0/542670412' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:49:04.998 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[29284]: audit 2024-03-26T03:49:04.405290+0000 mon.b (mon.1) 80 : audit [INF] from='client.? 172.21.15.175:0/60974697' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi175-64538-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-03-26T03:49:04.998 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[29284]: audit 2024-03-26T03:49:04.406653+0000 mon.b (mon.1) 81 : audit [INF] from='client.? 172.21.15.175:0/60974697' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi175-64538-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-26T03:49:04.998 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[29284]: audit 2024-03-26T03:49:04.407452+0000 mon.b (mon.1) 82 : audit [INF] from='client.? 172.21.15.175:0/2671281638' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi175-65296-2"}]: dispatch 2024-03-26T03:49:04.998 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[29284]: audit 2024-03-26T03:49:04.412345+0000 mon.a (mon.0) 1072 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi175-65233-2"}]: dispatch 2024-03-26T03:49:04.998 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[29284]: audit 2024-03-26T03:49:04.412804+0000 mon.a (mon.0) 1073 : audit [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi175-64470-10"}]: dispatch 2024-03-26T03:49:04.999 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[29284]: audit 2024-03-26T03:49:04.412883+0000 mon.c (mon.2) 154 : audit [INF] from='client.? 172.21.15.175:0/4207065844' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64770-6", "mode": "writeback"}]: dispatch 2024-03-26T03:49:04.999 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[21527]: audit 2024-03-26T03:49:04.413099+0000 mon.a (mon.0) 1074 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:49:04.999 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[21527]: audit 2024-03-26T03:49:04.413506+0000 mon.a (mon.0) 1075 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi175-64538-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-26T03:49:04.999 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[21527]: audit 2024-03-26T03:49:04.414707+0000 mon.a (mon.0) 1076 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi175-65296-2"}]: dispatch 2024-03-26T03:49:04.999 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[21527]: audit 2024-03-26T03:49:04.415016+0000 mon.a (mon.0) 1077 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64770-6", "mode": "writeback"}]: dispatch 2024-03-26T03:49:04.999 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[21527]: audit 2024-03-26T03:49:04.424094+0000 mon.a (mon.0) 1078 : audit [INF] from='client.? 172.21.15.175:0/175418209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi175-64639-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:04.999 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[29284]: audit 2024-03-26T03:49:04.413099+0000 mon.a (mon.0) 1074 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:49:05.000 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[29284]: audit 2024-03-26T03:49:04.413506+0000 mon.a (mon.0) 1075 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi175-64538-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-26T03:49:05.000 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[29284]: audit 2024-03-26T03:49:04.414707+0000 mon.a (mon.0) 1076 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi175-65296-2"}]: dispatch 2024-03-26T03:49:05.000 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[29284]: audit 2024-03-26T03:49:04.415016+0000 mon.a (mon.0) 1077 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64770-6", "mode": "writeback"}]: dispatch 2024-03-26T03:49:05.000 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:04 smithi175 bash[29284]: audit 2024-03-26T03:49:04.424094+0000 mon.a (mon.0) 1078 : audit [INF] from='client.? 172.21.15.175:0/175418209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi175-64639-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:05.409 INFO:tasks.workunit.client.0.smithi175.stdout: ec_io: Running main() from gmock_main.cc 2024-03-26T03:49:05.409 INFO:tasks.workunit.client.0.smithi175.stdout: ec_io: [==========] Running 2 tests from 1 test suite. 2024-03-26T03:49:05.409 INFO:tasks.workunit.client.0.smithi175.stdout: ec_io: [----------] Global test environment set-up. 2024-03-26T03:49:05.409 INFO:tasks.workunit.client.0.smithi175.stdout: ec_io: [----------] 2 tests from NeoRadosECIo 2024-03-26T03:49:05.409 INFO:tasks.workunit.client.0.smithi175.stdout: ec_io: [ RUN ] NeoRadosECIo.SimpleWrite 2024-03-26T03:49:05.409 INFO:tasks.workunit.client.0.smithi175.stdout: ec_io: [ OK ] NeoRadosECIo.SimpleWrite (5777 ms) 2024-03-26T03:49:05.409 INFO:tasks.workunit.client.0.smithi175.stdout: ec_io: [ RUN ] NeoRadosECIo.ReadOp 2024-03-26T03:49:05.409 INFO:tasks.workunit.client.0.smithi175.stdout: ec_io: [ OK ] NeoRadosECIo.ReadOp (7067 ms) 2024-03-26T03:49:05.409 INFO:tasks.workunit.client.0.smithi175.stdout: ec_io: [----------] 2 tests from NeoRadosECIo (12844 ms total) 2024-03-26T03:49:05.409 INFO:tasks.workunit.client.0.smithi175.stdout: ec_io: 2024-03-26T03:49:05.410 INFO:tasks.workunit.client.0.smithi175.stdout: ec_io: [----------] Global test environment tear-down 2024-03-26T03:49:05.410 INFO:tasks.workunit.client.0.smithi175.stdout: ec_io: [==========] 2 tests from 1 test suite ran. (12844 ms total) 2024-03-26T03:49:05.410 INFO:tasks.workunit.client.0.smithi175.stdout: ec_io: [ PASSED ] 2 tests. 2024-03-26T03:49:05.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:05 smithi190 bash[23392]: audit 2024-03-26T03:49:04.517067+0000 mon.a (mon.0) 1079 : audit [INF] from='client.? 172.21.15.175:0/4199038823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi175-64735-12"}]: dispatch 2024-03-26T03:49:05.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:05 smithi190 bash[23392]: audit 2024-03-26T03:49:04.531373+0000 mon.a (mon.0) 1080 : audit [INF] from='client.? 172.21.15.175:0/4199038823' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi175-64735-12"}]: dispatch 2024-03-26T03:49:05.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:05 smithi190 bash[23392]: audit 2024-03-26T03:49:04.532494+0000 mon.a (mon.0) 1081 : audit [INF] from='client.? 172.21.15.175:0/4199038823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi175-64735-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:49:05.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:05 smithi190 bash[23392]: cluster 2024-03-26T03:49:04.575053+0000 mgr.y (mgr.24431) 183 : cluster [DBG] pgmap v129: 616 pgs: 20 creating+peering, 1 creating+activating, 511 active+clean, 84 unknown; 145 MiB data, 755 MiB used, 714 GiB / 715 GiB avail 2024-03-26T03:49:05.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:05 smithi190 bash[23392]: audit 2024-03-26T03:49:04.681738+0000 mon.c (mon.2) 155 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:05.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:05 smithi190 bash[23392]: cluster 2024-03-26T03:49:05.390400+0000 mon.a (mon.0) 1082 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:49:05.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:05 smithi190 bash[23392]: audit 2024-03-26T03:49:05.394502+0000 mon.a (mon.0) 1083 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi175-65233-2"}]': finished 2024-03-26T03:49:05.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:05 smithi190 bash[23392]: audit 2024-03-26T03:49:05.394756+0000 mon.a (mon.0) 1084 : audit [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi175-64470-10"}]': finished 2024-03-26T03:49:05.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:05 smithi190 bash[23392]: audit 2024-03-26T03:49:05.394976+0000 mon.a (mon.0) 1085 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi175-64501-10"}]': finished 2024-03-26T03:49:05.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:05 smithi190 bash[23392]: audit 2024-03-26T03:49:05.395204+0000 mon.a (mon.0) 1086 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi175-64538-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-03-26T03:49:05.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:05 smithi190 bash[23392]: audit 2024-03-26T03:49:05.395487+0000 mon.a (mon.0) 1087 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi175-65296-2"}]': finished 2024-03-26T03:49:05.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:05 smithi190 bash[23392]: audit 2024-03-26T03:49:05.395697+0000 mon.a (mon.0) 1088 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64770-6", "mode": "writeback"}]': finished 2024-03-26T03:49:05.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:05 smithi190 bash[23392]: audit 2024-03-26T03:49:05.395935+0000 mon.a (mon.0) 1089 : audit [INF] from='client.? 172.21.15.175:0/175418209' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi175-64639-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:05.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:05 smithi190 bash[23392]: audit 2024-03-26T03:49:05.396172+0000 mon.a (mon.0) 1090 : audit [INF] from='client.? 172.21.15.175:0/4199038823' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi175-64735-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:49:05.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:05 smithi190 bash[23392]: audit 2024-03-26T03:49:05.409648+0000 mon.b (mon.1) 83 : audit [INF] from='client.? 172.21.15.175:0/542670412' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:49:05.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:05 smithi190 bash[23392]: audit 2024-03-26T03:49:05.410158+0000 mon.b (mon.1) 84 : audit [INF] from='client.? 172.21.15.175:0/2671281638' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi175-65296-2"}]: dispatch 2024-03-26T03:49:05.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:05 smithi190 bash[23392]: cluster 2024-03-26T03:49:05.418911+0000 mon.a (mon.0) 1091 : cluster [DBG] osdmap e89: 8 total, 8 up, 8 in 2024-03-26T03:49:05.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:05 smithi190 bash[23392]: audit 2024-03-26T03:49:05.419572+0000 mon.c (mon.2) 156 : audit [INF] from='client.? 172.21.15.175:0/3873547907' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi175-64398-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:05.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:05 smithi190 bash[23392]: audit 2024-03-26T03:49:05.427945+0000 mon.b (mon.1) 85 : audit [INF] from='client.? 172.21.15.175:0/60974697' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi175-64538-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-26T03:49:05.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:05 smithi190 bash[23392]: audit 2024-03-26T03:49:05.437112+0000 mon.a (mon.0) 1092 : audit [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi175-64470-10"}]: dispatch 2024-03-26T03:49:05.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:05 smithi190 bash[23392]: audit 2024-03-26T03:49:05.440399+0000 mon.a (mon.0) 1093 : audit [INF] from='client.? 172.21.15.175:0/4199038823' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi175-64735-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi175-64735-12"}]: dispatch 2024-03-26T03:49:05.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:05 smithi190 bash[23392]: audit 2024-03-26T03:49:05.440871+0000 mon.a (mon.0) 1094 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:49:05.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:05 smithi190 bash[23392]: audit 2024-03-26T03:49:05.442024+0000 mon.a (mon.0) 1095 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi175-65296-2"}]: dispatch 2024-03-26T03:49:05.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:05 smithi190 bash[23392]: audit 2024-03-26T03:49:05.442447+0000 mon.a (mon.0) 1096 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi175-64398-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:05.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:05 smithi190 bash[23392]: audit 2024-03-26T03:49:05.443346+0000 mon.a (mon.0) 1097 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi175-64538-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-26T03:49:05.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:05 smithi190 bash[23392]: audit 2024-03-26T03:49:05.444836+0000 mon.a (mon.0) 1098 : audit [INF] from='client.? 172.21.15.175:0/1683135378' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:05.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:05 smithi190 bash[23392]: audit 2024-03-26T03:49:05.446914+0000 mon.a (mon.0) 1099 : audit [INF] from='client.? 172.21.15.175:0/1703995240' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[29284]: audit 2024-03-26T03:49:04.517067+0000 mon.a (mon.0) 1079 : audit [INF] from='client.? 172.21.15.175:0/4199038823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi175-64735-12"}]: dispatch 2024-03-26T03:49:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[29284]: audit 2024-03-26T03:49:04.531373+0000 mon.a (mon.0) 1080 : audit [INF] from='client.? 172.21.15.175:0/4199038823' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi175-64735-12"}]: dispatch 2024-03-26T03:49:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[29284]: audit 2024-03-26T03:49:04.532494+0000 mon.a (mon.0) 1081 : audit [INF] from='client.? 172.21.15.175:0/4199038823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi175-64735-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:49:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[29284]: cluster 2024-03-26T03:49:04.575053+0000 mgr.y (mgr.24431) 183 : cluster [DBG] pgmap v129: 616 pgs: 20 creating+peering, 1 creating+activating, 511 active+clean, 84 unknown; 145 MiB data, 755 MiB used, 714 GiB / 715 GiB avail 2024-03-26T03:49:05.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[29284]: audit 2024-03-26T03:49:04.681738+0000 mon.c (mon.2) 155 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:05.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[29284]: cluster 2024-03-26T03:49:05.390400+0000 mon.a (mon.0) 1082 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:49:05.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[29284]: audit 2024-03-26T03:49:05.394502+0000 mon.a (mon.0) 1083 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi175-65233-2"}]': finished 2024-03-26T03:49:05.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[29284]: audit 2024-03-26T03:49:05.394756+0000 mon.a (mon.0) 1084 : audit [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi175-64470-10"}]': finished 2024-03-26T03:49:05.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[29284]: audit 2024-03-26T03:49:05.394976+0000 mon.a (mon.0) 1085 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi175-64501-10"}]': finished 2024-03-26T03:49:05.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[29284]: audit 2024-03-26T03:49:05.395204+0000 mon.a (mon.0) 1086 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi175-64538-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-03-26T03:49:05.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[29284]: audit 2024-03-26T03:49:05.395487+0000 mon.a (mon.0) 1087 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi175-65296-2"}]': finished 2024-03-26T03:49:05.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[29284]: audit 2024-03-26T03:49:05.395697+0000 mon.a (mon.0) 1088 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64770-6", "mode": "writeback"}]': finished 2024-03-26T03:49:05.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[29284]: audit 2024-03-26T03:49:05.395935+0000 mon.a (mon.0) 1089 : audit [INF] from='client.? 172.21.15.175:0/175418209' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi175-64639-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:05.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[29284]: audit 2024-03-26T03:49:05.396172+0000 mon.a (mon.0) 1090 : audit [INF] from='client.? 172.21.15.175:0/4199038823' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi175-64735-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:49:05.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[21527]: audit 2024-03-26T03:49:04.517067+0000 mon.a (mon.0) 1079 : audit [INF] from='client.? 172.21.15.175:0/4199038823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi175-64735-12"}]: dispatch 2024-03-26T03:49:05.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[21527]: audit 2024-03-26T03:49:04.531373+0000 mon.a (mon.0) 1080 : audit [INF] from='client.? 172.21.15.175:0/4199038823' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi175-64735-12"}]: dispatch 2024-03-26T03:49:05.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[21527]: audit 2024-03-26T03:49:04.532494+0000 mon.a (mon.0) 1081 : audit [INF] from='client.? 172.21.15.175:0/4199038823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi175-64735-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:49:05.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[21527]: cluster 2024-03-26T03:49:04.575053+0000 mgr.y (mgr.24431) 183 : cluster [DBG] pgmap v129: 616 pgs: 20 creating+peering, 1 creating+activating, 511 active+clean, 84 unknown; 145 MiB data, 755 MiB used, 714 GiB / 715 GiB avail 2024-03-26T03:49:05.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[21527]: audit 2024-03-26T03:49:04.681738+0000 mon.c (mon.2) 155 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:05.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[21527]: cluster 2024-03-26T03:49:05.390400+0000 mon.a (mon.0) 1082 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:49:05.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[21527]: audit 2024-03-26T03:49:05.394502+0000 mon.a (mon.0) 1083 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi175-65233-2"}]': finished 2024-03-26T03:49:05.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[21527]: audit 2024-03-26T03:49:05.394756+0000 mon.a (mon.0) 1084 : audit [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi175-64470-10"}]': finished 2024-03-26T03:49:05.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[21527]: audit 2024-03-26T03:49:05.394976+0000 mon.a (mon.0) 1085 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi175-64501-10"}]': finished 2024-03-26T03:49:05.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[21527]: audit 2024-03-26T03:49:05.395204+0000 mon.a (mon.0) 1086 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi175-64538-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-03-26T03:49:05.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[21527]: audit 2024-03-26T03:49:05.395487+0000 mon.a (mon.0) 1087 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi175-65296-2"}]': finished 2024-03-26T03:49:05.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[21527]: audit 2024-03-26T03:49:05.395697+0000 mon.a (mon.0) 1088 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64770-6", "mode": "writeback"}]': finished 2024-03-26T03:49:05.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[21527]: audit 2024-03-26T03:49:05.395935+0000 mon.a (mon.0) 1089 : audit [INF] from='client.? 172.21.15.175:0/175418209' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi175-64639-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:05.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[21527]: audit 2024-03-26T03:49:05.396172+0000 mon.a (mon.0) 1090 : audit [INF] from='client.? 172.21.15.175:0/4199038823' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi175-64735-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:49:05.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[21527]: audit 2024-03-26T03:49:05.409648+0000 mon.b (mon.1) 83 : audit [INF] from='client.? 172.21.15.175:0/542670412' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:49:05.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[21527]: audit 2024-03-26T03:49:05.410158+0000 mon.b (mon.1) 84 : audit [INF] from='client.? 172.21.15.175:0/2671281638' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi175-65296-2"}]: dispatch 2024-03-26T03:49:05.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[21527]: cluster 2024-03-26T03:49:05.418911+0000 mon.a (mon.0) 1091 : cluster [DBG] osdmap e89: 8 total, 8 up, 8 in 2024-03-26T03:49:05.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[21527]: audit 2024-03-26T03:49:05.419572+0000 mon.c (mon.2) 156 : audit [INF] from='client.? 172.21.15.175:0/3873547907' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi175-64398-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:05.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[21527]: audit 2024-03-26T03:49:05.427945+0000 mon.b (mon.1) 85 : audit [INF] from='client.? 172.21.15.175:0/60974697' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi175-64538-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-26T03:49:05.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[21527]: audit 2024-03-26T03:49:05.437112+0000 mon.a (mon.0) 1092 : audit [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi175-64470-10"}]: dispatch 2024-03-26T03:49:05.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[21527]: audit 2024-03-26T03:49:05.440399+0000 mon.a (mon.0) 1093 : audit [INF] from='client.? 172.21.15.175:0/4199038823' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi175-64735-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi175-64735-12"}]: dispatch 2024-03-26T03:49:05.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[21527]: audit 2024-03-26T03:49:05.440871+0000 mon.a (mon.0) 1094 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:49:05.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[21527]: audit 2024-03-26T03:49:05.442024+0000 mon.a (mon.0) 1095 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi175-65296-2"}]: dispatch 2024-03-26T03:49:05.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[21527]: audit 2024-03-26T03:49:05.442447+0000 mon.a (mon.0) 1096 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi175-64398-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:05.997 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[21527]: audit 2024-03-26T03:49:05.443346+0000 mon.a (mon.0) 1097 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi175-64538-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-26T03:49:05.997 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[21527]: audit 2024-03-26T03:49:05.444836+0000 mon.a (mon.0) 1098 : audit [INF] from='client.? 172.21.15.175:0/1683135378' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:05.997 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[21527]: audit 2024-03-26T03:49:05.446914+0000 mon.a (mon.0) 1099 : audit [INF] from='client.? 172.21.15.175:0/1703995240' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:05.997 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[29284]: audit 2024-03-26T03:49:05.409648+0000 mon.b (mon.1) 83 : audit [INF] from='client.? 172.21.15.175:0/542670412' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:49:05.997 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[29284]: audit 2024-03-26T03:49:05.410158+0000 mon.b (mon.1) 84 : audit [INF] from='client.? 172.21.15.175:0/2671281638' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi175-65296-2"}]: dispatch 2024-03-26T03:49:05.997 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[29284]: cluster 2024-03-26T03:49:05.418911+0000 mon.a (mon.0) 1091 : cluster [DBG] osdmap e89: 8 total, 8 up, 8 in 2024-03-26T03:49:05.997 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[29284]: audit 2024-03-26T03:49:05.419572+0000 mon.c (mon.2) 156 : audit [INF] from='client.? 172.21.15.175:0/3873547907' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi175-64398-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:05.997 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[29284]: audit 2024-03-26T03:49:05.427945+0000 mon.b (mon.1) 85 : audit [INF] from='client.? 172.21.15.175:0/60974697' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi175-64538-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-26T03:49:05.997 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[29284]: audit 2024-03-26T03:49:05.437112+0000 mon.a (mon.0) 1092 : audit [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi175-64470-10"}]: dispatch 2024-03-26T03:49:05.997 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[29284]: audit 2024-03-26T03:49:05.440399+0000 mon.a (mon.0) 1093 : audit [INF] from='client.? 172.21.15.175:0/4199038823' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi175-64735-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi175-64735-12"}]: dispatch 2024-03-26T03:49:05.997 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[29284]: audit 2024-03-26T03:49:05.440871+0000 mon.a (mon.0) 1094 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi175-64501-10"}]: dispatch 2024-03-26T03:49:05.997 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[29284]: audit 2024-03-26T03:49:05.442024+0000 mon.a (mon.0) 1095 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi175-65296-2"}]: dispatch 2024-03-26T03:49:05.998 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[29284]: audit 2024-03-26T03:49:05.442447+0000 mon.a (mon.0) 1096 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi175-64398-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:05.998 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[29284]: audit 2024-03-26T03:49:05.443346+0000 mon.a (mon.0) 1097 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi175-64538-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-26T03:49:05.998 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[29284]: audit 2024-03-26T03:49:05.444836+0000 mon.a (mon.0) 1098 : audit [INF] from='client.? 172.21.15.175:0/1683135378' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:05.998 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:05 smithi175 bash[29284]: audit 2024-03-26T03:49:05.446914+0000 mon.a (mon.0) 1099 : audit [INF] from='client.? 172.21.15.175:0/1703995240' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:06.393 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: Running main() from gmock_main.cc 2024-03-26T03:49:06.393 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2024-03-26T03:49:06.394 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [----------] Global test environment set-up. 2024-03-26T03:49:06.394 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2024-03-26T03:49:06.394 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: seed 64501 2024-03-26T03:49:06.394 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2024-03-26T03:49:06.394 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (459 ms) 2024-03-26T03:49:06.394 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2024-03-26T03:49:06.394 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (6 ms) 2024-03-26T03:49:06.394 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-03-26T03:49:06.394 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (1005 ms) 2024-03-26T03:49:06.394 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2024-03-26T03:49:06.394 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (1007 ms) 2024-03-26T03:49:06.394 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2024-03-26T03:49:06.394 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (8 ms) 2024-03-26T03:49:06.395 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2024-03-26T03:49:06.395 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (6 ms) 2024-03-26T03:49:06.395 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2024-03-26T03:49:06.395 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (9 ms) 2024-03-26T03:49:06.395 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2024-03-26T03:49:06.395 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (7 ms) 2024-03-26T03:49:06.395 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (2507 ms total) 2024-03-26T03:49:06.395 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: 2024-03-26T03:49:06.395 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2024-03-26T03:49:06.395 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2024-03-26T03:49:06.395 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (1171 ms) 2024-03-26T03:49:06.395 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2024-03-26T03:49:06.395 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (7 ms) 2024-03-26T03:49:06.395 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-03-26T03:49:06.395 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1087 ms) 2024-03-26T03:49:06.396 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-03-26T03:49:06.396 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1025 ms) 2024-03-26T03:49:06.396 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-03-26T03:49:06.396 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (10 ms) 2024-03-26T03:49:06.396 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2024-03-26T03:49:06.396 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (10 ms) 2024-03-26T03:49:06.396 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2024-03-26T03:49:06.396 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (10 ms) 2024-03-26T03:49:06.396 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2024-03-26T03:49:06.396 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (7 ms) 2024-03-26T03:49:06.396 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (3327 ms total) 2024-03-26T03:49:06.396 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: 2024-03-26T03:49:06.396 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [----------] Global test environment tear-down 2024-03-26T03:49:06.396 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (14140 ms total) 2024-03-26T03:49:06.396 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock_pp: [ PASSED ] 16 tests. 2024-03-26T03:49:06.412 INFO:tasks.workunit.client.0.smithi175.stdout: pool: Running main() from gmock_main.cc 2024-03-26T03:49:06.412 INFO:tasks.workunit.client.0.smithi175.stdout: pool: [==========] Running 6 tests from 1 test suite. 2024-03-26T03:49:06.413 INFO:tasks.workunit.client.0.smithi175.stdout: pool: [----------] Global test environment set-up. 2024-03-26T03:49:06.413 INFO:tasks.workunit.client.0.smithi175.stdout: pool: [----------] 6 tests from NeoRadosPools 2024-03-26T03:49:06.413 INFO:tasks.workunit.client.0.smithi175.stdout: pool: [ RUN ] NeoRadosPools.PoolList 2024-03-26T03:49:06.413 INFO:tasks.workunit.client.0.smithi175.stdout: pool: [ OK ] NeoRadosPools.PoolList (1755 ms) 2024-03-26T03:49:06.413 INFO:tasks.workunit.client.0.smithi175.stdout: pool: [ RUN ] NeoRadosPools.PoolLookup 2024-03-26T03:49:06.413 INFO:tasks.workunit.client.0.smithi175.stdout: pool: [ OK ] NeoRadosPools.PoolLookup (1985 ms) 2024-03-26T03:49:06.413 INFO:tasks.workunit.client.0.smithi175.stdout: pool: [ RUN ] NeoRadosPools.PoolLookupOtherInstance 2024-03-26T03:49:06.413 INFO:tasks.workunit.client.0.smithi175.stdout: pool: [ OK ] NeoRadosPools.PoolLookupOtherInstance (2061 ms) 2024-03-26T03:49:06.413 INFO:tasks.workunit.client.0.smithi175.stdout: pool: [ RUN ] NeoRadosPools.PoolDelete 2024-03-26T03:49:06.413 INFO:tasks.workunit.client.0.smithi175.stdout: pool: [ OK ] NeoRadosPools.PoolDelete (4008 ms) 2024-03-26T03:49:06.413 INFO:tasks.workunit.client.0.smithi175.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateDelete 2024-03-26T03:49:06.413 INFO:tasks.workunit.client.0.smithi175.stdout: pool: [ OK ] NeoRadosPools.PoolCreateDelete (2015 ms) 2024-03-26T03:49:06.413 INFO:tasks.workunit.client.0.smithi175.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateWithCrushRule 2024-03-26T03:49:06.413 INFO:tasks.workunit.client.0.smithi175.stdout: pool: [ OK ] NeoRadosPools.PoolCreateWithCrushRule (1986 ms) 2024-03-26T03:49:06.414 INFO:tasks.workunit.client.0.smithi175.stdout: pool: [----------] 6 tests from NeoRadosPools (13810 ms total) 2024-03-26T03:49:06.414 INFO:tasks.workunit.client.0.smithi175.stdout: pool: 2024-03-26T03:49:06.414 INFO:tasks.workunit.client.0.smithi175.stdout: pool: [----------] Global test environment tear-down 2024-03-26T03:49:06.414 INFO:tasks.workunit.client.0.smithi175.stdout: pool: [==========] 6 tests from 1 test suite ran. (13810 ms total) 2024-03-26T03:49:06.414 INFO:tasks.workunit.client.0.smithi175.stdout: pool: [ PASSED ] 6 tests. 2024-03-26T03:49:06.421 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: Running main() from gmock_main.cc 2024-03-26T03:49:06.421 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2024-03-26T03:49:06.421 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [----------] Global test environment set-up. 2024-03-26T03:49:06.421 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [----------] 8 tests from LibRadosLock 2024-03-26T03:49:06.421 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2024-03-26T03:49:06.421 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (474 ms) 2024-03-26T03:49:06.421 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2024-03-26T03:49:06.421 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ OK ] LibRadosLock.LockShared (11 ms) 2024-03-26T03:49:06.421 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2024-03-26T03:49:06.421 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (1012 ms) 2024-03-26T03:49:06.421 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2024-03-26T03:49:06.421 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1004 ms) 2024-03-26T03:49:06.422 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2024-03-26T03:49:06.422 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (4 ms) 2024-03-26T03:49:06.422 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2024-03-26T03:49:06.422 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ OK ] LibRadosLock.Unlock (8 ms) 2024-03-26T03:49:06.422 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2024-03-26T03:49:06.422 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (4 ms) 2024-03-26T03:49:06.422 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2024-03-26T03:49:06.422 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (6 ms) 2024-03-26T03:49:06.422 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [----------] 8 tests from LibRadosLock (2523 ms total) 2024-03-26T03:49:06.422 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: 2024-03-26T03:49:06.422 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2024-03-26T03:49:06.422 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2024-03-26T03:49:06.422 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (1155 ms) 2024-03-26T03:49:06.423 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2024-03-26T03:49:06.425 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (9 ms) 2024-03-26T03:49:06.426 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2024-03-26T03:49:06.426 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1070 ms) 2024-03-26T03:49:06.426 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2024-03-26T03:49:06.426 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1037 ms) 2024-03-26T03:49:06.426 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2024-03-26T03:49:06.426 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (11 ms) 2024-03-26T03:49:06.427 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2024-03-26T03:49:06.427 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (9 ms) 2024-03-26T03:49:06.427 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2024-03-26T03:49:06.427 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (11 ms) 2024-03-26T03:49:06.427 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2024-03-26T03:49:06.427 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (7 ms) 2024-03-26T03:49:06.427 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (3310 ms total) 2024-03-26T03:49:06.427 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: 2024-03-26T03:49:06.427 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [----------] Global test environment tear-down 2024-03-26T03:49:06.427 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (14182 ms total) 2024-03-26T03:49:06.427 INFO:tasks.workunit.client.0.smithi175.stdout: api_lock: [ PASSED ] 16 tests. 2024-03-26T03:49:06.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:06 smithi190 bash[23392]: cluster 2024-03-26T03:49:05.518845+0000 mon.a (mon.0) 1100 : cluster [WRN] Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:06.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:06 smithi190 bash[23392]: audit 2024-03-26T03:49:05.683725+0000 mon.c (mon.2) 157 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:06.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:06 smithi190 bash[23392]: cluster 2024-03-26T03:49:06.362040+0000 mon.a (mon.0) 1101 : cluster [WRN] pool 'PoolQuotaPP_smithi175-64406-3' is full (reached quota's max_bytes: 4 KiB) 2024-03-26T03:49:06.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:06 smithi190 bash[23392]: cluster 2024-03-26T03:49:06.366099+0000 mon.a (mon.0) 1102 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-26T03:49:06.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:06 smithi190 bash[23392]: audit 2024-03-26T03:49:06.369990+0000 mon.a (mon.0) 1103 : audit [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi175-64470-10"}]': finished 2024-03-26T03:49:06.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:06 smithi190 bash[23392]: audit 2024-03-26T03:49:06.370242+0000 mon.a (mon.0) 1104 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi175-64501-10"}]': finished 2024-03-26T03:49:06.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:06 smithi190 bash[23392]: audit 2024-03-26T03:49:06.370422+0000 mon.a (mon.0) 1105 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi175-65296-2"}]': finished 2024-03-26T03:49:06.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:06 smithi190 bash[23392]: audit 2024-03-26T03:49:06.370666+0000 mon.a (mon.0) 1106 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi175-64398-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:06.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:06 smithi190 bash[23392]: audit 2024-03-26T03:49:06.370902+0000 mon.a (mon.0) 1107 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi175-64538-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-03-26T03:49:06.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:06 smithi190 bash[23392]: audit 2024-03-26T03:49:06.371083+0000 mon.a (mon.0) 1108 : audit [INF] from='client.? 172.21.15.175:0/1683135378' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:06.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:06 smithi190 bash[23392]: audit 2024-03-26T03:49:06.371241+0000 mon.a (mon.0) 1109 : audit [INF] from='client.? 172.21.15.175:0/1703995240' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:06.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:06 smithi190 bash[23392]: audit 2024-03-26T03:49:06.389372+0000 mon.b (mon.1) 86 : audit [INF] from='client.? 172.21.15.175:0/60974697' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi175-64538-1","app":"app1","key":"key1"}]: dispatch 2024-03-26T03:49:06.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:06 smithi190 bash[23392]: cluster 2024-03-26T03:49:06.396502+0000 mon.a (mon.0) 1110 : cluster [DBG] osdmap e90: 8 total, 8 up, 8 in 2024-03-26T03:49:06.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:06 smithi190 bash[23392]: audit 2024-03-26T03:49:06.410576+0000 mon.a (mon.0) 1111 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi175-64538-1","app":"app1","key":"key1"}]: dispatch 2024-03-26T03:49:06.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:06 smithi190 bash[23392]: audit 2024-03-26T03:49:06.445409+0000 mon.a (mon.0) 1112 : audit [INF] from='client.? 172.21.15.175:0/1076098358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi175-65296-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:49:06.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:06 smithi175 bash[21527]: cluster 2024-03-26T03:49:05.518845+0000 mon.a (mon.0) 1100 : cluster [WRN] Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:06.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:06 smithi175 bash[21527]: audit 2024-03-26T03:49:05.683725+0000 mon.c (mon.2) 157 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:06.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:06 smithi175 bash[21527]: cluster 2024-03-26T03:49:06.362040+0000 mon.a (mon.0) 1101 : cluster [WRN] pool 'PoolQuotaPP_smithi175-64406-3' is full (reached quota's max_bytes: 4 KiB) 2024-03-26T03:49:06.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:06 smithi175 bash[21527]: cluster 2024-03-26T03:49:06.366099+0000 mon.a (mon.0) 1102 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-26T03:49:06.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:06 smithi175 bash[21527]: audit 2024-03-26T03:49:06.369990+0000 mon.a (mon.0) 1103 : audit [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi175-64470-10"}]': finished 2024-03-26T03:49:06.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:06 smithi175 bash[21527]: audit 2024-03-26T03:49:06.370242+0000 mon.a (mon.0) 1104 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi175-64501-10"}]': finished 2024-03-26T03:49:06.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:06 smithi175 bash[21527]: audit 2024-03-26T03:49:06.370422+0000 mon.a (mon.0) 1105 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi175-65296-2"}]': finished 2024-03-26T03:49:06.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:06 smithi175 bash[21527]: audit 2024-03-26T03:49:06.370666+0000 mon.a (mon.0) 1106 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi175-64398-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:06.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:06 smithi175 bash[21527]: audit 2024-03-26T03:49:06.370902+0000 mon.a (mon.0) 1107 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi175-64538-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-03-26T03:49:06.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:06 smithi175 bash[21527]: audit 2024-03-26T03:49:06.371083+0000 mon.a (mon.0) 1108 : audit [INF] from='client.? 172.21.15.175:0/1683135378' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:06.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:06 smithi175 bash[21527]: audit 2024-03-26T03:49:06.371241+0000 mon.a (mon.0) 1109 : audit [INF] from='client.? 172.21.15.175:0/1703995240' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:06.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:06 smithi175 bash[21527]: audit 2024-03-26T03:49:06.389372+0000 mon.b (mon.1) 86 : audit [INF] from='client.? 172.21.15.175:0/60974697' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi175-64538-1","app":"app1","key":"key1"}]: dispatch 2024-03-26T03:49:06.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:06 smithi175 bash[21527]: cluster 2024-03-26T03:49:06.396502+0000 mon.a (mon.0) 1110 : cluster [DBG] osdmap e90: 8 total, 8 up, 8 in 2024-03-26T03:49:06.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:06 smithi175 bash[21527]: audit 2024-03-26T03:49:06.410576+0000 mon.a (mon.0) 1111 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi175-64538-1","app":"app1","key":"key1"}]: dispatch 2024-03-26T03:49:06.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:06 smithi175 bash[21527]: audit 2024-03-26T03:49:06.445409+0000 mon.a (mon.0) 1112 : audit [INF] from='client.? 172.21.15.175:0/1076098358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi175-65296-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:49:06.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:06 smithi175 bash[29284]: cluster 2024-03-26T03:49:05.518845+0000 mon.a (mon.0) 1100 : cluster [WRN] Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:06.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:06 smithi175 bash[29284]: audit 2024-03-26T03:49:05.683725+0000 mon.c (mon.2) 157 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:06.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:06 smithi175 bash[29284]: cluster 2024-03-26T03:49:06.362040+0000 mon.a (mon.0) 1101 : cluster [WRN] pool 'PoolQuotaPP_smithi175-64406-3' is full (reached quota's max_bytes: 4 KiB) 2024-03-26T03:49:06.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:06 smithi175 bash[29284]: cluster 2024-03-26T03:49:06.366099+0000 mon.a (mon.0) 1102 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-26T03:49:06.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:06 smithi175 bash[29284]: audit 2024-03-26T03:49:06.369990+0000 mon.a (mon.0) 1103 : audit [INF] from='client.? 172.21.15.175:0/482986724' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi175-64470-10"}]': finished 2024-03-26T03:49:06.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:06 smithi175 bash[29284]: audit 2024-03-26T03:49:06.370242+0000 mon.a (mon.0) 1104 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi175-64501-10"}]': finished 2024-03-26T03:49:06.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:06 smithi175 bash[29284]: audit 2024-03-26T03:49:06.370422+0000 mon.a (mon.0) 1105 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi175-65296-2"}]': finished 2024-03-26T03:49:06.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:06 smithi175 bash[29284]: audit 2024-03-26T03:49:06.370666+0000 mon.a (mon.0) 1106 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi175-64398-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:06.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:06 smithi175 bash[29284]: audit 2024-03-26T03:49:06.370902+0000 mon.a (mon.0) 1107 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi175-64538-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-03-26T03:49:06.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:06 smithi175 bash[29284]: audit 2024-03-26T03:49:06.371083+0000 mon.a (mon.0) 1108 : audit [INF] from='client.? 172.21.15.175:0/1683135378' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:06.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:06 smithi175 bash[29284]: audit 2024-03-26T03:49:06.371241+0000 mon.a (mon.0) 1109 : audit [INF] from='client.? 172.21.15.175:0/1703995240' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:06.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:06 smithi175 bash[29284]: audit 2024-03-26T03:49:06.389372+0000 mon.b (mon.1) 86 : audit [INF] from='client.? 172.21.15.175:0/60974697' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi175-64538-1","app":"app1","key":"key1"}]: dispatch 2024-03-26T03:49:06.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:06 smithi175 bash[29284]: cluster 2024-03-26T03:49:06.396502+0000 mon.a (mon.0) 1110 : cluster [DBG] osdmap e90: 8 total, 8 up, 8 in 2024-03-26T03:49:06.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:06 smithi175 bash[29284]: audit 2024-03-26T03:49:06.410576+0000 mon.a (mon.0) 1111 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi175-64538-1","app":"app1","key":"key1"}]: dispatch 2024-03-26T03:49:06.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:06 smithi175 bash[29284]: audit 2024-03-26T03:49:06.445409+0000 mon.a (mon.0) 1112 : audit [INF] from='client.? 172.21.15.175:0/1076098358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi175-65296-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:49:07.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:07 smithi175 bash[21527]: cluster 2024-03-26T03:49:06.575715+0000 mgr.y (mgr.24431) 184 : cluster [DBG] pgmap v132: 680 pgs: 32 creating+peering, 456 active+clean, 192 unknown; 145 MiB data, 765 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-03-26T03:49:07.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:07 smithi175 bash[21527]: audit 2024-03-26T03:49:06.685744+0000 mon.c (mon.2) 158 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:07.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:07 smithi175 bash[21527]: audit 2024-03-26T03:49:07.375202+0000 mon.a (mon.0) 1113 : audit [INF] from='client.? 172.21.15.175:0/4199038823' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi175-64735-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi175-64735-12"}]': finished 2024-03-26T03:49:07.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:07 smithi175 bash[21527]: audit 2024-03-26T03:49:07.375468+0000 mon.a (mon.0) 1114 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi175-64538-1","app":"app1","key":"key1"}]': finished 2024-03-26T03:49:07.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:07 smithi175 bash[21527]: audit 2024-03-26T03:49:07.375620+0000 mon.a (mon.0) 1115 : audit [INF] from='client.? 172.21.15.175:0/1076098358' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi175-65296-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:49:07.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:07 smithi175 bash[21527]: cluster 2024-03-26T03:49:07.384380+0000 mon.a (mon.0) 1116 : cluster [DBG] osdmap e91: 8 total, 8 up, 8 in 2024-03-26T03:49:07.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:07 smithi175 bash[21527]: audit 2024-03-26T03:49:07.385896+0000 mon.a (mon.0) 1117 : audit [INF] from='client.? 172.21.15.175:0/1076098358' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi175-65296-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi175-65296-3"}]: dispatch 2024-03-26T03:49:07.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:07 smithi175 bash[21527]: audit 2024-03-26T03:49:07.391966+0000 mon.a (mon.0) 1118 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:07.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:07 smithi190 bash[23392]: cluster 2024-03-26T03:49:06.575715+0000 mgr.y (mgr.24431) 184 : cluster [DBG] pgmap v132: 680 pgs: 32 creating+peering, 456 active+clean, 192 unknown; 145 MiB data, 765 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-03-26T03:49:07.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:07 smithi190 bash[23392]: audit 2024-03-26T03:49:06.685744+0000 mon.c (mon.2) 158 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:07.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:07 smithi190 bash[23392]: audit 2024-03-26T03:49:07.375202+0000 mon.a (mon.0) 1113 : audit [INF] from='client.? 172.21.15.175:0/4199038823' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi175-64735-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi175-64735-12"}]': finished 2024-03-26T03:49:07.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:07 smithi190 bash[23392]: audit 2024-03-26T03:49:07.375468+0000 mon.a (mon.0) 1114 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi175-64538-1","app":"app1","key":"key1"}]': finished 2024-03-26T03:49:07.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:07 smithi190 bash[23392]: audit 2024-03-26T03:49:07.375620+0000 mon.a (mon.0) 1115 : audit [INF] from='client.? 172.21.15.175:0/1076098358' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi175-65296-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:49:07.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:07 smithi190 bash[23392]: cluster 2024-03-26T03:49:07.384380+0000 mon.a (mon.0) 1116 : cluster [DBG] osdmap e91: 8 total, 8 up, 8 in 2024-03-26T03:49:07.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:07 smithi190 bash[23392]: audit 2024-03-26T03:49:07.385896+0000 mon.a (mon.0) 1117 : audit [INF] from='client.? 172.21.15.175:0/1076098358' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi175-65296-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi175-65296-3"}]: dispatch 2024-03-26T03:49:07.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:07 smithi190 bash[23392]: audit 2024-03-26T03:49:07.391966+0000 mon.a (mon.0) 1118 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:07.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:07 smithi175 bash[29284]: cluster 2024-03-26T03:49:06.575715+0000 mgr.y (mgr.24431) 184 : cluster [DBG] pgmap v132: 680 pgs: 32 creating+peering, 456 active+clean, 192 unknown; 145 MiB data, 765 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-03-26T03:49:07.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:07 smithi175 bash[29284]: audit 2024-03-26T03:49:06.685744+0000 mon.c (mon.2) 158 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:07.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:07 smithi175 bash[29284]: audit 2024-03-26T03:49:07.375202+0000 mon.a (mon.0) 1113 : audit [INF] from='client.? 172.21.15.175:0/4199038823' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi175-64735-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi175-64735-12"}]': finished 2024-03-26T03:49:07.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:07 smithi175 bash[29284]: audit 2024-03-26T03:49:07.375468+0000 mon.a (mon.0) 1114 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi175-64538-1","app":"app1","key":"key1"}]': finished 2024-03-26T03:49:07.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:07 smithi175 bash[29284]: audit 2024-03-26T03:49:07.375620+0000 mon.a (mon.0) 1115 : audit [INF] from='client.? 172.21.15.175:0/1076098358' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi175-65296-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:49:07.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:07 smithi175 bash[29284]: cluster 2024-03-26T03:49:07.384380+0000 mon.a (mon.0) 1116 : cluster [DBG] osdmap e91: 8 total, 8 up, 8 in 2024-03-26T03:49:07.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:07 smithi175 bash[29284]: audit 2024-03-26T03:49:07.385896+0000 mon.a (mon.0) 1117 : audit [INF] from='client.? 172.21.15.175:0/1076098358' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi175-65296-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi175-65296-3"}]: dispatch 2024-03-26T03:49:07.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:07 smithi175 bash[29284]: audit 2024-03-26T03:49:07.391966+0000 mon.a (mon.0) 1118 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:08.565 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: Running main() from gmock_main.cc 2024-03-26T03:49:08.566 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2024-03-26T03:49:08.566 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2024-03-26T03:49:08.566 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2024-03-26T03:49:08.566 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-03-26T03:49:08.566 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: notify 2024-03-26T03:49:08.566 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (1120 ms) 2024-03-26T03:49:08.566 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-03-26T03:49:08.566 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (20 ms) 2024-03-26T03:49:08.566 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (1140 ms total) 2024-03-26T03:49:08.566 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: 2024-03-26T03:49:08.566 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-03-26T03:49:08.566 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-03-26T03:49:08.566 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: notify 2024-03-26T03:49:08.567 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (172 ms) 2024-03-26T03:49:08.567 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-03-26T03:49:08.567 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: notify 2024-03-26T03:49:08.567 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (3963 ms) 2024-03-26T03:49:08.567 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-03-26T03:49:08.567 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (15 ms) 2024-03-26T03:49:08.567 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-03-26T03:49:08.567 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (9 ms) 2024-03-26T03:49:08.567 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-03-26T03:49:08.567 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: handle_notify cookie 94378448984768 notify_id 382252089345 notifier_gid 25187 2024-03-26T03:49:08.567 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (6 ms) 2024-03-26T03:49:08.567 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-03-26T03:49:08.567 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: handle_notify cookie 94378449006688 notify_id 382252089347 notifier_gid 25187 2024-03-26T03:49:08.568 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (6 ms) 2024-03-26T03:49:08.568 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-03-26T03:49:08.568 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: handle_notify cookie 94378449006688 notify_id 382252089346 notifier_gid 25187 2024-03-26T03:49:08.568 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (5 ms) 2024-03-26T03:49:08.568 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-03-26T03:49:08.568 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: handle_notify cookie 94378449006688 notify_id 382252089347 notifier_gid 25187 2024-03-26T03:49:08.568 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (6 ms) 2024-03-26T03:49:08.568 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-03-26T03:49:08.568 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: handle_notify cookie 94378449006688 notify_id 382252089348 notifier_gid 25187 2024-03-26T03:49:08.568 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (5 ms) 2024-03-26T03:49:08.568 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-03-26T03:49:08.568 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: handle_notify cookie 94378449006688 notify_id 382252089346 notifier_gid 25187 2024-03-26T03:49:08.568 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (6 ms) 2024-03-26T03:49:08.568 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-03-26T03:49:08.569 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: trying... 2024-03-26T03:49:08.569 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: handle_notify cookie 94378449006688 notify_id 382252089349 notifier_gid 25187 2024-03-26T03:49:08.569 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: timed out 2024-03-26T03:49:08.569 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: flushing 2024-03-26T03:49:08.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:08 smithi190 bash[23392]: audit 2024-03-26T03:49:07.687361+0000 mon.c (mon.2) 159 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:08.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:08 smithi190 bash[23392]: audit 2024-03-26T03:49:08.381456+0000 mon.a (mon.0) 1119 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:08.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:08 smithi190 bash[23392]: cluster 2024-03-26T03:49:08.398504+0000 mon.a (mon.0) 1120 : cluster [DBG] osdmap e92: 8 total, 8 up, 8 in 2024-03-26T03:49:08.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:08 smithi190 bash[23392]: audit 2024-03-26T03:49:08.413559+0000 mon.b (mon.1) 87 : audit [INF] from='client.? 172.21.15.175:0/1363718088' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi175-64636-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:08.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:08 smithi190 bash[23392]: audit 2024-03-26T03:49:08.416375+0000 mon.b (mon.1) 88 : audit [INF] from='client.? 172.21.15.175:0/3970241970' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:08.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:08 smithi190 bash[23392]: audit 2024-03-26T03:49:08.418042+0000 mon.b (mon.1) 89 : audit [INF] from='client.? 172.21.15.175:0/2978308738' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi175-64398-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:08.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:08 smithi190 bash[23392]: audit 2024-03-26T03:49:08.420179+0000 mon.c (mon.2) 160 : audit [INF] from='client.? 172.21.15.175:0/1411564448' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:08.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:08 smithi190 bash[23392]: audit 2024-03-26T03:49:08.425177+0000 mon.a (mon.0) 1121 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi175-64636-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:08.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:08 smithi190 bash[23392]: audit 2024-03-26T03:49:08.425492+0000 mon.a (mon.0) 1122 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:08.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:08 smithi190 bash[23392]: audit 2024-03-26T03:49:08.425762+0000 mon.a (mon.0) 1123 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi175-64398-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:08.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:08 smithi190 bash[23392]: audit 2024-03-26T03:49:08.426430+0000 mon.a (mon.0) 1124 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:08.991 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:49:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:49:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:49:08.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:08 smithi175 bash[21527]: audit 2024-03-26T03:49:07.687361+0000 mon.c (mon.2) 159 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:08.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:08 smithi175 bash[21527]: audit 2024-03-26T03:49:08.381456+0000 mon.a (mon.0) 1119 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:08.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:08 smithi175 bash[21527]: cluster 2024-03-26T03:49:08.398504+0000 mon.a (mon.0) 1120 : cluster [DBG] osdmap e92: 8 total, 8 up, 8 in 2024-03-26T03:49:08.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:08 smithi175 bash[21527]: audit 2024-03-26T03:49:08.413559+0000 mon.b (mon.1) 87 : audit [INF] from='client.? 172.21.15.175:0/1363718088' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi175-64636-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:08.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:08 smithi175 bash[21527]: audit 2024-03-26T03:49:08.416375+0000 mon.b (mon.1) 88 : audit [INF] from='client.? 172.21.15.175:0/3970241970' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:08.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:08 smithi175 bash[21527]: audit 2024-03-26T03:49:08.418042+0000 mon.b (mon.1) 89 : audit [INF] from='client.? 172.21.15.175:0/2978308738' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi175-64398-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:08.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:08 smithi175 bash[21527]: audit 2024-03-26T03:49:08.420179+0000 mon.c (mon.2) 160 : audit [INF] from='client.? 172.21.15.175:0/1411564448' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:08.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:08 smithi175 bash[21527]: audit 2024-03-26T03:49:08.425177+0000 mon.a (mon.0) 1121 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi175-64636-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:08.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:08 smithi175 bash[21527]: audit 2024-03-26T03:49:08.425492+0000 mon.a (mon.0) 1122 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:08.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:08 smithi175 bash[21527]: audit 2024-03-26T03:49:08.425762+0000 mon.a (mon.0) 1123 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi175-64398-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:08.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:08 smithi175 bash[21527]: audit 2024-03-26T03:49:08.426430+0000 mon.a (mon.0) 1124 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:08.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:08 smithi175 bash[29284]: audit 2024-03-26T03:49:07.687361+0000 mon.c (mon.2) 159 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:08.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:08 smithi175 bash[29284]: audit 2024-03-26T03:49:08.381456+0000 mon.a (mon.0) 1119 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:08.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:08 smithi175 bash[29284]: cluster 2024-03-26T03:49:08.398504+0000 mon.a (mon.0) 1120 : cluster [DBG] osdmap e92: 8 total, 8 up, 8 in 2024-03-26T03:49:08.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:08 smithi175 bash[29284]: audit 2024-03-26T03:49:08.413559+0000 mon.b (mon.1) 87 : audit [INF] from='client.? 172.21.15.175:0/1363718088' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi175-64636-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:08.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:08 smithi175 bash[29284]: audit 2024-03-26T03:49:08.416375+0000 mon.b (mon.1) 88 : audit [INF] from='client.? 172.21.15.175:0/3970241970' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:08.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:08 smithi175 bash[29284]: audit 2024-03-26T03:49:08.418042+0000 mon.b (mon.1) 89 : audit [INF] from='client.? 172.21.15.175:0/2978308738' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi175-64398-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:08.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:08 smithi175 bash[29284]: audit 2024-03-26T03:49:08.420179+0000 mon.c (mon.2) 160 : audit [INF] from='client.? 172.21.15.175:0/1411564448' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:08.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:08 smithi175 bash[29284]: audit 2024-03-26T03:49:08.425177+0000 mon.a (mon.0) 1121 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi175-64636-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:08.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:08 smithi175 bash[29284]: audit 2024-03-26T03:49:08.425492+0000 mon.a (mon.0) 1122 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:08.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:08 smithi175 bash[29284]: audit 2024-03-26T03:49:08.425762+0000 mon.a (mon.0) 1123 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi175-64398-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:08.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:08 smithi175 bash[29284]: audit 2024-03-26T03:49:08.426430+0000 mon.a (mon.0) 1124 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:09.990 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:49:09 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:49:09.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:09 smithi190 bash[23392]: cluster 2024-03-26T03:49:08.576746+0000 mgr.y (mgr.24431) 185 : cluster [DBG] pgmap v135: 720 pgs: 32 creating+peering, 424 active+clean, 264 unknown; 145 MiB data, 765 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-26T03:49:09.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:09 smithi190 bash[23392]: audit 2024-03-26T03:49:08.688540+0000 mon.c (mon.2) 161 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:09.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:09 smithi190 bash[23392]: audit 2024-03-26T03:49:09.395032+0000 mon.a (mon.0) 1125 : audit [INF] from='client.? 172.21.15.175:0/1076098358' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi175-65296-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi175-65296-3"}]': finished 2024-03-26T03:49:09.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:09 smithi190 bash[23392]: audit 2024-03-26T03:49:09.395148+0000 mon.a (mon.0) 1126 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi175-64636-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:09.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:09 smithi190 bash[23392]: audit 2024-03-26T03:49:09.395252+0000 mon.a (mon.0) 1127 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:09.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:09 smithi190 bash[23392]: audit 2024-03-26T03:49:09.395345+0000 mon.a (mon.0) 1128 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi175-64398-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:09.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:09 smithi190 bash[23392]: audit 2024-03-26T03:49:09.395443+0000 mon.a (mon.0) 1129 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:09.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:09 smithi190 bash[23392]: cluster 2024-03-26T03:49:09.402243+0000 mon.a (mon.0) 1130 : cluster [DBG] osdmap e93: 8 total, 8 up, 8 in 2024-03-26T03:49:09.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:09 smithi190 bash[23392]: audit 2024-03-26T03:49:09.421083+0000 mon.a (mon.0) 1131 : audit [INF] from='client.? 172.21.15.175:0/4199038823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi175-64735-12"}]: dispatch 2024-03-26T03:49:09.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:09 smithi190 bash[23392]: audit 2024-03-26T03:49:09.425784+0000 mon.a (mon.0) 1132 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:09.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:09 smithi175 bash[21527]: cluster 2024-03-26T03:49:08.576746+0000 mgr.y (mgr.24431) 185 : cluster [DBG] pgmap v135: 720 pgs: 32 creating+peering, 424 active+clean, 264 unknown; 145 MiB data, 765 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-26T03:49:09.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:09 smithi175 bash[21527]: audit 2024-03-26T03:49:08.688540+0000 mon.c (mon.2) 161 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:09.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:09 smithi175 bash[21527]: audit 2024-03-26T03:49:09.395032+0000 mon.a (mon.0) 1125 : audit [INF] from='client.? 172.21.15.175:0/1076098358' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi175-65296-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi175-65296-3"}]': finished 2024-03-26T03:49:09.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:09 smithi175 bash[21527]: audit 2024-03-26T03:49:09.395148+0000 mon.a (mon.0) 1126 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi175-64636-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:09.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:09 smithi175 bash[21527]: audit 2024-03-26T03:49:09.395252+0000 mon.a (mon.0) 1127 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:09.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:09 smithi175 bash[21527]: audit 2024-03-26T03:49:09.395345+0000 mon.a (mon.0) 1128 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi175-64398-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:09.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:09 smithi175 bash[21527]: audit 2024-03-26T03:49:09.395443+0000 mon.a (mon.0) 1129 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:09.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:09 smithi175 bash[21527]: cluster 2024-03-26T03:49:09.402243+0000 mon.a (mon.0) 1130 : cluster [DBG] osdmap e93: 8 total, 8 up, 8 in 2024-03-26T03:49:09.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:09 smithi175 bash[21527]: audit 2024-03-26T03:49:09.421083+0000 mon.a (mon.0) 1131 : audit [INF] from='client.? 172.21.15.175:0/4199038823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi175-64735-12"}]: dispatch 2024-03-26T03:49:09.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:09 smithi175 bash[21527]: audit 2024-03-26T03:49:09.425784+0000 mon.a (mon.0) 1132 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:09.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:09 smithi175 bash[29284]: cluster 2024-03-26T03:49:08.576746+0000 mgr.y (mgr.24431) 185 : cluster [DBG] pgmap v135: 720 pgs: 32 creating+peering, 424 active+clean, 264 unknown; 145 MiB data, 765 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-26T03:49:09.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:09 smithi175 bash[29284]: audit 2024-03-26T03:49:08.688540+0000 mon.c (mon.2) 161 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:09.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:09 smithi175 bash[29284]: audit 2024-03-26T03:49:09.395032+0000 mon.a (mon.0) 1125 : audit [INF] from='client.? 172.21.15.175:0/1076098358' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi175-65296-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi175-65296-3"}]': finished 2024-03-26T03:49:09.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:09 smithi175 bash[29284]: audit 2024-03-26T03:49:09.395148+0000 mon.a (mon.0) 1126 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi175-64636-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:09.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:09 smithi175 bash[29284]: audit 2024-03-26T03:49:09.395252+0000 mon.a (mon.0) 1127 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:09.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:09 smithi175 bash[29284]: audit 2024-03-26T03:49:09.395345+0000 mon.a (mon.0) 1128 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi175-64398-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:09.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:09 smithi175 bash[29284]: audit 2024-03-26T03:49:09.395443+0000 mon.a (mon.0) 1129 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:09.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:09 smithi175 bash[29284]: cluster 2024-03-26T03:49:09.402243+0000 mon.a (mon.0) 1130 : cluster [DBG] osdmap e93: 8 total, 8 up, 8 in 2024-03-26T03:49:09.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:09 smithi175 bash[29284]: audit 2024-03-26T03:49:09.421083+0000 mon.a (mon.0) 1131 : audit [INF] from='client.? 172.21.15.175:0/4199038823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi175-64735-12"}]: dispatch 2024-03-26T03:49:09.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:09 smithi175 bash[29284]: audit 2024-03-26T03:49:09.425784+0000 mon.a (mon.0) 1132 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:10.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:10 smithi190 bash[23392]: audit 2024-03-26T03:49:09.690369+0000 mon.c (mon.2) 162 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:10.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:10 smithi190 bash[23392]: audit 2024-03-26T03:49:09.693795+0000 mgr.y (mgr.24431) 186 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:49:10.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:10 smithi190 bash[23392]: audit 2024-03-26T03:49:10.397410+0000 mon.a (mon.0) 1133 : audit [INF] from='client.? 172.21.15.175:0/4199038823' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi175-64735-12"}]': finished 2024-03-26T03:49:10.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:10 smithi190 bash[23392]: audit 2024-03-26T03:49:10.397495+0000 mon.a (mon.0) 1134 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:10.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:10 smithi190 bash[23392]: cluster 2024-03-26T03:49:10.413351+0000 mon.a (mon.0) 1135 : cluster [DBG] osdmap e94: 8 total, 8 up, 8 in 2024-03-26T03:49:10.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:10 smithi190 bash[23392]: audit 2024-03-26T03:49:10.416574+0000 mon.a (mon.0) 1136 : audit [INF] from='client.? 172.21.15.175:0/4199038823' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi175-64735-12"}]: dispatch 2024-03-26T03:49:10.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:10 smithi175 bash[21527]: audit 2024-03-26T03:49:09.690369+0000 mon.c (mon.2) 162 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:10.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:10 smithi175 bash[21527]: audit 2024-03-26T03:49:09.693795+0000 mgr.y (mgr.24431) 186 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:49:10.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:10 smithi175 bash[21527]: audit 2024-03-26T03:49:10.397410+0000 mon.a (mon.0) 1133 : audit [INF] from='client.? 172.21.15.175:0/4199038823' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi175-64735-12"}]': finished 2024-03-26T03:49:10.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:10 smithi175 bash[21527]: audit 2024-03-26T03:49:10.397495+0000 mon.a (mon.0) 1134 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:10.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:10 smithi175 bash[21527]: cluster 2024-03-26T03:49:10.413351+0000 mon.a (mon.0) 1135 : cluster [DBG] osdmap e94: 8 total, 8 up, 8 in 2024-03-26T03:49:10.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:10 smithi175 bash[21527]: audit 2024-03-26T03:49:10.416574+0000 mon.a (mon.0) 1136 : audit [INF] from='client.? 172.21.15.175:0/4199038823' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi175-64735-12"}]: dispatch 2024-03-26T03:49:10.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:10 smithi175 bash[29284]: audit 2024-03-26T03:49:09.690369+0000 mon.c (mon.2) 162 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:10.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:10 smithi175 bash[29284]: audit 2024-03-26T03:49:09.693795+0000 mgr.y (mgr.24431) 186 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:49:10.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:10 smithi175 bash[29284]: audit 2024-03-26T03:49:10.397410+0000 mon.a (mon.0) 1133 : audit [INF] from='client.? 172.21.15.175:0/4199038823' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi175-64735-12"}]': finished 2024-03-26T03:49:10.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:10 smithi175 bash[29284]: audit 2024-03-26T03:49:10.397495+0000 mon.a (mon.0) 1134 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:10.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:10 smithi175 bash[29284]: cluster 2024-03-26T03:49:10.413351+0000 mon.a (mon.0) 1135 : cluster [DBG] osdmap e94: 8 total, 8 up, 8 in 2024-03-26T03:49:10.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:10 smithi175 bash[29284]: audit 2024-03-26T03:49:10.416574+0000 mon.a (mon.0) 1136 : audit [INF] from='client.? 172.21.15.175:0/4199038823' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi175-64735-12"}]: dispatch 2024-03-26T03:49:11.438 INFO:tasks.workunit.client.0.smithi175.stdout: api_wa api_watch_notify: Running main() from gmock_main.cc 2024-03-26T03:49:11.439 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2024-03-26T03:49:11.439 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: [----------] Global test environment set-up. 2024-03-26T03:49:11.439 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2024-03-26T03:49:11.439 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2024-03-26T03:49:11.439 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: watch_notify_test_cb 2024-03-26T03:49:11.439 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (403 ms) 2024-03-26T03:49:11.439 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2024-03-26T03:49:11.439 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94621885765072 err -107 2024-03-26T03:49:11.439 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-03-26T03:49:11.439 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (5 ms) 2024-03-26T03:49:11.439 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-03-26T03:49:11.439 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94621885765072 err -107 2024-03-26T03:49:11.439 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-03-26T03:49:11.440 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (18 ms) 2024-03-26T03:49:11.440 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2024-03-26T03:49:11.440 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: watch_notify2_test_cb from 24701 notify_id 335007449089 cookie 94621887154144 2024-03-26T03:49:11.440 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (6 ms) 2024-03-26T03:49:11.440 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-03-26T03:49:11.440 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: watch_notify2_test_cb from 24701 notify_id 335007449088 cookie 94621887170032 2024-03-26T03:49:11.440 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (39 ms) 2024-03-26T03:49:11.440 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2024-03-26T03:49:11.440 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: watch_notify2_test_cb from 24701 notify_id 335007449088 cookie 94621887173360 2024-03-26T03:49:11.440 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (19 ms) 2024-03-26T03:49:11.440 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-03-26T03:49:11.440 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: watch_notify2_test_cb from 24701 notify_id 335007449089 cookie 94621887173360 2024-03-26T03:49:11.440 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: watch_notify2_test_cb from 24701 notify_id 335007449089 cookie 94621887188448 2024-03-26T03:49:11.440 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (16 ms) 2024-03-26T03:49:11.440 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-03-26T03:49:11.441 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: watch_notify2_test_cb from 24701 notify_id 335007449088 cookie 94621887188448 2024-03-26T03:49:11.441 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: watch_notify2_test_cb from 24701 notify_id 339302416385 cookie 94621887188448 2024-03-26T03:49:11.441 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3020 ms) 2024-03-26T03:49:11.441 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-03-26T03:49:11.441 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2024-03-26T03:49:11.441 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94621887188368 err -107 2024-03-26T03:49:11.441 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: watch_notify2_test_cb from 24701 notify_id 369367187458 cookie 94621887188320 2024-03-26T03:49:11.441 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (5017 ms) 2024-03-26T03:49:11.441 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-03-26T03:49:11.441 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2024-03-26T03:49:11.441 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94621887188320 err -107 2024-03-26T03:49:11.441 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (1011 ms) 2024-03-26T03:49:11.441 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (9554 ms total) 2024-03-26T03:49:11.441 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: 2024-03-26T03:49:11.441 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2024-03-26T03:49:11.442 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-03-26T03:49:11.442 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: watch_notify_test_cb 2024-03-26T03:49:11.442 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (1120 ms) 2024-03-26T03:49:11.442 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (1120 ms total) 2024-03-26T03:49:11.442 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: 2024-03-26T03:49:11.442 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: [----------] Global test environment tear-down 2024-03-26T03:49:11.442 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (19079 ms total) 2024-03-26T03:49:11.442 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify: [ PASSED ] 11 tests. 2024-03-26T03:49:11.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:11 smithi190 bash[23392]: cluster 2024-03-26T03:49:10.578184+0000 mgr.y (mgr.24431) 187 : cluster [DBG] pgmap v138: 624 pgs: 72 creating+peering, 1 creating+activating, 551 active+clean; 145 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-03-26T03:49:11.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:11 smithi190 bash[23392]: audit 2024-03-26T03:49:10.610356+0000 mon.a (mon.0) 1137 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:49:11.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:11 smithi190 bash[23392]: audit 2024-03-26T03:49:10.692960+0000 mon.c (mon.2) 163 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:11.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:11 smithi190 bash[23392]: cluster 2024-03-26T03:49:11.369496+0000 mon.a (mon.0) 1138 : cluster [WRN] Health check update: 11 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:11.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:11 smithi190 bash[23392]: audit 2024-03-26T03:49:11.405318+0000 mon.a (mon.0) 1139 : audit [INF] from='client.? 172.21.15.175:0/4199038823' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi175-64735-12"}]': finished 2024-03-26T03:49:11.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:11 smithi190 bash[23392]: audit 2024-03-26T03:49:11.405381+0000 mon.a (mon.0) 1140 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-7", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:49:11.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:11 smithi190 bash[23392]: cluster 2024-03-26T03:49:11.410046+0000 mon.a (mon.0) 1141 : cluster [DBG] osdmap e95: 8 total, 8 up, 8 in 2024-03-26T03:49:11.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:11 smithi190 bash[23392]: audit 2024-03-26T03:49:11.418117+0000 mon.c (mon.2) 164 : audit [INF] from='client.? 172.21.15.175:0/4012108150' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:11.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:11 smithi190 bash[23392]: audit 2024-03-26T03:49:11.430073+0000 mon.a (mon.0) 1142 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-7"}]: dispatch 2024-03-26T03:49:11.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:11 smithi190 bash[23392]: audit 2024-03-26T03:49:11.441295+0000 mon.a (mon.0) 1143 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:11.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:11 smithi190 bash[23392]: audit 2024-03-26T03:49:11.448670+0000 mon.a (mon.0) 1144 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi175-64398-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:11.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:11 smithi190 bash[23392]: audit 2024-03-26T03:49:11.453450+0000 mon.b (mon.1) 90 : audit [INF] from='client.? 172.21.15.175:0/2431415732' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi175-64398-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:11.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:11 smithi175 bash[29284]: cluster 2024-03-26T03:49:10.578184+0000 mgr.y (mgr.24431) 187 : cluster [DBG] pgmap v138: 624 pgs: 72 creating+peering, 1 creating+activating, 551 active+clean; 145 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-03-26T03:49:11.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:11 smithi175 bash[29284]: audit 2024-03-26T03:49:10.610356+0000 mon.a (mon.0) 1137 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:49:11.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:11 smithi175 bash[29284]: audit 2024-03-26T03:49:10.692960+0000 mon.c (mon.2) 163 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:11.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:11 smithi175 bash[29284]: cluster 2024-03-26T03:49:11.369496+0000 mon.a (mon.0) 1138 : cluster [WRN] Health check update: 11 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:11.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:11 smithi175 bash[29284]: audit 2024-03-26T03:49:11.405318+0000 mon.a (mon.0) 1139 : audit [INF] from='client.? 172.21.15.175:0/4199038823' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi175-64735-12"}]': finished 2024-03-26T03:49:11.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:11 smithi175 bash[29284]: audit 2024-03-26T03:49:11.405381+0000 mon.a (mon.0) 1140 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-7", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:49:11.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:11 smithi175 bash[29284]: cluster 2024-03-26T03:49:11.410046+0000 mon.a (mon.0) 1141 : cluster [DBG] osdmap e95: 8 total, 8 up, 8 in 2024-03-26T03:49:11.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:11 smithi175 bash[29284]: audit 2024-03-26T03:49:11.418117+0000 mon.c (mon.2) 164 : audit [INF] from='client.? 172.21.15.175:0/4012108150' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:11.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:11 smithi175 bash[29284]: audit 2024-03-26T03:49:11.430073+0000 mon.a (mon.0) 1142 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-7"}]: dispatch 2024-03-26T03:49:11.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:11 smithi175 bash[29284]: audit 2024-03-26T03:49:11.441295+0000 mon.a (mon.0) 1143 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:11.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:11 smithi175 bash[29284]: audit 2024-03-26T03:49:11.448670+0000 mon.a (mon.0) 1144 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi175-64398-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:11.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:11 smithi175 bash[29284]: audit 2024-03-26T03:49:11.453450+0000 mon.b (mon.1) 90 : audit [INF] from='client.? 172.21.15.175:0/2431415732' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi175-64398-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:11.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:11 smithi175 bash[21527]: cluster 2024-03-26T03:49:10.578184+0000 mgr.y (mgr.24431) 187 : cluster [DBG] pgmap v138: 624 pgs: 72 creating+peering, 1 creating+activating, 551 active+clean; 145 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-03-26T03:49:11.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:11 smithi175 bash[21527]: audit 2024-03-26T03:49:10.610356+0000 mon.a (mon.0) 1137 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:49:11.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:11 smithi175 bash[21527]: audit 2024-03-26T03:49:10.692960+0000 mon.c (mon.2) 163 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:11.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:11 smithi175 bash[21527]: cluster 2024-03-26T03:49:11.369496+0000 mon.a (mon.0) 1138 : cluster [WRN] Health check update: 11 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:11.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:11 smithi175 bash[21527]: audit 2024-03-26T03:49:11.405318+0000 mon.a (mon.0) 1139 : audit [INF] from='client.? 172.21.15.175:0/4199038823' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi175-64735-12"}]': finished 2024-03-26T03:49:11.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:11 smithi175 bash[21527]: audit 2024-03-26T03:49:11.405381+0000 mon.a (mon.0) 1140 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-7", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:49:11.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:11 smithi175 bash[21527]: cluster 2024-03-26T03:49:11.410046+0000 mon.a (mon.0) 1141 : cluster [DBG] osdmap e95: 8 total, 8 up, 8 in 2024-03-26T03:49:11.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:11 smithi175 bash[21527]: audit 2024-03-26T03:49:11.418117+0000 mon.c (mon.2) 164 : audit [INF] from='client.? 172.21.15.175:0/4012108150' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:11.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:11 smithi175 bash[21527]: audit 2024-03-26T03:49:11.430073+0000 mon.a (mon.0) 1142 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-7"}]: dispatch 2024-03-26T03:49:11.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:11 smithi175 bash[21527]: audit 2024-03-26T03:49:11.441295+0000 mon.a (mon.0) 1143 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:11.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:11 smithi175 bash[21527]: audit 2024-03-26T03:49:11.448670+0000 mon.a (mon.0) 1144 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi175-64398-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:11.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:11 smithi175 bash[21527]: audit 2024-03-26T03:49:11.453450+0000 mon.b (mon.1) 90 : audit [INF] from='client.? 172.21.15.175:0/2431415732' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi175-64398-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:12.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:12 smithi190 bash[23392]: audit 2024-03-26T03:49:11.695694+0000 mon.c (mon.2) 165 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:12.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:12 smithi190 bash[23392]: cluster 2024-03-26T03:49:12.407332+0000 mon.a (mon.0) 1145 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-26T03:49:12.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:12 smithi190 bash[23392]: audit 2024-03-26T03:49:12.412193+0000 mon.a (mon.0) 1146 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-7"}]': finished 2024-03-26T03:49:12.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:12 smithi190 bash[23392]: audit 2024-03-26T03:49:12.412294+0000 mon.a (mon.0) 1147 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:12.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:12 smithi190 bash[23392]: audit 2024-03-26T03:49:12.412380+0000 mon.a (mon.0) 1148 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi175-64398-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:12.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:12 smithi190 bash[23392]: cluster 2024-03-26T03:49:12.416962+0000 mon.a (mon.0) 1149 : cluster [DBG] osdmap e96: 8 total, 8 up, 8 in 2024-03-26T03:49:12.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:12 smithi190 bash[23392]: audit 2024-03-26T03:49:12.624548+0000 mon.a (mon.0) 1150 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:49:12.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:12 smithi175 bash[21527]: audit 2024-03-26T03:49:11.695694+0000 mon.c (mon.2) 165 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:12.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:12 smithi175 bash[21527]: cluster 2024-03-26T03:49:12.407332+0000 mon.a (mon.0) 1145 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-26T03:49:12.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:12 smithi175 bash[21527]: audit 2024-03-26T03:49:12.412193+0000 mon.a (mon.0) 1146 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-7"}]': finished 2024-03-26T03:49:12.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:12 smithi175 bash[21527]: audit 2024-03-26T03:49:12.412294+0000 mon.a (mon.0) 1147 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:12.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:12 smithi175 bash[21527]: audit 2024-03-26T03:49:12.412380+0000 mon.a (mon.0) 1148 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi175-64398-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:12.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:12 smithi175 bash[21527]: cluster 2024-03-26T03:49:12.416962+0000 mon.a (mon.0) 1149 : cluster [DBG] osdmap e96: 8 total, 8 up, 8 in 2024-03-26T03:49:12.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:12 smithi175 bash[21527]: audit 2024-03-26T03:49:12.624548+0000 mon.a (mon.0) 1150 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:49:12.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:12 smithi175 bash[29284]: audit 2024-03-26T03:49:11.695694+0000 mon.c (mon.2) 165 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:12.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:12 smithi175 bash[29284]: cluster 2024-03-26T03:49:12.407332+0000 mon.a (mon.0) 1145 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-26T03:49:12.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:12 smithi175 bash[29284]: audit 2024-03-26T03:49:12.412193+0000 mon.a (mon.0) 1146 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-7"}]': finished 2024-03-26T03:49:12.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:12 smithi175 bash[29284]: audit 2024-03-26T03:49:12.412294+0000 mon.a (mon.0) 1147 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64886-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:12.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:12 smithi175 bash[29284]: audit 2024-03-26T03:49:12.412380+0000 mon.a (mon.0) 1148 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi175-64398-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:12.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:12 smithi175 bash[29284]: cluster 2024-03-26T03:49:12.416962+0000 mon.a (mon.0) 1149 : cluster [DBG] osdmap e96: 8 total, 8 up, 8 in 2024-03-26T03:49:12.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:12 smithi175 bash[29284]: audit 2024-03-26T03:49:12.624548+0000 mon.a (mon.0) 1150 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:49:13.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:13 smithi190 bash[23392]: cluster 2024-03-26T03:49:12.580207+0000 mgr.y (mgr.24431) 188 : cluster [DBG] pgmap v141: 720 pgs: 192 unknown, 40 creating+peering, 1 creating+activating, 487 active+clean; 145 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-26T03:49:13.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:13 smithi190 bash[23392]: audit 2024-03-26T03:49:12.703577+0000 mon.c (mon.2) 166 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:13.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:13 smithi190 bash[23392]: audit 2024-03-26T03:49:13.424457+0000 mon.a (mon.0) 1151 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:49:13.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:13 smithi190 bash[23392]: cluster 2024-03-26T03:49:13.443858+0000 mon.a (mon.0) 1152 : cluster [DBG] osdmap e97: 8 total, 8 up, 8 in 2024-03-26T03:49:13.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:13 smithi190 bash[23392]: audit 2024-03-26T03:49:13.446955+0000 mon.a (mon.0) 1153 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-7"}]: dispatch 2024-03-26T03:49:13.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:13 smithi190 bash[23392]: audit 2024-03-26T03:49:13.448244+0000 mon.a (mon.0) 1154 : audit [INF] from='client.? 172.21.15.175:0/1076098358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi175-65296-3"}]: dispatch 2024-03-26T03:49:13.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:13 smithi190 bash[23392]: audit 2024-03-26T03:49:13.459692+0000 mon.a (mon.0) 1155 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:13.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:13 smithi190 bash[23392]: audit 2024-03-26T03:49:13.460700+0000 mon.b (mon.1) 91 : audit [INF] from='client.? 172.21.15.175:0/1172143020' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:13.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:13 smithi175 bash[29284]: cluster 2024-03-26T03:49:12.580207+0000 mgr.y (mgr.24431) 188 : cluster [DBG] pgmap v141: 720 pgs: 192 unknown, 40 creating+peering, 1 creating+activating, 487 active+clean; 145 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-26T03:49:13.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:13 smithi175 bash[29284]: audit 2024-03-26T03:49:12.703577+0000 mon.c (mon.2) 166 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:13.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:13 smithi175 bash[29284]: audit 2024-03-26T03:49:13.424457+0000 mon.a (mon.0) 1151 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:49:13.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:13 smithi175 bash[29284]: cluster 2024-03-26T03:49:13.443858+0000 mon.a (mon.0) 1152 : cluster [DBG] osdmap e97: 8 total, 8 up, 8 in 2024-03-26T03:49:13.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:13 smithi175 bash[29284]: audit 2024-03-26T03:49:13.446955+0000 mon.a (mon.0) 1153 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-7"}]: dispatch 2024-03-26T03:49:13.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:13 smithi175 bash[29284]: audit 2024-03-26T03:49:13.448244+0000 mon.a (mon.0) 1154 : audit [INF] from='client.? 172.21.15.175:0/1076098358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi175-65296-3"}]: dispatch 2024-03-26T03:49:13.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:13 smithi175 bash[29284]: audit 2024-03-26T03:49:13.459692+0000 mon.a (mon.0) 1155 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:13.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:13 smithi175 bash[29284]: audit 2024-03-26T03:49:13.460700+0000 mon.b (mon.1) 91 : audit [INF] from='client.? 172.21.15.175:0/1172143020' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:13.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:13 smithi175 bash[21527]: cluster 2024-03-26T03:49:12.580207+0000 mgr.y (mgr.24431) 188 : cluster [DBG] pgmap v141: 720 pgs: 192 unknown, 40 creating+peering, 1 creating+activating, 487 active+clean; 145 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-26T03:49:13.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:13 smithi175 bash[21527]: audit 2024-03-26T03:49:12.703577+0000 mon.c (mon.2) 166 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:13.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:13 smithi175 bash[21527]: audit 2024-03-26T03:49:13.424457+0000 mon.a (mon.0) 1151 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:49:13.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:13 smithi175 bash[21527]: cluster 2024-03-26T03:49:13.443858+0000 mon.a (mon.0) 1152 : cluster [DBG] osdmap e97: 8 total, 8 up, 8 in 2024-03-26T03:49:13.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:13 smithi175 bash[21527]: audit 2024-03-26T03:49:13.446955+0000 mon.a (mon.0) 1153 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-7"}]: dispatch 2024-03-26T03:49:13.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:13 smithi175 bash[21527]: audit 2024-03-26T03:49:13.448244+0000 mon.a (mon.0) 1154 : audit [INF] from='client.? 172.21.15.175:0/1076098358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi175-65296-3"}]: dispatch 2024-03-26T03:49:13.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:13 smithi175 bash[21527]: audit 2024-03-26T03:49:13.459692+0000 mon.a (mon.0) 1155 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:13.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:13 smithi175 bash[21527]: audit 2024-03-26T03:49:13.460700+0000 mon.b (mon.1) 91 : audit [INF] from='client.? 172.21.15.175:0/1172143020' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:14.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:14 smithi190 bash[23392]: audit 2024-03-26T03:49:13.705748+0000 mon.c (mon.2) 167 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:14.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:14 smithi190 bash[23392]: audit 2024-03-26T03:49:14.428578+0000 mon.a (mon.0) 1156 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-7"}]': finished 2024-03-26T03:49:14.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:14 smithi190 bash[23392]: audit 2024-03-26T03:49:14.428682+0000 mon.a (mon.0) 1157 : audit [INF] from='client.? 172.21.15.175:0/1076098358' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi175-65296-3"}]': finished 2024-03-26T03:49:14.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:14 smithi190 bash[23392]: audit 2024-03-26T03:49:14.428757+0000 mon.a (mon.0) 1158 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:14.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:14 smithi190 bash[23392]: cluster 2024-03-26T03:49:14.432817+0000 mon.a (mon.0) 1159 : cluster [DBG] osdmap e98: 8 total, 8 up, 8 in 2024-03-26T03:49:14.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:14 smithi190 bash[23392]: audit 2024-03-26T03:49:14.436144+0000 mon.a (mon.0) 1160 : audit [INF] from='client.? 172.21.15.175:0/1076098358' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi175-65296-3"}]: dispatch 2024-03-26T03:49:14.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:14 smithi190 bash[23392]: audit 2024-03-26T03:49:14.441122+0000 mon.a (mon.0) 1161 : audit [INF] from='client.? 172.21.15.175:0/2142539938' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi175-64406-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:14.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:14 smithi190 bash[23392]: audit 2024-03-26T03:49:14.492543+0000 mon.a (mon.0) 1162 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi175-64398-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:14.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:14 smithi190 bash[23392]: audit 2024-03-26T03:49:14.496457+0000 mon.b (mon.1) 92 : audit [INF] from='client.? 172.21.15.175:0/507486799' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi175-64398-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:14.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:14 smithi175 bash[21527]: audit 2024-03-26T03:49:13.705748+0000 mon.c (mon.2) 167 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:14.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:14 smithi175 bash[21527]: audit 2024-03-26T03:49:14.428578+0000 mon.a (mon.0) 1156 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-7"}]': finished 2024-03-26T03:49:14.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:14 smithi175 bash[21527]: audit 2024-03-26T03:49:14.428682+0000 mon.a (mon.0) 1157 : audit [INF] from='client.? 172.21.15.175:0/1076098358' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi175-65296-3"}]': finished 2024-03-26T03:49:14.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:14 smithi175 bash[21527]: audit 2024-03-26T03:49:14.428757+0000 mon.a (mon.0) 1158 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:14.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:14 smithi175 bash[21527]: cluster 2024-03-26T03:49:14.432817+0000 mon.a (mon.0) 1159 : cluster [DBG] osdmap e98: 8 total, 8 up, 8 in 2024-03-26T03:49:14.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:14 smithi175 bash[21527]: audit 2024-03-26T03:49:14.436144+0000 mon.a (mon.0) 1160 : audit [INF] from='client.? 172.21.15.175:0/1076098358' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi175-65296-3"}]: dispatch 2024-03-26T03:49:14.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:14 smithi175 bash[21527]: audit 2024-03-26T03:49:14.441122+0000 mon.a (mon.0) 1161 : audit [INF] from='client.? 172.21.15.175:0/2142539938' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi175-64406-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:14.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:14 smithi175 bash[21527]: audit 2024-03-26T03:49:14.492543+0000 mon.a (mon.0) 1162 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi175-64398-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:14.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:14 smithi175 bash[21527]: audit 2024-03-26T03:49:14.496457+0000 mon.b (mon.1) 92 : audit [INF] from='client.? 172.21.15.175:0/507486799' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi175-64398-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:14.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:14 smithi175 bash[29284]: audit 2024-03-26T03:49:13.705748+0000 mon.c (mon.2) 167 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:14.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:14 smithi175 bash[29284]: audit 2024-03-26T03:49:14.428578+0000 mon.a (mon.0) 1156 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-7"}]': finished 2024-03-26T03:49:14.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:14 smithi175 bash[29284]: audit 2024-03-26T03:49:14.428682+0000 mon.a (mon.0) 1157 : audit [INF] from='client.? 172.21.15.175:0/1076098358' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi175-65296-3"}]': finished 2024-03-26T03:49:14.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:14 smithi175 bash[29284]: audit 2024-03-26T03:49:14.428757+0000 mon.a (mon.0) 1158 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:14.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:14 smithi175 bash[29284]: cluster 2024-03-26T03:49:14.432817+0000 mon.a (mon.0) 1159 : cluster [DBG] osdmap e98: 8 total, 8 up, 8 in 2024-03-26T03:49:14.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:14 smithi175 bash[29284]: audit 2024-03-26T03:49:14.436144+0000 mon.a (mon.0) 1160 : audit [INF] from='client.? 172.21.15.175:0/1076098358' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi175-65296-3"}]: dispatch 2024-03-26T03:49:14.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:14 smithi175 bash[29284]: audit 2024-03-26T03:49:14.441122+0000 mon.a (mon.0) 1161 : audit [INF] from='client.? 172.21.15.175:0/2142539938' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi175-64406-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:14.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:14 smithi175 bash[29284]: audit 2024-03-26T03:49:14.492543+0000 mon.a (mon.0) 1162 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi175-64398-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:14.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:14 smithi175 bash[29284]: audit 2024-03-26T03:49:14.496457+0000 mon.b (mon.1) 92 : audit [INF] from='client.? 172.21.15.175:0/507486799' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi175-64398-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:15.465 INFO:tasks.workunit.client.0.smithi175.stdout: ec_list: Running main() from gmock_main.cc 2024-03-26T03:49:15.465 INFO:tasks.workunit.client.0.smithi175.stdout: ec_list: [==========] Running 3 tests from 1 test suite. 2024-03-26T03:49:15.465 INFO:tasks.workunit.client.0.smithi175.stdout: ec_list: [----------] Global test environment set-up. 2024-03-26T03:49:15.465 INFO:tasks.workunit.client.0.smithi175.stdout: ec_list: [----------] 3 tests from NeoradosECList 2024-03-26T03:49:15.465 INFO:tasks.workunit.client.0.smithi175.stdout: ec_list: [ RUN ] NeoradosECList.ListObjects 2024-03-26T03:49:15.465 INFO:tasks.workunit.client.0.smithi175.stdout: ec_list: [ OK ] NeoradosECList.ListObjects (6782 ms) 2024-03-26T03:49:15.466 INFO:tasks.workunit.client.0.smithi175.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsNS 2024-03-26T03:49:15.466 INFO:tasks.workunit.client.0.smithi175.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsNS (7024 ms) 2024-03-26T03:49:15.466 INFO:tasks.workunit.client.0.smithi175.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsMany 2024-03-26T03:49:15.466 INFO:tasks.workunit.client.0.smithi175.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsMany (9060 ms) 2024-03-26T03:49:15.466 INFO:tasks.workunit.client.0.smithi175.stdout: ec_list: [----------] 3 tests from NeoradosECList (22866 ms total) 2024-03-26T03:49:15.466 INFO:tasks.workunit.client.0.smithi175.stdout: ec_list: 2024-03-26T03:49:15.466 INFO:tasks.workunit.client.0.smithi175.stdout: ec_list: [----------] Global test environment tear-down 2024-03-26T03:49:15.466 INFO:tasks.workunit.client.0.smithi175.stdout: ec_list: [==========] 3 tests from 1 test suite ran. (22866 ms total) 2024-03-26T03:49:15.466 INFO:tasks.workunit.client.0.smithi175.stdout: ec_list: [ PASSED ] 3 tests. 2024-03-26T03:49:15.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:15 smithi190 bash[23392]: cluster 2024-03-26T03:49:14.581897+0000 mgr.y (mgr.24431) 189 : cluster [DBG] pgmap v144: 712 pgs: 103 unknown, 85 creating+peering, 524 active+clean; 145 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 B/s wr, 2 op/s 2024-03-26T03:49:15.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:15 smithi190 bash[23392]: audit 2024-03-26T03:49:14.710321+0000 mon.c (mon.2) 168 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:15.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:15 smithi190 bash[23392]: audit 2024-03-26T03:49:15.441864+0000 mon.a (mon.0) 1163 : audit [INF] from='client.? 172.21.15.175:0/1076098358' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi175-65296-3"}]': finished 2024-03-26T03:49:15.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:15 smithi190 bash[23392]: audit 2024-03-26T03:49:15.441957+0000 mon.a (mon.0) 1164 : audit [INF] from='client.? 172.21.15.175:0/2142539938' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi175-64406-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:15.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:15 smithi190 bash[23392]: audit 2024-03-26T03:49:15.442051+0000 mon.a (mon.0) 1165 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi175-64398-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:15.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:15 smithi190 bash[23392]: cluster 2024-03-26T03:49:15.448934+0000 mon.a (mon.0) 1166 : cluster [DBG] osdmap e99: 8 total, 8 up, 8 in 2024-03-26T03:49:15.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:15 smithi175 bash[21527]: cluster 2024-03-26T03:49:14.581897+0000 mgr.y (mgr.24431) 189 : cluster [DBG] pgmap v144: 712 pgs: 103 unknown, 85 creating+peering, 524 active+clean; 145 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 B/s wr, 2 op/s 2024-03-26T03:49:15.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:15 smithi175 bash[21527]: audit 2024-03-26T03:49:14.710321+0000 mon.c (mon.2) 168 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:15.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:15 smithi175 bash[21527]: audit 2024-03-26T03:49:15.441864+0000 mon.a (mon.0) 1163 : audit [INF] from='client.? 172.21.15.175:0/1076098358' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi175-65296-3"}]': finished 2024-03-26T03:49:15.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:15 smithi175 bash[21527]: audit 2024-03-26T03:49:15.441957+0000 mon.a (mon.0) 1164 : audit [INF] from='client.? 172.21.15.175:0/2142539938' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi175-64406-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:15.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:15 smithi175 bash[21527]: audit 2024-03-26T03:49:15.442051+0000 mon.a (mon.0) 1165 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi175-64398-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:15.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:15 smithi175 bash[21527]: cluster 2024-03-26T03:49:15.448934+0000 mon.a (mon.0) 1166 : cluster [DBG] osdmap e99: 8 total, 8 up, 8 in 2024-03-26T03:49:15.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:15 smithi175 bash[29284]: cluster 2024-03-26T03:49:14.581897+0000 mgr.y (mgr.24431) 189 : cluster [DBG] pgmap v144: 712 pgs: 103 unknown, 85 creating+peering, 524 active+clean; 145 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 B/s wr, 2 op/s 2024-03-26T03:49:15.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:15 smithi175 bash[29284]: audit 2024-03-26T03:49:14.710321+0000 mon.c (mon.2) 168 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:15.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:15 smithi175 bash[29284]: audit 2024-03-26T03:49:15.441864+0000 mon.a (mon.0) 1163 : audit [INF] from='client.? 172.21.15.175:0/1076098358' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi175-65296-3"}]': finished 2024-03-26T03:49:15.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:15 smithi175 bash[29284]: audit 2024-03-26T03:49:15.441957+0000 mon.a (mon.0) 1164 : audit [INF] from='client.? 172.21.15.175:0/2142539938' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi175-64406-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:15.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:15 smithi175 bash[29284]: audit 2024-03-26T03:49:15.442051+0000 mon.a (mon.0) 1165 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi175-64398-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:15.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:15 smithi175 bash[29284]: cluster 2024-03-26T03:49:15.448934+0000 mon.a (mon.0) 1166 : cluster [DBG] osdmap e99: 8 total, 8 up, 8 in 2024-03-26T03:49:16.444 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_write_operations: Running main() from gmock_main.cc 2024-03-26T03:49:16.444 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2024-03-26T03:49:16.444 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_write_operations: [----------] Global test environment set-up. 2024-03-26T03:49:16.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2024-03-26T03:49:16.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2024-03-26T03:49:16.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-03-26T03:49:16.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2024-03-26T03:49:16.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_write_operations: 2024-03-26T03:49:16.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2024-03-26T03:49:16.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2024-03-26T03:49:16.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (2900 ms) 2024-03-26T03:49:16.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-03-26T03:49:16.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (3090 ms) 2024-03-26T03:49:16.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2024-03-26T03:49:16.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (2979 ms) 2024-03-26T03:49:16.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2024-03-26T03:49:16.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (3026 ms) 2024-03-26T03:49:16.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2024-03-26T03:49:16.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (2970 ms) 2024-03-26T03:49:16.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2024-03-26T03:49:16.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (3038 ms) 2024-03-26T03:49:16.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2024-03-26T03:49:16.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (5988 ms) 2024-03-26T03:49:16.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (24022 ms total) 2024-03-26T03:49:16.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_write_operations: 2024-03-26T03:49:16.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_write_operations: [----------] Global test environment tear-down 2024-03-26T03:49:16.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (24023 ms total) 2024-03-26T03:49:16.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2024-03-26T03:49:16.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:16 smithi190 bash[23392]: audit 2024-03-26T03:49:15.714671+0000 mon.c (mon.2) 169 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:16.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:16 smithi190 bash[23392]: cluster 2024-03-26T03:49:16.370244+0000 mon.a (mon.0) 1167 : cluster [WRN] Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:16.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:16 smithi190 bash[23392]: cluster 2024-03-26T03:49:16.378391+0000 mon.a (mon.0) 1168 : cluster [DBG] osdmap e100: 8 total, 8 up, 8 in 2024-03-26T03:49:16.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:16 smithi190 bash[23392]: audit 2024-03-26T03:49:16.397768+0000 mon.a (mon.0) 1169 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:16.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:16 smithi175 bash[29284]: audit 2024-03-26T03:49:15.714671+0000 mon.c (mon.2) 169 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:16.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:16 smithi175 bash[29284]: cluster 2024-03-26T03:49:16.370244+0000 mon.a (mon.0) 1167 : cluster [WRN] Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:16.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:16 smithi175 bash[29284]: cluster 2024-03-26T03:49:16.378391+0000 mon.a (mon.0) 1168 : cluster [DBG] osdmap e100: 8 total, 8 up, 8 in 2024-03-26T03:49:16.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:16 smithi175 bash[29284]: audit 2024-03-26T03:49:16.397768+0000 mon.a (mon.0) 1169 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:16.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:16 smithi175 bash[21527]: audit 2024-03-26T03:49:15.714671+0000 mon.c (mon.2) 169 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:16.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:16 smithi175 bash[21527]: cluster 2024-03-26T03:49:16.370244+0000 mon.a (mon.0) 1167 : cluster [WRN] Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:16.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:16 smithi175 bash[21527]: cluster 2024-03-26T03:49:16.378391+0000 mon.a (mon.0) 1168 : cluster [DBG] osdmap e100: 8 total, 8 up, 8 in 2024-03-26T03:49:16.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:16 smithi175 bash[21527]: audit 2024-03-26T03:49:16.397768+0000 mon.a (mon.0) 1169 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:17.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:17 smithi175 bash[29284]: cluster 2024-03-26T03:49:16.583233+0000 mgr.y (mgr.24431) 190 : cluster [DBG] pgmap v147: 584 pgs: 64 unknown, 520 active+clean; 145 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-03-26T03:49:17.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:17 smithi175 bash[29284]: audit 2024-03-26T03:49:16.718216+0000 mon.c (mon.2) 170 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:17.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:17 smithi175 bash[29284]: audit 2024-03-26T03:49:17.386716+0000 mon.a (mon.0) 1170 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:17.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:17 smithi175 bash[29284]: cluster 2024-03-26T03:49:17.394282+0000 mon.a (mon.0) 1171 : cluster [DBG] osdmap e101: 8 total, 8 up, 8 in 2024-03-26T03:49:17.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:17 smithi175 bash[29284]: audit 2024-03-26T03:49:17.428584+0000 mon.b (mon.1) 93 : audit [INF] from='client.? 172.21.15.175:0/133141350' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi175-64406-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:17.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:17 smithi175 bash[29284]: audit 2024-03-26T03:49:17.428951+0000 mon.b (mon.1) 94 : audit [INF] from='client.? 172.21.15.175:0/1311370472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi175-64398-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:17.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:17 smithi175 bash[29284]: audit 2024-03-26T03:49:17.432500+0000 mon.a (mon.0) 1172 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi175-64406-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:17.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:17 smithi175 bash[29284]: audit 2024-03-26T03:49:17.432868+0000 mon.a (mon.0) 1173 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi175-64398-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:17.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:17 smithi175 bash[29284]: audit 2024-03-26T03:49:17.499788+0000 mon.a (mon.0) 1174 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:49:17.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:17 smithi175 bash[29284]: audit 2024-03-26T03:49:17.612134+0000 mon.c (mon.2) 171 : audit [INF] from='client.? 172.21.15.175:0/4207065844' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4"}]: dispatch 2024-03-26T03:49:17.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:17 smithi175 bash[29284]: audit 2024-03-26T03:49:17.613619+0000 mon.a (mon.0) 1175 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4"}]: dispatch 2024-03-26T03:49:17.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:17 smithi190 bash[23392]: cluster 2024-03-26T03:49:16.583233+0000 mgr.y (mgr.24431) 190 : cluster [DBG] pgmap v147: 584 pgs: 64 unknown, 520 active+clean; 145 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-03-26T03:49:17.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:17 smithi190 bash[23392]: audit 2024-03-26T03:49:16.718216+0000 mon.c (mon.2) 170 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:17.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:17 smithi190 bash[23392]: audit 2024-03-26T03:49:17.386716+0000 mon.a (mon.0) 1170 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:17.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:17 smithi190 bash[23392]: cluster 2024-03-26T03:49:17.394282+0000 mon.a (mon.0) 1171 : cluster [DBG] osdmap e101: 8 total, 8 up, 8 in 2024-03-26T03:49:17.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:17 smithi190 bash[23392]: audit 2024-03-26T03:49:17.428584+0000 mon.b (mon.1) 93 : audit [INF] from='client.? 172.21.15.175:0/133141350' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi175-64406-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:17.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:17 smithi190 bash[23392]: audit 2024-03-26T03:49:17.428951+0000 mon.b (mon.1) 94 : audit [INF] from='client.? 172.21.15.175:0/1311370472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi175-64398-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:17.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:17 smithi190 bash[23392]: audit 2024-03-26T03:49:17.432500+0000 mon.a (mon.0) 1172 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi175-64406-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:17.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:17 smithi190 bash[23392]: audit 2024-03-26T03:49:17.432868+0000 mon.a (mon.0) 1173 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi175-64398-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:17.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:17 smithi190 bash[23392]: audit 2024-03-26T03:49:17.499788+0000 mon.a (mon.0) 1174 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:49:17.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:17 smithi190 bash[23392]: audit 2024-03-26T03:49:17.612134+0000 mon.c (mon.2) 171 : audit [INF] from='client.? 172.21.15.175:0/4207065844' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4"}]: dispatch 2024-03-26T03:49:17.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:17 smithi190 bash[23392]: audit 2024-03-26T03:49:17.613619+0000 mon.a (mon.0) 1175 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4"}]: dispatch 2024-03-26T03:49:17.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:17 smithi175 bash[21527]: cluster 2024-03-26T03:49:16.583233+0000 mgr.y (mgr.24431) 190 : cluster [DBG] pgmap v147: 584 pgs: 64 unknown, 520 active+clean; 145 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-03-26T03:49:17.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:17 smithi175 bash[21527]: audit 2024-03-26T03:49:16.718216+0000 mon.c (mon.2) 170 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:17.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:17 smithi175 bash[21527]: audit 2024-03-26T03:49:17.386716+0000 mon.a (mon.0) 1170 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:17.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:17 smithi175 bash[21527]: cluster 2024-03-26T03:49:17.394282+0000 mon.a (mon.0) 1171 : cluster [DBG] osdmap e101: 8 total, 8 up, 8 in 2024-03-26T03:49:17.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:17 smithi175 bash[21527]: audit 2024-03-26T03:49:17.428584+0000 mon.b (mon.1) 93 : audit [INF] from='client.? 172.21.15.175:0/133141350' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi175-64406-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:17.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:17 smithi175 bash[21527]: audit 2024-03-26T03:49:17.428951+0000 mon.b (mon.1) 94 : audit [INF] from='client.? 172.21.15.175:0/1311370472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi175-64398-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:17.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:17 smithi175 bash[21527]: audit 2024-03-26T03:49:17.432500+0000 mon.a (mon.0) 1172 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi175-64406-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:17.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:17 smithi175 bash[21527]: audit 2024-03-26T03:49:17.432868+0000 mon.a (mon.0) 1173 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi175-64398-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:17.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:17 smithi175 bash[21527]: audit 2024-03-26T03:49:17.499788+0000 mon.a (mon.0) 1174 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:49:17.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:17 smithi175 bash[21527]: audit 2024-03-26T03:49:17.612134+0000 mon.c (mon.2) 171 : audit [INF] from='client.? 172.21.15.175:0/4207065844' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4"}]: dispatch 2024-03-26T03:49:17.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:17 smithi175 bash[21527]: audit 2024-03-26T03:49:17.613619+0000 mon.a (mon.0) 1175 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4"}]: dispatch 2024-03-26T03:49:18.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:18 smithi190 bash[23392]: audit 2024-03-26T03:49:17.719996+0000 mon.c (mon.2) 172 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:18.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:18 smithi190 bash[23392]: audit 2024-03-26T03:49:18.392143+0000 mon.a (mon.0) 1176 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi175-64406-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:18.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:18 smithi190 bash[23392]: audit 2024-03-26T03:49:18.392247+0000 mon.a (mon.0) 1177 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi175-64398-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:18.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:18 smithi190 bash[23392]: audit 2024-03-26T03:49:18.392323+0000 mon.a (mon.0) 1178 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-9", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:49:18.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:18 smithi190 bash[23392]: audit 2024-03-26T03:49:18.392389+0000 mon.a (mon.0) 1179 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4"}]': finished 2024-03-26T03:49:18.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:18 smithi190 bash[23392]: cluster 2024-03-26T03:49:18.397372+0000 mon.a (mon.0) 1180 : cluster [DBG] osdmap e102: 8 total, 8 up, 8 in 2024-03-26T03:49:18.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:18 smithi190 bash[23392]: audit 2024-03-26T03:49:18.418734+0000 mon.c (mon.2) 173 : audit [INF] from='client.? 172.21.15.175:0/4207065844' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4", "tierpool": "test-rados-api-smithi175-64770-6"}]: dispatch 2024-03-26T03:49:18.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:18 smithi190 bash[23392]: audit 2024-03-26T03:49:18.419913+0000 mon.c (mon.2) 174 : audit [INF] from='client.? 172.21.15.175:0/2241429380' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:18.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:18 smithi190 bash[23392]: audit 2024-03-26T03:49:18.420697+0000 mon.a (mon.0) 1181 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-9"}]: dispatch 2024-03-26T03:49:18.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:18 smithi190 bash[23392]: audit 2024-03-26T03:49:18.421097+0000 mon.a (mon.0) 1182 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4", "tierpool": "test-rados-api-smithi175-64770-6"}]: dispatch 2024-03-26T03:49:18.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:18 smithi190 bash[23392]: audit 2024-03-26T03:49:18.421434+0000 mon.a (mon.0) 1183 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:18.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:18 smithi175 bash[29284]: audit 2024-03-26T03:49:17.719996+0000 mon.c (mon.2) 172 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:18.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:18 smithi175 bash[29284]: audit 2024-03-26T03:49:18.392143+0000 mon.a (mon.0) 1176 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi175-64406-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:18.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:18 smithi175 bash[29284]: audit 2024-03-26T03:49:18.392247+0000 mon.a (mon.0) 1177 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi175-64398-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:18.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:18 smithi175 bash[29284]: audit 2024-03-26T03:49:18.392323+0000 mon.a (mon.0) 1178 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-9", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:49:18.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:18 smithi175 bash[29284]: audit 2024-03-26T03:49:18.392389+0000 mon.a (mon.0) 1179 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4"}]': finished 2024-03-26T03:49:18.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:18 smithi175 bash[29284]: cluster 2024-03-26T03:49:18.397372+0000 mon.a (mon.0) 1180 : cluster [DBG] osdmap e102: 8 total, 8 up, 8 in 2024-03-26T03:49:18.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:18 smithi175 bash[29284]: audit 2024-03-26T03:49:18.418734+0000 mon.c (mon.2) 173 : audit [INF] from='client.? 172.21.15.175:0/4207065844' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4", "tierpool": "test-rados-api-smithi175-64770-6"}]: dispatch 2024-03-26T03:49:18.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:18 smithi175 bash[29284]: audit 2024-03-26T03:49:18.419913+0000 mon.c (mon.2) 174 : audit [INF] from='client.? 172.21.15.175:0/2241429380' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:18.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:18 smithi175 bash[29284]: audit 2024-03-26T03:49:18.420697+0000 mon.a (mon.0) 1181 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-9"}]: dispatch 2024-03-26T03:49:18.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:18 smithi175 bash[29284]: audit 2024-03-26T03:49:18.421097+0000 mon.a (mon.0) 1182 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4", "tierpool": "test-rados-api-smithi175-64770-6"}]: dispatch 2024-03-26T03:49:18.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:18 smithi175 bash[29284]: audit 2024-03-26T03:49:18.421434+0000 mon.a (mon.0) 1183 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:18.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:18 smithi175 bash[21527]: audit 2024-03-26T03:49:17.719996+0000 mon.c (mon.2) 172 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:18.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:18 smithi175 bash[21527]: audit 2024-03-26T03:49:18.392143+0000 mon.a (mon.0) 1176 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi175-64406-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:18.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:18 smithi175 bash[21527]: audit 2024-03-26T03:49:18.392247+0000 mon.a (mon.0) 1177 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi175-64398-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:18.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:18 smithi175 bash[21527]: audit 2024-03-26T03:49:18.392323+0000 mon.a (mon.0) 1178 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-9", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:49:18.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:18 smithi175 bash[21527]: audit 2024-03-26T03:49:18.392389+0000 mon.a (mon.0) 1179 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4"}]': finished 2024-03-26T03:49:18.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:18 smithi175 bash[21527]: cluster 2024-03-26T03:49:18.397372+0000 mon.a (mon.0) 1180 : cluster [DBG] osdmap e102: 8 total, 8 up, 8 in 2024-03-26T03:49:18.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:18 smithi175 bash[21527]: audit 2024-03-26T03:49:18.418734+0000 mon.c (mon.2) 173 : audit [INF] from='client.? 172.21.15.175:0/4207065844' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4", "tierpool": "test-rados-api-smithi175-64770-6"}]: dispatch 2024-03-26T03:49:18.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:18 smithi175 bash[21527]: audit 2024-03-26T03:49:18.419913+0000 mon.c (mon.2) 174 : audit [INF] from='client.? 172.21.15.175:0/2241429380' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:18.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:18 smithi175 bash[21527]: audit 2024-03-26T03:49:18.420697+0000 mon.a (mon.0) 1181 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-9"}]: dispatch 2024-03-26T03:49:18.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:18 smithi175 bash[21527]: audit 2024-03-26T03:49:18.421097+0000 mon.a (mon.0) 1182 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4", "tierpool": "test-rados-api-smithi175-64770-6"}]: dispatch 2024-03-26T03:49:18.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:18 smithi175 bash[21527]: audit 2024-03-26T03:49:18.421434+0000 mon.a (mon.0) 1183 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:18.994 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:49:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:49:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:49:19.964 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: [==========] Running 4 tests from 1 test suite. 2024-03-26T03:49:19.964 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: [----------] Global test environment set-up. 2024-03-26T03:49:19.965 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: [----------] 4 tests from LibRadosService 2024-03-26T03:49:19.965 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2024-03-26T03:49:19.965 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5095 ms) 2024-03-26T03:49:19.965 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2024-03-26T03:49:19.965 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: [ OK ] LibRadosService.RegisterLate (73 ms) 2024-03-26T03:49:19.965 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2024-03-26T03:49:19.965 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: cluster: 2024-03-26T03:49:19.965 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: id: 762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T03:49:19.965 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: health: HEALTH_WARN 2024-03-26T03:49:19.965 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: 17 pool(s) do not have an application enabled 2024-03-26T03:49:19.965 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: 2024-03-26T03:49:19.965 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: services: 2024-03-26T03:49:19.965 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 12m) 2024-03-26T03:49:19.965 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: mgr: y(active, since 3m), standbys: x 2024-03-26T03:49:19.966 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: osd: 8 osds: 8 up (since 5m), 8 in (since 5m) 2024-03-26T03:49:19.966 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-03-26T03:49:19.966 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-03-26T03:49:19.966 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: 2024-03-26T03:49:19.966 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: data: 2024-03-26T03:49:19.966 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: pools: 33 pools, 948 pgs 2024-03-26T03:49:19.966 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: objects: 236 objects, 583 KiB 2024-03-26T03:49:19.966 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: usage: 236 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:49:19.966 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: pgs: 45.570% pgs unknown 2024-03-26T03:49:19.966 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: 25.738% pgs not active 2024-03-26T03:49:19.966 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: 432 unknown 2024-03-26T03:49:19.966 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: 263 active+clean 2024-03-26T03:49:19.966 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: 180 creating+activating 2024-03-26T03:49:19.966 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: 64 creating+peering 2024-03-26T03:49:19.966 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: 9 active 2024-03-26T03:49:19.967 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: 2024-03-26T03:49:19.967 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: io: 2024-03-26T03:49:19.967 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: client: 2.7 KiB/s rd, 2.7 KiB/s wr, 2 op/s rd, 2 op/s wr 2024-03-26T03:49:19.967 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: 2024-03-26T03:49:19.967 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: 2024-03-26T03:49:19.967 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: cluster: 2024-03-26T03:49:19.967 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: id: 762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T03:49:19.967 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: health: HEALTH_WARN 2024-03-26T03:49:19.967 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: 12 pool(s) do not have an application enabled 2024-03-26T03:49:19.967 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: 2024-03-26T03:49:19.967 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: services: 2024-03-26T03:49:19.967 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 12m) 2024-03-26T03:49:19.967 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: mgr: y(active, since 3m), standbys: x 2024-03-26T03:49:19.967 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: osd: 8 osds: 8 up (since 5m), 8 in (since 5m) 2024-03-26T03:49:19.968 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2024-03-26T03:49:19.968 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-03-26T03:49:19.968 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-03-26T03:49:19.968 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: 2024-03-26T03:49:19.968 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: data: 2024-03-26T03:49:19.968 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: pools: 28 pools, 740 pgs 2024-03-26T03:49:19.968 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: objects: 235 objects, 583 KiB 2024-03-26T03:49:19.968 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: usage: 236 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:49:19.968 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: pgs: 43.243% pgs unknown 2024-03-26T03:49:19.968 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: 25.135% pgs not active 2024-03-26T03:49:19.968 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: 320 unknown 2024-03-26T03:49:19.968 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: 228 active+clean 2024-03-26T03:49:19.968 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: 122 creating+activating 2024-03-26T03:49:19.968 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: 64 creating+peering 2024-03-26T03:49:19.968 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: 6 active 2024-03-26T03:49:19.969 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: 2024-03-26T03:49:19.969 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: io: 2024-03-26T03:49:19.969 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: client: 2.7 KiB/s rd, 2.7 KiB/s wr, 2 op/s rd, 2 op/s wr 2024-03-26T03:49:19.969 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: 2024-03-26T03:49:19.969 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: 2024-03-26T03:49:19.969 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: [ OK ] LibRadosService.StatusFormat (2328 ms) 2024-03-26T03:49:19.969 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: [ RUN ] LibRadosService.Status 2024-03-26T03:49:19.969 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: [ OK ] LibRadosService.Status (20041 ms) 2024-03-26T03:49:19.969 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: [----------] 4 tests from LibRadosService (27537 ms total) 2024-03-26T03:49:19.970 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: 2024-03-26T03:49:19.970 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: [----------] Global test environment tear-down 2024-03-26T03:49:19.970 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: [==========] 4 tests from 1 test suite ran. (27537 ms total) 2024-03-26T03:49:19.971 INFO:tasks.workunit.client.0.smithi175.stdout: api_service: [ PASSED ] 4 tests. 2024-03-26T03:49:19.990 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:49:19 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:49:19.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:19 smithi190 bash[23392]: cluster 2024-03-26T03:49:18.584312+0000 mgr.y (mgr.24431) 191 : cluster [DBG] pgmap v150: 680 pgs: 224 unknown, 456 active+clean; 145 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-26T03:49:19.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:19 smithi190 bash[23392]: audit 2024-03-26T03:49:18.724322+0000 mon.c (mon.2) 175 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:19.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:19 smithi190 bash[23392]: audit 2024-03-26T03:49:19.057846+0000 mon.a (mon.0) 1184 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:49:19.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:19 smithi190 bash[23392]: audit 2024-03-26T03:49:19.060074+0000 mon.c (mon.2) 176 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:49:19.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:19 smithi190 bash[23392]: cluster 2024-03-26T03:49:19.393288+0000 mon.a (mon.0) 1185 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:49:19.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:19 smithi190 bash[23392]: audit 2024-03-26T03:49:19.395523+0000 mon.a (mon.0) 1186 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-9"}]': finished 2024-03-26T03:49:19.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:19 smithi190 bash[23392]: audit 2024-03-26T03:49:19.395593+0000 mon.a (mon.0) 1187 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4", "tierpool": "test-rados-api-smithi175-64770-6"}]': finished 2024-03-26T03:49:19.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:19 smithi190 bash[23392]: audit 2024-03-26T03:49:19.395676+0000 mon.a (mon.0) 1188 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:19.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:19 smithi190 bash[23392]: cluster 2024-03-26T03:49:19.401971+0000 mon.a (mon.0) 1189 : cluster [DBG] osdmap e103: 8 total, 8 up, 8 in 2024-03-26T03:49:19.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:19 smithi190 bash[23392]: audit 2024-03-26T03:49:19.429052+0000 mon.c (mon.2) 177 : audit [INF] from='client.? 172.21.15.175:0/4207065844' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi175-64770-6", "pool2": "test-rados-api-smithi175-64770-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T03:49:19.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:19 smithi190 bash[23392]: audit 2024-03-26T03:49:19.444242+0000 mon.a (mon.0) 1190 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-9", "mode": "writeback"}]: dispatch 2024-03-26T03:49:19.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:19 smithi190 bash[23392]: audit 2024-03-26T03:49:19.445108+0000 mon.a (mon.0) 1191 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi175-64770-6", "pool2": "test-rados-api-smithi175-64770-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T03:49:19.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:19 smithi175 bash[29284]: cluster 2024-03-26T03:49:18.584312+0000 mgr.y (mgr.24431) 191 : cluster [DBG] pgmap v150: 680 pgs: 224 unknown, 456 active+clean; 145 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-26T03:49:19.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:19 smithi175 bash[29284]: audit 2024-03-26T03:49:18.724322+0000 mon.c (mon.2) 175 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:19.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:19 smithi175 bash[29284]: audit 2024-03-26T03:49:19.057846+0000 mon.a (mon.0) 1184 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:49:19.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:19 smithi175 bash[29284]: audit 2024-03-26T03:49:19.060074+0000 mon.c (mon.2) 176 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:49:19.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:19 smithi175 bash[29284]: cluster 2024-03-26T03:49:19.393288+0000 mon.a (mon.0) 1185 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:49:19.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:19 smithi175 bash[29284]: audit 2024-03-26T03:49:19.395523+0000 mon.a (mon.0) 1186 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-9"}]': finished 2024-03-26T03:49:19.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:19 smithi175 bash[29284]: audit 2024-03-26T03:49:19.395593+0000 mon.a (mon.0) 1187 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4", "tierpool": "test-rados-api-smithi175-64770-6"}]': finished 2024-03-26T03:49:19.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:19 smithi175 bash[29284]: audit 2024-03-26T03:49:19.395676+0000 mon.a (mon.0) 1188 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:19.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:19 smithi175 bash[29284]: cluster 2024-03-26T03:49:19.401971+0000 mon.a (mon.0) 1189 : cluster [DBG] osdmap e103: 8 total, 8 up, 8 in 2024-03-26T03:49:19.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:19 smithi175 bash[29284]: audit 2024-03-26T03:49:19.429052+0000 mon.c (mon.2) 177 : audit [INF] from='client.? 172.21.15.175:0/4207065844' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi175-64770-6", "pool2": "test-rados-api-smithi175-64770-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T03:49:19.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:19 smithi175 bash[29284]: audit 2024-03-26T03:49:19.444242+0000 mon.a (mon.0) 1190 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-9", "mode": "writeback"}]: dispatch 2024-03-26T03:49:19.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:19 smithi175 bash[29284]: audit 2024-03-26T03:49:19.445108+0000 mon.a (mon.0) 1191 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi175-64770-6", "pool2": "test-rados-api-smithi175-64770-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T03:49:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:19 smithi175 bash[21527]: cluster 2024-03-26T03:49:18.584312+0000 mgr.y (mgr.24431) 191 : cluster [DBG] pgmap v150: 680 pgs: 224 unknown, 456 active+clean; 145 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-26T03:49:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:19 smithi175 bash[21527]: audit 2024-03-26T03:49:18.724322+0000 mon.c (mon.2) 175 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:19 smithi175 bash[21527]: audit 2024-03-26T03:49:19.057846+0000 mon.a (mon.0) 1184 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:49:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:19 smithi175 bash[21527]: audit 2024-03-26T03:49:19.060074+0000 mon.c (mon.2) 176 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:49:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:19 smithi175 bash[21527]: cluster 2024-03-26T03:49:19.393288+0000 mon.a (mon.0) 1185 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:49:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:19 smithi175 bash[21527]: audit 2024-03-26T03:49:19.395523+0000 mon.a (mon.0) 1186 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-9"}]': finished 2024-03-26T03:49:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:19 smithi175 bash[21527]: audit 2024-03-26T03:49:19.395593+0000 mon.a (mon.0) 1187 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi175-64770-4", "tierpool": "test-rados-api-smithi175-64770-6"}]': finished 2024-03-26T03:49:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:19 smithi175 bash[21527]: audit 2024-03-26T03:49:19.395676+0000 mon.a (mon.0) 1188 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:19 smithi175 bash[21527]: cluster 2024-03-26T03:49:19.401971+0000 mon.a (mon.0) 1189 : cluster [DBG] osdmap e103: 8 total, 8 up, 8 in 2024-03-26T03:49:19.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:19 smithi175 bash[21527]: audit 2024-03-26T03:49:19.429052+0000 mon.c (mon.2) 177 : audit [INF] from='client.? 172.21.15.175:0/4207065844' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi175-64770-6", "pool2": "test-rados-api-smithi175-64770-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T03:49:19.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:19 smithi175 bash[21527]: audit 2024-03-26T03:49:19.444242+0000 mon.a (mon.0) 1190 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-9", "mode": "writeback"}]: dispatch 2024-03-26T03:49:19.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:19 smithi175 bash[21527]: audit 2024-03-26T03:49:19.445108+0000 mon.a (mon.0) 1191 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi175-64770-6", "pool2": "test-rados-api-smithi175-64770-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T03:49:20.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:20 smithi190 bash[23392]: audit 2024-03-26T03:49:19.703332+0000 mon.b (mon.1) 95 : audit [INF] from='client.? 172.21.15.175:0/2989316611' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi175-64639-10"}]: dispatch 2024-03-26T03:49:20.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:20 smithi190 bash[23392]: audit 2024-03-26T03:49:19.703537+0000 mgr.y (mgr.24431) 192 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:49:20.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:20 smithi190 bash[23392]: audit 2024-03-26T03:49:19.719246+0000 mon.a (mon.0) 1192 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi175-64639-10"}]: dispatch 2024-03-26T03:49:20.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:20 smithi190 bash[23392]: audit 2024-03-26T03:49:19.725030+0000 mon.a (mon.0) 1193 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi175-64639-10"}]: dispatch 2024-03-26T03:49:20.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:20 smithi190 bash[23392]: audit 2024-03-26T03:49:19.725966+0000 mon.c (mon.2) 178 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:20.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:20 smithi190 bash[23392]: audit 2024-03-26T03:49:19.729286+0000 mon.b (mon.1) 96 : audit [INF] from='client.? 172.21.15.175:0/2989316611' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi175-64639-10"}]: dispatch 2024-03-26T03:49:20.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:20 smithi190 bash[23392]: audit 2024-03-26T03:49:19.730586+0000 mon.a (mon.0) 1194 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi175-64639-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:49:20.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:20 smithi190 bash[23392]: audit 2024-03-26T03:49:19.734096+0000 mon.b (mon.1) 97 : audit [INF] from='client.? 172.21.15.175:0/2989316611' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi175-64639-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:49:20.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:20 smithi190 bash[23392]: cluster 2024-03-26T03:49:20.397631+0000 mon.a (mon.0) 1195 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:49:20.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:20 smithi190 bash[23392]: audit 2024-03-26T03:49:20.400379+0000 mon.a (mon.0) 1196 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-9", "mode": "writeback"}]': finished 2024-03-26T03:49:20.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:20 smithi190 bash[23392]: audit 2024-03-26T03:49:20.400473+0000 mon.a (mon.0) 1197 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi175-64770-6", "pool2": "test-rados-api-smithi175-64770-6", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T03:49:20.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:20 smithi190 bash[23392]: audit 2024-03-26T03:49:20.400573+0000 mon.a (mon.0) 1198 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi175-64639-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:49:20.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:20 smithi190 bash[23392]: cluster 2024-03-26T03:49:20.407258+0000 mon.a (mon.0) 1199 : cluster [DBG] osdmap e104: 8 total, 8 up, 8 in 2024-03-26T03:49:20.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:20 smithi190 bash[23392]: audit 2024-03-26T03:49:20.438464+0000 mon.c (mon.2) 179 : audit [INF] from='client.? 172.21.15.175:0/717714646' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi175-64398-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:20.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:20 smithi190 bash[23392]: audit 2024-03-26T03:49:20.438951+0000 mon.a (mon.0) 1200 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi175-64398-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:20.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:20 smithi190 bash[23392]: audit 2024-03-26T03:49:20.439150+0000 mon.b (mon.1) 98 : audit [INF] from='client.? 172.21.15.175:0/2989316611' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi175-64639-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi175-64639-10"}]: dispatch 2024-03-26T03:49:20.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:20 smithi190 bash[23392]: audit 2024-03-26T03:49:20.451932+0000 mon.a (mon.0) 1201 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi175-64639-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi175-64639-10"}]: dispatch 2024-03-26T03:49:20.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:20 smithi190 bash[23392]: audit 2024-03-26T03:49:20.462575+0000 mon.a (mon.0) 1202 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi175-64406-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:20.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:20 smithi190 bash[23392]: audit 2024-03-26T03:49:20.467316+0000 mon.b (mon.1) 99 : audit [INF] from='client.? 172.21.15.175:0/1024118157' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi175-64406-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:20.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:20 smithi190 bash[23392]: audit 2024-03-26T03:49:20.607647+0000 mon.a (mon.0) 1203 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:49:20.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[21527]: audit 2024-03-26T03:49:19.703332+0000 mon.b (mon.1) 95 : audit [INF] from='client.? 172.21.15.175:0/2989316611' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi175-64639-10"}]: dispatch 2024-03-26T03:49:20.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[21527]: audit 2024-03-26T03:49:19.703537+0000 mgr.y (mgr.24431) 192 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:49:20.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[21527]: audit 2024-03-26T03:49:19.719246+0000 mon.a (mon.0) 1192 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi175-64639-10"}]: dispatch 2024-03-26T03:49:20.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[21527]: audit 2024-03-26T03:49:19.725030+0000 mon.a (mon.0) 1193 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi175-64639-10"}]: dispatch 2024-03-26T03:49:20.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[21527]: audit 2024-03-26T03:49:19.725966+0000 mon.c (mon.2) 178 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:20.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[21527]: audit 2024-03-26T03:49:19.729286+0000 mon.b (mon.1) 96 : audit [INF] from='client.? 172.21.15.175:0/2989316611' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi175-64639-10"}]: dispatch 2024-03-26T03:49:20.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[21527]: audit 2024-03-26T03:49:19.730586+0000 mon.a (mon.0) 1194 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi175-64639-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:49:20.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[21527]: audit 2024-03-26T03:49:19.734096+0000 mon.b (mon.1) 97 : audit [INF] from='client.? 172.21.15.175:0/2989316611' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi175-64639-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:49:20.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[21527]: cluster 2024-03-26T03:49:20.397631+0000 mon.a (mon.0) 1195 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:49:20.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[21527]: audit 2024-03-26T03:49:20.400379+0000 mon.a (mon.0) 1196 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-9", "mode": "writeback"}]': finished 2024-03-26T03:49:20.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[21527]: audit 2024-03-26T03:49:20.400473+0000 mon.a (mon.0) 1197 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi175-64770-6", "pool2": "test-rados-api-smithi175-64770-6", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T03:49:20.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[21527]: audit 2024-03-26T03:49:20.400573+0000 mon.a (mon.0) 1198 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi175-64639-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:49:20.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[21527]: cluster 2024-03-26T03:49:20.407258+0000 mon.a (mon.0) 1199 : cluster [DBG] osdmap e104: 8 total, 8 up, 8 in 2024-03-26T03:49:20.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[21527]: audit 2024-03-26T03:49:20.438464+0000 mon.c (mon.2) 179 : audit [INF] from='client.? 172.21.15.175:0/717714646' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi175-64398-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:20.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[21527]: audit 2024-03-26T03:49:20.438951+0000 mon.a (mon.0) 1200 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi175-64398-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:20.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[21527]: audit 2024-03-26T03:49:20.439150+0000 mon.b (mon.1) 98 : audit [INF] from='client.? 172.21.15.175:0/2989316611' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi175-64639-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi175-64639-10"}]: dispatch 2024-03-26T03:49:20.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[21527]: audit 2024-03-26T03:49:20.451932+0000 mon.a (mon.0) 1201 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi175-64639-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi175-64639-10"}]: dispatch 2024-03-26T03:49:20.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[21527]: audit 2024-03-26T03:49:20.462575+0000 mon.a (mon.0) 1202 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi175-64406-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:20.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[21527]: audit 2024-03-26T03:49:20.467316+0000 mon.b (mon.1) 99 : audit [INF] from='client.? 172.21.15.175:0/1024118157' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi175-64406-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:20.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[21527]: audit 2024-03-26T03:49:20.607647+0000 mon.a (mon.0) 1203 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:49:20.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[29284]: audit 2024-03-26T03:49:19.703332+0000 mon.b (mon.1) 95 : audit [INF] from='client.? 172.21.15.175:0/2989316611' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi175-64639-10"}]: dispatch 2024-03-26T03:49:20.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[29284]: audit 2024-03-26T03:49:19.703537+0000 mgr.y (mgr.24431) 192 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:49:20.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[29284]: audit 2024-03-26T03:49:19.719246+0000 mon.a (mon.0) 1192 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi175-64639-10"}]: dispatch 2024-03-26T03:49:20.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[29284]: audit 2024-03-26T03:49:19.725030+0000 mon.a (mon.0) 1193 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi175-64639-10"}]: dispatch 2024-03-26T03:49:20.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[29284]: audit 2024-03-26T03:49:19.725966+0000 mon.c (mon.2) 178 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:20.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[29284]: audit 2024-03-26T03:49:19.729286+0000 mon.b (mon.1) 96 : audit [INF] from='client.? 172.21.15.175:0/2989316611' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi175-64639-10"}]: dispatch 2024-03-26T03:49:20.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[29284]: audit 2024-03-26T03:49:19.730586+0000 mon.a (mon.0) 1194 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi175-64639-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:49:20.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[29284]: audit 2024-03-26T03:49:19.734096+0000 mon.b (mon.1) 97 : audit [INF] from='client.? 172.21.15.175:0/2989316611' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi175-64639-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:49:20.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[29284]: cluster 2024-03-26T03:49:20.397631+0000 mon.a (mon.0) 1195 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:49:20.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[29284]: audit 2024-03-26T03:49:20.400379+0000 mon.a (mon.0) 1196 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-9", "mode": "writeback"}]': finished 2024-03-26T03:49:20.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[29284]: audit 2024-03-26T03:49:20.400473+0000 mon.a (mon.0) 1197 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi175-64770-6", "pool2": "test-rados-api-smithi175-64770-6", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T03:49:20.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[29284]: audit 2024-03-26T03:49:20.400573+0000 mon.a (mon.0) 1198 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi175-64639-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:49:20.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[29284]: cluster 2024-03-26T03:49:20.407258+0000 mon.a (mon.0) 1199 : cluster [DBG] osdmap e104: 8 total, 8 up, 8 in 2024-03-26T03:49:20.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[29284]: audit 2024-03-26T03:49:20.438464+0000 mon.c (mon.2) 179 : audit [INF] from='client.? 172.21.15.175:0/717714646' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi175-64398-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:20.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[29284]: audit 2024-03-26T03:49:20.438951+0000 mon.a (mon.0) 1200 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi175-64398-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:20.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[29284]: audit 2024-03-26T03:49:20.439150+0000 mon.b (mon.1) 98 : audit [INF] from='client.? 172.21.15.175:0/2989316611' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi175-64639-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi175-64639-10"}]: dispatch 2024-03-26T03:49:20.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[29284]: audit 2024-03-26T03:49:20.451932+0000 mon.a (mon.0) 1201 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi175-64639-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi175-64639-10"}]: dispatch 2024-03-26T03:49:20.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[29284]: audit 2024-03-26T03:49:20.462575+0000 mon.a (mon.0) 1202 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi175-64406-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:20.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[29284]: audit 2024-03-26T03:49:20.467316+0000 mon.b (mon.1) 99 : audit [INF] from='client.? 172.21.15.175:0/1024118157' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi175-64406-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:20.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:20 smithi175 bash[29284]: audit 2024-03-26T03:49:20.607647+0000 mon.a (mon.0) 1203 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:49:21.424 INFO:tasks.workunit.client.0.smithi175.stdout:tch_notify_pp: flushed 2024-03-26T03:49:21.424 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3004 ms) 2024-03-26T03:49:21.424 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-03-26T03:49:21.424 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: trying... 2024-03-26T03:49:21.424 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: handle_notify cookie 94378449006688 notify_id 395136991238 notifier_gid 25187 2024-03-26T03:49:21.424 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: timed out 2024-03-26T03:49:21.425 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: flushing 2024-03-26T03:49:21.425 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: flushed 2024-03-26T03:49:21.425 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3015 ms) 2024-03-26T03:49:21.425 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-03-26T03:49:21.425 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: List watches 2024-03-26T03:49:21.425 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: notify2 2024-03-26T03:49:21.425 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: handle_notify cookie 94378449006688 notify_id 408021893127 notifier_gid 25187 2024-03-26T03:49:21.425 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: notify2 done 2024-03-26T03:49:21.425 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: watch_check 2024-03-26T03:49:21.425 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: unwatch2 2024-03-26T03:49:21.425 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: flushing 2024-03-26T03:49:21.425 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: done 2024-03-26T03:49:21.425 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3014 ms) 2024-03-26T03:49:21.425 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-03-26T03:49:21.425 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: List watches 2024-03-26T03:49:21.426 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: notify2 2024-03-26T03:49:21.426 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: handle_notify cookie 94378449006688 notify_id 420906795012 notifier_gid 25187 2024-03-26T03:49:21.426 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: notify2 done 2024-03-26T03:49:21.426 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: watch_check 2024-03-26T03:49:21.426 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: unwatch2 2024-03-26T03:49:21.426 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: flushing 2024-03-26T03:49:21.426 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: done 2024-03-26T03:49:21.426 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3012 ms) 2024-03-26T03:49:21.426 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (16238 ms total) 2024-03-26T03:49:21.426 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: 2024-03-26T03:49:21.426 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2024-03-26T03:49:21.426 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (29040 ms total) 2024-03-26T03:49:21.426 INFO:tasks.workunit.client.0.smithi175.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2024-03-26T03:49:21.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:21 smithi190 bash[23392]: cluster 2024-03-26T03:49:20.585224+0000 mgr.y (mgr.24431) 193 : cluster [DBG] pgmap v153: 676 pgs: 32 creating+peering, 2 creating+activating, 160 unknown, 482 active+clean; 145 MiB data, 764 MiB used, 714 GiB / 715 GiB avail 2024-03-26T03:49:21.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:21 smithi190 bash[23392]: audit 2024-03-26T03:49:20.735980+0000 mon.c (mon.2) 180 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:21.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:21 smithi190 bash[23392]: cluster 2024-03-26T03:49:21.372609+0000 mon.a (mon.0) 1204 : cluster [WRN] Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:21 smithi190 bash[23392]: audit 2024-03-26T03:49:21.379541+0000 mon.a (mon.0) 1205 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi175-64398-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:21 smithi190 bash[23392]: audit 2024-03-26T03:49:21.379647+0000 mon.a (mon.0) 1206 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi175-64406-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:21 smithi190 bash[23392]: audit 2024-03-26T03:49:21.379734+0000 mon.a (mon.0) 1207 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:49:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:21 smithi190 bash[23392]: cluster 2024-03-26T03:49:21.402762+0000 mon.a (mon.0) 1208 : cluster [DBG] osdmap e105: 8 total, 8 up, 8 in 2024-03-26T03:49:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:21 smithi190 bash[23392]: audit 2024-03-26T03:49:21.411156+0000 mon.a (mon.0) 1209 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-9"}]: dispatch 2024-03-26T03:49:21.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:21 smithi175 bash[29284]: cluster 2024-03-26T03:49:20.585224+0000 mgr.y (mgr.24431) 193 : cluster [DBG] pgmap v153: 676 pgs: 32 creating+peering, 2 creating+activating, 160 unknown, 482 active+clean; 145 MiB data, 764 MiB used, 714 GiB / 715 GiB avail 2024-03-26T03:49:21.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:21 smithi175 bash[29284]: audit 2024-03-26T03:49:20.735980+0000 mon.c (mon.2) 180 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:21.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:21 smithi175 bash[29284]: cluster 2024-03-26T03:49:21.372609+0000 mon.a (mon.0) 1204 : cluster [WRN] Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:21.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:21 smithi175 bash[29284]: audit 2024-03-26T03:49:21.379541+0000 mon.a (mon.0) 1205 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi175-64398-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:21.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:21 smithi175 bash[29284]: audit 2024-03-26T03:49:21.379647+0000 mon.a (mon.0) 1206 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi175-64406-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:21.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:21 smithi175 bash[29284]: audit 2024-03-26T03:49:21.379734+0000 mon.a (mon.0) 1207 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:49:21.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:21 smithi175 bash[29284]: cluster 2024-03-26T03:49:21.402762+0000 mon.a (mon.0) 1208 : cluster [DBG] osdmap e105: 8 total, 8 up, 8 in 2024-03-26T03:49:21.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:21 smithi175 bash[29284]: audit 2024-03-26T03:49:21.411156+0000 mon.a (mon.0) 1209 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-9"}]: dispatch 2024-03-26T03:49:21.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:21 smithi175 bash[21527]: cluster 2024-03-26T03:49:20.585224+0000 mgr.y (mgr.24431) 193 : cluster [DBG] pgmap v153: 676 pgs: 32 creating+peering, 2 creating+activating, 160 unknown, 482 active+clean; 145 MiB data, 764 MiB used, 714 GiB / 715 GiB avail 2024-03-26T03:49:21.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:21 smithi175 bash[21527]: audit 2024-03-26T03:49:20.735980+0000 mon.c (mon.2) 180 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:21.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:21 smithi175 bash[21527]: cluster 2024-03-26T03:49:21.372609+0000 mon.a (mon.0) 1204 : cluster [WRN] Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:21.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:21 smithi175 bash[21527]: audit 2024-03-26T03:49:21.379541+0000 mon.a (mon.0) 1205 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi175-64398-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:21.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:21 smithi175 bash[21527]: audit 2024-03-26T03:49:21.379647+0000 mon.a (mon.0) 1206 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi175-64406-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:21.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:21 smithi175 bash[21527]: audit 2024-03-26T03:49:21.379734+0000 mon.a (mon.0) 1207 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:49:21.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:21 smithi175 bash[21527]: cluster 2024-03-26T03:49:21.402762+0000 mon.a (mon.0) 1208 : cluster [DBG] osdmap e105: 8 total, 8 up, 8 in 2024-03-26T03:49:21.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:21 smithi175 bash[21527]: audit 2024-03-26T03:49:21.411156+0000 mon.a (mon.0) 1209 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-9"}]: dispatch 2024-03-26T03:49:22.811 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: [==========] Running 4 tests from 1 test suite. 2024-03-26T03:49:22.811 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: [----------] Global test environment set-up. 2024-03-26T03:49:22.811 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2024-03-26T03:49:22.811 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2024-03-26T03:49:22.811 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5083 ms) 2024-03-26T03:49:22.811 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2024-03-26T03:49:22.811 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (72 ms) 2024-03-26T03:49:22.811 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2024-03-26T03:49:22.811 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (20062 ms) 2024-03-26T03:49:22.811 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2024-03-26T03:49:22.812 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: attempt 0 of 20 2024-03-26T03:49:22.812 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: failed to deregister: 2024-03-26T03:49:22.812 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: { 2024-03-26T03:49:22.812 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "epoch": 20, 2024-03-26T03:49:22.812 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "modified": "2024-03-26T03:49:20.584566+0000", 2024-03-26T03:49:22.812 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "services": { 2024-03-26T03:49:22.812 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "laundry": { 2024-03-26T03:49:22.812 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "daemons": { 2024-03-26T03:49:22.812 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "summary": "", 2024-03-26T03:49:22.812 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "close-test-pid64880": { 2024-03-26T03:49:22.812 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "start_epoch": 19, 2024-03-26T03:49:22.812 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "start_stamp": "2024-03-26T03:49:17.762795+0000", 2024-03-26T03:49:22.812 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "gid": 32344, 2024-03-26T03:49:22.812 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "addr": "172.21.15.175:0/2431829388", 2024-03-26T03:49:22.813 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "metadata": { 2024-03-26T03:49:22.813 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "arch": "x86_64", 2024-03-26T03:49:22.813 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "ceph_release": "squid", 2024-03-26T03:49:22.813 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-2475-ge42b8ea3 (e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003) squid (dev)", 2024-03-26T03:49:22.813 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "ceph_version_short": "19.0.0-2475-ge42b8ea3", 2024-03-26T03:49:22.813 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-03-26T03:49:22.813 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "distro": "ubuntu", 2024-03-26T03:49:22.813 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "distro_description": "Ubuntu 22.04.1 LTS", 2024-03-26T03:49:22.813 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "distro_version": "22.04", 2024-03-26T03:49:22.813 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "foo": "bar", 2024-03-26T03:49:22.813 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "hostname": "smithi175", 2024-03-26T03:49:22.813 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "kernel_description": "#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024", 2024-03-26T03:49:22.813 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "kernel_version": "5.15.0-101-generic", 2024-03-26T03:49:22.814 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "mem_swap_kb": "2097148", 2024-03-26T03:49:22.814 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "mem_total_kb": "32746836", 2024-03-26T03:49:22.814 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "os": "Linux", 2024-03-26T03:49:22.814 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "this": "that" 2024-03-26T03:49:22.814 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: }, 2024-03-26T03:49:22.814 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "task_status": {} 2024-03-26T03:49:22.814 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: } 2024-03-26T03:49:22.814 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: } 2024-03-26T03:49:22.814 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: }, 2024-03-26T03:49:22.814 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "rgw": { 2024-03-26T03:49:22.814 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "daemons": { 2024-03-26T03:49:22.814 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "summary": "", 2024-03-26T03:49:22.814 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "14418": { 2024-03-26T03:49:22.814 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "start_epoch": 13, 2024-03-26T03:49:22.815 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "start_stamp": "2024-03-26T03:44:23.501796+0000", 2024-03-26T03:49:22.815 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "gid": 14418, 2024-03-26T03:49:22.815 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "addr": "172.21.15.175:0/1435275536", 2024-03-26T03:49:22.815 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "metadata": { 2024-03-26T03:49:22.815 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "arch": "x86_64", 2024-03-26T03:49:22.815 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "ceph_release": "squid", 2024-03-26T03:49:22.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:22 smithi190 bash[23392]: audit 2024-03-26T03:49:21.738935+0000 mon.c (mon.2) 181 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:22.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:22 smithi190 bash[23392]: cluster 2024-03-26T03:49:22.380171+0000 mon.a (mon.0) 1210 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:49:22.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:22 smithi190 bash[23392]: audit 2024-03-26T03:49:22.382238+0000 mon.a (mon.0) 1211 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi175-64639-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi175-64639-10"}]': finished 2024-03-26T03:49:22.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:22 smithi190 bash[23392]: audit 2024-03-26T03:49:22.382361+0000 mon.a (mon.0) 1212 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-9"}]': finished 2024-03-26T03:49:22.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:22 smithi190 bash[23392]: cluster 2024-03-26T03:49:22.387055+0000 mon.a (mon.0) 1213 : cluster [DBG] osdmap e106: 8 total, 8 up, 8 in 2024-03-26T03:49:22.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:22 smithi175 bash[29284]: audit 2024-03-26T03:49:21.738935+0000 mon.c (mon.2) 181 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:22.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:22 smithi175 bash[29284]: cluster 2024-03-26T03:49:22.380171+0000 mon.a (mon.0) 1210 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:49:22.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:22 smithi175 bash[29284]: audit 2024-03-26T03:49:22.382238+0000 mon.a (mon.0) 1211 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi175-64639-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi175-64639-10"}]': finished 2024-03-26T03:49:22.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:22 smithi175 bash[29284]: audit 2024-03-26T03:49:22.382361+0000 mon.a (mon.0) 1212 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-9"}]': finished 2024-03-26T03:49:22.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:22 smithi175 bash[29284]: cluster 2024-03-26T03:49:22.387055+0000 mon.a (mon.0) 1213 : cluster [DBG] osdmap e106: 8 total, 8 up, 8 in 2024-03-26T03:49:22.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:22 smithi175 bash[21527]: audit 2024-03-26T03:49:21.738935+0000 mon.c (mon.2) 181 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:22.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:22 smithi175 bash[21527]: cluster 2024-03-26T03:49:22.380171+0000 mon.a (mon.0) 1210 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:49:22.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:22 smithi175 bash[21527]: audit 2024-03-26T03:49:22.382238+0000 mon.a (mon.0) 1211 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi175-64639-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi175-64639-10"}]': finished 2024-03-26T03:49:22.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:22 smithi175 bash[21527]: audit 2024-03-26T03:49:22.382361+0000 mon.a (mon.0) 1212 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-9"}]': finished 2024-03-26T03:49:22.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:22 smithi175 bash[21527]: cluster 2024-03-26T03:49:22.387055+0000 mon.a (mon.0) 1213 : cluster [DBG] osdmap e106: 8 total, 8 up, 8 in 2024-03-26T03:49:23.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:23 smithi190 bash[23392]: cluster 2024-03-26T03:49:22.585879+0000 mgr.y (mgr.24431) 194 : cluster [DBG] pgmap v156: 460 pgs: 32 creating+peering, 8 unknown, 420 active+clean; 145 MiB data, 764 MiB used, 714 GiB / 715 GiB avail 2024-03-26T03:49:23.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:23 smithi190 bash[23392]: audit 2024-03-26T03:49:22.740240+0000 mon.c (mon.2) 182 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:23.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:23 smithi190 bash[23392]: audit 2024-03-26T03:49:22.810770+0000 mon.a (mon.0) 1214 : audit [DBG] from='client.? 172.21.15.175:0/3600614687' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-26T03:49:23.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:23 smithi190 bash[23392]: audit 2024-03-26T03:49:22.810969+0000 mgr.y (mgr.24431) 195 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-26T03:49:23.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:23 smithi190 bash[23392]: cluster 2024-03-26T03:49:23.391734+0000 mon.a (mon.0) 1215 : cluster [DBG] osdmap e107: 8 total, 8 up, 8 in 2024-03-26T03:49:23.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:23 smithi190 bash[23392]: audit 2024-03-26T03:49:23.407317+0000 mon.b (mon.1) 100 : audit [INF] from='client.? 172.21.15.175:0/2781238020' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi175-64406-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:23.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:23 smithi190 bash[23392]: audit 2024-03-26T03:49:23.407639+0000 mon.a (mon.0) 1216 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi175-64406-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:23.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:23 smithi190 bash[23392]: audit 2024-03-26T03:49:23.408086+0000 mon.b (mon.1) 101 : audit [INF] from='client.? 172.21.15.175:0/548513411' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi175-64398-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:23.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:23 smithi190 bash[23392]: audit 2024-03-26T03:49:23.415850+0000 mon.a (mon.0) 1217 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi175-64398-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:23.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:23 smithi190 bash[23392]: audit 2024-03-26T03:49:23.437002+0000 mon.b (mon.1) 102 : audit [INF] from='client.? 172.21.15.175:0/2686632908' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:23.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:23 smithi190 bash[23392]: audit 2024-03-26T03:49:23.443755+0000 mon.a (mon.0) 1218 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:23.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:23 smithi175 bash[29284]: cluster 2024-03-26T03:49:22.585879+0000 mgr.y (mgr.24431) 194 : cluster [DBG] pgmap v156: 460 pgs: 32 creating+peering, 8 unknown, 420 active+clean; 145 MiB data, 764 MiB used, 714 GiB / 715 GiB avail 2024-03-26T03:49:23.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:23 smithi175 bash[29284]: audit 2024-03-26T03:49:22.740240+0000 mon.c (mon.2) 182 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:23.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:23 smithi175 bash[29284]: audit 2024-03-26T03:49:22.810770+0000 mon.a (mon.0) 1214 : audit [DBG] from='client.? 172.21.15.175:0/3600614687' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-26T03:49:23.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:23 smithi175 bash[29284]: audit 2024-03-26T03:49:22.810969+0000 mgr.y (mgr.24431) 195 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-26T03:49:23.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:23 smithi175 bash[29284]: cluster 2024-03-26T03:49:23.391734+0000 mon.a (mon.0) 1215 : cluster [DBG] osdmap e107: 8 total, 8 up, 8 in 2024-03-26T03:49:23.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:23 smithi175 bash[29284]: audit 2024-03-26T03:49:23.407317+0000 mon.b (mon.1) 100 : audit [INF] from='client.? 172.21.15.175:0/2781238020' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi175-64406-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:23.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:23 smithi175 bash[29284]: audit 2024-03-26T03:49:23.407639+0000 mon.a (mon.0) 1216 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi175-64406-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:23.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:23 smithi175 bash[29284]: audit 2024-03-26T03:49:23.408086+0000 mon.b (mon.1) 101 : audit [INF] from='client.? 172.21.15.175:0/548513411' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi175-64398-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:23.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:23 smithi175 bash[29284]: audit 2024-03-26T03:49:23.415850+0000 mon.a (mon.0) 1217 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi175-64398-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:23.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:23 smithi175 bash[29284]: audit 2024-03-26T03:49:23.437002+0000 mon.b (mon.1) 102 : audit [INF] from='client.? 172.21.15.175:0/2686632908' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:23.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:23 smithi175 bash[29284]: audit 2024-03-26T03:49:23.443755+0000 mon.a (mon.0) 1218 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:23.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:23 smithi175 bash[21527]: cluster 2024-03-26T03:49:22.585879+0000 mgr.y (mgr.24431) 194 : cluster [DBG] pgmap v156: 460 pgs: 32 creating+peering, 8 unknown, 420 active+clean; 145 MiB data, 764 MiB used, 714 GiB / 715 GiB avail 2024-03-26T03:49:23.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:23 smithi175 bash[21527]: audit 2024-03-26T03:49:22.740240+0000 mon.c (mon.2) 182 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:23.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:23 smithi175 bash[21527]: audit 2024-03-26T03:49:22.810770+0000 mon.a (mon.0) 1214 : audit [DBG] from='client.? 172.21.15.175:0/3600614687' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-26T03:49:23.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:23 smithi175 bash[21527]: audit 2024-03-26T03:49:22.810969+0000 mgr.y (mgr.24431) 195 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-26T03:49:23.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:23 smithi175 bash[21527]: cluster 2024-03-26T03:49:23.391734+0000 mon.a (mon.0) 1215 : cluster [DBG] osdmap e107: 8 total, 8 up, 8 in 2024-03-26T03:49:23.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:23 smithi175 bash[21527]: audit 2024-03-26T03:49:23.407317+0000 mon.b (mon.1) 100 : audit [INF] from='client.? 172.21.15.175:0/2781238020' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi175-64406-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:23.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:23 smithi175 bash[21527]: audit 2024-03-26T03:49:23.407639+0000 mon.a (mon.0) 1216 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi175-64406-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:23.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:23 smithi175 bash[21527]: audit 2024-03-26T03:49:23.408086+0000 mon.b (mon.1) 101 : audit [INF] from='client.? 172.21.15.175:0/548513411' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi175-64398-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:23.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:23 smithi175 bash[21527]: audit 2024-03-26T03:49:23.415850+0000 mon.a (mon.0) 1217 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi175-64398-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:23.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:23 smithi175 bash[21527]: audit 2024-03-26T03:49:23.437002+0000 mon.b (mon.1) 102 : audit [INF] from='client.? 172.21.15.175:0/2686632908' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:23.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:23 smithi175 bash[21527]: audit 2024-03-26T03:49:23.443755+0000 mon.a (mon.0) 1218 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:24.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:24 smithi190 bash[23392]: audit 2024-03-26T03:49:23.741727+0000 mon.c (mon.2) 183 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:24.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:24 smithi190 bash[23392]: audit 2024-03-26T03:49:24.392042+0000 mon.a (mon.0) 1219 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi175-64406-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:24.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:24 smithi190 bash[23392]: audit 2024-03-26T03:49:24.392133+0000 mon.a (mon.0) 1220 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi175-64398-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:24.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:24 smithi190 bash[23392]: audit 2024-03-26T03:49:24.392204+0000 mon.a (mon.0) 1221 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:24.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:24 smithi190 bash[23392]: cluster 2024-03-26T03:49:24.398939+0000 mon.a (mon.0) 1222 : cluster [DBG] osdmap e108: 8 total, 8 up, 8 in 2024-03-26T03:49:24.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:24 smithi190 bash[23392]: audit 2024-03-26T03:49:24.416033+0000 mon.a (mon.0) 1223 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:24.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:24 smithi175 bash[21527]: audit 2024-03-26T03:49:23.741727+0000 mon.c (mon.2) 183 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:24.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:24 smithi175 bash[21527]: audit 2024-03-26T03:49:24.392042+0000 mon.a (mon.0) 1219 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi175-64406-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:24.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:24 smithi175 bash[21527]: audit 2024-03-26T03:49:24.392133+0000 mon.a (mon.0) 1220 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi175-64398-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:24.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:24 smithi175 bash[21527]: audit 2024-03-26T03:49:24.392204+0000 mon.a (mon.0) 1221 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:24.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:24 smithi175 bash[21527]: cluster 2024-03-26T03:49:24.398939+0000 mon.a (mon.0) 1222 : cluster [DBG] osdmap e108: 8 total, 8 up, 8 in 2024-03-26T03:49:24.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:24 smithi175 bash[21527]: audit 2024-03-26T03:49:24.416033+0000 mon.a (mon.0) 1223 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:24.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:24 smithi175 bash[29284]: audit 2024-03-26T03:49:23.741727+0000 mon.c (mon.2) 183 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:24.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:24 smithi175 bash[29284]: audit 2024-03-26T03:49:24.392042+0000 mon.a (mon.0) 1219 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi175-64406-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:24.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:24 smithi175 bash[29284]: audit 2024-03-26T03:49:24.392133+0000 mon.a (mon.0) 1220 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi175-64398-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:24.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:24 smithi175 bash[29284]: audit 2024-03-26T03:49:24.392204+0000 mon.a (mon.0) 1221 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64590-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:24.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:24 smithi175 bash[29284]: cluster 2024-03-26T03:49:24.398939+0000 mon.a (mon.0) 1222 : cluster [DBG] osdmap e108: 8 total, 8 up, 8 in 2024-03-26T03:49:24.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:24 smithi175 bash[29284]: audit 2024-03-26T03:49:24.416033+0000 mon.a (mon.0) 1223 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:25.459 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-2475-ge42b8ea3 (e42b8ea316eec6c6f4fae026f6d6 misc: Running main() from gmock_main.cc 2024-03-26T03:49:25.459 INFO:tasks.workunit.client.0.smithi175.stdout: misc: [==========] Running 12 tests from 1 test suite. 2024-03-26T03:49:25.459 INFO:tasks.workunit.client.0.smithi175.stdout: misc: [----------] Global test environment set-up. 2024-03-26T03:49:25.460 INFO:tasks.workunit.client.0.smithi175.stdout: misc: [----------] 12 tests from NeoRadosMisc 2024-03-26T03:49:25.460 INFO:tasks.workunit.client.0.smithi175.stdout: misc: [ RUN ] NeoRadosMisc.Version 2024-03-26T03:49:25.460 INFO:tasks.workunit.client.0.smithi175.stdout: misc: [ OK ] NeoRadosMisc.Version (1763 ms) 2024-03-26T03:49:25.460 INFO:tasks.workunit.client.0.smithi175.stdout: misc: [ RUN ] NeoRadosMisc.WaitOSDMap 2024-03-26T03:49:25.460 INFO:tasks.workunit.client.0.smithi175.stdout: misc: [ OK ] NeoRadosMisc.WaitOSDMap (1991 ms) 2024-03-26T03:49:25.460 INFO:tasks.workunit.client.0.smithi175.stdout: misc: [ RUN ] NeoRadosMisc.LongName 2024-03-26T03:49:25.460 INFO:tasks.workunit.client.0.smithi175.stdout: misc: [ OK ] NeoRadosMisc.LongName (3035 ms) 2024-03-26T03:49:25.460 INFO:tasks.workunit.client.0.smithi175.stdout: misc: [ RUN ] NeoRadosMisc.LongLocator 2024-03-26T03:49:25.460 INFO:tasks.workunit.client.0.smithi175.stdout: misc: [ OK ] NeoRadosMisc.LongLocator (3028 ms) 2024-03-26T03:49:25.460 INFO:tasks.workunit.client.0.smithi175.stdout: misc: [ RUN ] NeoRadosMisc.LongNamespace 2024-03-26T03:49:25.460 INFO:tasks.workunit.client.0.smithi175.stdout: misc: [ OK ] NeoRadosMisc.LongNamespace (3014 ms) 2024-03-26T03:49:25.460 INFO:tasks.workunit.client.0.smithi175.stdout: misc: [ RUN ] NeoRadosMisc.LongAttrName 2024-03-26T03:49:25.460 INFO:tasks.workunit.client.0.smithi175.stdout: misc: [ OK ] NeoRadosMisc.LongAttrName (2999 ms) 2024-03-26T03:49:25.460 INFO:tasks.workunit.client.0.smithi175.stdout: misc: [ RUN ] NeoRadosMisc.Exec 2024-03-26T03:49:25.460 INFO:tasks.workunit.client.0.smithi175.stdout: misc: [ OK ] NeoRadosMisc.Exec (3006 ms) 2024-03-26T03:49:25.461 INFO:tasks.workunit.client.0.smithi175.stdout: misc: [ RUN ] NeoRadosMisc.Operate1 2024-03-26T03:49:25.461 INFO:tasks.workunit.client.0.smithi175.stdout: misc: [ OK ] NeoRadosMisc.Operate1 (3055 ms) 2024-03-26T03:49:25.461 INFO:tasks.workunit.client.0.smithi175.stdout: misc: [ RUN ] NeoRadosMisc.Operate2 2024-03-26T03:49:25.461 INFO:tasks.workunit.client.0.smithi175.stdout: misc: [ OK ] NeoRadosMisc.Operate2 (2968 ms) 2024-03-26T03:49:25.461 INFO:tasks.workunit.client.0.smithi175.stdout: misc: [ RUN ] NeoRadosMisc.BigObject 2024-03-26T03:49:25.461 INFO:tasks.workunit.client.0.smithi175.stdout: misc: [ OK ] NeoRadosMisc.BigObject (2991 ms) 2024-03-26T03:49:25.461 INFO:tasks.workunit.client.0.smithi175.stdout: misc: [ RUN ] NeoRadosMisc.BigAttr 2024-03-26T03:49:25.461 INFO:tasks.workunit.client.0.smithi175.stdout: misc: [ OK ] NeoRadosMisc.BigAttr (1977 ms) 2024-03-26T03:49:25.461 INFO:tasks.workunit.client.0.smithi175.stdout: misc: [ RUN ] NeoRadosMisc.WriteSame 2024-03-26T03:49:25.461 INFO:tasks.workunit.client.0.smithi175.stdout: misc: [ OK ] NeoRadosMisc.WriteSame (3012 ms) 2024-03-26T03:49:25.461 INFO:tasks.workunit.client.0.smithi175.stdout: misc: [----------] 12 tests from NeoRadosMisc (32839 ms total) 2024-03-26T03:49:25.461 INFO:tasks.workunit.client.0.smithi175.stdout: misc: 2024-03-26T03:49:25.461 INFO:tasks.workunit.client.0.smithi175.stdout: misc: [----------] Global test environment tear-down 2024-03-26T03:49:25.461 INFO:tasks.workunit.client.0.smithi175.stdout: misc: [==========] 12 tests from 1 test suite ran. (32839 ms total) 2024-03-26T03:49:25.461 INFO:tasks.workunit.client.0.smithi175.stdout: misc: [ PASSED ] 12 tests. 2024-03-26T03:49:25.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:25 smithi190 bash[23392]: cluster 2024-03-26T03:49:24.587277+0000 mgr.y (mgr.24431) 196 : cluster [DBG] pgmap v159: 652 pgs: 34 creating+peering, 135 unknown, 483 active+clean; 146 MiB data, 769 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 258 KiB/s wr, 3 op/s 2024-03-26T03:49:25.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:25 smithi190 bash[23392]: audit 2024-03-26T03:49:24.744270+0000 mon.c (mon.2) 184 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:25.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:25 smithi190 bash[23392]: audit 2024-03-26T03:49:25.395778+0000 mon.a (mon.0) 1224 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:25.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:25 smithi190 bash[23392]: cluster 2024-03-26T03:49:25.406039+0000 mon.a (mon.0) 1225 : cluster [DBG] osdmap e109: 8 total, 8 up, 8 in 2024-03-26T03:49:25.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:25 smithi190 bash[23392]: audit 2024-03-26T03:49:25.423837+0000 mon.b (mon.1) 103 : audit [INF] from='client.? 172.21.15.175:0/2686632908' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64590-10", "tierpool":"test-rados-api-smithi175-64590-10-cache", "force_nonempty":""}]: dispatch 2024-03-26T03:49:25.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:25 smithi190 bash[23392]: audit 2024-03-26T03:49:25.430572+0000 mon.a (mon.0) 1226 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64590-10", "tierpool":"test-rados-api-smithi175-64590-10-cache", "force_nonempty":""}]: dispatch 2024-03-26T03:49:25.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:25 smithi175 bash[29284]: cluster 2024-03-26T03:49:24.587277+0000 mgr.y (mgr.24431) 196 : cluster [DBG] pgmap v159: 652 pgs: 34 creating+peering, 135 unknown, 483 active+clean; 146 MiB data, 769 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 258 KiB/s wr, 3 op/s 2024-03-26T03:49:25.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:25 smithi175 bash[29284]: audit 2024-03-26T03:49:24.744270+0000 mon.c (mon.2) 184 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:25.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:25 smithi175 bash[29284]: audit 2024-03-26T03:49:25.395778+0000 mon.a (mon.0) 1224 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:25.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:25 smithi175 bash[29284]: cluster 2024-03-26T03:49:25.406039+0000 mon.a (mon.0) 1225 : cluster [DBG] osdmap e109: 8 total, 8 up, 8 in 2024-03-26T03:49:25.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:25 smithi175 bash[29284]: audit 2024-03-26T03:49:25.423837+0000 mon.b (mon.1) 103 : audit [INF] from='client.? 172.21.15.175:0/2686632908' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64590-10", "tierpool":"test-rados-api-smithi175-64590-10-cache", "force_nonempty":""}]: dispatch 2024-03-26T03:49:25.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:25 smithi175 bash[29284]: audit 2024-03-26T03:49:25.430572+0000 mon.a (mon.0) 1226 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64590-10", "tierpool":"test-rados-api-smithi175-64590-10-cache", "force_nonempty":""}]: dispatch 2024-03-26T03:49:25.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:25 smithi175 bash[21527]: cluster 2024-03-26T03:49:24.587277+0000 mgr.y (mgr.24431) 196 : cluster [DBG] pgmap v159: 652 pgs: 34 creating+peering, 135 unknown, 483 active+clean; 146 MiB data, 769 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 258 KiB/s wr, 3 op/s 2024-03-26T03:49:25.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:25 smithi175 bash[21527]: audit 2024-03-26T03:49:24.744270+0000 mon.c (mon.2) 184 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:25.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:25 smithi175 bash[21527]: audit 2024-03-26T03:49:25.395778+0000 mon.a (mon.0) 1224 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:25.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:25 smithi175 bash[21527]: cluster 2024-03-26T03:49:25.406039+0000 mon.a (mon.0) 1225 : cluster [DBG] osdmap e109: 8 total, 8 up, 8 in 2024-03-26T03:49:25.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:25 smithi175 bash[21527]: audit 2024-03-26T03:49:25.423837+0000 mon.b (mon.1) 103 : audit [INF] from='client.? 172.21.15.175:0/2686632908' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64590-10", "tierpool":"test-rados-api-smithi175-64590-10-cache", "force_nonempty":""}]: dispatch 2024-03-26T03:49:25.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:25 smithi175 bash[21527]: audit 2024-03-26T03:49:25.430572+0000 mon.a (mon.0) 1226 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64590-10", "tierpool":"test-rados-api-smithi175-64590-10-cache", "force_nonempty":""}]: dispatch 2024-03-26T03:49:26.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:26 smithi190 bash[23392]: audit 2024-03-26T03:49:25.746966+0000 mon.c (mon.2) 185 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:26.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:26 smithi190 bash[23392]: cluster 2024-03-26T03:49:26.376515+0000 mon.a (mon.0) 1227 : cluster [WRN] Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:26.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:26 smithi190 bash[23392]: audit 2024-03-26T03:49:26.381956+0000 mon.a (mon.0) 1228 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64590-10", "tierpool":"test-rados-api-smithi175-64590-10-cache", "force_nonempty":""}]': finished 2024-03-26T03:49:26.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:26 smithi190 bash[23392]: cluster 2024-03-26T03:49:26.387108+0000 mon.a (mon.0) 1229 : cluster [DBG] osdmap e110: 8 total, 8 up, 8 in 2024-03-26T03:49:26.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:26 smithi190 bash[23392]: audit 2024-03-26T03:49:26.401781+0000 mon.b (mon.1) 104 : audit [INF] from='client.? 172.21.15.175:0/2686632908' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64590-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:26.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:26 smithi190 bash[23392]: audit 2024-03-26T03:49:26.402713+0000 mon.a (mon.0) 1230 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64590-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:26.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:26 smithi190 bash[23392]: audit 2024-03-26T03:49:26.409870+0000 mon.a (mon.0) 1231 : audit [INF] from='client.? 172.21.15.175:0/3019427948' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi175-64406-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:26.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:26 smithi190 bash[23392]: audit 2024-03-26T03:49:26.415889+0000 mon.c (mon.2) 186 : audit [INF] from='client.? 172.21.15.175:0/208685589' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi175-64398-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:26.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:26 smithi190 bash[23392]: audit 2024-03-26T03:49:26.422989+0000 mon.a (mon.0) 1232 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi175-64398-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:26.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:26 smithi190 bash[23392]: audit 2024-03-26T03:49:26.476889+0000 mon.a (mon.0) 1233 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:49:26.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:26 smithi175 bash[29284]: audit 2024-03-26T03:49:25.746966+0000 mon.c (mon.2) 185 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:26.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:26 smithi175 bash[29284]: cluster 2024-03-26T03:49:26.376515+0000 mon.a (mon.0) 1227 : cluster [WRN] Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:26.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:26 smithi175 bash[29284]: audit 2024-03-26T03:49:26.381956+0000 mon.a (mon.0) 1228 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64590-10", "tierpool":"test-rados-api-smithi175-64590-10-cache", "force_nonempty":""}]': finished 2024-03-26T03:49:26.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:26 smithi175 bash[29284]: cluster 2024-03-26T03:49:26.387108+0000 mon.a (mon.0) 1229 : cluster [DBG] osdmap e110: 8 total, 8 up, 8 in 2024-03-26T03:49:26.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:26 smithi175 bash[29284]: audit 2024-03-26T03:49:26.401781+0000 mon.b (mon.1) 104 : audit [INF] from='client.? 172.21.15.175:0/2686632908' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64590-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:26.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:26 smithi175 bash[29284]: audit 2024-03-26T03:49:26.402713+0000 mon.a (mon.0) 1230 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64590-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:26.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:26 smithi175 bash[29284]: audit 2024-03-26T03:49:26.409870+0000 mon.a (mon.0) 1231 : audit [INF] from='client.? 172.21.15.175:0/3019427948' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi175-64406-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:26.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:26 smithi175 bash[29284]: audit 2024-03-26T03:49:26.415889+0000 mon.c (mon.2) 186 : audit [INF] from='client.? 172.21.15.175:0/208685589' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi175-64398-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:26.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:26 smithi175 bash[29284]: audit 2024-03-26T03:49:26.422989+0000 mon.a (mon.0) 1232 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi175-64398-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:26.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:26 smithi175 bash[29284]: audit 2024-03-26T03:49:26.476889+0000 mon.a (mon.0) 1233 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:49:26.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:26 smithi175 bash[21527]: audit 2024-03-26T03:49:25.746966+0000 mon.c (mon.2) 185 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:26.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:26 smithi175 bash[21527]: cluster 2024-03-26T03:49:26.376515+0000 mon.a (mon.0) 1227 : cluster [WRN] Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:26.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:26 smithi175 bash[21527]: audit 2024-03-26T03:49:26.381956+0000 mon.a (mon.0) 1228 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64590-10", "tierpool":"test-rados-api-smithi175-64590-10-cache", "force_nonempty":""}]': finished 2024-03-26T03:49:26.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:26 smithi175 bash[21527]: cluster 2024-03-26T03:49:26.387108+0000 mon.a (mon.0) 1229 : cluster [DBG] osdmap e110: 8 total, 8 up, 8 in 2024-03-26T03:49:26.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:26 smithi175 bash[21527]: audit 2024-03-26T03:49:26.401781+0000 mon.b (mon.1) 104 : audit [INF] from='client.? 172.21.15.175:0/2686632908' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64590-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:26.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:26 smithi175 bash[21527]: audit 2024-03-26T03:49:26.402713+0000 mon.a (mon.0) 1230 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64590-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:26.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:26 smithi175 bash[21527]: audit 2024-03-26T03:49:26.409870+0000 mon.a (mon.0) 1231 : audit [INF] from='client.? 172.21.15.175:0/3019427948' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi175-64406-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:26.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:26 smithi175 bash[21527]: audit 2024-03-26T03:49:26.415889+0000 mon.c (mon.2) 186 : audit [INF] from='client.? 172.21.15.175:0/208685589' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi175-64398-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:26.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:26 smithi175 bash[21527]: audit 2024-03-26T03:49:26.422989+0000 mon.a (mon.0) 1232 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi175-64398-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:26.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:26 smithi175 bash[21527]: audit 2024-03-26T03:49:26.476889+0000 mon.a (mon.0) 1233 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:49:27.914 INFO:tasks.workunit.client.0.smithi175.stdout:b62fa5fe7003) squid (dev)", 2024-03-26T03:49:27.914 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "ceph_version_short": "19.0.0-2475-ge42b8ea3", 2024-03-26T03:49:27.914 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "container_hostname": "smithi175", 2024-03-26T03:49:27.915 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:036ff263f269d03295f88ce388558d8d1cbe14b2ee02cd78a973d9f233151717", 2024-03-26T03:49:27.915 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-03-26T03:49:27.915 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "distro": "centos", 2024-03-26T03:49:27.915 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "distro_description": "CentOS Stream 8", 2024-03-26T03:49:27.915 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "distro_version": "8", 2024-03-26T03:49:27.915 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "frontend_config#0": "beast port=80", 2024-03-26T03:49:27.915 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "frontend_type#0": "beast", 2024-03-26T03:49:27.915 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "hostname": "smithi175", 2024-03-26T03:49:27.915 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "id": "foo.a", 2024-03-26T03:49:27.915 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "kernel_description": "#111-Ubuntu SMP Tue Mar 5 20:16:58 UTC 2024", 2024-03-26T03:49:27.915 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "kernel_version": "5.15.0-101-generic", 2024-03-26T03:49:27.915 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "mem_swap_kb": "2097148", 2024-03-26T03:49:27.915 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "mem_total_kb": "32746836", 2024-03-26T03:49:27.916 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "num_handles": "1", 2024-03-26T03:49:27.916 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "os": "Linux", 2024-03-26T03:49:27.916 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "pid": "7", 2024-03-26T03:49:27.916 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "realm_id": "", 2024-03-26T03:49:27.916 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "realm_name": "", 2024-03-26T03:49:27.916 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "zone_id": "306aec61-9fdf-481e-9f4e-fd73f841949d", 2024-03-26T03:49:27.916 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "zone_name": "default", 2024-03-26T03:49:27.916 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "zonegroup_id": "b636cc1d-23c6-4059-8ac4-ccf1aaa9c5ca", 2024-03-26T03:49:27.916 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "zonegroup_name": "default" 2024-03-26T03:49:27.916 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: }, 2024-03-26T03:49:27.916 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: "task_status": {} 2024-03-26T03:49:27.916 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: } 2024-03-26T03:49:27.916 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: } 2024-03-26T03:49:27.917 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: } 2024-03-26T03:49:27.917 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: } 2024-03-26T03:49:27.917 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: } 2024-03-26T03:49:27.917 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: 2024-03-26T03:49:27.917 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: attempt 1 of 20 2024-03-26T03:49:27.917 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (10258 ms) 2024-03-26T03:49:27.917 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (35475 ms total) 2024-03-26T03:49:27.917 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: 2024-03-26T03:49:27.917 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: [----------] Global test environment tear-down 2024-03-26T03:49:27.917 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (35475 ms total) 2024-03-26T03:49:27.917 INFO:tasks.workunit.client.0.smithi175.stdout: api_service_pp: [ PASSED ] 4 tests. 2024-03-26T03:49:27.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:27 smithi190 bash[23392]: cluster 2024-03-26T03:49:26.587927+0000 mgr.y (mgr.24431) 197 : cluster [DBG] pgmap v162: 652 pgs: 64 creating+peering, 160 unknown, 428 active+clean; 146 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 4 op/s 2024-03-26T03:49:27.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:27 smithi190 bash[23392]: audit 2024-03-26T03:49:26.751328+0000 mon.c (mon.2) 187 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:27.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:27 smithi190 bash[23392]: audit 2024-03-26T03:49:27.386874+0000 mon.a (mon.0) 1234 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64590-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:27.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:27 smithi190 bash[23392]: audit 2024-03-26T03:49:27.386956+0000 mon.a (mon.0) 1235 : audit [INF] from='client.? 172.21.15.175:0/3019427948' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi175-64406-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:27.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:27 smithi190 bash[23392]: audit 2024-03-26T03:49:27.387011+0000 mon.a (mon.0) 1236 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi175-64398-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:27.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:27 smithi190 bash[23392]: audit 2024-03-26T03:49:27.387075+0000 mon.a (mon.0) 1237 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-11", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:49:27.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:27 smithi190 bash[23392]: cluster 2024-03-26T03:49:27.393168+0000 mon.a (mon.0) 1238 : cluster [DBG] osdmap e111: 8 total, 8 up, 8 in 2024-03-26T03:49:27.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:27 smithi190 bash[23392]: audit 2024-03-26T03:49:27.408871+0000 mon.a (mon.0) 1239 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-11"}]: dispatch 2024-03-26T03:49:27.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:27 smithi190 bash[23392]: audit 2024-03-26T03:49:27.434151+0000 mon.a (mon.0) 1240 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64590-10", "tierpool":"test-rados-api-smithi175-64590-10-cache"}]: dispatch 2024-03-26T03:49:27.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:27 smithi190 bash[23392]: audit 2024-03-26T03:49:27.439017+0000 mon.b (mon.1) 105 : audit [INF] from='client.? 172.21.15.175:0/2686632908' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64590-10", "tierpool":"test-rados-api-smithi175-64590-10-cache"}]: dispatch 2024-03-26T03:49:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:27 smithi175 bash[21527]: cluster 2024-03-26T03:49:26.587927+0000 mgr.y (mgr.24431) 197 : cluster [DBG] pgmap v162: 652 pgs: 64 creating+peering, 160 unknown, 428 active+clean; 146 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 4 op/s 2024-03-26T03:49:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:27 smithi175 bash[21527]: audit 2024-03-26T03:49:26.751328+0000 mon.c (mon.2) 187 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:27 smithi175 bash[21527]: audit 2024-03-26T03:49:27.386874+0000 mon.a (mon.0) 1234 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64590-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:27 smithi175 bash[21527]: audit 2024-03-26T03:49:27.386956+0000 mon.a (mon.0) 1235 : audit [INF] from='client.? 172.21.15.175:0/3019427948' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi175-64406-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:27 smithi175 bash[21527]: audit 2024-03-26T03:49:27.387011+0000 mon.a (mon.0) 1236 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi175-64398-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:27 smithi175 bash[21527]: audit 2024-03-26T03:49:27.387075+0000 mon.a (mon.0) 1237 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-11", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:49:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:27 smithi175 bash[21527]: cluster 2024-03-26T03:49:27.393168+0000 mon.a (mon.0) 1238 : cluster [DBG] osdmap e111: 8 total, 8 up, 8 in 2024-03-26T03:49:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:27 smithi175 bash[21527]: audit 2024-03-26T03:49:27.408871+0000 mon.a (mon.0) 1239 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-11"}]: dispatch 2024-03-26T03:49:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:27 smithi175 bash[21527]: audit 2024-03-26T03:49:27.434151+0000 mon.a (mon.0) 1240 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64590-10", "tierpool":"test-rados-api-smithi175-64590-10-cache"}]: dispatch 2024-03-26T03:49:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:27 smithi175 bash[21527]: audit 2024-03-26T03:49:27.439017+0000 mon.b (mon.1) 105 : audit [INF] from='client.? 172.21.15.175:0/2686632908' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64590-10", "tierpool":"test-rados-api-smithi175-64590-10-cache"}]: dispatch 2024-03-26T03:49:27.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:27 smithi175 bash[29284]: cluster 2024-03-26T03:49:26.587927+0000 mgr.y (mgr.24431) 197 : cluster [DBG] pgmap v162: 652 pgs: 64 creating+peering, 160 unknown, 428 active+clean; 146 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 4 op/s 2024-03-26T03:49:27.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:27 smithi175 bash[29284]: audit 2024-03-26T03:49:26.751328+0000 mon.c (mon.2) 187 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:27.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:27 smithi175 bash[29284]: audit 2024-03-26T03:49:27.386874+0000 mon.a (mon.0) 1234 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64590-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:27.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:27 smithi175 bash[29284]: audit 2024-03-26T03:49:27.386956+0000 mon.a (mon.0) 1235 : audit [INF] from='client.? 172.21.15.175:0/3019427948' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi175-64406-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:27.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:27 smithi175 bash[29284]: audit 2024-03-26T03:49:27.387011+0000 mon.a (mon.0) 1236 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi175-64398-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:27.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:27 smithi175 bash[29284]: audit 2024-03-26T03:49:27.387075+0000 mon.a (mon.0) 1237 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-11", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:49:27.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:27 smithi175 bash[29284]: cluster 2024-03-26T03:49:27.393168+0000 mon.a (mon.0) 1238 : cluster [DBG] osdmap e111: 8 total, 8 up, 8 in 2024-03-26T03:49:27.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:27 smithi175 bash[29284]: audit 2024-03-26T03:49:27.408871+0000 mon.a (mon.0) 1239 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-11"}]: dispatch 2024-03-26T03:49:27.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:27 smithi175 bash[29284]: audit 2024-03-26T03:49:27.434151+0000 mon.a (mon.0) 1240 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64590-10", "tierpool":"test-rados-api-smithi175-64590-10-cache"}]: dispatch 2024-03-26T03:49:27.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:27 smithi175 bash[29284]: audit 2024-03-26T03:49:27.439017+0000 mon.b (mon.1) 105 : audit [INF] from='client.? 172.21.15.175:0/2686632908' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64590-10", "tierpool":"test-rados-api-smithi175-64590-10-cache"}]: dispatch 2024-03-26T03:49:28.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:28 smithi190 bash[23392]: audit 2024-03-26T03:49:27.754092+0000 mon.c (mon.2) 188 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:28.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:28 smithi190 bash[23392]: audit 2024-03-26T03:49:27.901297+0000 mon.a (mon.0) 1241 : audit [DBG] from='client.? 172.21.15.175:0/811501118' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-26T03:49:28.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:28 smithi190 bash[23392]: audit 2024-03-26T03:49:27.901782+0000 mgr.y (mgr.24431) 198 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-26T03:49:28.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:28 smithi190 bash[23392]: audit 2024-03-26T03:49:28.389892+0000 mon.a (mon.0) 1242 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-11"}]': finished 2024-03-26T03:49:28.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:28 smithi190 bash[23392]: audit 2024-03-26T03:49:28.389977+0000 mon.a (mon.0) 1243 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64590-10", "tierpool":"test-rados-api-smithi175-64590-10-cache"}]': finished 2024-03-26T03:49:28.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:28 smithi190 bash[23392]: cluster 2024-03-26T03:49:28.396336+0000 mon.a (mon.0) 1244 : cluster [DBG] osdmap e112: 8 total, 8 up, 8 in 2024-03-26T03:49:28.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:28 smithi190 bash[23392]: audit 2024-03-26T03:49:28.396571+0000 mon.a (mon.0) 1245 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-11", "mode": "writeback"}]: dispatch 2024-03-26T03:49:28.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:28 smithi175 bash[29284]: audit 2024-03-26T03:49:27.754092+0000 mon.c (mon.2) 188 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:28.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:28 smithi175 bash[29284]: audit 2024-03-26T03:49:27.901297+0000 mon.a (mon.0) 1241 : audit [DBG] from='client.? 172.21.15.175:0/811501118' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-26T03:49:28.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:28 smithi175 bash[29284]: audit 2024-03-26T03:49:27.901782+0000 mgr.y (mgr.24431) 198 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-26T03:49:28.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:28 smithi175 bash[29284]: audit 2024-03-26T03:49:28.389892+0000 mon.a (mon.0) 1242 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-11"}]': finished 2024-03-26T03:49:28.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:28 smithi175 bash[29284]: audit 2024-03-26T03:49:28.389977+0000 mon.a (mon.0) 1243 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64590-10", "tierpool":"test-rados-api-smithi175-64590-10-cache"}]': finished 2024-03-26T03:49:28.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:28 smithi175 bash[29284]: cluster 2024-03-26T03:49:28.396336+0000 mon.a (mon.0) 1244 : cluster [DBG] osdmap e112: 8 total, 8 up, 8 in 2024-03-26T03:49:28.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:28 smithi175 bash[29284]: audit 2024-03-26T03:49:28.396571+0000 mon.a (mon.0) 1245 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-11", "mode": "writeback"}]: dispatch 2024-03-26T03:49:28.995 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:49:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:49:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:49:28.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:28 smithi175 bash[21527]: audit 2024-03-26T03:49:27.754092+0000 mon.c (mon.2) 188 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:28.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:28 smithi175 bash[21527]: audit 2024-03-26T03:49:27.901297+0000 mon.a (mon.0) 1241 : audit [DBG] from='client.? 172.21.15.175:0/811501118' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-26T03:49:28.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:28 smithi175 bash[21527]: audit 2024-03-26T03:49:27.901782+0000 mgr.y (mgr.24431) 198 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-26T03:49:28.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:28 smithi175 bash[21527]: audit 2024-03-26T03:49:28.389892+0000 mon.a (mon.0) 1242 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-11"}]': finished 2024-03-26T03:49:28.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:28 smithi175 bash[21527]: audit 2024-03-26T03:49:28.389977+0000 mon.a (mon.0) 1243 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64590-10", "tierpool":"test-rados-api-smithi175-64590-10-cache"}]': finished 2024-03-26T03:49:28.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:28 smithi175 bash[21527]: cluster 2024-03-26T03:49:28.396336+0000 mon.a (mon.0) 1244 : cluster [DBG] osdmap e112: 8 total, 8 up, 8 in 2024-03-26T03:49:28.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:28 smithi175 bash[21527]: audit 2024-03-26T03:49:28.396571+0000 mon.a (mon.0) 1245 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-11", "mode": "writeback"}]: dispatch 2024-03-26T03:49:29.990 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:49:29 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:49:29.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:29 smithi190 bash[23392]: cluster 2024-03-26T03:49:28.588547+0000 mgr.y (mgr.24431) 199 : cluster [DBG] pgmap v165: 524 pgs: 64 creating+peering, 32 unknown, 428 active+clean; 146 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T03:49:29.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:29 smithi190 bash[23392]: audit 2024-03-26T03:49:28.756006+0000 mon.c (mon.2) 189 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:29.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:29 smithi190 bash[23392]: audit 2024-03-26T03:49:28.983742+0000 mon.c (mon.2) 190 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.0"}]: dispatch 2024-03-26T03:49:29.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:29 smithi190 bash[23392]: audit 2024-03-26T03:49:28.986026+0000 mgr.y (mgr.24431) 200 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.0"}]: dispatch 2024-03-26T03:49:29.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:29 smithi190 bash[23392]: audit 2024-03-26T03:49:28.988688+0000 mon.c (mon.2) 191 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.1"}]: dispatch 2024-03-26T03:49:29.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:29 smithi190 bash[23392]: audit 2024-03-26T03:49:28.989205+0000 mgr.y (mgr.24431) 201 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.1"}]: dispatch 2024-03-26T03:49:29.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:29 smithi190 bash[23392]: audit 2024-03-26T03:49:28.991708+0000 mon.c (mon.2) 192 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.2"}]: dispatch 2024-03-26T03:49:29.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:29 smithi190 bash[23392]: audit 2024-03-26T03:49:28.993594+0000 mgr.y (mgr.24431) 202 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.2"}]: dispatch 2024-03-26T03:49:29.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:29 smithi190 bash[23392]: audit 2024-03-26T03:49:28.994579+0000 mon.c (mon.2) 193 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.3"}]: dispatch 2024-03-26T03:49:29.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:29 smithi190 bash[23392]: audit 2024-03-26T03:49:28.994726+0000 mgr.y (mgr.24431) 203 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.3"}]: dispatch 2024-03-26T03:49:29.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:29 smithi190 bash[23392]: audit 2024-03-26T03:49:28.995311+0000 mon.c (mon.2) 194 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.4"}]: dispatch 2024-03-26T03:49:29.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:29 smithi190 bash[23392]: audit 2024-03-26T03:49:28.995525+0000 mgr.y (mgr.24431) 204 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.4"}]: dispatch 2024-03-26T03:49:29.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:29 smithi190 bash[23392]: audit 2024-03-26T03:49:28.995985+0000 mon.c (mon.2) 195 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.5"}]: dispatch 2024-03-26T03:49:29.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:29 smithi190 bash[23392]: audit 2024-03-26T03:49:28.996642+0000 mgr.y (mgr.24431) 205 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.5"}]: dispatch 2024-03-26T03:49:29.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:29 smithi190 bash[23392]: audit 2024-03-26T03:49:28.997353+0000 mon.c (mon.2) 196 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.6"}]: dispatch 2024-03-26T03:49:29.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:29 smithi190 bash[23392]: audit 2024-03-26T03:49:28.998719+0000 mgr.y (mgr.24431) 206 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.6"}]: dispatch 2024-03-26T03:49:29.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:29 smithi190 bash[23392]: audit 2024-03-26T03:49:28.999910+0000 mon.c (mon.2) 197 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.7"}]: dispatch 2024-03-26T03:49:29.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:29 smithi190 bash[23392]: audit 2024-03-26T03:49:29.000115+0000 mgr.y (mgr.24431) 207 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.7"}]: dispatch 2024-03-26T03:49:29.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[21527]: cluster 2024-03-26T03:49:28.588547+0000 mgr.y (mgr.24431) 199 : cluster [DBG] pgmap v165: 524 pgs: 64 creating+peering, 32 unknown, 428 active+clean; 146 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T03:49:29.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[21527]: audit 2024-03-26T03:49:28.756006+0000 mon.c (mon.2) 189 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:29.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[21527]: audit 2024-03-26T03:49:28.983742+0000 mon.c (mon.2) 190 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.0"}]: dispatch 2024-03-26T03:49:29.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[21527]: audit 2024-03-26T03:49:28.986026+0000 mgr.y (mgr.24431) 200 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.0"}]: dispatch 2024-03-26T03:49:29.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[21527]: audit 2024-03-26T03:49:28.988688+0000 mon.c (mon.2) 191 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.1"}]: dispatch 2024-03-26T03:49:29.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[21527]: audit 2024-03-26T03:49:28.989205+0000 mgr.y (mgr.24431) 201 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.1"}]: dispatch 2024-03-26T03:49:29.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[21527]: audit 2024-03-26T03:49:28.991708+0000 mon.c (mon.2) 192 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.2"}]: dispatch 2024-03-26T03:49:29.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[21527]: audit 2024-03-26T03:49:28.993594+0000 mgr.y (mgr.24431) 202 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.2"}]: dispatch 2024-03-26T03:49:29.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[21527]: audit 2024-03-26T03:49:28.994579+0000 mon.c (mon.2) 193 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.3"}]: dispatch 2024-03-26T03:49:29.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[21527]: audit 2024-03-26T03:49:28.994726+0000 mgr.y (mgr.24431) 203 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.3"}]: dispatch 2024-03-26T03:49:29.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[21527]: audit 2024-03-26T03:49:28.995311+0000 mon.c (mon.2) 194 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.4"}]: dispatch 2024-03-26T03:49:29.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[21527]: audit 2024-03-26T03:49:28.995525+0000 mgr.y (mgr.24431) 204 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.4"}]: dispatch 2024-03-26T03:49:29.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[21527]: audit 2024-03-26T03:49:28.995985+0000 mon.c (mon.2) 195 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.5"}]: dispatch 2024-03-26T03:49:29.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[21527]: audit 2024-03-26T03:49:28.996642+0000 mgr.y (mgr.24431) 205 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.5"}]: dispatch 2024-03-26T03:49:29.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[21527]: audit 2024-03-26T03:49:28.997353+0000 mon.c (mon.2) 196 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.6"}]: dispatch 2024-03-26T03:49:29.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[21527]: audit 2024-03-26T03:49:28.998719+0000 mgr.y (mgr.24431) 206 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.6"}]: dispatch 2024-03-26T03:49:29.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[21527]: audit 2024-03-26T03:49:28.999910+0000 mon.c (mon.2) 197 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.7"}]: dispatch 2024-03-26T03:49:29.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[21527]: audit 2024-03-26T03:49:29.000115+0000 mgr.y (mgr.24431) 207 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.7"}]: dispatch 2024-03-26T03:49:29.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[21527]: audit 2024-03-26T03:49:29.001631+0000 mon.c (mon.2) 198 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.8"}]: dispatch 2024-03-26T03:49:29.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[21527]: audit 2024-03-26T03:49:29.002221+0000 mgr.y (mgr.24431) 208 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.8"}]: dispatch 2024-03-26T03:49:29.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[21527]: audit 2024-03-26T03:49:29.004916+0000 mon.c (mon.2) 199 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.9"}]: dispatch 2024-03-26T03:49:29.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[21527]: audit 2024-03-26T03:49:29.005216+0000 mgr.y (mgr.24431) 209 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.9"}]: dispatch 2024-03-26T03:49:29.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[21527]: cluster 2024-03-26T03:49:29.391493+0000 mon.a (mon.0) 1246 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:49:29.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[21527]: audit 2024-03-26T03:49:29.393505+0000 mon.a (mon.0) 1247 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-11", "mode": "writeback"}]': finished 2024-03-26T03:49:29.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[21527]: cluster 2024-03-26T03:49:29.397692+0000 mon.a (mon.0) 1248 : cluster [DBG] osdmap e113: 8 total, 8 up, 8 in 2024-03-26T03:49:29.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[21527]: audit 2024-03-26T03:49:29.421897+0000 mon.a (mon.0) 1249 : audit [INF] from='client.? 172.21.15.175:0/1689806230' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi175-64398-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:29.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[21527]: audit 2024-03-26T03:49:29.438688+0000 mon.c (mon.2) 200 : audit [INF] from='client.? 172.21.15.175:0/358691016' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi175-64406-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:29.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[21527]: audit 2024-03-26T03:49:29.447591+0000 mon.a (mon.0) 1250 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi175-64406-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:29.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:29 smithi190 bash[23392]: audit 2024-03-26T03:49:29.001631+0000 mon.c (mon.2) 198 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.8"}]: dispatch 2024-03-26T03:49:29.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:29 smithi190 bash[23392]: audit 2024-03-26T03:49:29.002221+0000 mgr.y (mgr.24431) 208 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.8"}]: dispatch 2024-03-26T03:49:29.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:29 smithi190 bash[23392]: audit 2024-03-26T03:49:29.004916+0000 mon.c (mon.2) 199 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.9"}]: dispatch 2024-03-26T03:49:29.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:29 smithi190 bash[23392]: audit 2024-03-26T03:49:29.005216+0000 mgr.y (mgr.24431) 209 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.9"}]: dispatch 2024-03-26T03:49:29.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:29 smithi190 bash[23392]: cluster 2024-03-26T03:49:29.391493+0000 mon.a (mon.0) 1246 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:49:29.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:29 smithi190 bash[23392]: audit 2024-03-26T03:49:29.393505+0000 mon.a (mon.0) 1247 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-11", "mode": "writeback"}]': finished 2024-03-26T03:49:29.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:29 smithi190 bash[23392]: cluster 2024-03-26T03:49:29.397692+0000 mon.a (mon.0) 1248 : cluster [DBG] osdmap e113: 8 total, 8 up, 8 in 2024-03-26T03:49:29.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:29 smithi190 bash[23392]: audit 2024-03-26T03:49:29.421897+0000 mon.a (mon.0) 1249 : audit [INF] from='client.? 172.21.15.175:0/1689806230' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi175-64398-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:29.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:29 smithi190 bash[23392]: audit 2024-03-26T03:49:29.438688+0000 mon.c (mon.2) 200 : audit [INF] from='client.? 172.21.15.175:0/358691016' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi175-64406-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:29.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:29 smithi190 bash[23392]: audit 2024-03-26T03:49:29.447591+0000 mon.a (mon.0) 1250 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi175-64406-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:29.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[29284]: cluster 2024-03-26T03:49:28.588547+0000 mgr.y (mgr.24431) 199 : cluster [DBG] pgmap v165: 524 pgs: 64 creating+peering, 32 unknown, 428 active+clean; 146 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T03:49:29.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[29284]: audit 2024-03-26T03:49:28.756006+0000 mon.c (mon.2) 189 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:29.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[29284]: audit 2024-03-26T03:49:28.983742+0000 mon.c (mon.2) 190 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.0"}]: dispatch 2024-03-26T03:49:29.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[29284]: audit 2024-03-26T03:49:28.986026+0000 mgr.y (mgr.24431) 200 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.0"}]: dispatch 2024-03-26T03:49:29.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[29284]: audit 2024-03-26T03:49:28.988688+0000 mon.c (mon.2) 191 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.1"}]: dispatch 2024-03-26T03:49:29.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[29284]: audit 2024-03-26T03:49:28.989205+0000 mgr.y (mgr.24431) 201 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.1"}]: dispatch 2024-03-26T03:49:29.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[29284]: audit 2024-03-26T03:49:28.991708+0000 mon.c (mon.2) 192 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.2"}]: dispatch 2024-03-26T03:49:29.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[29284]: audit 2024-03-26T03:49:28.993594+0000 mgr.y (mgr.24431) 202 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.2"}]: dispatch 2024-03-26T03:49:29.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[29284]: audit 2024-03-26T03:49:28.994579+0000 mon.c (mon.2) 193 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.3"}]: dispatch 2024-03-26T03:49:29.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[29284]: audit 2024-03-26T03:49:28.994726+0000 mgr.y (mgr.24431) 203 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.3"}]: dispatch 2024-03-26T03:49:29.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[29284]: audit 2024-03-26T03:49:28.995311+0000 mon.c (mon.2) 194 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.4"}]: dispatch 2024-03-26T03:49:29.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[29284]: audit 2024-03-26T03:49:28.995525+0000 mgr.y (mgr.24431) 204 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.4"}]: dispatch 2024-03-26T03:49:29.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[29284]: audit 2024-03-26T03:49:28.995985+0000 mon.c (mon.2) 195 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.5"}]: dispatch 2024-03-26T03:49:29.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[29284]: audit 2024-03-26T03:49:28.996642+0000 mgr.y (mgr.24431) 205 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.5"}]: dispatch 2024-03-26T03:49:29.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[29284]: audit 2024-03-26T03:49:28.997353+0000 mon.c (mon.2) 196 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.6"}]: dispatch 2024-03-26T03:49:29.997 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[29284]: audit 2024-03-26T03:49:28.998719+0000 mgr.y (mgr.24431) 206 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.6"}]: dispatch 2024-03-26T03:49:29.997 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[29284]: audit 2024-03-26T03:49:28.999910+0000 mon.c (mon.2) 197 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.7"}]: dispatch 2024-03-26T03:49:29.997 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[29284]: audit 2024-03-26T03:49:29.000115+0000 mgr.y (mgr.24431) 207 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.7"}]: dispatch 2024-03-26T03:49:29.997 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[29284]: audit 2024-03-26T03:49:29.001631+0000 mon.c (mon.2) 198 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.8"}]: dispatch 2024-03-26T03:49:29.997 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[29284]: audit 2024-03-26T03:49:29.002221+0000 mgr.y (mgr.24431) 208 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.8"}]: dispatch 2024-03-26T03:49:29.997 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[29284]: audit 2024-03-26T03:49:29.004916+0000 mon.c (mon.2) 199 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.9"}]: dispatch 2024-03-26T03:49:29.997 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[29284]: audit 2024-03-26T03:49:29.005216+0000 mgr.y (mgr.24431) 209 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.9"}]: dispatch 2024-03-26T03:49:29.997 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[29284]: cluster 2024-03-26T03:49:29.391493+0000 mon.a (mon.0) 1246 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:49:29.997 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[29284]: audit 2024-03-26T03:49:29.393505+0000 mon.a (mon.0) 1247 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-11", "mode": "writeback"}]': finished 2024-03-26T03:49:29.997 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[29284]: cluster 2024-03-26T03:49:29.397692+0000 mon.a (mon.0) 1248 : cluster [DBG] osdmap e113: 8 total, 8 up, 8 in 2024-03-26T03:49:29.997 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[29284]: audit 2024-03-26T03:49:29.421897+0000 mon.a (mon.0) 1249 : audit [INF] from='client.? 172.21.15.175:0/1689806230' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi175-64398-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:29.997 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[29284]: audit 2024-03-26T03:49:29.438688+0000 mon.c (mon.2) 200 : audit [INF] from='client.? 172.21.15.175:0/358691016' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi175-64406-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:29.997 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:29 smithi175 bash[29284]: audit 2024-03-26T03:49:29.447591+0000 mon.a (mon.0) 1250 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi175-64406-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:30.422 INFO:tasks.workunit.client.0.smithi175.stdout: api_pool: Running main() from gmock_main.cc 2024-03-26T03:49:30.422 INFO:tasks.workunit.client.0.smithi175.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2024-03-26T03:49:30.423 INFO:tasks.workunit.client.0.smithi175.stdout: api_pool: [----------] Global test environment set-up. 2024-03-26T03:49:30.423 INFO:tasks.workunit.client.0.smithi175.stdout: api_pool: [----------] 9 tests from LibRadosPools 2024-03-26T03:49:30.423 INFO:tasks.workunit.client.0.smithi175.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2024-03-26T03:49:30.423 INFO:tasks.workunit.client.0.smithi175.stdout: api_pool: [ OK ] LibRadosPools.PoolList (3024 ms) 2024-03-26T03:49:30.423 INFO:tasks.workunit.client.0.smithi175.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2024-03-26T03:49:30.423 INFO:tasks.workunit.client.0.smithi175.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (3083 ms) 2024-03-26T03:49:30.423 INFO:tasks.workunit.client.0.smithi175.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2024-03-26T03:49:30.423 INFO:tasks.workunit.client.0.smithi175.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (2976 ms) 2024-03-26T03:49:30.423 INFO:tasks.workunit.client.0.smithi175.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-03-26T03:49:30.423 INFO:tasks.workunit.client.0.smithi175.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (3018 ms) 2024-03-26T03:49:30.423 INFO:tasks.workunit.client.0.smithi175.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-03-26T03:49:30.423 INFO:tasks.workunit.client.0.smithi175.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (3015 ms) 2024-03-26T03:49:30.423 INFO:tasks.workunit.client.0.smithi175.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2024-03-26T03:49:30.423 INFO:tasks.workunit.client.0.smithi175.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (5025 ms) 2024-03-26T03:49:30.424 INFO:tasks.workunit.client.0.smithi175.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2024-03-26T03:49:30.424 INFO:tasks.workunit.client.0.smithi175.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (4968 ms) 2024-03-26T03:49:30.424 INFO:tasks.workunit.client.0.smithi175.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-03-26T03:49:30.424 INFO:tasks.workunit.client.0.smithi175.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (5008 ms) 2024-03-26T03:49:30.424 INFO:tasks.workunit.client.0.smithi175.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2024-03-26T03:49:30.424 INFO:tasks.workunit.client.0.smithi175.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (7997 ms) 2024-03-26T03:49:30.424 INFO:tasks.workunit.client.0.smithi175.stdout: api_pool: [----------] 9 tests from LibRadosPools (38114 ms total) 2024-03-26T03:49:30.424 INFO:tasks.workunit.client.0.smithi175.stdout: api_pool: 2024-03-26T03:49:30.424 INFO:tasks.workunit.client.0.smithi175.stdout: api_pool: [----------] Global test environment tear-down 2024-03-26T03:49:30.424 INFO:tasks.workunit.client.0.smithi175.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (38114 ms total) 2024-03-26T03:49:30.424 INFO:tasks.workunit.client.0.smithi175.stdout: api_pool: [ PASSED ] 9 tests. 2024-03-26T03:49:30.460 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2024-03-26T03:49:30.461 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [----------] Global test environment set-up. 2024-03-26T03:49:30.461 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2024-03-26T03:49:30.461 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2024-03-26T03:49:30.461 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2024-03-26T03:49:30.461 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-03-26T03:49:30.461 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: 2024-03-26T03:49:30.461 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2024-03-26T03:49:30.461 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-03-26T03:49:30.461 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2024-03-26T03:49:30.461 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: 2024-03-26T03:48:52.363+0000 7f43f96d2880 -1 failed for service _ceph-mon._tcp 2024-03-26T03:49:30.461 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: 2024-03-26T03:48:52.363+0000 7f43f96d2880 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-03-26T03:49:30.461 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (60 ms) 2024-03-26T03:49:30.461 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-03-26T03:49:30.461 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (2020 ms) 2024-03-26T03:49:30.462 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (2080 ms total) 2024-03-26T03:49:30.462 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: 2024-03-26T03:49:30.462 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2024-03-26T03:49:30.462 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2024-03-26T03:49:30.462 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: started 0x563073aba300 2024-03-26T03:49:30.462 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: started 0x563073d5e390 2024-03-26T03:49:30.462 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: started 2 aios 2024-03-26T03:49:30.462 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: waiting 0x563073aba300 2024-03-26T03:49:30.462 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: waiting 0x563073d5e390 2024-03-26T03:49:30.462 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: done. 2024-03-26T03:49:30.462 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (6002 ms) 2024-03-26T03:49:30.462 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (6002 ms total) 2024-03-26T03:49:30.462 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: 2024-03-26T03:49:30.462 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2024-03-26T03:49:30.462 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2024-03-26T03:49:30.463 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-03-26T03:49:30.463 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2024-03-26T03:49:30.463 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [ OK ] LibRadosMisc.Exec (138 ms) 2024-03-26T03:49:30.463 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2024-03-26T03:49:30.463 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (16 ms) 2024-03-26T03:49:30.463 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2024-03-26T03:49:30.463 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (4 ms) 2024-03-26T03:49:30.463 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2024-03-26T03:49:30.463 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [ OK ] LibRadosMisc.Applications (4847 ms) 2024-03-26T03:49:30.463 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2024-03-26T03:49:30.463 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-03-26T03:49:30.463 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2024-03-26T03:49:30.463 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-03-26T03:49:30.463 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2024-03-26T03:49:30.463 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (22819 ms) 2024-03-26T03:49:30.464 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [----------] 8 tests from LibRadosMisc (27825 ms total) 2024-03-26T03:49:30.464 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: 2024-03-26T03:49:30.464 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [----------] Global test environment tear-down 2024-03-26T03:49:30.464 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (38121 ms total) 2024-03-26T03:49:30.464 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc: [ PASSED ] 12 tests. 2024-03-26T03:49:30.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:30 smithi190 bash[23392]: cluster 2024-03-26T03:49:29.233868+0000 osd.0 (osd.0) 9 : cluster [DBG] 16.5 deep-scrub starts 2024-03-26T03:49:30.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:30 smithi190 bash[23392]: cluster 2024-03-26T03:49:29.236086+0000 osd.0 (osd.0) 10 : cluster [DBG] 16.5 deep-scrub ok 2024-03-26T03:49:30.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:30 smithi190 bash[23392]: cluster 2024-03-26T03:49:29.339710+0000 osd.2 (osd.2) 5 : cluster [DBG] 16.1 deep-scrub starts 2024-03-26T03:49:30.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:30 smithi190 bash[23392]: cluster 2024-03-26T03:49:29.345208+0000 osd.2 (osd.2) 6 : cluster [DBG] 16.1 deep-scrub ok 2024-03-26T03:49:30.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:30 smithi190 bash[23392]: cluster 2024-03-26T03:49:29.574966+0000 osd.6 (osd.6) 5 : cluster [DBG] 16.6 deep-scrub starts 2024-03-26T03:49:30.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:30 smithi190 bash[23392]: cluster 2024-03-26T03:49:29.576289+0000 osd.6 (osd.6) 6 : cluster [DBG] 16.6 deep-scrub ok 2024-03-26T03:49:30.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:30 smithi190 bash[23392]: cluster 2024-03-26T03:49:29.611411+0000 osd.3 (osd.3) 7 : cluster [DBG] 16.4 deep-scrub starts 2024-03-26T03:49:30.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:30 smithi190 bash[23392]: cluster 2024-03-26T03:49:29.684472+0000 osd.3 (osd.3) 8 : cluster [DBG] 16.4 deep-scrub ok 2024-03-26T03:49:30.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:30 smithi190 bash[23392]: audit 2024-03-26T03:49:29.715483+0000 mgr.y (mgr.24431) 210 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:49:30.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:30 smithi190 bash[23392]: audit 2024-03-26T03:49:29.759743+0000 mon.c (mon.2) 201 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:30.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:30 smithi190 bash[23392]: cluster 2024-03-26T03:49:29.874913+0000 osd.1 (osd.1) 5 : cluster [DBG] 16.9 deep-scrub starts 2024-03-26T03:49:30.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:30 smithi190 bash[23392]: cluster 2024-03-26T03:49:29.876677+0000 osd.1 (osd.1) 6 : cluster [DBG] 16.9 deep-scrub ok 2024-03-26T03:49:30.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:30 smithi190 bash[23392]: cluster 2024-03-26T03:49:29.972922+0000 osd.5 (osd.5) 5 : cluster [DBG] 16.3 deep-scrub starts 2024-03-26T03:49:30.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:30 smithi190 bash[23392]: cluster 2024-03-26T03:49:29.975845+0000 osd.5 (osd.5) 6 : cluster [DBG] 16.3 deep-scrub ok 2024-03-26T03:49:30.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:30 smithi190 bash[23392]: cluster 2024-03-26T03:49:29.981002+0000 osd.4 (osd.4) 5 : cluster [DBG] 16.2 deep-scrub starts 2024-03-26T03:49:30.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:30 smithi190 bash[23392]: cluster 2024-03-26T03:49:29.989077+0000 osd.4 (osd.4) 6 : cluster [DBG] 16.2 deep-scrub ok 2024-03-26T03:49:30.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:30 smithi190 bash[23392]: audit 2024-03-26T03:49:30.398682+0000 mon.a (mon.0) 1251 : audit [INF] from='client.? 172.21.15.175:0/1689806230' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi175-64398-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:30.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:30 smithi190 bash[23392]: audit 2024-03-26T03:49:30.398897+0000 mon.a (mon.0) 1252 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi175-64406-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:30.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:30 smithi190 bash[23392]: cluster 2024-03-26T03:49:30.407284+0000 mon.a (mon.0) 1253 : cluster [DBG] osdmap e114: 8 total, 8 up, 8 in 2024-03-26T03:49:30.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:30 smithi190 bash[23392]: audit 2024-03-26T03:49:30.562972+0000 mon.a (mon.0) 1254 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:49:30.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[21527]: cluster 2024-03-26T03:49:29.233868+0000 osd.0 (osd.0) 9 : cluster [DBG] 16.5 deep-scrub starts 2024-03-26T03:49:30.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[21527]: cluster 2024-03-26T03:49:29.236086+0000 osd.0 (osd.0) 10 : cluster [DBG] 16.5 deep-scrub ok 2024-03-26T03:49:30.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[21527]: cluster 2024-03-26T03:49:29.339710+0000 osd.2 (osd.2) 5 : cluster [DBG] 16.1 deep-scrub starts 2024-03-26T03:49:30.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[21527]: cluster 2024-03-26T03:49:29.345208+0000 osd.2 (osd.2) 6 : cluster [DBG] 16.1 deep-scrub ok 2024-03-26T03:49:30.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[21527]: cluster 2024-03-26T03:49:29.574966+0000 osd.6 (osd.6) 5 : cluster [DBG] 16.6 deep-scrub starts 2024-03-26T03:49:30.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[21527]: cluster 2024-03-26T03:49:29.576289+0000 osd.6 (osd.6) 6 : cluster [DBG] 16.6 deep-scrub ok 2024-03-26T03:49:30.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[21527]: cluster 2024-03-26T03:49:29.611411+0000 osd.3 (osd.3) 7 : cluster [DBG] 16.4 deep-scrub starts 2024-03-26T03:49:30.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[21527]: cluster 2024-03-26T03:49:29.684472+0000 osd.3 (osd.3) 8 : cluster [DBG] 16.4 deep-scrub ok 2024-03-26T03:49:30.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[21527]: audit 2024-03-26T03:49:29.715483+0000 mgr.y (mgr.24431) 210 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:49:30.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[21527]: audit 2024-03-26T03:49:29.759743+0000 mon.c (mon.2) 201 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:30.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[21527]: cluster 2024-03-26T03:49:29.874913+0000 osd.1 (osd.1) 5 : cluster [DBG] 16.9 deep-scrub starts 2024-03-26T03:49:30.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[21527]: cluster 2024-03-26T03:49:29.876677+0000 osd.1 (osd.1) 6 : cluster [DBG] 16.9 deep-scrub ok 2024-03-26T03:49:30.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[21527]: cluster 2024-03-26T03:49:29.972922+0000 osd.5 (osd.5) 5 : cluster [DBG] 16.3 deep-scrub starts 2024-03-26T03:49:30.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[21527]: cluster 2024-03-26T03:49:29.975845+0000 osd.5 (osd.5) 6 : cluster [DBG] 16.3 deep-scrub ok 2024-03-26T03:49:30.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[21527]: cluster 2024-03-26T03:49:29.981002+0000 osd.4 (osd.4) 5 : cluster [DBG] 16.2 deep-scrub starts 2024-03-26T03:49:30.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[21527]: cluster 2024-03-26T03:49:29.989077+0000 osd.4 (osd.4) 6 : cluster [DBG] 16.2 deep-scrub ok 2024-03-26T03:49:30.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[21527]: audit 2024-03-26T03:49:30.398682+0000 mon.a (mon.0) 1251 : audit [INF] from='client.? 172.21.15.175:0/1689806230' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi175-64398-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:30.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[21527]: audit 2024-03-26T03:49:30.398897+0000 mon.a (mon.0) 1252 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi175-64406-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:30.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[21527]: cluster 2024-03-26T03:49:30.407284+0000 mon.a (mon.0) 1253 : cluster [DBG] osdmap e114: 8 total, 8 up, 8 in 2024-03-26T03:49:30.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[21527]: audit 2024-03-26T03:49:30.562972+0000 mon.a (mon.0) 1254 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:49:30.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[29284]: cluster 2024-03-26T03:49:29.233868+0000 osd.0 (osd.0) 9 : cluster [DBG] 16.5 deep-scrub starts 2024-03-26T03:49:30.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[29284]: cluster 2024-03-26T03:49:29.236086+0000 osd.0 (osd.0) 10 : cluster [DBG] 16.5 deep-scrub ok 2024-03-26T03:49:30.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[29284]: cluster 2024-03-26T03:49:29.339710+0000 osd.2 (osd.2) 5 : cluster [DBG] 16.1 deep-scrub starts 2024-03-26T03:49:30.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[29284]: cluster 2024-03-26T03:49:29.345208+0000 osd.2 (osd.2) 6 : cluster [DBG] 16.1 deep-scrub ok 2024-03-26T03:49:30.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[29284]: cluster 2024-03-26T03:49:29.574966+0000 osd.6 (osd.6) 5 : cluster [DBG] 16.6 deep-scrub starts 2024-03-26T03:49:30.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[29284]: cluster 2024-03-26T03:49:29.576289+0000 osd.6 (osd.6) 6 : cluster [DBG] 16.6 deep-scrub ok 2024-03-26T03:49:30.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[29284]: cluster 2024-03-26T03:49:29.611411+0000 osd.3 (osd.3) 7 : cluster [DBG] 16.4 deep-scrub starts 2024-03-26T03:49:30.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[29284]: cluster 2024-03-26T03:49:29.684472+0000 osd.3 (osd.3) 8 : cluster [DBG] 16.4 deep-scrub ok 2024-03-26T03:49:30.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[29284]: audit 2024-03-26T03:49:29.715483+0000 mgr.y (mgr.24431) 210 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:49:30.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[29284]: audit 2024-03-26T03:49:29.759743+0000 mon.c (mon.2) 201 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:30.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[29284]: cluster 2024-03-26T03:49:29.874913+0000 osd.1 (osd.1) 5 : cluster [DBG] 16.9 deep-scrub starts 2024-03-26T03:49:30.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[29284]: cluster 2024-03-26T03:49:29.876677+0000 osd.1 (osd.1) 6 : cluster [DBG] 16.9 deep-scrub ok 2024-03-26T03:49:30.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[29284]: cluster 2024-03-26T03:49:29.972922+0000 osd.5 (osd.5) 5 : cluster [DBG] 16.3 deep-scrub starts 2024-03-26T03:49:30.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[29284]: cluster 2024-03-26T03:49:29.975845+0000 osd.5 (osd.5) 6 : cluster [DBG] 16.3 deep-scrub ok 2024-03-26T03:49:30.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[29284]: cluster 2024-03-26T03:49:29.981002+0000 osd.4 (osd.4) 5 : cluster [DBG] 16.2 deep-scrub starts 2024-03-26T03:49:30.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[29284]: cluster 2024-03-26T03:49:29.989077+0000 osd.4 (osd.4) 6 : cluster [DBG] 16.2 deep-scrub ok 2024-03-26T03:49:30.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[29284]: audit 2024-03-26T03:49:30.398682+0000 mon.a (mon.0) 1251 : audit [INF] from='client.? 172.21.15.175:0/1689806230' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi175-64398-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:30.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[29284]: audit 2024-03-26T03:49:30.398897+0000 mon.a (mon.0) 1252 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi175-64406-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:30.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[29284]: cluster 2024-03-26T03:49:30.407284+0000 mon.a (mon.0) 1253 : cluster [DBG] osdmap e114: 8 total, 8 up, 8 in 2024-03-26T03:49:30.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:30 smithi175 bash[29284]: audit 2024-03-26T03:49:30.562972+0000 mon.a (mon.0) 1254 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:49:31.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:31 smithi190 bash[23392]: cluster 2024-03-26T03:49:30.210193+0000 osd.0 (osd.0) 11 : cluster [DBG] 16.8 deep-scrub starts 2024-03-26T03:49:31.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:31 smithi190 bash[23392]: cluster 2024-03-26T03:49:30.214403+0000 osd.0 (osd.0) 12 : cluster [DBG] 16.8 deep-scrub ok 2024-03-26T03:49:31.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:31 smithi190 bash[23392]: cluster 2024-03-26T03:49:30.589219+0000 mgr.y (mgr.24431) 211 : cluster [DBG] pgmap v168: 556 pgs: 32 unknown, 128 creating+peering, 396 active+clean; 217 MiB data, 998 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 18 MiB/s wr, 6 op/s 2024-03-26T03:49:31.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:31 smithi190 bash[23392]: cluster 2024-03-26T03:49:30.617424+0000 osd.3 (osd.3) 9 : cluster [DBG] 16.7 deep-scrub starts 2024-03-26T03:49:31.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:31 smithi190 bash[23392]: cluster 2024-03-26T03:49:30.620385+0000 osd.3 (osd.3) 10 : cluster [DBG] 16.7 deep-scrub ok 2024-03-26T03:49:31.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:31 smithi190 bash[23392]: audit 2024-03-26T03:49:30.761025+0000 mon.c (mon.2) 202 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:31.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:31 smithi190 bash[23392]: cluster 2024-03-26T03:49:31.380048+0000 mon.a (mon.0) 1255 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:31.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:31 smithi190 bash[23392]: audit 2024-03-26T03:49:31.410452+0000 mon.a (mon.0) 1256 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:49:31.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:31 smithi190 bash[23392]: cluster 2024-03-26T03:49:31.418486+0000 mon.a (mon.0) 1257 : cluster [DBG] osdmap e115: 8 total, 8 up, 8 in 2024-03-26T03:49:31.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:31 smithi190 bash[23392]: audit 2024-03-26T03:49:31.419384+0000 mon.a (mon.0) 1258 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-11"}]: dispatch 2024-03-26T03:49:31.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:31 smithi175 bash[21527]: cluster 2024-03-26T03:49:30.210193+0000 osd.0 (osd.0) 11 : cluster [DBG] 16.8 deep-scrub starts 2024-03-26T03:49:31.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:31 smithi175 bash[21527]: cluster 2024-03-26T03:49:30.214403+0000 osd.0 (osd.0) 12 : cluster [DBG] 16.8 deep-scrub ok 2024-03-26T03:49:31.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:31 smithi175 bash[21527]: cluster 2024-03-26T03:49:30.589219+0000 mgr.y (mgr.24431) 211 : cluster [DBG] pgmap v168: 556 pgs: 32 unknown, 128 creating+peering, 396 active+clean; 217 MiB data, 998 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 18 MiB/s wr, 6 op/s 2024-03-26T03:49:31.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:31 smithi175 bash[21527]: cluster 2024-03-26T03:49:30.617424+0000 osd.3 (osd.3) 9 : cluster [DBG] 16.7 deep-scrub starts 2024-03-26T03:49:31.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:31 smithi175 bash[21527]: cluster 2024-03-26T03:49:30.620385+0000 osd.3 (osd.3) 10 : cluster [DBG] 16.7 deep-scrub ok 2024-03-26T03:49:31.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:31 smithi175 bash[21527]: audit 2024-03-26T03:49:30.761025+0000 mon.c (mon.2) 202 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:31.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:31 smithi175 bash[21527]: cluster 2024-03-26T03:49:31.380048+0000 mon.a (mon.0) 1255 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:31.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:31 smithi175 bash[21527]: audit 2024-03-26T03:49:31.410452+0000 mon.a (mon.0) 1256 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:49:31.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:31 smithi175 bash[21527]: cluster 2024-03-26T03:49:31.418486+0000 mon.a (mon.0) 1257 : cluster [DBG] osdmap e115: 8 total, 8 up, 8 in 2024-03-26T03:49:31.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:31 smithi175 bash[21527]: audit 2024-03-26T03:49:31.419384+0000 mon.a (mon.0) 1258 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-11"}]: dispatch 2024-03-26T03:49:31.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:31 smithi175 bash[29284]: cluster 2024-03-26T03:49:30.210193+0000 osd.0 (osd.0) 11 : cluster [DBG] 16.8 deep-scrub starts 2024-03-26T03:49:31.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:31 smithi175 bash[29284]: cluster 2024-03-26T03:49:30.214403+0000 osd.0 (osd.0) 12 : cluster [DBG] 16.8 deep-scrub ok 2024-03-26T03:49:31.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:31 smithi175 bash[29284]: cluster 2024-03-26T03:49:30.589219+0000 mgr.y (mgr.24431) 211 : cluster [DBG] pgmap v168: 556 pgs: 32 unknown, 128 creating+peering, 396 active+clean; 217 MiB data, 998 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 18 MiB/s wr, 6 op/s 2024-03-26T03:49:31.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:31 smithi175 bash[29284]: cluster 2024-03-26T03:49:30.617424+0000 osd.3 (osd.3) 9 : cluster [DBG] 16.7 deep-scrub starts 2024-03-26T03:49:31.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:31 smithi175 bash[29284]: cluster 2024-03-26T03:49:30.620385+0000 osd.3 (osd.3) 10 : cluster [DBG] 16.7 deep-scrub ok 2024-03-26T03:49:31.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:31 smithi175 bash[29284]: audit 2024-03-26T03:49:30.761025+0000 mon.c (mon.2) 202 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:31.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:31 smithi175 bash[29284]: cluster 2024-03-26T03:49:31.380048+0000 mon.a (mon.0) 1255 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:31.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:31 smithi175 bash[29284]: audit 2024-03-26T03:49:31.410452+0000 mon.a (mon.0) 1256 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:49:31.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:31 smithi175 bash[29284]: cluster 2024-03-26T03:49:31.418486+0000 mon.a (mon.0) 1257 : cluster [DBG] osdmap e115: 8 total, 8 up, 8 in 2024-03-26T03:49:31.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:31 smithi175 bash[29284]: audit 2024-03-26T03:49:31.419384+0000 mon.a (mon.0) 1258 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-11"}]: dispatch 2024-03-26T03:49:32.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:32 smithi190 bash[23392]: cluster 2024-03-26T03:49:31.196590+0000 osd.0 (osd.0) 13 : cluster [DBG] 16.0 deep-scrub starts 2024-03-26T03:49:32.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:32 smithi190 bash[23392]: cluster 2024-03-26T03:49:31.199116+0000 osd.0 (osd.0) 14 : cluster [DBG] 16.0 deep-scrub ok 2024-03-26T03:49:32.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:32 smithi190 bash[23392]: audit 2024-03-26T03:49:31.762751+0000 mon.c (mon.2) 203 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:32.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:32 smithi190 bash[23392]: cluster 2024-03-26T03:49:32.413034+0000 mon.a (mon.0) 1259 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:49:32.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:32 smithi190 bash[23392]: audit 2024-03-26T03:49:32.416841+0000 mon.a (mon.0) 1260 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-11"}]': finished 2024-03-26T03:49:32.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:32 smithi190 bash[23392]: cluster 2024-03-26T03:49:32.433028+0000 mon.a (mon.0) 1261 : cluster [DBG] osdmap e116: 8 total, 8 up, 8 in 2024-03-26T03:49:32.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:32 smithi190 bash[23392]: audit 2024-03-26T03:49:32.448449+0000 mon.a (mon.0) 1262 : audit [INF] from='client.? 172.21.15.175:0/2491267037' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi175-64406-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:32.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:32 smithi190 bash[23392]: audit 2024-03-26T03:49:32.451491+0000 mon.a (mon.0) 1263 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi175-64398-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:32.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:32 smithi190 bash[23392]: audit 2024-03-26T03:49:32.452106+0000 mon.a (mon.0) 1264 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi175-64639-10"}]: dispatch 2024-03-26T03:49:32.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:32 smithi190 bash[23392]: audit 2024-03-26T03:49:32.454232+0000 mon.b (mon.1) 106 : audit [INF] from='client.? 172.21.15.175:0/910016756' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi175-64398-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:32.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:32 smithi190 bash[23392]: audit 2024-03-26T03:49:32.455309+0000 mon.b (mon.1) 107 : audit [INF] from='client.? 172.21.15.175:0/2989316611' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi175-64639-10"}]: dispatch 2024-03-26T03:49:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:32 smithi175 bash[21527]: cluster 2024-03-26T03:49:31.196590+0000 osd.0 (osd.0) 13 : cluster [DBG] 16.0 deep-scrub starts 2024-03-26T03:49:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:32 smithi175 bash[21527]: cluster 2024-03-26T03:49:31.199116+0000 osd.0 (osd.0) 14 : cluster [DBG] 16.0 deep-scrub ok 2024-03-26T03:49:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:32 smithi175 bash[21527]: audit 2024-03-26T03:49:31.762751+0000 mon.c (mon.2) 203 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:32 smithi175 bash[21527]: cluster 2024-03-26T03:49:32.413034+0000 mon.a (mon.0) 1259 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:49:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:32 smithi175 bash[21527]: audit 2024-03-26T03:49:32.416841+0000 mon.a (mon.0) 1260 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-11"}]': finished 2024-03-26T03:49:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:32 smithi175 bash[21527]: cluster 2024-03-26T03:49:32.433028+0000 mon.a (mon.0) 1261 : cluster [DBG] osdmap e116: 8 total, 8 up, 8 in 2024-03-26T03:49:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:32 smithi175 bash[21527]: audit 2024-03-26T03:49:32.448449+0000 mon.a (mon.0) 1262 : audit [INF] from='client.? 172.21.15.175:0/2491267037' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi175-64406-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:32 smithi175 bash[21527]: audit 2024-03-26T03:49:32.451491+0000 mon.a (mon.0) 1263 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi175-64398-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:32 smithi175 bash[21527]: audit 2024-03-26T03:49:32.452106+0000 mon.a (mon.0) 1264 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi175-64639-10"}]: dispatch 2024-03-26T03:49:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:32 smithi175 bash[21527]: audit 2024-03-26T03:49:32.454232+0000 mon.b (mon.1) 106 : audit [INF] from='client.? 172.21.15.175:0/910016756' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi175-64398-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:32 smithi175 bash[21527]: audit 2024-03-26T03:49:32.455309+0000 mon.b (mon.1) 107 : audit [INF] from='client.? 172.21.15.175:0/2989316611' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi175-64639-10"}]: dispatch 2024-03-26T03:49:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:32 smithi175 bash[29284]: cluster 2024-03-26T03:49:31.196590+0000 osd.0 (osd.0) 13 : cluster [DBG] 16.0 deep-scrub starts 2024-03-26T03:49:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:32 smithi175 bash[29284]: cluster 2024-03-26T03:49:31.199116+0000 osd.0 (osd.0) 14 : cluster [DBG] 16.0 deep-scrub ok 2024-03-26T03:49:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:32 smithi175 bash[29284]: audit 2024-03-26T03:49:31.762751+0000 mon.c (mon.2) 203 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:32 smithi175 bash[29284]: cluster 2024-03-26T03:49:32.413034+0000 mon.a (mon.0) 1259 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:49:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:32 smithi175 bash[29284]: audit 2024-03-26T03:49:32.416841+0000 mon.a (mon.0) 1260 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-11"}]': finished 2024-03-26T03:49:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:32 smithi175 bash[29284]: cluster 2024-03-26T03:49:32.433028+0000 mon.a (mon.0) 1261 : cluster [DBG] osdmap e116: 8 total, 8 up, 8 in 2024-03-26T03:49:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:32 smithi175 bash[29284]: audit 2024-03-26T03:49:32.448449+0000 mon.a (mon.0) 1262 : audit [INF] from='client.? 172.21.15.175:0/2491267037' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi175-64406-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:32 smithi175 bash[29284]: audit 2024-03-26T03:49:32.451491+0000 mon.a (mon.0) 1263 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi175-64398-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:32 smithi175 bash[29284]: audit 2024-03-26T03:49:32.452106+0000 mon.a (mon.0) 1264 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi175-64639-10"}]: dispatch 2024-03-26T03:49:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:32 smithi175 bash[29284]: audit 2024-03-26T03:49:32.454232+0000 mon.b (mon.1) 106 : audit [INF] from='client.? 172.21.15.175:0/910016756' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi175-64398-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:32 smithi175 bash[29284]: audit 2024-03-26T03:49:32.455309+0000 mon.b (mon.1) 107 : audit [INF] from='client.? 172.21.15.175:0/2989316611' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi175-64639-10"}]: dispatch 2024-03-26T03:49:33.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:33 smithi190 bash[23392]: cluster 2024-03-26T03:49:32.590338+0000 mgr.y (mgr.24431) 212 : cluster [DBG] pgmap v171: 548 pgs: 160 unknown, 388 active+clean; 217 MiB data, 998 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 18 MiB/s wr, 5 op/s 2024-03-26T03:49:33.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:33 smithi190 bash[23392]: audit 2024-03-26T03:49:32.764548+0000 mon.c (mon.2) 204 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:33.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:33 smithi190 bash[23392]: audit 2024-03-26T03:49:33.427132+0000 mon.a (mon.0) 1265 : audit [INF] from='client.? 172.21.15.175:0/2491267037' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi175-64406-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:33.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:33 smithi190 bash[23392]: audit 2024-03-26T03:49:33.427308+0000 mon.a (mon.0) 1266 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi175-64398-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:33.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:33 smithi190 bash[23392]: audit 2024-03-26T03:49:33.427484+0000 mon.a (mon.0) 1267 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi175-64639-10"}]': finished 2024-03-26T03:49:33.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:33 smithi190 bash[23392]: cluster 2024-03-26T03:49:33.436694+0000 mon.a (mon.0) 1268 : cluster [DBG] osdmap e117: 8 total, 8 up, 8 in 2024-03-26T03:49:33.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:33 smithi190 bash[23392]: audit 2024-03-26T03:49:33.449381+0000 mon.b (mon.1) 108 : audit [INF] from='client.? 172.21.15.175:0/2989316611' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi175-64639-10"}]: dispatch 2024-03-26T03:49:33.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:33 smithi190 bash[23392]: audit 2024-03-26T03:49:33.455380+0000 mon.a (mon.0) 1269 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi175-64639-10"}]: dispatch 2024-03-26T03:49:33.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:33 smithi175 bash[21527]: cluster 2024-03-26T03:49:32.590338+0000 mgr.y (mgr.24431) 212 : cluster [DBG] pgmap v171: 548 pgs: 160 unknown, 388 active+clean; 217 MiB data, 998 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 18 MiB/s wr, 5 op/s 2024-03-26T03:49:33.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:33 smithi175 bash[21527]: audit 2024-03-26T03:49:32.764548+0000 mon.c (mon.2) 204 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:33.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:33 smithi175 bash[21527]: audit 2024-03-26T03:49:33.427132+0000 mon.a (mon.0) 1265 : audit [INF] from='client.? 172.21.15.175:0/2491267037' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi175-64406-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:33.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:33 smithi175 bash[21527]: audit 2024-03-26T03:49:33.427308+0000 mon.a (mon.0) 1266 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi175-64398-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:33.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:33 smithi175 bash[21527]: audit 2024-03-26T03:49:33.427484+0000 mon.a (mon.0) 1267 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi175-64639-10"}]': finished 2024-03-26T03:49:33.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:33 smithi175 bash[21527]: cluster 2024-03-26T03:49:33.436694+0000 mon.a (mon.0) 1268 : cluster [DBG] osdmap e117: 8 total, 8 up, 8 in 2024-03-26T03:49:33.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:33 smithi175 bash[21527]: audit 2024-03-26T03:49:33.449381+0000 mon.b (mon.1) 108 : audit [INF] from='client.? 172.21.15.175:0/2989316611' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi175-64639-10"}]: dispatch 2024-03-26T03:49:33.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:33 smithi175 bash[21527]: audit 2024-03-26T03:49:33.455380+0000 mon.a (mon.0) 1269 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi175-64639-10"}]: dispatch 2024-03-26T03:49:33.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:33 smithi175 bash[29284]: cluster 2024-03-26T03:49:32.590338+0000 mgr.y (mgr.24431) 212 : cluster [DBG] pgmap v171: 548 pgs: 160 unknown, 388 active+clean; 217 MiB data, 998 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 18 MiB/s wr, 5 op/s 2024-03-26T03:49:33.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:33 smithi175 bash[29284]: audit 2024-03-26T03:49:32.764548+0000 mon.c (mon.2) 204 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:33.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:33 smithi175 bash[29284]: audit 2024-03-26T03:49:33.427132+0000 mon.a (mon.0) 1265 : audit [INF] from='client.? 172.21.15.175:0/2491267037' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi175-64406-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:33.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:33 smithi175 bash[29284]: audit 2024-03-26T03:49:33.427308+0000 mon.a (mon.0) 1266 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi175-64398-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:33.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:33 smithi175 bash[29284]: audit 2024-03-26T03:49:33.427484+0000 mon.a (mon.0) 1267 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi175-64639-10"}]': finished 2024-03-26T03:49:33.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:33 smithi175 bash[29284]: cluster 2024-03-26T03:49:33.436694+0000 mon.a (mon.0) 1268 : cluster [DBG] osdmap e117: 8 total, 8 up, 8 in 2024-03-26T03:49:33.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:33 smithi175 bash[29284]: audit 2024-03-26T03:49:33.449381+0000 mon.b (mon.1) 108 : audit [INF] from='client.? 172.21.15.175:0/2989316611' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi175-64639-10"}]: dispatch 2024-03-26T03:49:33.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:33 smithi175 bash[29284]: audit 2024-03-26T03:49:33.455380+0000 mon.a (mon.0) 1269 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi175-64639-10"}]: dispatch 2024-03-26T03:49:34.475 INFO:tasks.workunit.client.0.smithi175.stdout: io: Running main() from gmock_main.cc 2024-03-26T03:49:34.475 INFO:tasks.workunit.client.0.smithi175.stdout: io: [==========] Running 14 tests from 1 test suite. 2024-03-26T03:49:34.475 INFO:tasks.workunit.client.0.smithi175.stdout: io: [----------] Global test environment set-up. 2024-03-26T03:49:34.475 INFO:tasks.workunit.client.0.smithi175.stdout: io: [----------] 14 tests from NeoRadosIo 2024-03-26T03:49:34.475 INFO:tasks.workunit.client.0.smithi175.stdout: io: [ RUN ] NeoRadosIo.Limits 2024-03-26T03:49:34.475 INFO:tasks.workunit.client.0.smithi175.stdout: io: [ OK ] NeoRadosIo.Limits (2774 ms) 2024-03-26T03:49:34.475 INFO:tasks.workunit.client.0.smithi175.stdout: io: [ RUN ] NeoRadosIo.SimpleWrite 2024-03-26T03:49:34.476 INFO:tasks.workunit.client.0.smithi175.stdout: io: [ OK ] NeoRadosIo.SimpleWrite (3061 ms) 2024-03-26T03:49:34.476 INFO:tasks.workunit.client.0.smithi175.stdout: io: [ RUN ] NeoRadosIo.ReadOp 2024-03-26T03:49:34.476 INFO:tasks.workunit.client.0.smithi175.stdout: io: [ OK ] NeoRadosIo.ReadOp (3021 ms) 2024-03-26T03:49:34.476 INFO:tasks.workunit.client.0.smithi175.stdout: io: [ RUN ] NeoRadosIo.SparseRead 2024-03-26T03:49:34.476 INFO:tasks.workunit.client.0.smithi175.stdout: io: [ OK ] NeoRadosIo.SparseRead (2989 ms) 2024-03-26T03:49:34.476 INFO:tasks.workunit.client.0.smithi175.stdout: io: [ RUN ] NeoRadosIo.RoundTrip 2024-03-26T03:49:34.476 INFO:tasks.workunit.client.0.smithi175.stdout: io: [ OK ] NeoRadosIo.RoundTrip (3025 ms) 2024-03-26T03:49:34.476 INFO:tasks.workunit.client.0.smithi175.stdout: io: [ RUN ] NeoRadosIo.ReadIntoBuufferlist 2024-03-26T03:49:34.476 INFO:tasks.workunit.client.0.smithi175.stdout: io: [ OK ] NeoRadosIo.ReadIntoBuufferlist (2990 ms) 2024-03-26T03:49:34.476 INFO:tasks.workunit.client.0.smithi175.stdout: io: [ RUN ] NeoRadosIo.OverlappingWriteRoundTrip 2024-03-26T03:49:34.476 INFO:tasks.workunit.client.0.smithi175.stdout: io: [ OK ] NeoRadosIo.OverlappingWriteRoundTrip (3059 ms) 2024-03-26T03:49:34.476 INFO:tasks.workunit.client.0.smithi175.stdout: io: [ RUN ] NeoRadosIo.WriteFullRoundTrip 2024-03-26T03:49:34.476 INFO:tasks.workunit.client.0.smithi175.stdout: io: [ OK ] NeoRadosIo.WriteFullRoundTrip (2992 ms) 2024-03-26T03:49:34.477 INFO:tasks.workunit.client.0.smithi175.stdout: io: [ RUN ] NeoRadosIo.AppendRoundTrip 2024-03-26T03:49:34.477 INFO:tasks.workunit.client.0.smithi175.stdout: io: [ OK ] NeoRadosIo.AppendRoundTrip (2951 ms) 2024-03-26T03:49:34.477 INFO:tasks.workunit.client.0.smithi175.stdout: io: [ RUN ] NeoRadosIo.Trunc 2024-03-26T03:49:34.477 INFO:tasks.workunit.client.0.smithi175.stdout: io: [ OK ] NeoRadosIo.Trunc (3013 ms) 2024-03-26T03:49:34.477 INFO:tasks.workunit.client.0.smithi175.stdout: io: [ RUN ] NeoRadosIo.Remove 2024-03-26T03:49:34.477 INFO:tasks.workunit.client.0.smithi175.stdout: io: [ OK ] NeoRadosIo.Remove (3036 ms) 2024-03-26T03:49:34.477 INFO:tasks.workunit.client.0.smithi175.stdout: io: [ RUN ] NeoRadosIo.XattrsRoundTrip 2024-03-26T03:49:34.477 INFO:tasks.workunit.client.0.smithi175.stdout: io: [ OK ] NeoRadosIo.XattrsRoundTrip (2961 ms) 2024-03-26T03:49:34.477 INFO:tasks.workunit.client.0.smithi175.stdout: io: [ RUN ] NeoRadosIo.RmXattr 2024-03-26T03:49:34.477 INFO:tasks.workunit.client.0.smithi175.stdout: io: [ OK ] NeoRadosIo.RmXattr (3044 ms) 2024-03-26T03:49:34.477 INFO:tasks.workunit.client.0.smithi175.stdout: io: [ RUN ] NeoRadosIo.GetXattrs 2024-03-26T03:49:34.477 INFO:tasks.workunit.client.0.smithi175.stdout: io: [ OK ] NeoRadosIo.GetXattrs (2997 ms) 2024-03-26T03:49:34.477 INFO:tasks.workunit.client.0.smithi175.stdout: io: [----------] 14 tests from NeoRadosIo (41914 ms total) 2024-03-26T03:49:34.477 INFO:tasks.workunit.client.0.smithi175.stdout: io: 2024-03-26T03:49:34.478 INFO:tasks.workunit.client.0.smithi175.stdout: io: [----------] Global test environment tear-down 2024-03-26T03:49:34.478 INFO:tasks.workunit.client.0.smithi175.stdout: io: [==========] 14 tests from 1 test suite ran. (41914 ms total) 2024-03-26T03:49:34.478 INFO:tasks.workunit.client.0.smithi175.stdout: io: [ PASSED ] 14 tests. 2024-03-26T03:49:34.484 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: Running main() from gmock_main.cc 2024-03-26T03:49:34.484 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [==========] Running 14 tests from 1 test suite. 2024-03-26T03:49:34.484 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [----------] Global test environment set-up. 2024-03-26T03:49:34.484 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps 2024-03-26T03:49:34.484 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [ RUN ] NeoRadosReadOps.SetOpFlags 2024-03-26T03:49:34.484 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [ OK ] NeoRadosReadOps.SetOpFlags (2790 ms) 2024-03-26T03:49:34.484 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertExists 2024-03-26T03:49:34.484 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertExists (3000 ms) 2024-03-26T03:49:34.484 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertVersion 2024-03-26T03:49:34.484 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertVersion (3001 ms) 2024-03-26T03:49:34.485 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpXattr 2024-03-26T03:49:34.485 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpXattr (3012 ms) 2024-03-26T03:49:34.485 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [ RUN ] NeoRadosReadOps.Read 2024-03-26T03:49:34.485 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [ OK ] NeoRadosReadOps.Read (2983 ms) 2024-03-26T03:49:34.485 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [ RUN ] NeoRadosReadOps.Checksum 2024-03-26T03:49:34.485 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [ OK ] NeoRadosReadOps.Checksum (3046 ms) 2024-03-26T03:49:34.485 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [ RUN ] NeoRadosReadOps.RWOrderedRead 2024-03-26T03:49:34.485 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [ OK ] NeoRadosReadOps.RWOrderedRead (3048 ms) 2024-03-26T03:49:34.485 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [ RUN ] NeoRadosReadOps.ShortRead 2024-03-26T03:49:34.485 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [ OK ] NeoRadosReadOps.ShortRead (2921 ms) 2024-03-26T03:49:34.485 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [ RUN ] NeoRadosReadOps.Exec 2024-03-26T03:49:34.485 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [ OK ] NeoRadosReadOps.Exec (3008 ms) 2024-03-26T03:49:34.485 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [ RUN ] NeoRadosReadOps.Stat 2024-03-26T03:49:34.485 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [ OK ] NeoRadosReadOps.Stat (3025 ms) 2024-03-26T03:49:34.485 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [ RUN ] NeoRadosReadOps.Omap 2024-03-26T03:49:34.486 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [ OK ] NeoRadosReadOps.Omap (3003 ms) 2024-03-26T03:49:34.486 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [ RUN ] NeoRadosReadOps.OmapNuls 2024-03-26T03:49:34.486 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [ OK ] NeoRadosReadOps.OmapNuls (2966 ms) 2024-03-26T03:49:34.486 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [ RUN ] NeoRadosReadOps.GetXattrs 2024-03-26T03:49:34.486 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [ OK ] NeoRadosReadOps.GetXattrs (3020 ms) 2024-03-26T03:49:34.487 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpExt 2024-03-26T03:49:34.487 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpExt (3056 ms) 2024-03-26T03:49:34.487 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps (41880 ms total) 2024-03-26T03:49:34.487 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: 2024-03-26T03:49:34.487 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [----------] Global test environment tear-down 2024-03-26T03:49:34.487 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [==========] 14 tests from 1 test suite ran. (41880 ms total) 2024-03-26T03:49:34.487 INFO:tasks.workunit.client.0.smithi175.stdout: read_operations: [ PASSED ] 14 tests. 2024-03-26T03:49:34.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:34 smithi190 bash[23392]: audit 2024-03-26T03:49:33.766187+0000 mon.c (mon.2) 205 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:34.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:34 smithi190 bash[23392]: audit 2024-03-26T03:49:34.089591+0000 mon.a (mon.0) 1270 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:49:34.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:34 smithi190 bash[23392]: audit 2024-03-26T03:49:34.093279+0000 mon.c (mon.2) 206 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:49:34.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:34 smithi190 bash[23392]: audit 2024-03-26T03:49:34.432131+0000 mon.a (mon.0) 1271 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi175-64639-10"}]': finished 2024-03-26T03:49:34.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:34 smithi190 bash[23392]: cluster 2024-03-26T03:49:34.440015+0000 mon.a (mon.0) 1272 : cluster [DBG] osdmap e118: 8 total, 8 up, 8 in 2024-03-26T03:49:34.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:34 smithi190 bash[23392]: audit 2024-03-26T03:49:34.450963+0000 mon.a (mon.0) 1273 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:34.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:34 smithi190 bash[23392]: audit 2024-03-26T03:49:34.519834+0000 mon.a (mon.0) 1274 : audit [INF] from='client.? 172.21.15.175:0/1231167386' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi175-64639-15"}]: dispatch 2024-03-26T03:49:34.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:34 smithi190 bash[23392]: audit 2024-03-26T03:49:34.530633+0000 mon.a (mon.0) 1275 : audit [INF] from='client.? 172.21.15.175:0/1231167386' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi175-64639-15"}]: dispatch 2024-03-26T03:49:34.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:34 smithi190 bash[23392]: audit 2024-03-26T03:49:34.536293+0000 mon.a (mon.0) 1276 : audit [INF] from='client.? 172.21.15.175:0/1231167386' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi175-64639-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:49:34.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:34 smithi175 bash[21527]: audit 2024-03-26T03:49:33.766187+0000 mon.c (mon.2) 205 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:34.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:34 smithi175 bash[21527]: audit 2024-03-26T03:49:34.089591+0000 mon.a (mon.0) 1270 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:49:34.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:34 smithi175 bash[21527]: audit 2024-03-26T03:49:34.093279+0000 mon.c (mon.2) 206 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:49:34.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:34 smithi175 bash[21527]: audit 2024-03-26T03:49:34.432131+0000 mon.a (mon.0) 1271 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi175-64639-10"}]': finished 2024-03-26T03:49:34.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:34 smithi175 bash[21527]: cluster 2024-03-26T03:49:34.440015+0000 mon.a (mon.0) 1272 : cluster [DBG] osdmap e118: 8 total, 8 up, 8 in 2024-03-26T03:49:34.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:34 smithi175 bash[21527]: audit 2024-03-26T03:49:34.450963+0000 mon.a (mon.0) 1273 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:34.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:34 smithi175 bash[21527]: audit 2024-03-26T03:49:34.519834+0000 mon.a (mon.0) 1274 : audit [INF] from='client.? 172.21.15.175:0/1231167386' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi175-64639-15"}]: dispatch 2024-03-26T03:49:34.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:34 smithi175 bash[21527]: audit 2024-03-26T03:49:34.530633+0000 mon.a (mon.0) 1275 : audit [INF] from='client.? 172.21.15.175:0/1231167386' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi175-64639-15"}]: dispatch 2024-03-26T03:49:34.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:34 smithi175 bash[21527]: audit 2024-03-26T03:49:34.536293+0000 mon.a (mon.0) 1276 : audit [INF] from='client.? 172.21.15.175:0/1231167386' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi175-64639-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:49:34.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:34 smithi175 bash[29284]: audit 2024-03-26T03:49:33.766187+0000 mon.c (mon.2) 205 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:34.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:34 smithi175 bash[29284]: audit 2024-03-26T03:49:34.089591+0000 mon.a (mon.0) 1270 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:49:34.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:34 smithi175 bash[29284]: audit 2024-03-26T03:49:34.093279+0000 mon.c (mon.2) 206 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:49:34.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:34 smithi175 bash[29284]: audit 2024-03-26T03:49:34.432131+0000 mon.a (mon.0) 1271 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi175-64639-10"}]': finished 2024-03-26T03:49:34.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:34 smithi175 bash[29284]: cluster 2024-03-26T03:49:34.440015+0000 mon.a (mon.0) 1272 : cluster [DBG] osdmap e118: 8 total, 8 up, 8 in 2024-03-26T03:49:34.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:34 smithi175 bash[29284]: audit 2024-03-26T03:49:34.450963+0000 mon.a (mon.0) 1273 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:34.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:34 smithi175 bash[29284]: audit 2024-03-26T03:49:34.519834+0000 mon.a (mon.0) 1274 : audit [INF] from='client.? 172.21.15.175:0/1231167386' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi175-64639-15"}]: dispatch 2024-03-26T03:49:34.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:34 smithi175 bash[29284]: audit 2024-03-26T03:49:34.530633+0000 mon.a (mon.0) 1275 : audit [INF] from='client.? 172.21.15.175:0/1231167386' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi175-64639-15"}]: dispatch 2024-03-26T03:49:34.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:34 smithi175 bash[29284]: audit 2024-03-26T03:49:34.536293+0000 mon.a (mon.0) 1276 : audit [INF] from='client.? 172.21.15.175:0/1231167386' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi175-64639-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:49:35.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:35 smithi175 bash[21527]: cluster 2024-03-26T03:49:34.591409+0000 mgr.y (mgr.24431) 213 : cluster [DBG] pgmap v174: 420 pgs: 20 creating+peering, 28 unknown, 372 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s wr, 2 op/s 2024-03-26T03:49:35.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:35 smithi175 bash[21527]: audit 2024-03-26T03:49:34.767793+0000 mon.c (mon.2) 207 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:35.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:35 smithi175 bash[21527]: audit 2024-03-26T03:49:35.442710+0000 mon.a (mon.0) 1277 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:35.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:35 smithi175 bash[21527]: audit 2024-03-26T03:49:35.442890+0000 mon.a (mon.0) 1278 : audit [INF] from='client.? 172.21.15.175:0/1231167386' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi175-64639-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:49:35.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:35 smithi175 bash[21527]: cluster 2024-03-26T03:49:35.452507+0000 mon.a (mon.0) 1279 : cluster [DBG] osdmap e119: 8 total, 8 up, 8 in 2024-03-26T03:49:35.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:35 smithi175 bash[21527]: audit 2024-03-26T03:49:35.459752+0000 mon.a (mon.0) 1280 : audit [INF] from='client.? 172.21.15.175:0/1231167386' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi175-64639-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi175-64639-15"}]: dispatch 2024-03-26T03:49:35.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:35 smithi175 bash[21527]: audit 2024-03-26T03:49:35.460950+0000 mon.a (mon.0) 1281 : audit [INF] from='client.? 172.21.15.175:0/801569162' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi175-64398-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:35.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:35 smithi175 bash[21527]: audit 2024-03-26T03:49:35.461483+0000 mon.a (mon.0) 1282 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi175-64406-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:35.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:35 smithi175 bash[21527]: audit 2024-03-26T03:49:35.462547+0000 mon.b (mon.1) 109 : audit [INF] from='client.? 172.21.15.175:0/1012916093' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi175-64406-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:35.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:35 smithi175 bash[29284]: cluster 2024-03-26T03:49:34.591409+0000 mgr.y (mgr.24431) 213 : cluster [DBG] pgmap v174: 420 pgs: 20 creating+peering, 28 unknown, 372 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s wr, 2 op/s 2024-03-26T03:49:35.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:35 smithi175 bash[29284]: audit 2024-03-26T03:49:34.767793+0000 mon.c (mon.2) 207 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:35.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:35 smithi175 bash[29284]: audit 2024-03-26T03:49:35.442710+0000 mon.a (mon.0) 1277 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:35.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:35 smithi175 bash[29284]: audit 2024-03-26T03:49:35.442890+0000 mon.a (mon.0) 1278 : audit [INF] from='client.? 172.21.15.175:0/1231167386' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi175-64639-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:49:35.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:35 smithi175 bash[29284]: cluster 2024-03-26T03:49:35.452507+0000 mon.a (mon.0) 1279 : cluster [DBG] osdmap e119: 8 total, 8 up, 8 in 2024-03-26T03:49:35.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:35 smithi175 bash[29284]: audit 2024-03-26T03:49:35.459752+0000 mon.a (mon.0) 1280 : audit [INF] from='client.? 172.21.15.175:0/1231167386' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi175-64639-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi175-64639-15"}]: dispatch 2024-03-26T03:49:35.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:35 smithi175 bash[29284]: audit 2024-03-26T03:49:35.460950+0000 mon.a (mon.0) 1281 : audit [INF] from='client.? 172.21.15.175:0/801569162' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi175-64398-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:35.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:35 smithi175 bash[29284]: audit 2024-03-26T03:49:35.461483+0000 mon.a (mon.0) 1282 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi175-64406-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:35.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:35 smithi175 bash[29284]: audit 2024-03-26T03:49:35.462547+0000 mon.b (mon.1) 109 : audit [INF] from='client.? 172.21.15.175:0/1012916093' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi175-64406-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:36.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:35 smithi190 bash[23392]: cluster 2024-03-26T03:49:34.591409+0000 mgr.y (mgr.24431) 213 : cluster [DBG] pgmap v174: 420 pgs: 20 creating+peering, 28 unknown, 372 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s wr, 2 op/s 2024-03-26T03:49:36.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:35 smithi190 bash[23392]: audit 2024-03-26T03:49:34.767793+0000 mon.c (mon.2) 207 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:36.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:35 smithi190 bash[23392]: audit 2024-03-26T03:49:35.442710+0000 mon.a (mon.0) 1277 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:36.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:35 smithi190 bash[23392]: audit 2024-03-26T03:49:35.442890+0000 mon.a (mon.0) 1278 : audit [INF] from='client.? 172.21.15.175:0/1231167386' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi175-64639-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:49:36.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:35 smithi190 bash[23392]: cluster 2024-03-26T03:49:35.452507+0000 mon.a (mon.0) 1279 : cluster [DBG] osdmap e119: 8 total, 8 up, 8 in 2024-03-26T03:49:36.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:35 smithi190 bash[23392]: audit 2024-03-26T03:49:35.459752+0000 mon.a (mon.0) 1280 : audit [INF] from='client.? 172.21.15.175:0/1231167386' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi175-64639-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi175-64639-15"}]: dispatch 2024-03-26T03:49:36.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:35 smithi190 bash[23392]: audit 2024-03-26T03:49:35.460950+0000 mon.a (mon.0) 1281 : audit [INF] from='client.? 172.21.15.175:0/801569162' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi175-64398-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:36.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:35 smithi190 bash[23392]: audit 2024-03-26T03:49:35.461483+0000 mon.a (mon.0) 1282 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi175-64406-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:36.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:35 smithi190 bash[23392]: audit 2024-03-26T03:49:35.462547+0000 mon.b (mon.1) 109 : audit [INF] from='client.? 172.21.15.175:0/1012916093' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi175-64406-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:37.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:36 smithi190 bash[23392]: audit 2024-03-26T03:49:35.769426+0000 mon.c (mon.2) 208 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:37.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:36 smithi190 bash[23392]: audit 2024-03-26T03:49:36.387037+0000 mon.a (mon.0) 1283 : audit [INF] from='client.? 172.21.15.175:0/801569162' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi175-64398-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:37.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:36 smithi190 bash[23392]: audit 2024-03-26T03:49:36.387216+0000 mon.a (mon.0) 1284 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi175-64406-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:37.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:36 smithi190 bash[23392]: cluster 2024-03-26T03:49:36.406383+0000 mon.a (mon.0) 1285 : cluster [DBG] osdmap e120: 8 total, 8 up, 8 in 2024-03-26T03:49:37.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:36 smithi175 bash[21527]: audit 2024-03-26T03:49:35.769426+0000 mon.c (mon.2) 208 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:37.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:36 smithi175 bash[21527]: audit 2024-03-26T03:49:36.387037+0000 mon.a (mon.0) 1283 : audit [INF] from='client.? 172.21.15.175:0/801569162' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi175-64398-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:37.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:36 smithi175 bash[21527]: audit 2024-03-26T03:49:36.387216+0000 mon.a (mon.0) 1284 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi175-64406-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:37.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:36 smithi175 bash[21527]: cluster 2024-03-26T03:49:36.406383+0000 mon.a (mon.0) 1285 : cluster [DBG] osdmap e120: 8 total, 8 up, 8 in 2024-03-26T03:49:37.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:36 smithi175 bash[29284]: audit 2024-03-26T03:49:35.769426+0000 mon.c (mon.2) 208 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:37.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:36 smithi175 bash[29284]: audit 2024-03-26T03:49:36.387037+0000 mon.a (mon.0) 1283 : audit [INF] from='client.? 172.21.15.175:0/801569162' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi175-64398-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:37.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:36 smithi175 bash[29284]: audit 2024-03-26T03:49:36.387216+0000 mon.a (mon.0) 1284 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi175-64406-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:37.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:36 smithi175 bash[29284]: cluster 2024-03-26T03:49:36.406383+0000 mon.a (mon.0) 1285 : cluster [DBG] osdmap e120: 8 total, 8 up, 8 in 2024-03-26T03:49:38.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:37 smithi190 bash[23392]: cluster 2024-03-26T03:49:36.592058+0000 mgr.y (mgr.24431) 214 : cluster [DBG] pgmap v177: 484 pgs: 64 unknown, 32 creating+peering, 388 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 3.7 KiB/s wr, 4 op/s 2024-03-26T03:49:38.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:37 smithi190 bash[23392]: audit 2024-03-26T03:49:36.771143+0000 mon.c (mon.2) 209 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:38.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:37 smithi190 bash[23392]: cluster 2024-03-26T03:49:37.387144+0000 mon.a (mon.0) 1286 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:38.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:37 smithi190 bash[23392]: audit 2024-03-26T03:49:37.397371+0000 mon.a (mon.0) 1287 : audit [INF] from='client.? 172.21.15.175:0/1231167386' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi175-64639-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi175-64639-15"}]': finished 2024-03-26T03:49:38.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:37 smithi190 bash[23392]: cluster 2024-03-26T03:49:37.406707+0000 mon.a (mon.0) 1288 : cluster [DBG] osdmap e121: 8 total, 8 up, 8 in 2024-03-26T03:49:38.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:37 smithi175 bash[21527]: cluster 2024-03-26T03:49:36.592058+0000 mgr.y (mgr.24431) 214 : cluster [DBG] pgmap v177: 484 pgs: 64 unknown, 32 creating+peering, 388 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 3.7 KiB/s wr, 4 op/s 2024-03-26T03:49:38.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:37 smithi175 bash[21527]: audit 2024-03-26T03:49:36.771143+0000 mon.c (mon.2) 209 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:38.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:37 smithi175 bash[21527]: cluster 2024-03-26T03:49:37.387144+0000 mon.a (mon.0) 1286 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:38.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:37 smithi175 bash[21527]: audit 2024-03-26T03:49:37.397371+0000 mon.a (mon.0) 1287 : audit [INF] from='client.? 172.21.15.175:0/1231167386' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi175-64639-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi175-64639-15"}]': finished 2024-03-26T03:49:38.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:37 smithi175 bash[21527]: cluster 2024-03-26T03:49:37.406707+0000 mon.a (mon.0) 1288 : cluster [DBG] osdmap e121: 8 total, 8 up, 8 in 2024-03-26T03:49:38.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:37 smithi175 bash[29284]: cluster 2024-03-26T03:49:36.592058+0000 mgr.y (mgr.24431) 214 : cluster [DBG] pgmap v177: 484 pgs: 64 unknown, 32 creating+peering, 388 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 3.7 KiB/s wr, 4 op/s 2024-03-26T03:49:38.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:37 smithi175 bash[29284]: audit 2024-03-26T03:49:36.771143+0000 mon.c (mon.2) 209 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:38.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:37 smithi175 bash[29284]: cluster 2024-03-26T03:49:37.387144+0000 mon.a (mon.0) 1286 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:38.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:37 smithi175 bash[29284]: audit 2024-03-26T03:49:37.397371+0000 mon.a (mon.0) 1287 : audit [INF] from='client.? 172.21.15.175:0/1231167386' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi175-64639-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi175-64639-15"}]': finished 2024-03-26T03:49:38.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:37 smithi175 bash[29284]: cluster 2024-03-26T03:49:37.406707+0000 mon.a (mon.0) 1288 : cluster [DBG] osdmap e121: 8 total, 8 up, 8 in 2024-03-26T03:49:39.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:38 smithi190 bash[23392]: audit 2024-03-26T03:49:37.772780+0000 mon.c (mon.2) 210 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:39.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:38 smithi190 bash[23392]: cluster 2024-03-26T03:49:38.412100+0000 mon.a (mon.0) 1289 : cluster [DBG] osdmap e122: 8 total, 8 up, 8 in 2024-03-26T03:49:39.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:38 smithi190 bash[23392]: audit 2024-03-26T03:49:38.416119+0000 mon.c (mon.2) 211 : audit [INF] from='client.? 172.21.15.175:0/2616272771' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi175-64406-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:39.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:38 smithi190 bash[23392]: audit 2024-03-26T03:49:38.417126+0000 mon.a (mon.0) 1290 : audit [INF] from='client.? 172.21.15.175:0/1397453119' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi175-64398-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:39.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:38 smithi190 bash[23392]: audit 2024-03-26T03:49:38.427418+0000 mon.a (mon.0) 1291 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi175-64406-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:39.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:49:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:49:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:49:39.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:38 smithi175 bash[29284]: audit 2024-03-26T03:49:37.772780+0000 mon.c (mon.2) 210 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:39.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:38 smithi175 bash[29284]: cluster 2024-03-26T03:49:38.412100+0000 mon.a (mon.0) 1289 : cluster [DBG] osdmap e122: 8 total, 8 up, 8 in 2024-03-26T03:49:39.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:38 smithi175 bash[29284]: audit 2024-03-26T03:49:38.416119+0000 mon.c (mon.2) 211 : audit [INF] from='client.? 172.21.15.175:0/2616272771' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi175-64406-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:39.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:38 smithi175 bash[29284]: audit 2024-03-26T03:49:38.417126+0000 mon.a (mon.0) 1290 : audit [INF] from='client.? 172.21.15.175:0/1397453119' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi175-64398-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:39.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:38 smithi175 bash[29284]: audit 2024-03-26T03:49:38.427418+0000 mon.a (mon.0) 1291 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi175-64406-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:39.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:38 smithi175 bash[21527]: audit 2024-03-26T03:49:37.772780+0000 mon.c (mon.2) 210 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:39.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:38 smithi175 bash[21527]: cluster 2024-03-26T03:49:38.412100+0000 mon.a (mon.0) 1289 : cluster [DBG] osdmap e122: 8 total, 8 up, 8 in 2024-03-26T03:49:39.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:38 smithi175 bash[21527]: audit 2024-03-26T03:49:38.416119+0000 mon.c (mon.2) 211 : audit [INF] from='client.? 172.21.15.175:0/2616272771' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi175-64406-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:39.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:38 smithi175 bash[21527]: audit 2024-03-26T03:49:38.417126+0000 mon.a (mon.0) 1290 : audit [INF] from='client.? 172.21.15.175:0/1397453119' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi175-64398-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:39.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:38 smithi175 bash[21527]: audit 2024-03-26T03:49:38.427418+0000 mon.a (mon.0) 1291 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi175-64406-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:39.990 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:49:39 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:49:39.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:39 smithi190 bash[23392]: cluster 2024-03-26T03:49:38.592822+0000 mgr.y (mgr.24431) 215 : cluster [DBG] pgmap v180: 492 pgs: 104 unknown, 32 creating+peering, 356 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-26T03:49:39.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:39 smithi190 bash[23392]: audit 2024-03-26T03:49:38.774623+0000 mon.c (mon.2) 212 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:39.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:39 smithi190 bash[23392]: audit 2024-03-26T03:49:39.414158+0000 mon.a (mon.0) 1292 : audit [INF] from='client.? 172.21.15.175:0/1397453119' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi175-64398-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:39.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:39 smithi190 bash[23392]: audit 2024-03-26T03:49:39.414344+0000 mon.a (mon.0) 1293 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi175-64406-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:39.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:39 smithi190 bash[23392]: cluster 2024-03-26T03:49:39.422744+0000 mon.a (mon.0) 1294 : cluster [DBG] osdmap e123: 8 total, 8 up, 8 in 2024-03-26T03:49:39.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:39 smithi190 bash[23392]: audit 2024-03-26T03:49:39.662500+0000 mon.a (mon.0) 1295 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:49:40.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:39 smithi175 bash[29284]: cluster 2024-03-26T03:49:38.592822+0000 mgr.y (mgr.24431) 215 : cluster [DBG] pgmap v180: 492 pgs: 104 unknown, 32 creating+peering, 356 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-26T03:49:40.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:39 smithi175 bash[29284]: audit 2024-03-26T03:49:38.774623+0000 mon.c (mon.2) 212 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:40.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:39 smithi175 bash[29284]: audit 2024-03-26T03:49:39.414158+0000 mon.a (mon.0) 1292 : audit [INF] from='client.? 172.21.15.175:0/1397453119' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi175-64398-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:40.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:39 smithi175 bash[29284]: audit 2024-03-26T03:49:39.414344+0000 mon.a (mon.0) 1293 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi175-64406-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:40.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:39 smithi175 bash[29284]: cluster 2024-03-26T03:49:39.422744+0000 mon.a (mon.0) 1294 : cluster [DBG] osdmap e123: 8 total, 8 up, 8 in 2024-03-26T03:49:40.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:39 smithi175 bash[29284]: audit 2024-03-26T03:49:39.662500+0000 mon.a (mon.0) 1295 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:49:40.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:39 smithi175 bash[21527]: cluster 2024-03-26T03:49:38.592822+0000 mgr.y (mgr.24431) 215 : cluster [DBG] pgmap v180: 492 pgs: 104 unknown, 32 creating+peering, 356 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-26T03:49:40.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:39 smithi175 bash[21527]: audit 2024-03-26T03:49:38.774623+0000 mon.c (mon.2) 212 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:40.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:39 smithi175 bash[21527]: audit 2024-03-26T03:49:39.414158+0000 mon.a (mon.0) 1292 : audit [INF] from='client.? 172.21.15.175:0/1397453119' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi175-64398-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:40.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:39 smithi175 bash[21527]: audit 2024-03-26T03:49:39.414344+0000 mon.a (mon.0) 1293 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi175-64406-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:40.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:39 smithi175 bash[21527]: cluster 2024-03-26T03:49:39.422744+0000 mon.a (mon.0) 1294 : cluster [DBG] osdmap e123: 8 total, 8 up, 8 in 2024-03-26T03:49:40.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:39 smithi175 bash[21527]: audit 2024-03-26T03:49:39.662500+0000 mon.a (mon.0) 1295 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:49:41.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:40 smithi190 bash[23392]: audit 2024-03-26T03:49:39.727818+0000 mgr.y (mgr.24431) 216 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:49:41.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:40 smithi190 bash[23392]: audit 2024-03-26T03:49:39.776341+0000 mon.c (mon.2) 213 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:41.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:40 smithi190 bash[23392]: audit 2024-03-26T03:49:40.417415+0000 mon.a (mon.0) 1296 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-13", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:49:41.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:40 smithi190 bash[23392]: cluster 2024-03-26T03:49:40.426495+0000 mon.a (mon.0) 1297 : cluster [DBG] osdmap e124: 8 total, 8 up, 8 in 2024-03-26T03:49:41.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:40 smithi190 bash[23392]: audit 2024-03-26T03:49:40.428536+0000 mon.a (mon.0) 1298 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-13"}]: dispatch 2024-03-26T03:49:41.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:40 smithi175 bash[21527]: audit 2024-03-26T03:49:39.727818+0000 mgr.y (mgr.24431) 216 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:49:41.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:40 smithi175 bash[21527]: audit 2024-03-26T03:49:39.776341+0000 mon.c (mon.2) 213 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:41.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:40 smithi175 bash[21527]: audit 2024-03-26T03:49:40.417415+0000 mon.a (mon.0) 1296 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-13", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:49:41.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:40 smithi175 bash[21527]: cluster 2024-03-26T03:49:40.426495+0000 mon.a (mon.0) 1297 : cluster [DBG] osdmap e124: 8 total, 8 up, 8 in 2024-03-26T03:49:41.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:40 smithi175 bash[21527]: audit 2024-03-26T03:49:40.428536+0000 mon.a (mon.0) 1298 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-13"}]: dispatch 2024-03-26T03:49:41.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:40 smithi175 bash[29284]: audit 2024-03-26T03:49:39.727818+0000 mgr.y (mgr.24431) 216 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:49:41.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:40 smithi175 bash[29284]: audit 2024-03-26T03:49:39.776341+0000 mon.c (mon.2) 213 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:41.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:40 smithi175 bash[29284]: audit 2024-03-26T03:49:40.417415+0000 mon.a (mon.0) 1296 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-13", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:49:41.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:40 smithi175 bash[29284]: cluster 2024-03-26T03:49:40.426495+0000 mon.a (mon.0) 1297 : cluster [DBG] osdmap e124: 8 total, 8 up, 8 in 2024-03-26T03:49:41.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:40 smithi175 bash[29284]: audit 2024-03-26T03:49:40.428536+0000 mon.a (mon.0) 1298 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-13"}]: dispatch 2024-03-26T03:49:42.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:41 smithi190 bash[23392]: cluster 2024-03-26T03:49:40.593635+0000 mgr.y (mgr.24431) 217 : cluster [DBG] pgmap v183: 428 pgs: 428 active+clean; 218 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 374 KiB/s wr, 115 op/s 2024-03-26T03:49:42.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:41 smithi190 bash[23392]: audit 2024-03-26T03:49:40.777926+0000 mon.c (mon.2) 214 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:42.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:41 smithi190 bash[23392]: audit 2024-03-26T03:49:41.391868+0000 mon.a (mon.0) 1299 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-13"}]': finished 2024-03-26T03:49:42.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:41 smithi190 bash[23392]: cluster 2024-03-26T03:49:41.399999+0000 mon.a (mon.0) 1300 : cluster [DBG] osdmap e125: 8 total, 8 up, 8 in 2024-03-26T03:49:42.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:41 smithi190 bash[23392]: audit 2024-03-26T03:49:41.403950+0000 mon.a (mon.0) 1301 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-13", "mode": "writeback"}]: dispatch 2024-03-26T03:49:42.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:41 smithi190 bash[23392]: audit 2024-03-26T03:49:41.404209+0000 mon.c (mon.2) 215 : audit [INF] from='client.? 172.21.15.175:0/3740803143' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi175-64406-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:42.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:41 smithi190 bash[23392]: audit 2024-03-26T03:49:41.404632+0000 mon.c (mon.2) 216 : audit [INF] from='client.? 172.21.15.175:0/719788658' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi175-64398-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:42.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:41 smithi190 bash[23392]: audit 2024-03-26T03:49:41.404965+0000 mon.a (mon.0) 1302 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi175-64406-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:42.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:41 smithi190 bash[23392]: audit 2024-03-26T03:49:41.405497+0000 mon.a (mon.0) 1303 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi175-64398-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:42.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:41 smithi175 bash[21527]: cluster 2024-03-26T03:49:40.593635+0000 mgr.y (mgr.24431) 217 : cluster [DBG] pgmap v183: 428 pgs: 428 active+clean; 218 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 374 KiB/s wr, 115 op/s 2024-03-26T03:49:42.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:41 smithi175 bash[21527]: audit 2024-03-26T03:49:40.777926+0000 mon.c (mon.2) 214 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:42.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:41 smithi175 bash[21527]: audit 2024-03-26T03:49:41.391868+0000 mon.a (mon.0) 1299 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-13"}]': finished 2024-03-26T03:49:42.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:41 smithi175 bash[21527]: cluster 2024-03-26T03:49:41.399999+0000 mon.a (mon.0) 1300 : cluster [DBG] osdmap e125: 8 total, 8 up, 8 in 2024-03-26T03:49:42.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:41 smithi175 bash[21527]: audit 2024-03-26T03:49:41.403950+0000 mon.a (mon.0) 1301 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-13", "mode": "writeback"}]: dispatch 2024-03-26T03:49:42.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:41 smithi175 bash[21527]: audit 2024-03-26T03:49:41.404209+0000 mon.c (mon.2) 215 : audit [INF] from='client.? 172.21.15.175:0/3740803143' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi175-64406-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:42.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:41 smithi175 bash[21527]: audit 2024-03-26T03:49:41.404632+0000 mon.c (mon.2) 216 : audit [INF] from='client.? 172.21.15.175:0/719788658' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi175-64398-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:42.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:41 smithi175 bash[21527]: audit 2024-03-26T03:49:41.404965+0000 mon.a (mon.0) 1302 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi175-64406-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:42.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:41 smithi175 bash[21527]: audit 2024-03-26T03:49:41.405497+0000 mon.a (mon.0) 1303 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi175-64398-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:42.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:41 smithi175 bash[29284]: cluster 2024-03-26T03:49:40.593635+0000 mgr.y (mgr.24431) 217 : cluster [DBG] pgmap v183: 428 pgs: 428 active+clean; 218 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 374 KiB/s wr, 115 op/s 2024-03-26T03:49:42.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:41 smithi175 bash[29284]: audit 2024-03-26T03:49:40.777926+0000 mon.c (mon.2) 214 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:42.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:41 smithi175 bash[29284]: audit 2024-03-26T03:49:41.391868+0000 mon.a (mon.0) 1299 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-13"}]': finished 2024-03-26T03:49:42.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:41 smithi175 bash[29284]: cluster 2024-03-26T03:49:41.399999+0000 mon.a (mon.0) 1300 : cluster [DBG] osdmap e125: 8 total, 8 up, 8 in 2024-03-26T03:49:42.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:41 smithi175 bash[29284]: audit 2024-03-26T03:49:41.403950+0000 mon.a (mon.0) 1301 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-13", "mode": "writeback"}]: dispatch 2024-03-26T03:49:42.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:41 smithi175 bash[29284]: audit 2024-03-26T03:49:41.404209+0000 mon.c (mon.2) 215 : audit [INF] from='client.? 172.21.15.175:0/3740803143' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi175-64406-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:42.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:41 smithi175 bash[29284]: audit 2024-03-26T03:49:41.404632+0000 mon.c (mon.2) 216 : audit [INF] from='client.? 172.21.15.175:0/719788658' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi175-64398-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:42.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:41 smithi175 bash[29284]: audit 2024-03-26T03:49:41.404965+0000 mon.a (mon.0) 1302 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi175-64406-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:42.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:41 smithi175 bash[29284]: audit 2024-03-26T03:49:41.405497+0000 mon.a (mon.0) 1303 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi175-64398-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:43.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:42 smithi190 bash[23392]: audit 2024-03-26T03:49:41.779601+0000 mon.c (mon.2) 217 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:43.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:42 smithi190 bash[23392]: audit 2024-03-26T03:49:41.781007+0000 mon.c (mon.2) 218 : audit [INF] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pg_num","val":"11"}]: dispatch 2024-03-26T03:49:43.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:42 smithi190 bash[23392]: audit 2024-03-26T03:49:41.781676+0000 mon.a (mon.0) 1304 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pg_num","val":"11"}]: dispatch 2024-03-26T03:49:43.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:42 smithi190 bash[23392]: cluster 2024-03-26T03:49:42.392841+0000 mon.a (mon.0) 1305 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:49:43.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:42 smithi190 bash[23392]: audit 2024-03-26T03:49:42.395643+0000 mon.a (mon.0) 1306 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-13", "mode": "writeback"}]': finished 2024-03-26T03:49:43.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:42 smithi190 bash[23392]: audit 2024-03-26T03:49:42.395862+0000 mon.a (mon.0) 1307 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi175-64406-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:43.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:42 smithi190 bash[23392]: audit 2024-03-26T03:49:42.396118+0000 mon.a (mon.0) 1308 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi175-64398-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:43.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:42 smithi190 bash[23392]: audit 2024-03-26T03:49:42.396375+0000 mon.a (mon.0) 1309 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pg_num","val":"11"}]': finished 2024-03-26T03:49:43.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:42 smithi190 bash[23392]: cluster 2024-03-26T03:49:42.405009+0000 mon.a (mon.0) 1310 : cluster [DBG] osdmap e126: 8 total, 8 up, 8 in 2024-03-26T03:49:43.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:42 smithi190 bash[23392]: audit 2024-03-26T03:49:42.407256+0000 mon.c (mon.2) 219 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:43.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:42 smithi190 bash[23392]: audit 2024-03-26T03:49:42.410868+0000 mon.c (mon.2) 220 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:43.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:42 smithi190 bash[23392]: audit 2024-03-26T03:49:42.412089+0000 mon.c (mon.2) 221 : audit [INF] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-26T03:49:43.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:42 smithi190 bash[23392]: audit 2024-03-26T03:49:42.412831+0000 mon.a (mon.0) 1311 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-26T03:49:43.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:42 smithi175 bash[29284]: audit 2024-03-26T03:49:41.779601+0000 mon.c (mon.2) 217 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:43.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:42 smithi175 bash[29284]: audit 2024-03-26T03:49:41.781007+0000 mon.c (mon.2) 218 : audit [INF] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pg_num","val":"11"}]: dispatch 2024-03-26T03:49:43.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:42 smithi175 bash[29284]: audit 2024-03-26T03:49:41.781676+0000 mon.a (mon.0) 1304 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pg_num","val":"11"}]: dispatch 2024-03-26T03:49:43.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:42 smithi175 bash[29284]: cluster 2024-03-26T03:49:42.392841+0000 mon.a (mon.0) 1305 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:49:43.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:42 smithi175 bash[29284]: audit 2024-03-26T03:49:42.395643+0000 mon.a (mon.0) 1306 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-13", "mode": "writeback"}]': finished 2024-03-26T03:49:43.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:42 smithi175 bash[29284]: audit 2024-03-26T03:49:42.395862+0000 mon.a (mon.0) 1307 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi175-64406-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:43.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:42 smithi175 bash[29284]: audit 2024-03-26T03:49:42.396118+0000 mon.a (mon.0) 1308 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi175-64398-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:43.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:42 smithi175 bash[29284]: audit 2024-03-26T03:49:42.396375+0000 mon.a (mon.0) 1309 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pg_num","val":"11"}]': finished 2024-03-26T03:49:43.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:42 smithi175 bash[29284]: cluster 2024-03-26T03:49:42.405009+0000 mon.a (mon.0) 1310 : cluster [DBG] osdmap e126: 8 total, 8 up, 8 in 2024-03-26T03:49:43.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:42 smithi175 bash[29284]: audit 2024-03-26T03:49:42.407256+0000 mon.c (mon.2) 219 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:43.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:42 smithi175 bash[29284]: audit 2024-03-26T03:49:42.410868+0000 mon.c (mon.2) 220 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:43.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:42 smithi175 bash[29284]: audit 2024-03-26T03:49:42.412089+0000 mon.c (mon.2) 221 : audit [INF] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-26T03:49:43.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:42 smithi175 bash[29284]: audit 2024-03-26T03:49:42.412831+0000 mon.a (mon.0) 1311 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-26T03:49:43.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:42 smithi175 bash[21527]: audit 2024-03-26T03:49:41.779601+0000 mon.c (mon.2) 217 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:43.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:42 smithi175 bash[21527]: audit 2024-03-26T03:49:41.781007+0000 mon.c (mon.2) 218 : audit [INF] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pg_num","val":"11"}]: dispatch 2024-03-26T03:49:43.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:42 smithi175 bash[21527]: audit 2024-03-26T03:49:41.781676+0000 mon.a (mon.0) 1304 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pg_num","val":"11"}]: dispatch 2024-03-26T03:49:43.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:42 smithi175 bash[21527]: cluster 2024-03-26T03:49:42.392841+0000 mon.a (mon.0) 1305 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:49:43.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:42 smithi175 bash[21527]: audit 2024-03-26T03:49:42.395643+0000 mon.a (mon.0) 1306 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-13", "mode": "writeback"}]': finished 2024-03-26T03:49:43.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:42 smithi175 bash[21527]: audit 2024-03-26T03:49:42.395862+0000 mon.a (mon.0) 1307 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi175-64406-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:43.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:42 smithi175 bash[21527]: audit 2024-03-26T03:49:42.396118+0000 mon.a (mon.0) 1308 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi175-64398-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:43.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:42 smithi175 bash[21527]: audit 2024-03-26T03:49:42.396375+0000 mon.a (mon.0) 1309 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pg_num","val":"11"}]': finished 2024-03-26T03:49:43.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:42 smithi175 bash[21527]: cluster 2024-03-26T03:49:42.405009+0000 mon.a (mon.0) 1310 : cluster [DBG] osdmap e126: 8 total, 8 up, 8 in 2024-03-26T03:49:43.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:42 smithi175 bash[21527]: audit 2024-03-26T03:49:42.407256+0000 mon.c (mon.2) 219 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:43.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:42 smithi175 bash[21527]: audit 2024-03-26T03:49:42.410868+0000 mon.c (mon.2) 220 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:43.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:42 smithi175 bash[21527]: audit 2024-03-26T03:49:42.412089+0000 mon.c (mon.2) 221 : audit [INF] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-26T03:49:43.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:42 smithi175 bash[21527]: audit 2024-03-26T03:49:42.412831+0000 mon.a (mon.0) 1311 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-26T03:49:44.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:44 smithi190 bash[23392]: cluster 2024-03-26T03:49:42.594505+0000 mgr.y (mgr.24431) 218 : cluster [DBG] pgmap v186: 492 pgs: 64 unknown, 428 active+clean; 218 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 377 KiB/s wr, 119 op/s 2024-03-26T03:49:44.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:44 smithi190 bash[23392]: audit 2024-03-26T03:49:43.397524+0000 mon.a (mon.0) 1312 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pgp_num","val":"11"}]': finished 2024-03-26T03:49:44.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:44 smithi190 bash[23392]: cluster 2024-03-26T03:49:43.403267+0000 mon.a (mon.0) 1313 : cluster [DBG] osdmap e127: 8 total, 8 up, 8 in 2024-03-26T03:49:44.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:44 smithi190 bash[23392]: audit 2024-03-26T03:49:43.432657+0000 mon.c (mon.2) 222 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:44.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:44 smithi190 bash[23392]: audit 2024-03-26T03:49:43.561194+0000 mon.a (mon.0) 1314 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "166.0"}]: dispatch 2024-03-26T03:49:44.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:44 smithi190 bash[23392]: audit 2024-03-26T03:49:43.561920+0000 mon.a (mon.0) 1315 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "166.1"}]: dispatch 2024-03-26T03:49:44.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:44 smithi190 bash[23392]: audit 2024-03-26T03:49:43.562396+0000 mon.a (mon.0) 1316 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "166.2"}]: dispatch 2024-03-26T03:49:44.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:44 smithi190 bash[23392]: audit 2024-03-26T03:49:43.562873+0000 mon.a (mon.0) 1317 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "166.3"}]: dispatch 2024-03-26T03:49:44.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:44 smithi190 bash[23392]: audit 2024-03-26T03:49:43.563330+0000 mon.a (mon.0) 1318 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "166.4"}]: dispatch 2024-03-26T03:49:44.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:44 smithi190 bash[23392]: audit 2024-03-26T03:49:43.563804+0000 mon.a (mon.0) 1319 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "166.5"}]: dispatch 2024-03-26T03:49:44.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:44 smithi190 bash[23392]: audit 2024-03-26T03:49:43.564260+0000 mon.a (mon.0) 1320 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "166.6"}]: dispatch 2024-03-26T03:49:44.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:44 smithi190 bash[23392]: audit 2024-03-26T03:49:43.564727+0000 mon.a (mon.0) 1321 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "166.7"}]: dispatch 2024-03-26T03:49:44.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:44 smithi190 bash[23392]: audit 2024-03-26T03:49:43.565146+0000 mon.a (mon.0) 1322 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "166.8"}]: dispatch 2024-03-26T03:49:44.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:44 smithi190 bash[23392]: audit 2024-03-26T03:49:43.565586+0000 mon.a (mon.0) 1323 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "166.9"}]: dispatch 2024-03-26T03:49:44.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:44 smithi190 bash[23392]: cluster 2024-03-26T03:49:43.644951+0000 osd.6 (osd.6) 7 : cluster [DBG] 166.9 deep-scrub starts 2024-03-26T03:49:44.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:44 smithi190 bash[23392]: cluster 2024-03-26T03:49:43.656626+0000 osd.6 (osd.6) 8 : cluster [DBG] 166.9 deep-scrub ok 2024-03-26T03:49:44.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:44 smithi190 bash[23392]: cluster 2024-03-26T03:49:43.697060+0000 osd.2 (osd.2) 7 : cluster [DBG] 166.8 scrub starts 2024-03-26T03:49:44.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:44 smithi190 bash[23392]: cluster 2024-03-26T03:49:43.699391+0000 osd.2 (osd.2) 8 : cluster [DBG] 166.8 scrub ok 2024-03-26T03:49:44.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:44 smithi190 bash[23392]: cluster 2024-03-26T03:49:43.944255+0000 osd.4 (osd.4) 7 : cluster [DBG] 166.3 scrub starts 2024-03-26T03:49:44.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:44 smithi190 bash[23392]: cluster 2024-03-26T03:49:43.950288+0000 osd.4 (osd.4) 8 : cluster [DBG] 166.3 scrub ok 2024-03-26T03:49:44.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[21527]: cluster 2024-03-26T03:49:42.594505+0000 mgr.y (mgr.24431) 218 : cluster [DBG] pgmap v186: 492 pgs: 64 unknown, 428 active+clean; 218 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 377 KiB/s wr, 119 op/s 2024-03-26T03:49:44.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[21527]: audit 2024-03-26T03:49:43.397524+0000 mon.a (mon.0) 1312 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pgp_num","val":"11"}]': finished 2024-03-26T03:49:44.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[21527]: cluster 2024-03-26T03:49:43.403267+0000 mon.a (mon.0) 1313 : cluster [DBG] osdmap e127: 8 total, 8 up, 8 in 2024-03-26T03:49:44.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[21527]: audit 2024-03-26T03:49:43.432657+0000 mon.c (mon.2) 222 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:44.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[21527]: audit 2024-03-26T03:49:43.561194+0000 mon.a (mon.0) 1314 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "166.0"}]: dispatch 2024-03-26T03:49:44.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[21527]: audit 2024-03-26T03:49:43.561920+0000 mon.a (mon.0) 1315 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "166.1"}]: dispatch 2024-03-26T03:49:44.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[21527]: audit 2024-03-26T03:49:43.562396+0000 mon.a (mon.0) 1316 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "166.2"}]: dispatch 2024-03-26T03:49:44.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[21527]: audit 2024-03-26T03:49:43.562873+0000 mon.a (mon.0) 1317 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "166.3"}]: dispatch 2024-03-26T03:49:44.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[21527]: audit 2024-03-26T03:49:43.563330+0000 mon.a (mon.0) 1318 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "166.4"}]: dispatch 2024-03-26T03:49:44.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[21527]: audit 2024-03-26T03:49:43.563804+0000 mon.a (mon.0) 1319 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "166.5"}]: dispatch 2024-03-26T03:49:44.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[21527]: audit 2024-03-26T03:49:43.564260+0000 mon.a (mon.0) 1320 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "166.6"}]: dispatch 2024-03-26T03:49:44.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[21527]: audit 2024-03-26T03:49:43.564727+0000 mon.a (mon.0) 1321 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "166.7"}]: dispatch 2024-03-26T03:49:44.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[21527]: audit 2024-03-26T03:49:43.565146+0000 mon.a (mon.0) 1322 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "166.8"}]: dispatch 2024-03-26T03:49:44.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[21527]: audit 2024-03-26T03:49:43.565586+0000 mon.a (mon.0) 1323 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "166.9"}]: dispatch 2024-03-26T03:49:44.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[21527]: cluster 2024-03-26T03:49:43.644951+0000 osd.6 (osd.6) 7 : cluster [DBG] 166.9 deep-scrub starts 2024-03-26T03:49:44.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[21527]: cluster 2024-03-26T03:49:43.656626+0000 osd.6 (osd.6) 8 : cluster [DBG] 166.9 deep-scrub ok 2024-03-26T03:49:44.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[21527]: cluster 2024-03-26T03:49:43.697060+0000 osd.2 (osd.2) 7 : cluster [DBG] 166.8 scrub starts 2024-03-26T03:49:44.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[21527]: cluster 2024-03-26T03:49:43.699391+0000 osd.2 (osd.2) 8 : cluster [DBG] 166.8 scrub ok 2024-03-26T03:49:44.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[21527]: cluster 2024-03-26T03:49:43.944255+0000 osd.4 (osd.4) 7 : cluster [DBG] 166.3 scrub starts 2024-03-26T03:49:44.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[21527]: cluster 2024-03-26T03:49:43.950288+0000 osd.4 (osd.4) 8 : cluster [DBG] 166.3 scrub ok 2024-03-26T03:49:44.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[29284]: cluster 2024-03-26T03:49:42.594505+0000 mgr.y (mgr.24431) 218 : cluster [DBG] pgmap v186: 492 pgs: 64 unknown, 428 active+clean; 218 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 377 KiB/s wr, 119 op/s 2024-03-26T03:49:44.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[29284]: audit 2024-03-26T03:49:43.397524+0000 mon.a (mon.0) 1312 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pgp_num","val":"11"}]': finished 2024-03-26T03:49:44.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[29284]: cluster 2024-03-26T03:49:43.403267+0000 mon.a (mon.0) 1313 : cluster [DBG] osdmap e127: 8 total, 8 up, 8 in 2024-03-26T03:49:44.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[29284]: audit 2024-03-26T03:49:43.432657+0000 mon.c (mon.2) 222 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:44.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[29284]: audit 2024-03-26T03:49:43.561194+0000 mon.a (mon.0) 1314 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "166.0"}]: dispatch 2024-03-26T03:49:44.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[29284]: audit 2024-03-26T03:49:43.561920+0000 mon.a (mon.0) 1315 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "166.1"}]: dispatch 2024-03-26T03:49:44.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[29284]: audit 2024-03-26T03:49:43.562396+0000 mon.a (mon.0) 1316 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "166.2"}]: dispatch 2024-03-26T03:49:44.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[29284]: audit 2024-03-26T03:49:43.562873+0000 mon.a (mon.0) 1317 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "166.3"}]: dispatch 2024-03-26T03:49:44.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[29284]: audit 2024-03-26T03:49:43.563330+0000 mon.a (mon.0) 1318 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "166.4"}]: dispatch 2024-03-26T03:49:44.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[29284]: audit 2024-03-26T03:49:43.563804+0000 mon.a (mon.0) 1319 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "166.5"}]: dispatch 2024-03-26T03:49:44.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[29284]: audit 2024-03-26T03:49:43.564260+0000 mon.a (mon.0) 1320 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "166.6"}]: dispatch 2024-03-26T03:49:44.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[29284]: audit 2024-03-26T03:49:43.564727+0000 mon.a (mon.0) 1321 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "166.7"}]: dispatch 2024-03-26T03:49:44.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[29284]: audit 2024-03-26T03:49:43.565146+0000 mon.a (mon.0) 1322 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "166.8"}]: dispatch 2024-03-26T03:49:44.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[29284]: audit 2024-03-26T03:49:43.565586+0000 mon.a (mon.0) 1323 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "166.9"}]: dispatch 2024-03-26T03:49:44.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[29284]: cluster 2024-03-26T03:49:43.644951+0000 osd.6 (osd.6) 7 : cluster [DBG] 166.9 deep-scrub starts 2024-03-26T03:49:44.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[29284]: cluster 2024-03-26T03:49:43.656626+0000 osd.6 (osd.6) 8 : cluster [DBG] 166.9 deep-scrub ok 2024-03-26T03:49:44.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[29284]: cluster 2024-03-26T03:49:43.697060+0000 osd.2 (osd.2) 7 : cluster [DBG] 166.8 scrub starts 2024-03-26T03:49:44.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[29284]: cluster 2024-03-26T03:49:43.699391+0000 osd.2 (osd.2) 8 : cluster [DBG] 166.8 scrub ok 2024-03-26T03:49:44.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[29284]: cluster 2024-03-26T03:49:43.944255+0000 osd.4 (osd.4) 7 : cluster [DBG] 166.3 scrub starts 2024-03-26T03:49:44.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:44 smithi175 bash[29284]: cluster 2024-03-26T03:49:43.950288+0000 osd.4 (osd.4) 8 : cluster [DBG] 166.3 scrub ok 2024-03-26T03:49:45.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:45 smithi190 bash[23392]: audit 2024-03-26T03:49:43.561394+0000 mgr.y (mgr.24431) 219 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "166.0"}]: dispatch 2024-03-26T03:49:45.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:45 smithi190 bash[23392]: audit 2024-03-26T03:49:43.562037+0000 mgr.y (mgr.24431) 220 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "166.1"}]: dispatch 2024-03-26T03:49:45.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:45 smithi190 bash[23392]: audit 2024-03-26T03:49:43.562505+0000 mgr.y (mgr.24431) 221 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "166.2"}]: dispatch 2024-03-26T03:49:45.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:45 smithi190 bash[23392]: audit 2024-03-26T03:49:43.562982+0000 mgr.y (mgr.24431) 222 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "166.3"}]: dispatch 2024-03-26T03:49:45.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:45 smithi190 bash[23392]: audit 2024-03-26T03:49:43.563454+0000 mgr.y (mgr.24431) 223 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "166.4"}]: dispatch 2024-03-26T03:49:45.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:45 smithi190 bash[23392]: audit 2024-03-26T03:49:43.563919+0000 mgr.y (mgr.24431) 224 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "166.5"}]: dispatch 2024-03-26T03:49:45.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:45 smithi190 bash[23392]: audit 2024-03-26T03:49:43.564367+0000 mgr.y (mgr.24431) 225 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "166.6"}]: dispatch 2024-03-26T03:49:45.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:45 smithi190 bash[23392]: audit 2024-03-26T03:49:43.564835+0000 mgr.y (mgr.24431) 226 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "166.7"}]: dispatch 2024-03-26T03:49:45.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:45 smithi190 bash[23392]: audit 2024-03-26T03:49:43.565248+0000 mgr.y (mgr.24431) 227 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "166.8"}]: dispatch 2024-03-26T03:49:45.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:45 smithi190 bash[23392]: audit 2024-03-26T03:49:43.565709+0000 mgr.y (mgr.24431) 228 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "166.9"}]: dispatch 2024-03-26T03:49:45.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:45 smithi190 bash[23392]: cluster 2024-03-26T03:49:43.626721+0000 osd.3 (osd.3) 11 : cluster [DBG] 166.0 scrub starts 2024-03-26T03:49:45.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:45 smithi190 bash[23392]: cluster 2024-03-26T03:49:43.628339+0000 osd.3 (osd.3) 12 : cluster [DBG] 166.0 scrub ok 2024-03-26T03:49:45.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:45 smithi190 bash[23392]: cluster 2024-03-26T03:49:43.879789+0000 osd.1 (osd.1) 7 : cluster [DBG] 166.6 scrub starts 2024-03-26T03:49:45.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:45 smithi190 bash[23392]: cluster 2024-03-26T03:49:43.885780+0000 osd.1 (osd.1) 8 : cluster [DBG] 166.6 scrub ok 2024-03-26T03:49:45.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:45 smithi190 bash[23392]: cluster 2024-03-26T03:49:44.431144+0000 mon.a (mon.0) 1324 : cluster [DBG] osdmap e128: 8 total, 8 up, 8 in 2024-03-26T03:49:45.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:45 smithi190 bash[23392]: audit 2024-03-26T03:49:44.437263+0000 mon.c (mon.2) 223 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:45.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:45 smithi190 bash[23392]: audit 2024-03-26T03:49:44.437629+0000 mon.a (mon.0) 1325 : audit [INF] from='client.? 172.21.15.175:0/1224308315' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi175-64398-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:45.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:45 smithi190 bash[23392]: audit 2024-03-26T03:49:44.443904+0000 mon.c (mon.2) 224 : audit [INF] from='client.? 172.21.15.175:0/2209130636' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi175-64406-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:45.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:45 smithi190 bash[23392]: audit 2024-03-26T03:49:44.444621+0000 mon.a (mon.0) 1326 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi175-64406-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:45.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:45 smithi190 bash[23392]: cluster 2024-03-26T03:49:44.498363+0000 osd.7 (osd.7) 3 : cluster [DBG] 166.1 scrub starts 2024-03-26T03:49:45.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:45 smithi190 bash[23392]: cluster 2024-03-26T03:49:44.500391+0000 osd.7 (osd.7) 4 : cluster [DBG] 166.1 scrub ok 2024-03-26T03:49:45.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:45 smithi190 bash[23392]: cluster 2024-03-26T03:49:44.655729+0000 osd.6 (osd.6) 9 : cluster [DBG] 166.5 scrub starts 2024-03-26T03:49:45.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:45 smithi190 bash[23392]: cluster 2024-03-26T03:49:44.661145+0000 osd.6 (osd.6) 10 : cluster [DBG] 166.5 scrub ok 2024-03-26T03:49:45.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:45 smithi190 bash[23392]: cluster 2024-03-26T03:49:44.715525+0000 osd.2 (osd.2) 9 : cluster [DBG] 166.2 scrub starts 2024-03-26T03:49:45.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:45 smithi190 bash[23392]: cluster 2024-03-26T03:49:44.721111+0000 osd.2 (osd.2) 10 : cluster [DBG] 166.2 scrub ok 2024-03-26T03:49:45.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:45 smithi190 bash[23392]: cluster 2024-03-26T03:49:44.895840+0000 osd.4 (osd.4) 9 : cluster [DBG] 166.7 scrub starts 2024-03-26T03:49:45.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:45 smithi190 bash[23392]: cluster 2024-03-26T03:49:44.902180+0000 osd.4 (osd.4) 10 : cluster [DBG] 166.7 scrub ok 2024-03-26T03:49:45.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[21527]: audit 2024-03-26T03:49:43.561394+0000 mgr.y (mgr.24431) 219 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "166.0"}]: dispatch 2024-03-26T03:49:45.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[21527]: audit 2024-03-26T03:49:43.562037+0000 mgr.y (mgr.24431) 220 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "166.1"}]: dispatch 2024-03-26T03:49:45.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[21527]: audit 2024-03-26T03:49:43.562505+0000 mgr.y (mgr.24431) 221 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "166.2"}]: dispatch 2024-03-26T03:49:45.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[21527]: audit 2024-03-26T03:49:43.562982+0000 mgr.y (mgr.24431) 222 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "166.3"}]: dispatch 2024-03-26T03:49:45.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[21527]: audit 2024-03-26T03:49:43.563454+0000 mgr.y (mgr.24431) 223 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "166.4"}]: dispatch 2024-03-26T03:49:45.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[21527]: audit 2024-03-26T03:49:43.563919+0000 mgr.y (mgr.24431) 224 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "166.5"}]: dispatch 2024-03-26T03:49:45.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[21527]: audit 2024-03-26T03:49:43.564367+0000 mgr.y (mgr.24431) 225 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "166.6"}]: dispatch 2024-03-26T03:49:45.750 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[21527]: audit 2024-03-26T03:49:43.564835+0000 mgr.y (mgr.24431) 226 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "166.7"}]: dispatch 2024-03-26T03:49:45.751 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[21527]: audit 2024-03-26T03:49:43.565248+0000 mgr.y (mgr.24431) 227 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "166.8"}]: dispatch 2024-03-26T03:49:45.751 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[21527]: audit 2024-03-26T03:49:43.565709+0000 mgr.y (mgr.24431) 228 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "166.9"}]: dispatch 2024-03-26T03:49:45.751 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[21527]: cluster 2024-03-26T03:49:43.626721+0000 osd.3 (osd.3) 11 : cluster [DBG] 166.0 scrub starts 2024-03-26T03:49:45.751 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[21527]: cluster 2024-03-26T03:49:43.628339+0000 osd.3 (osd.3) 12 : cluster [DBG] 166.0 scrub ok 2024-03-26T03:49:45.751 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[21527]: cluster 2024-03-26T03:49:43.879789+0000 osd.1 (osd.1) 7 : cluster [DBG] 166.6 scrub starts 2024-03-26T03:49:45.751 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[21527]: cluster 2024-03-26T03:49:43.885780+0000 osd.1 (osd.1) 8 : cluster [DBG] 166.6 scrub ok 2024-03-26T03:49:45.751 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[21527]: cluster 2024-03-26T03:49:44.431144+0000 mon.a (mon.0) 1324 : cluster [DBG] osdmap e128: 8 total, 8 up, 8 in 2024-03-26T03:49:45.751 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[21527]: audit 2024-03-26T03:49:44.437263+0000 mon.c (mon.2) 223 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:45.751 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[21527]: audit 2024-03-26T03:49:44.437629+0000 mon.a (mon.0) 1325 : audit [INF] from='client.? 172.21.15.175:0/1224308315' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi175-64398-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:45.751 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[21527]: audit 2024-03-26T03:49:44.443904+0000 mon.c (mon.2) 224 : audit [INF] from='client.? 172.21.15.175:0/2209130636' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi175-64406-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:45.751 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[21527]: audit 2024-03-26T03:49:44.444621+0000 mon.a (mon.0) 1326 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi175-64406-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:45.751 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[21527]: cluster 2024-03-26T03:49:44.498363+0000 osd.7 (osd.7) 3 : cluster [DBG] 166.1 scrub starts 2024-03-26T03:49:45.751 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[21527]: cluster 2024-03-26T03:49:44.500391+0000 osd.7 (osd.7) 4 : cluster [DBG] 166.1 scrub ok 2024-03-26T03:49:45.752 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[29284]: audit 2024-03-26T03:49:43.561394+0000 mgr.y (mgr.24431) 219 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "166.0"}]: dispatch 2024-03-26T03:49:45.752 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[29284]: audit 2024-03-26T03:49:43.562037+0000 mgr.y (mgr.24431) 220 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "166.1"}]: dispatch 2024-03-26T03:49:45.752 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[29284]: audit 2024-03-26T03:49:43.562505+0000 mgr.y (mgr.24431) 221 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "166.2"}]: dispatch 2024-03-26T03:49:45.752 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[29284]: audit 2024-03-26T03:49:43.562982+0000 mgr.y (mgr.24431) 222 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "166.3"}]: dispatch 2024-03-26T03:49:45.752 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[29284]: audit 2024-03-26T03:49:43.563454+0000 mgr.y (mgr.24431) 223 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "166.4"}]: dispatch 2024-03-26T03:49:45.752 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[29284]: audit 2024-03-26T03:49:43.563919+0000 mgr.y (mgr.24431) 224 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "166.5"}]: dispatch 2024-03-26T03:49:45.752 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[29284]: audit 2024-03-26T03:49:43.564367+0000 mgr.y (mgr.24431) 225 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "166.6"}]: dispatch 2024-03-26T03:49:45.752 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[29284]: audit 2024-03-26T03:49:43.564835+0000 mgr.y (mgr.24431) 226 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "166.7"}]: dispatch 2024-03-26T03:49:45.752 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[29284]: audit 2024-03-26T03:49:43.565248+0000 mgr.y (mgr.24431) 227 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "166.8"}]: dispatch 2024-03-26T03:49:45.752 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[29284]: audit 2024-03-26T03:49:43.565709+0000 mgr.y (mgr.24431) 228 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "166.9"}]: dispatch 2024-03-26T03:49:45.753 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[29284]: cluster 2024-03-26T03:49:43.626721+0000 osd.3 (osd.3) 11 : cluster [DBG] 166.0 scrub starts 2024-03-26T03:49:45.753 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[29284]: cluster 2024-03-26T03:49:43.628339+0000 osd.3 (osd.3) 12 : cluster [DBG] 166.0 scrub ok 2024-03-26T03:49:45.753 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[29284]: cluster 2024-03-26T03:49:43.879789+0000 osd.1 (osd.1) 7 : cluster [DBG] 166.6 scrub starts 2024-03-26T03:49:45.753 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[29284]: cluster 2024-03-26T03:49:43.885780+0000 osd.1 (osd.1) 8 : cluster [DBG] 166.6 scrub ok 2024-03-26T03:49:45.753 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[29284]: cluster 2024-03-26T03:49:44.431144+0000 mon.a (mon.0) 1324 : cluster [DBG] osdmap e128: 8 total, 8 up, 8 in 2024-03-26T03:49:45.753 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[29284]: audit 2024-03-26T03:49:44.437263+0000 mon.c (mon.2) 223 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:45.753 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[29284]: audit 2024-03-26T03:49:44.437629+0000 mon.a (mon.0) 1325 : audit [INF] from='client.? 172.21.15.175:0/1224308315' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi175-64398-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:45.753 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[29284]: audit 2024-03-26T03:49:44.443904+0000 mon.c (mon.2) 224 : audit [INF] from='client.? 172.21.15.175:0/2209130636' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi175-64406-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:45.753 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[29284]: audit 2024-03-26T03:49:44.444621+0000 mon.a (mon.0) 1326 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi175-64406-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:45.753 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[29284]: cluster 2024-03-26T03:49:44.498363+0000 osd.7 (osd.7) 3 : cluster [DBG] 166.1 scrub starts 2024-03-26T03:49:45.753 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[29284]: cluster 2024-03-26T03:49:44.500391+0000 osd.7 (osd.7) 4 : cluster [DBG] 166.1 scrub ok 2024-03-26T03:49:45.753 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[21527]: cluster 2024-03-26T03:49:44.655729+0000 osd.6 (osd.6) 9 : cluster [DBG] 166.5 scrub starts 2024-03-26T03:49:45.754 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[21527]: cluster 2024-03-26T03:49:44.661145+0000 osd.6 (osd.6) 10 : cluster [DBG] 166.5 scrub ok 2024-03-26T03:49:45.754 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[21527]: cluster 2024-03-26T03:49:44.715525+0000 osd.2 (osd.2) 9 : cluster [DBG] 166.2 scrub starts 2024-03-26T03:49:45.754 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[21527]: cluster 2024-03-26T03:49:44.721111+0000 osd.2 (osd.2) 10 : cluster [DBG] 166.2 scrub ok 2024-03-26T03:49:45.754 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[21527]: cluster 2024-03-26T03:49:44.895840+0000 osd.4 (osd.4) 9 : cluster [DBG] 166.7 scrub starts 2024-03-26T03:49:45.754 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[21527]: cluster 2024-03-26T03:49:44.902180+0000 osd.4 (osd.4) 10 : cluster [DBG] 166.7 scrub ok 2024-03-26T03:49:45.754 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[29284]: cluster 2024-03-26T03:49:44.655729+0000 osd.6 (osd.6) 9 : cluster [DBG] 166.5 scrub starts 2024-03-26T03:49:45.754 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[29284]: cluster 2024-03-26T03:49:44.661145+0000 osd.6 (osd.6) 10 : cluster [DBG] 166.5 scrub ok 2024-03-26T03:49:45.754 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[29284]: cluster 2024-03-26T03:49:44.715525+0000 osd.2 (osd.2) 9 : cluster [DBG] 166.2 scrub starts 2024-03-26T03:49:45.754 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[29284]: cluster 2024-03-26T03:49:44.721111+0000 osd.2 (osd.2) 10 : cluster [DBG] 166.2 scrub ok 2024-03-26T03:49:45.754 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[29284]: cluster 2024-03-26T03:49:44.895840+0000 osd.4 (osd.4) 9 : cluster [DBG] 166.7 scrub starts 2024-03-26T03:49:45.754 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:45 smithi175 bash[29284]: cluster 2024-03-26T03:49:44.902180+0000 osd.4 (osd.4) 10 : cluster [DBG] 166.7 scrub ok 2024-03-26T03:49:46.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:46 smithi190 bash[23392]: cluster 2024-03-26T03:49:44.597045+0000 mgr.y (mgr.24431) 229 : cluster [DBG] pgmap v189: 492 pgs: 51 creating+peering, 45 unknown, 396 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 45 KiB/s rd, 0 B/s wr, 58 op/s 2024-03-26T03:49:46.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:46 smithi190 bash[23392]: cluster 2024-03-26T03:49:45.420958+0000 mon.a (mon.0) 1327 : cluster [WRN] Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:46.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:46 smithi190 bash[23392]: audit 2024-03-26T03:49:45.422837+0000 mon.a (mon.0) 1328 : audit [INF] from='client.? 172.21.15.175:0/1224308315' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi175-64398-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:46.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:46 smithi190 bash[23392]: audit 2024-03-26T03:49:45.423112+0000 mon.a (mon.0) 1329 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi175-64406-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:46.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:46 smithi190 bash[23392]: cluster 2024-03-26T03:49:45.431932+0000 mon.a (mon.0) 1330 : cluster [DBG] osdmap e129: 8 total, 8 up, 8 in 2024-03-26T03:49:46.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:46 smithi190 bash[23392]: audit 2024-03-26T03:49:45.457788+0000 mon.c (mon.2) 225 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:46.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:46 smithi190 bash[23392]: cluster 2024-03-26T03:49:45.734969+0000 osd.2 (osd.2) 11 : cluster [DBG] 166.4 scrub starts 2024-03-26T03:49:46.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:46 smithi190 bash[23392]: cluster 2024-03-26T03:49:45.741151+0000 osd.2 (osd.2) 12 : cluster [DBG] 166.4 scrub ok 2024-03-26T03:49:46.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:46 smithi190 bash[23392]: cluster 2024-03-26T03:49:46.404088+0000 mon.a (mon.0) 1331 : cluster [DBG] osdmap e130: 8 total, 8 up, 8 in 2024-03-26T03:49:46.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:46 smithi190 bash[23392]: audit 2024-03-26T03:49:46.408626+0000 mon.a (mon.0) 1332 : audit [INF] from='client.? 172.21.15.175:0/1231167386' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi175-64639-15"}]: dispatch 2024-03-26T03:49:46.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:46 smithi175 bash[21527]: cluster 2024-03-26T03:49:44.597045+0000 mgr.y (mgr.24431) 229 : cluster [DBG] pgmap v189: 492 pgs: 51 creating+peering, 45 unknown, 396 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 45 KiB/s rd, 0 B/s wr, 58 op/s 2024-03-26T03:49:46.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:46 smithi175 bash[21527]: cluster 2024-03-26T03:49:45.420958+0000 mon.a (mon.0) 1327 : cluster [WRN] Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:46.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:46 smithi175 bash[21527]: audit 2024-03-26T03:49:45.422837+0000 mon.a (mon.0) 1328 : audit [INF] from='client.? 172.21.15.175:0/1224308315' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi175-64398-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:46.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:46 smithi175 bash[21527]: audit 2024-03-26T03:49:45.423112+0000 mon.a (mon.0) 1329 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi175-64406-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:46.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:46 smithi175 bash[21527]: cluster 2024-03-26T03:49:45.431932+0000 mon.a (mon.0) 1330 : cluster [DBG] osdmap e129: 8 total, 8 up, 8 in 2024-03-26T03:49:46.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:46 smithi175 bash[21527]: audit 2024-03-26T03:49:45.457788+0000 mon.c (mon.2) 225 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:46.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:46 smithi175 bash[21527]: cluster 2024-03-26T03:49:45.734969+0000 osd.2 (osd.2) 11 : cluster [DBG] 166.4 scrub starts 2024-03-26T03:49:46.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:46 smithi175 bash[21527]: cluster 2024-03-26T03:49:45.741151+0000 osd.2 (osd.2) 12 : cluster [DBG] 166.4 scrub ok 2024-03-26T03:49:46.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:46 smithi175 bash[21527]: cluster 2024-03-26T03:49:46.404088+0000 mon.a (mon.0) 1331 : cluster [DBG] osdmap e130: 8 total, 8 up, 8 in 2024-03-26T03:49:46.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:46 smithi175 bash[21527]: audit 2024-03-26T03:49:46.408626+0000 mon.a (mon.0) 1332 : audit [INF] from='client.? 172.21.15.175:0/1231167386' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi175-64639-15"}]: dispatch 2024-03-26T03:49:46.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:46 smithi175 bash[29284]: cluster 2024-03-26T03:49:44.597045+0000 mgr.y (mgr.24431) 229 : cluster [DBG] pgmap v189: 492 pgs: 51 creating+peering, 45 unknown, 396 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 45 KiB/s rd, 0 B/s wr, 58 op/s 2024-03-26T03:49:46.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:46 smithi175 bash[29284]: cluster 2024-03-26T03:49:45.420958+0000 mon.a (mon.0) 1327 : cluster [WRN] Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:46.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:46 smithi175 bash[29284]: audit 2024-03-26T03:49:45.422837+0000 mon.a (mon.0) 1328 : audit [INF] from='client.? 172.21.15.175:0/1224308315' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi175-64398-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:46.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:46 smithi175 bash[29284]: audit 2024-03-26T03:49:45.423112+0000 mon.a (mon.0) 1329 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi175-64406-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:46.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:46 smithi175 bash[29284]: cluster 2024-03-26T03:49:45.431932+0000 mon.a (mon.0) 1330 : cluster [DBG] osdmap e129: 8 total, 8 up, 8 in 2024-03-26T03:49:46.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:46 smithi175 bash[29284]: audit 2024-03-26T03:49:45.457788+0000 mon.c (mon.2) 225 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:46.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:46 smithi175 bash[29284]: cluster 2024-03-26T03:49:45.734969+0000 osd.2 (osd.2) 11 : cluster [DBG] 166.4 scrub starts 2024-03-26T03:49:46.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:46 smithi175 bash[29284]: cluster 2024-03-26T03:49:45.741151+0000 osd.2 (osd.2) 12 : cluster [DBG] 166.4 scrub ok 2024-03-26T03:49:46.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:46 smithi175 bash[29284]: cluster 2024-03-26T03:49:46.404088+0000 mon.a (mon.0) 1331 : cluster [DBG] osdmap e130: 8 total, 8 up, 8 in 2024-03-26T03:49:46.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:46 smithi175 bash[29284]: audit 2024-03-26T03:49:46.408626+0000 mon.a (mon.0) 1332 : audit [INF] from='client.? 172.21.15.175:0/1231167386' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi175-64639-15"}]: dispatch 2024-03-26T03:49:47.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:47 smithi190 bash[23392]: audit 2024-03-26T03:49:46.459234+0000 mon.c (mon.2) 226 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:47.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:47 smithi190 bash[23392]: cluster 2024-03-26T03:49:46.598017+0000 mgr.y (mgr.24431) 230 : cluster [DBG] pgmap v192: 420 pgs: 32 creating+peering, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 55 KiB/s rd, 0 B/s wr, 83 op/s 2024-03-26T03:49:47.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:47 smithi190 bash[23392]: audit 2024-03-26T03:49:47.400866+0000 mon.a (mon.0) 1333 : audit [INF] from='client.? 172.21.15.175:0/1231167386' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi175-64639-15"}]': finished 2024-03-26T03:49:47.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:47 smithi190 bash[23392]: cluster 2024-03-26T03:49:47.415918+0000 mon.a (mon.0) 1334 : cluster [DBG] osdmap e131: 8 total, 8 up, 8 in 2024-03-26T03:49:47.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:47 smithi190 bash[23392]: audit 2024-03-26T03:49:47.417148+0000 mon.a (mon.0) 1335 : audit [INF] from='client.? 172.21.15.175:0/1231167386' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi175-64639-15"}]: dispatch 2024-03-26T03:49:47.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:47 smithi190 bash[23392]: audit 2024-03-26T03:49:47.420786+0000 mon.a (mon.0) 1336 : audit [INF] from='client.? 172.21.15.175:0/1417513312' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi175-64406-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:47.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:47 smithi190 bash[23392]: audit 2024-03-26T03:49:47.421069+0000 mon.a (mon.0) 1337 : audit [INF] from='client.? 172.21.15.175:0/595977124' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi175-64398-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:47.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:47 smithi175 bash[21527]: audit 2024-03-26T03:49:46.459234+0000 mon.c (mon.2) 226 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:47.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:47 smithi175 bash[21527]: cluster 2024-03-26T03:49:46.598017+0000 mgr.y (mgr.24431) 230 : cluster [DBG] pgmap v192: 420 pgs: 32 creating+peering, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 55 KiB/s rd, 0 B/s wr, 83 op/s 2024-03-26T03:49:47.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:47 smithi175 bash[21527]: audit 2024-03-26T03:49:47.400866+0000 mon.a (mon.0) 1333 : audit [INF] from='client.? 172.21.15.175:0/1231167386' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi175-64639-15"}]': finished 2024-03-26T03:49:47.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:47 smithi175 bash[21527]: cluster 2024-03-26T03:49:47.415918+0000 mon.a (mon.0) 1334 : cluster [DBG] osdmap e131: 8 total, 8 up, 8 in 2024-03-26T03:49:47.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:47 smithi175 bash[21527]: audit 2024-03-26T03:49:47.417148+0000 mon.a (mon.0) 1335 : audit [INF] from='client.? 172.21.15.175:0/1231167386' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi175-64639-15"}]: dispatch 2024-03-26T03:49:47.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:47 smithi175 bash[21527]: audit 2024-03-26T03:49:47.420786+0000 mon.a (mon.0) 1336 : audit [INF] from='client.? 172.21.15.175:0/1417513312' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi175-64406-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:47.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:47 smithi175 bash[21527]: audit 2024-03-26T03:49:47.421069+0000 mon.a (mon.0) 1337 : audit [INF] from='client.? 172.21.15.175:0/595977124' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi175-64398-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:47.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:47 smithi175 bash[29284]: audit 2024-03-26T03:49:46.459234+0000 mon.c (mon.2) 226 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:47.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:47 smithi175 bash[29284]: cluster 2024-03-26T03:49:46.598017+0000 mgr.y (mgr.24431) 230 : cluster [DBG] pgmap v192: 420 pgs: 32 creating+peering, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 55 KiB/s rd, 0 B/s wr, 83 op/s 2024-03-26T03:49:47.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:47 smithi175 bash[29284]: audit 2024-03-26T03:49:47.400866+0000 mon.a (mon.0) 1333 : audit [INF] from='client.? 172.21.15.175:0/1231167386' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi175-64639-15"}]': finished 2024-03-26T03:49:47.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:47 smithi175 bash[29284]: cluster 2024-03-26T03:49:47.415918+0000 mon.a (mon.0) 1334 : cluster [DBG] osdmap e131: 8 total, 8 up, 8 in 2024-03-26T03:49:47.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:47 smithi175 bash[29284]: audit 2024-03-26T03:49:47.417148+0000 mon.a (mon.0) 1335 : audit [INF] from='client.? 172.21.15.175:0/1231167386' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi175-64639-15"}]: dispatch 2024-03-26T03:49:47.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:47 smithi175 bash[29284]: audit 2024-03-26T03:49:47.420786+0000 mon.a (mon.0) 1336 : audit [INF] from='client.? 172.21.15.175:0/1417513312' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi175-64406-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:47.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:47 smithi175 bash[29284]: audit 2024-03-26T03:49:47.421069+0000 mon.a (mon.0) 1337 : audit [INF] from='client.? 172.21.15.175:0/595977124' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi175-64398-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:48.443 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: Running main() from gmock_main.cc 2024-03-26T03:49:48.443 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [==========] Running 13 tests from 4 test suites. 2024-03-26T03:49:48.443 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [----------] Global test environment set-up. 2024-03-26T03:49:48.443 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2024-03-26T03:49:48.444 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2024-03-26T03:49:48.444 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (1996 ms) 2024-03-26T03:49:48.444 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2024-03-26T03:49:48.444 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (2044 ms) 2024-03-26T03:49:48.444 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2024-03-26T03:49:48.444 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (2003 ms) 2024-03-26T03:49:48.444 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2024-03-26T03:49:48.444 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (2011 ms) 2024-03-26T03:49:48.444 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (8055 ms total) 2024-03-26T03:49:48.444 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: 2024-03-26T03:49:48.444 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged 2024-03-26T03:49:48.444 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-03-26T03:49:48.444 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (4085 ms) 2024-03-26T03:49:48.444 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-03-26T03:49:48.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (4050 ms) 2024-03-26T03:49:48.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.FutureSnapRollback 2024-03-26T03:49:48.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.FutureSnapRollback (4941 ms) 2024-03-26T03:49:48.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged (13076 ms total) 2024-03-26T03:49:48.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: 2024-03-26T03:49:48.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2024-03-26T03:49:48.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2024-03-26T03:49:48.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (3017 ms) 2024-03-26T03:49:48.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-03-26T03:49:48.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (1984 ms) 2024-03-26T03:49:48.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2024-03-26T03:49:48.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (2008 ms) 2024-03-26T03:49:48.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-03-26T03:49:48.445 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (1994 ms) 2024-03-26T03:49:48.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (9003 ms total) 2024-03-26T03:49:48.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: 2024-03-26T03:49:48.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-03-26T03:49:48.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-03-26T03:49:48.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (4058 ms) 2024-03-26T03:49:48.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-03-26T03:49:48.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (4031 ms) 2024-03-26T03:49:48.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (8089 ms total) 2024-03-26T03:49:48.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: 2024-03-26T03:49:48.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [----------] Global test environment tear-down 2024-03-26T03:49:48.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [==========] 13 tests from 4 test suites ran. (56123 ms total) 2024-03-26T03:49:48.446 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots: [ PASSED ] 13 tests. 2024-03-26T03:49:48.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:48 smithi190 bash[23392]: audit 2024-03-26T03:49:47.465381+0000 mon.c (mon.2) 227 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:48.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:48 smithi190 bash[23392]: audit 2024-03-26T03:49:48.406858+0000 mon.a (mon.0) 1338 : audit [INF] from='client.? 172.21.15.175:0/1231167386' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi175-64639-15"}]': finished 2024-03-26T03:49:48.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:48 smithi190 bash[23392]: audit 2024-03-26T03:49:48.406999+0000 mon.a (mon.0) 1339 : audit [INF] from='client.? 172.21.15.175:0/1417513312' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi175-64406-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:48.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:48 smithi190 bash[23392]: audit 2024-03-26T03:49:48.407198+0000 mon.a (mon.0) 1340 : audit [INF] from='client.? 172.21.15.175:0/595977124' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi175-64398-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:48.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:48 smithi190 bash[23392]: cluster 2024-03-26T03:49:48.416154+0000 mon.a (mon.0) 1341 : cluster [DBG] osdmap e132: 8 total, 8 up, 8 in 2024-03-26T03:49:48.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:48 smithi175 bash[21527]: audit 2024-03-26T03:49:47.465381+0000 mon.c (mon.2) 227 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:48.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:48 smithi175 bash[21527]: audit 2024-03-26T03:49:48.406858+0000 mon.a (mon.0) 1338 : audit [INF] from='client.? 172.21.15.175:0/1231167386' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi175-64639-15"}]': finished 2024-03-26T03:49:48.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:48 smithi175 bash[21527]: audit 2024-03-26T03:49:48.406999+0000 mon.a (mon.0) 1339 : audit [INF] from='client.? 172.21.15.175:0/1417513312' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi175-64406-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:48.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:48 smithi175 bash[21527]: audit 2024-03-26T03:49:48.407198+0000 mon.a (mon.0) 1340 : audit [INF] from='client.? 172.21.15.175:0/595977124' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi175-64398-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:48.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:48 smithi175 bash[21527]: cluster 2024-03-26T03:49:48.416154+0000 mon.a (mon.0) 1341 : cluster [DBG] osdmap e132: 8 total, 8 up, 8 in 2024-03-26T03:49:48.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:48 smithi175 bash[29284]: audit 2024-03-26T03:49:47.465381+0000 mon.c (mon.2) 227 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:48.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:48 smithi175 bash[29284]: audit 2024-03-26T03:49:48.406858+0000 mon.a (mon.0) 1338 : audit [INF] from='client.? 172.21.15.175:0/1231167386' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi175-64639-15"}]': finished 2024-03-26T03:49:48.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:48 smithi175 bash[29284]: audit 2024-03-26T03:49:48.406999+0000 mon.a (mon.0) 1339 : audit [INF] from='client.? 172.21.15.175:0/1417513312' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi175-64406-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:48.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:48 smithi175 bash[29284]: audit 2024-03-26T03:49:48.407198+0000 mon.a (mon.0) 1340 : audit [INF] from='client.? 172.21.15.175:0/595977124' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi175-64398-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:48.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:48 smithi175 bash[29284]: cluster 2024-03-26T03:49:48.416154+0000 mon.a (mon.0) 1341 : cluster [DBG] osdmap e132: 8 total, 8 up, 8 in 2024-03-26T03:49:49.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:49:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:49:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:49:49.737 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:49 smithi190 bash[23392]: audit 2024-03-26T03:49:48.470263+0000 mon.c (mon.2) 228 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:49.737 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:49 smithi190 bash[23392]: cluster 2024-03-26T03:49:48.598998+0000 mgr.y (mgr.24431) 231 : cluster [DBG] pgmap v195: 452 pgs: 96 unknown, 356 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 9.0 KiB/s rd, 20 op/s 2024-03-26T03:49:49.737 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:49 smithi190 bash[23392]: audit 2024-03-26T03:49:49.139816+0000 mon.a (mon.0) 1342 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:49:49.737 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:49 smithi190 bash[23392]: audit 2024-03-26T03:49:49.142947+0000 mon.c (mon.2) 229 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:49:49.737 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:49 smithi190 bash[23392]: cluster 2024-03-26T03:49:49.425731+0000 mon.a (mon.0) 1343 : cluster [DBG] osdmap e133: 8 total, 8 up, 8 in 2024-03-26T03:49:49.737 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:49 smithi190 bash[23392]: audit 2024-03-26T03:49:49.427716+0000 mon.c (mon.2) 230 : audit [INF] from='client.? 172.21.15.175:0/3469526107' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64636-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:49.737 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:49 smithi190 bash[23392]: audit 2024-03-26T03:49:49.448103+0000 mon.a (mon.0) 1344 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64636-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:49.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:49 smithi175 bash[29284]: audit 2024-03-26T03:49:48.470263+0000 mon.c (mon.2) 228 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:49.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:49 smithi175 bash[29284]: cluster 2024-03-26T03:49:48.598998+0000 mgr.y (mgr.24431) 231 : cluster [DBG] pgmap v195: 452 pgs: 96 unknown, 356 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 9.0 KiB/s rd, 20 op/s 2024-03-26T03:49:49.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:49 smithi175 bash[29284]: audit 2024-03-26T03:49:49.139816+0000 mon.a (mon.0) 1342 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:49:49.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:49 smithi175 bash[29284]: audit 2024-03-26T03:49:49.142947+0000 mon.c (mon.2) 229 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:49:49.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:49 smithi175 bash[29284]: cluster 2024-03-26T03:49:49.425731+0000 mon.a (mon.0) 1343 : cluster [DBG] osdmap e133: 8 total, 8 up, 8 in 2024-03-26T03:49:49.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:49 smithi175 bash[29284]: audit 2024-03-26T03:49:49.427716+0000 mon.c (mon.2) 230 : audit [INF] from='client.? 172.21.15.175:0/3469526107' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64636-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:49.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:49 smithi175 bash[29284]: audit 2024-03-26T03:49:49.448103+0000 mon.a (mon.0) 1344 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64636-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:49.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:49 smithi175 bash[21527]: audit 2024-03-26T03:49:48.470263+0000 mon.c (mon.2) 228 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:49.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:49 smithi175 bash[21527]: cluster 2024-03-26T03:49:48.598998+0000 mgr.y (mgr.24431) 231 : cluster [DBG] pgmap v195: 452 pgs: 96 unknown, 356 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 9.0 KiB/s rd, 20 op/s 2024-03-26T03:49:49.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:49 smithi175 bash[21527]: audit 2024-03-26T03:49:49.139816+0000 mon.a (mon.0) 1342 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:49:49.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:49 smithi175 bash[21527]: audit 2024-03-26T03:49:49.142947+0000 mon.c (mon.2) 229 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:49:49.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:49 smithi175 bash[21527]: cluster 2024-03-26T03:49:49.425731+0000 mon.a (mon.0) 1343 : cluster [DBG] osdmap e133: 8 total, 8 up, 8 in 2024-03-26T03:49:49.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:49 smithi175 bash[21527]: audit 2024-03-26T03:49:49.427716+0000 mon.c (mon.2) 230 : audit [INF] from='client.? 172.21.15.175:0/3469526107' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64636-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:49.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:49 smithi175 bash[21527]: audit 2024-03-26T03:49:49.448103+0000 mon.a (mon.0) 1344 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64636-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:49.990 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:49:49 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:49:50.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:50 smithi190 bash[23392]: audit 2024-03-26T03:49:49.477069+0000 mon.c (mon.2) 231 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:50.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:50 smithi190 bash[23392]: audit 2024-03-26T03:49:49.739225+0000 mgr.y (mgr.24431) 232 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:49:50.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:50 smithi190 bash[23392]: audit 2024-03-26T03:49:50.257243+0000 mon.c (mon.2) 232 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:49:50.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:50 smithi190 bash[23392]: audit 2024-03-26T03:49:50.414714+0000 mon.a (mon.0) 1345 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64636-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:50.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:50 smithi190 bash[23392]: cluster 2024-03-26T03:49:50.437995+0000 mon.a (mon.0) 1346 : cluster [DBG] osdmap e134: 8 total, 8 up, 8 in 2024-03-26T03:49:50.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:50 smithi190 bash[23392]: audit 2024-03-26T03:49:50.439201+0000 mon.c (mon.2) 233 : audit [INF] from='client.? 172.21.15.175:0/1747256013' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi175-64406-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:50.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:50 smithi190 bash[23392]: audit 2024-03-26T03:49:50.439561+0000 mon.c (mon.2) 234 : audit [INF] from='client.? 172.21.15.175:0/162707968' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi175-64398-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:50.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:50 smithi190 bash[23392]: audit 2024-03-26T03:49:50.449516+0000 mon.a (mon.0) 1347 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi175-64406-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:50.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:50 smithi190 bash[23392]: audit 2024-03-26T03:49:50.449931+0000 mon.a (mon.0) 1348 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi175-64398-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:50.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:50 smithi175 bash[21527]: audit 2024-03-26T03:49:49.477069+0000 mon.c (mon.2) 231 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:50.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:50 smithi175 bash[21527]: audit 2024-03-26T03:49:49.739225+0000 mgr.y (mgr.24431) 232 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:49:50.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:50 smithi175 bash[21527]: audit 2024-03-26T03:49:50.257243+0000 mon.c (mon.2) 232 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:49:50.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:50 smithi175 bash[21527]: audit 2024-03-26T03:49:50.414714+0000 mon.a (mon.0) 1345 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64636-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:50.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:50 smithi175 bash[21527]: cluster 2024-03-26T03:49:50.437995+0000 mon.a (mon.0) 1346 : cluster [DBG] osdmap e134: 8 total, 8 up, 8 in 2024-03-26T03:49:50.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:50 smithi175 bash[21527]: audit 2024-03-26T03:49:50.439201+0000 mon.c (mon.2) 233 : audit [INF] from='client.? 172.21.15.175:0/1747256013' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi175-64406-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:50.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:50 smithi175 bash[21527]: audit 2024-03-26T03:49:50.439561+0000 mon.c (mon.2) 234 : audit [INF] from='client.? 172.21.15.175:0/162707968' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi175-64398-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:50.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:50 smithi175 bash[21527]: audit 2024-03-26T03:49:50.449516+0000 mon.a (mon.0) 1347 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi175-64406-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:50.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:50 smithi175 bash[21527]: audit 2024-03-26T03:49:50.449931+0000 mon.a (mon.0) 1348 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi175-64398-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:50.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:50 smithi175 bash[29284]: audit 2024-03-26T03:49:49.477069+0000 mon.c (mon.2) 231 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:50.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:50 smithi175 bash[29284]: audit 2024-03-26T03:49:49.739225+0000 mgr.y (mgr.24431) 232 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:49:50.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:50 smithi175 bash[29284]: audit 2024-03-26T03:49:50.257243+0000 mon.c (mon.2) 232 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:49:50.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:50 smithi175 bash[29284]: audit 2024-03-26T03:49:50.414714+0000 mon.a (mon.0) 1345 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64636-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:50.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:50 smithi175 bash[29284]: cluster 2024-03-26T03:49:50.437995+0000 mon.a (mon.0) 1346 : cluster [DBG] osdmap e134: 8 total, 8 up, 8 in 2024-03-26T03:49:50.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:50 smithi175 bash[29284]: audit 2024-03-26T03:49:50.439201+0000 mon.c (mon.2) 233 : audit [INF] from='client.? 172.21.15.175:0/1747256013' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi175-64406-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:50.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:50 smithi175 bash[29284]: audit 2024-03-26T03:49:50.439561+0000 mon.c (mon.2) 234 : audit [INF] from='client.? 172.21.15.175:0/162707968' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi175-64398-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:50.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:50 smithi175 bash[29284]: audit 2024-03-26T03:49:50.449516+0000 mon.a (mon.0) 1347 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi175-64406-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:50.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:50 smithi175 bash[29284]: audit 2024-03-26T03:49:50.449931+0000 mon.a (mon.0) 1348 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi175-64398-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:51.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:51 smithi190 bash[23392]: audit 2024-03-26T03:49:50.478385+0000 mon.c (mon.2) 235 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:51.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:51 smithi190 bash[23392]: cluster 2024-03-26T03:49:50.599753+0000 mgr.y (mgr.24431) 233 : cluster [DBG] pgmap v198: 484 pgs: 64 unknown, 32 creating+peering, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-26T03:49:51.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:51 smithi190 bash[23392]: audit 2024-03-26T03:49:51.035105+0000 mon.c (mon.2) 236 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:49:51.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:51 smithi190 bash[23392]: audit 2024-03-26T03:49:51.036972+0000 mon.c (mon.2) 237 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:49:51.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:51 smithi190 bash[23392]: audit 2024-03-26T03:49:51.048237+0000 mon.a (mon.0) 1349 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:49:51.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:51 smithi190 bash[23392]: cluster 2024-03-26T03:49:51.395461+0000 mon.a (mon.0) 1350 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:51.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:51 smithi190 bash[23392]: audit 2024-03-26T03:49:51.420012+0000 mon.a (mon.0) 1351 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi175-64406-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:51.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:51 smithi190 bash[23392]: audit 2024-03-26T03:49:51.420241+0000 mon.a (mon.0) 1352 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi175-64398-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:51.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:51 smithi190 bash[23392]: cluster 2024-03-26T03:49:51.430927+0000 mon.a (mon.0) 1353 : cluster [DBG] osdmap e135: 8 total, 8 up, 8 in 2024-03-26T03:49:51.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:51 smithi175 bash[29284]: audit 2024-03-26T03:49:50.478385+0000 mon.c (mon.2) 235 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:51.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:51 smithi175 bash[29284]: cluster 2024-03-26T03:49:50.599753+0000 mgr.y (mgr.24431) 233 : cluster [DBG] pgmap v198: 484 pgs: 64 unknown, 32 creating+peering, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-26T03:49:51.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:51 smithi175 bash[29284]: audit 2024-03-26T03:49:51.035105+0000 mon.c (mon.2) 236 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:49:51.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:51 smithi175 bash[29284]: audit 2024-03-26T03:49:51.036972+0000 mon.c (mon.2) 237 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:49:51.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:51 smithi175 bash[29284]: audit 2024-03-26T03:49:51.048237+0000 mon.a (mon.0) 1349 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:49:51.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:51 smithi175 bash[29284]: cluster 2024-03-26T03:49:51.395461+0000 mon.a (mon.0) 1350 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:51.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:51 smithi175 bash[29284]: audit 2024-03-26T03:49:51.420012+0000 mon.a (mon.0) 1351 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi175-64406-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:51.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:51 smithi175 bash[29284]: audit 2024-03-26T03:49:51.420241+0000 mon.a (mon.0) 1352 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi175-64398-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:51.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:51 smithi175 bash[29284]: cluster 2024-03-26T03:49:51.430927+0000 mon.a (mon.0) 1353 : cluster [DBG] osdmap e135: 8 total, 8 up, 8 in 2024-03-26T03:49:51.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:51 smithi175 bash[21527]: audit 2024-03-26T03:49:50.478385+0000 mon.c (mon.2) 235 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:51.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:51 smithi175 bash[21527]: cluster 2024-03-26T03:49:50.599753+0000 mgr.y (mgr.24431) 233 : cluster [DBG] pgmap v198: 484 pgs: 64 unknown, 32 creating+peering, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-26T03:49:51.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:51 smithi175 bash[21527]: audit 2024-03-26T03:49:51.035105+0000 mon.c (mon.2) 236 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:49:51.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:51 smithi175 bash[21527]: audit 2024-03-26T03:49:51.036972+0000 mon.c (mon.2) 237 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:49:51.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:51 smithi175 bash[21527]: audit 2024-03-26T03:49:51.048237+0000 mon.a (mon.0) 1349 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:49:51.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:51 smithi175 bash[21527]: cluster 2024-03-26T03:49:51.395461+0000 mon.a (mon.0) 1350 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:51.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:51 smithi175 bash[21527]: audit 2024-03-26T03:49:51.420012+0000 mon.a (mon.0) 1351 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi175-64406-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:51.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:51 smithi175 bash[21527]: audit 2024-03-26T03:49:51.420241+0000 mon.a (mon.0) 1352 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi175-64398-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:51.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:51 smithi175 bash[21527]: cluster 2024-03-26T03:49:51.430927+0000 mon.a (mon.0) 1353 : cluster [DBG] osdmap e135: 8 total, 8 up, 8 in 2024-03-26T03:49:52.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:52 smithi190 bash[23392]: audit 2024-03-26T03:49:51.483719+0000 mon.c (mon.2) 238 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:52.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:52 smithi190 bash[23392]: cluster 2024-03-26T03:49:52.430453+0000 mon.a (mon.0) 1354 : cluster [DBG] osdmap e136: 8 total, 8 up, 8 in 2024-03-26T03:49:52.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:52 smithi175 bash[21527]: audit 2024-03-26T03:49:51.483719+0000 mon.c (mon.2) 238 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:52.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:52 smithi175 bash[21527]: cluster 2024-03-26T03:49:52.430453+0000 mon.a (mon.0) 1354 : cluster [DBG] osdmap e136: 8 total, 8 up, 8 in 2024-03-26T03:49:52.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:52 smithi175 bash[29284]: audit 2024-03-26T03:49:51.483719+0000 mon.c (mon.2) 238 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:52.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:52 smithi175 bash[29284]: cluster 2024-03-26T03:49:52.430453+0000 mon.a (mon.0) 1354 : cluster [DBG] osdmap e136: 8 total, 8 up, 8 in 2024-03-26T03:49:53.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:53 smithi190 bash[23392]: audit 2024-03-26T03:49:52.485637+0000 mon.c (mon.2) 239 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:53.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:53 smithi190 bash[23392]: cluster 2024-03-26T03:49:52.600684+0000 mgr.y (mgr.24431) 234 : cluster [DBG] pgmap v201: 420 pgs: 32 creating+peering, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:49:53.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:53 smithi175 bash[21527]: audit 2024-03-26T03:49:52.485637+0000 mon.c (mon.2) 239 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:53.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:53 smithi175 bash[21527]: cluster 2024-03-26T03:49:52.600684+0000 mgr.y (mgr.24431) 234 : cluster [DBG] pgmap v201: 420 pgs: 32 creating+peering, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:49:53.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:53 smithi175 bash[29284]: audit 2024-03-26T03:49:52.485637+0000 mon.c (mon.2) 239 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:53.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:53 smithi175 bash[29284]: cluster 2024-03-26T03:49:52.600684+0000 mgr.y (mgr.24431) 234 : cluster [DBG] pgmap v201: 420 pgs: 32 creating+peering, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:49:54.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:54 smithi190 bash[23392]: audit 2024-03-26T03:49:53.490211+0000 mon.c (mon.2) 240 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:54.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:54 smithi190 bash[23392]: cluster 2024-03-26T03:49:53.499485+0000 mon.a (mon.0) 1355 : cluster [DBG] osdmap e137: 8 total, 8 up, 8 in 2024-03-26T03:49:54.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:54 smithi190 bash[23392]: audit 2024-03-26T03:49:53.520391+0000 mon.a (mon.0) 1356 : audit [INF] from='client.? 172.21.15.175:0/3807629499' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi175-64406-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:54.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:54 smithi190 bash[23392]: audit 2024-03-26T03:49:53.520948+0000 mon.a (mon.0) 1357 : audit [INF] from='client.? 172.21.15.175:0/1114022281' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi175-64398-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:54.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:54 smithi190 bash[23392]: audit 2024-03-26T03:49:53.569619+0000 mon.a (mon.0) 1358 : audit [INF] from='client.? 172.21.15.175:0/692307030' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi175-64636-16"}]: dispatch 2024-03-26T03:49:54.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:54 smithi190 bash[23392]: audit 2024-03-26T03:49:53.570732+0000 mon.a (mon.0) 1359 : audit [INF] from='client.? 172.21.15.175:0/692307030' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi175-64636-16"}]: dispatch 2024-03-26T03:49:54.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:54 smithi190 bash[23392]: audit 2024-03-26T03:49:53.571081+0000 mon.a (mon.0) 1360 : audit [INF] from='client.? 172.21.15.175:0/692307030' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi175-64636-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:49:54.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:54 smithi175 bash[21527]: audit 2024-03-26T03:49:53.490211+0000 mon.c (mon.2) 240 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:54.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:54 smithi175 bash[21527]: cluster 2024-03-26T03:49:53.499485+0000 mon.a (mon.0) 1355 : cluster [DBG] osdmap e137: 8 total, 8 up, 8 in 2024-03-26T03:49:54.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:54 smithi175 bash[21527]: audit 2024-03-26T03:49:53.520391+0000 mon.a (mon.0) 1356 : audit [INF] from='client.? 172.21.15.175:0/3807629499' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi175-64406-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:54.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:54 smithi175 bash[21527]: audit 2024-03-26T03:49:53.520948+0000 mon.a (mon.0) 1357 : audit [INF] from='client.? 172.21.15.175:0/1114022281' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi175-64398-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:54.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:54 smithi175 bash[21527]: audit 2024-03-26T03:49:53.569619+0000 mon.a (mon.0) 1358 : audit [INF] from='client.? 172.21.15.175:0/692307030' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi175-64636-16"}]: dispatch 2024-03-26T03:49:54.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:54 smithi175 bash[21527]: audit 2024-03-26T03:49:53.570732+0000 mon.a (mon.0) 1359 : audit [INF] from='client.? 172.21.15.175:0/692307030' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi175-64636-16"}]: dispatch 2024-03-26T03:49:54.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:54 smithi175 bash[21527]: audit 2024-03-26T03:49:53.571081+0000 mon.a (mon.0) 1360 : audit [INF] from='client.? 172.21.15.175:0/692307030' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi175-64636-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:49:54.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:54 smithi175 bash[29284]: audit 2024-03-26T03:49:53.490211+0000 mon.c (mon.2) 240 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:54.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:54 smithi175 bash[29284]: cluster 2024-03-26T03:49:53.499485+0000 mon.a (mon.0) 1355 : cluster [DBG] osdmap e137: 8 total, 8 up, 8 in 2024-03-26T03:49:54.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:54 smithi175 bash[29284]: audit 2024-03-26T03:49:53.520391+0000 mon.a (mon.0) 1356 : audit [INF] from='client.? 172.21.15.175:0/3807629499' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi175-64406-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:54.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:54 smithi175 bash[29284]: audit 2024-03-26T03:49:53.520948+0000 mon.a (mon.0) 1357 : audit [INF] from='client.? 172.21.15.175:0/1114022281' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi175-64398-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:54.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:54 smithi175 bash[29284]: audit 2024-03-26T03:49:53.569619+0000 mon.a (mon.0) 1358 : audit [INF] from='client.? 172.21.15.175:0/692307030' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi175-64636-16"}]: dispatch 2024-03-26T03:49:54.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:54 smithi175 bash[29284]: audit 2024-03-26T03:49:53.570732+0000 mon.a (mon.0) 1359 : audit [INF] from='client.? 172.21.15.175:0/692307030' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi175-64636-16"}]: dispatch 2024-03-26T03:49:54.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:54 smithi175 bash[29284]: audit 2024-03-26T03:49:53.571081+0000 mon.a (mon.0) 1360 : audit [INF] from='client.? 172.21.15.175:0/692307030' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi175-64636-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:49:55.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:55 smithi190 bash[23392]: audit 2024-03-26T03:49:54.493017+0000 mon.c (mon.2) 241 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:55.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:55 smithi190 bash[23392]: audit 2024-03-26T03:49:54.510388+0000 mon.a (mon.0) 1361 : audit [INF] from='client.? 172.21.15.175:0/3807629499' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi175-64406-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:55.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:55 smithi190 bash[23392]: audit 2024-03-26T03:49:54.510563+0000 mon.a (mon.0) 1362 : audit [INF] from='client.? 172.21.15.175:0/1114022281' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi175-64398-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:55.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:55 smithi190 bash[23392]: audit 2024-03-26T03:49:54.510691+0000 mon.a (mon.0) 1363 : audit [INF] from='client.? 172.21.15.175:0/692307030' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi175-64636-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:49:55.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:55 smithi190 bash[23392]: cluster 2024-03-26T03:49:54.520514+0000 mon.a (mon.0) 1364 : cluster [DBG] osdmap e138: 8 total, 8 up, 8 in 2024-03-26T03:49:55.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:55 smithi190 bash[23392]: audit 2024-03-26T03:49:54.525049+0000 mon.a (mon.0) 1365 : audit [INF] from='client.? 172.21.15.175:0/692307030' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi175-64636-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi175-64636-16"}]: dispatch 2024-03-26T03:49:55.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:55 smithi190 bash[23392]: cluster 2024-03-26T03:49:54.601856+0000 mgr.y (mgr.24431) 235 : cluster [DBG] pgmap v204: 452 pgs: 1 active, 8 creating+activating, 42 unknown, 401 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-26T03:49:55.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:55 smithi175 bash[21527]: audit 2024-03-26T03:49:54.493017+0000 mon.c (mon.2) 241 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:55.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:55 smithi175 bash[21527]: audit 2024-03-26T03:49:54.510388+0000 mon.a (mon.0) 1361 : audit [INF] from='client.? 172.21.15.175:0/3807629499' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi175-64406-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:55.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:55 smithi175 bash[21527]: audit 2024-03-26T03:49:54.510563+0000 mon.a (mon.0) 1362 : audit [INF] from='client.? 172.21.15.175:0/1114022281' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi175-64398-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:55.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:55 smithi175 bash[21527]: audit 2024-03-26T03:49:54.510691+0000 mon.a (mon.0) 1363 : audit [INF] from='client.? 172.21.15.175:0/692307030' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi175-64636-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:49:55.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:55 smithi175 bash[21527]: cluster 2024-03-26T03:49:54.520514+0000 mon.a (mon.0) 1364 : cluster [DBG] osdmap e138: 8 total, 8 up, 8 in 2024-03-26T03:49:55.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:55 smithi175 bash[21527]: audit 2024-03-26T03:49:54.525049+0000 mon.a (mon.0) 1365 : audit [INF] from='client.? 172.21.15.175:0/692307030' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi175-64636-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi175-64636-16"}]: dispatch 2024-03-26T03:49:55.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:55 smithi175 bash[21527]: cluster 2024-03-26T03:49:54.601856+0000 mgr.y (mgr.24431) 235 : cluster [DBG] pgmap v204: 452 pgs: 1 active, 8 creating+activating, 42 unknown, 401 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-26T03:49:55.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:55 smithi175 bash[29284]: audit 2024-03-26T03:49:54.493017+0000 mon.c (mon.2) 241 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:55.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:55 smithi175 bash[29284]: audit 2024-03-26T03:49:54.510388+0000 mon.a (mon.0) 1361 : audit [INF] from='client.? 172.21.15.175:0/3807629499' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi175-64406-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:55.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:55 smithi175 bash[29284]: audit 2024-03-26T03:49:54.510563+0000 mon.a (mon.0) 1362 : audit [INF] from='client.? 172.21.15.175:0/1114022281' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi175-64398-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:55.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:55 smithi175 bash[29284]: audit 2024-03-26T03:49:54.510691+0000 mon.a (mon.0) 1363 : audit [INF] from='client.? 172.21.15.175:0/692307030' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi175-64636-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:49:55.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:55 smithi175 bash[29284]: cluster 2024-03-26T03:49:54.520514+0000 mon.a (mon.0) 1364 : cluster [DBG] osdmap e138: 8 total, 8 up, 8 in 2024-03-26T03:49:55.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:55 smithi175 bash[29284]: audit 2024-03-26T03:49:54.525049+0000 mon.a (mon.0) 1365 : audit [INF] from='client.? 172.21.15.175:0/692307030' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi175-64636-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi175-64636-16"}]: dispatch 2024-03-26T03:49:55.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:55 smithi175 bash[29284]: cluster 2024-03-26T03:49:54.601856+0000 mgr.y (mgr.24431) 235 : cluster [DBG] pgmap v204: 452 pgs: 1 active, 8 creating+activating, 42 unknown, 401 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-26T03:49:56.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:56 smithi190 bash[23392]: audit 2024-03-26T03:49:55.496442+0000 mon.c (mon.2) 242 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:56.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:56 smithi190 bash[23392]: cluster 2024-03-26T03:49:55.544350+0000 mon.a (mon.0) 1366 : cluster [DBG] osdmap e139: 8 total, 8 up, 8 in 2024-03-26T03:49:56.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:56 smithi175 bash[29284]: audit 2024-03-26T03:49:55.496442+0000 mon.c (mon.2) 242 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:56.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:56 smithi175 bash[29284]: cluster 2024-03-26T03:49:55.544350+0000 mon.a (mon.0) 1366 : cluster [DBG] osdmap e139: 8 total, 8 up, 8 in 2024-03-26T03:49:56.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:56 smithi175 bash[21527]: audit 2024-03-26T03:49:55.496442+0000 mon.c (mon.2) 242 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:56.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:56 smithi175 bash[21527]: cluster 2024-03-26T03:49:55.544350+0000 mon.a (mon.0) 1366 : cluster [DBG] osdmap e139: 8 total, 8 up, 8 in 2024-03-26T03:49:57.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:57 smithi190 bash[23392]: audit 2024-03-26T03:49:56.498054+0000 mon.c (mon.2) 243 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:57.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:57 smithi190 bash[23392]: audit 2024-03-26T03:49:56.527628+0000 mon.a (mon.0) 1367 : audit [INF] from='client.? 172.21.15.175:0/692307030' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi175-64636-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi175-64636-16"}]': finished 2024-03-26T03:49:57.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:57 smithi190 bash[23392]: cluster 2024-03-26T03:49:56.537337+0000 mon.a (mon.0) 1368 : cluster [DBG] osdmap e140: 8 total, 8 up, 8 in 2024-03-26T03:49:57.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:57 smithi190 bash[23392]: audit 2024-03-26T03:49:56.541016+0000 mon.c (mon.2) 244 : audit [INF] from='client.? 172.21.15.175:0/3555130207' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi175-64406-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:57.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:57 smithi190 bash[23392]: audit 2024-03-26T03:49:56.541880+0000 mon.c (mon.2) 245 : audit [INF] from='client.? 172.21.15.175:0/1164463712' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi175-64398-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:57.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:57 smithi190 bash[23392]: audit 2024-03-26T03:49:56.550999+0000 mon.a (mon.0) 1369 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi175-64406-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:57.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:57 smithi190 bash[23392]: audit 2024-03-26T03:49:56.551377+0000 mon.a (mon.0) 1370 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi175-64398-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:57.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:57 smithi190 bash[23392]: cluster 2024-03-26T03:49:56.602573+0000 mgr.y (mgr.24431) 236 : cluster [DBG] pgmap v207: 460 pgs: 72 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:49:57.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:57 smithi175 bash[29284]: audit 2024-03-26T03:49:56.498054+0000 mon.c (mon.2) 243 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:57.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:57 smithi175 bash[29284]: audit 2024-03-26T03:49:56.527628+0000 mon.a (mon.0) 1367 : audit [INF] from='client.? 172.21.15.175:0/692307030' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi175-64636-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi175-64636-16"}]': finished 2024-03-26T03:49:57.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:57 smithi175 bash[29284]: cluster 2024-03-26T03:49:56.537337+0000 mon.a (mon.0) 1368 : cluster [DBG] osdmap e140: 8 total, 8 up, 8 in 2024-03-26T03:49:57.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:57 smithi175 bash[29284]: audit 2024-03-26T03:49:56.541016+0000 mon.c (mon.2) 244 : audit [INF] from='client.? 172.21.15.175:0/3555130207' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi175-64406-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:57.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:57 smithi175 bash[29284]: audit 2024-03-26T03:49:56.541880+0000 mon.c (mon.2) 245 : audit [INF] from='client.? 172.21.15.175:0/1164463712' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi175-64398-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:57.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:57 smithi175 bash[29284]: audit 2024-03-26T03:49:56.550999+0000 mon.a (mon.0) 1369 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi175-64406-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:57.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:57 smithi175 bash[29284]: audit 2024-03-26T03:49:56.551377+0000 mon.a (mon.0) 1370 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi175-64398-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:57.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:57 smithi175 bash[29284]: cluster 2024-03-26T03:49:56.602573+0000 mgr.y (mgr.24431) 236 : cluster [DBG] pgmap v207: 460 pgs: 72 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:49:57.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:57 smithi175 bash[21527]: audit 2024-03-26T03:49:56.498054+0000 mon.c (mon.2) 243 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:57.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:57 smithi175 bash[21527]: audit 2024-03-26T03:49:56.527628+0000 mon.a (mon.0) 1367 : audit [INF] from='client.? 172.21.15.175:0/692307030' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi175-64636-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi175-64636-16"}]': finished 2024-03-26T03:49:57.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:57 smithi175 bash[21527]: cluster 2024-03-26T03:49:56.537337+0000 mon.a (mon.0) 1368 : cluster [DBG] osdmap e140: 8 total, 8 up, 8 in 2024-03-26T03:49:57.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:57 smithi175 bash[21527]: audit 2024-03-26T03:49:56.541016+0000 mon.c (mon.2) 244 : audit [INF] from='client.? 172.21.15.175:0/3555130207' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi175-64406-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:57.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:57 smithi175 bash[21527]: audit 2024-03-26T03:49:56.541880+0000 mon.c (mon.2) 245 : audit [INF] from='client.? 172.21.15.175:0/1164463712' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi175-64398-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:57.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:57 smithi175 bash[21527]: audit 2024-03-26T03:49:56.550999+0000 mon.a (mon.0) 1369 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi175-64406-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:57.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:57 smithi175 bash[21527]: audit 2024-03-26T03:49:56.551377+0000 mon.a (mon.0) 1370 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi175-64398-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:57.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:57 smithi175 bash[21527]: cluster 2024-03-26T03:49:56.602573+0000 mgr.y (mgr.24431) 236 : cluster [DBG] pgmap v207: 460 pgs: 72 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:49:58.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:58 smithi175 bash[21527]: audit 2024-03-26T03:49:57.503334+0000 mon.c (mon.2) 246 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:58.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:58 smithi175 bash[21527]: cluster 2024-03-26T03:49:57.527611+0000 mon.a (mon.0) 1371 : cluster [WRN] Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:58.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:58 smithi175 bash[21527]: audit 2024-03-26T03:49:57.540203+0000 mon.a (mon.0) 1372 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi175-64406-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:58.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:58 smithi175 bash[21527]: audit 2024-03-26T03:49:57.540422+0000 mon.a (mon.0) 1373 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi175-64398-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:58.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:58 smithi175 bash[21527]: cluster 2024-03-26T03:49:57.547385+0000 mon.a (mon.0) 1374 : cluster [DBG] osdmap e141: 8 total, 8 up, 8 in 2024-03-26T03:49:58.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:58 smithi175 bash[29284]: audit 2024-03-26T03:49:57.503334+0000 mon.c (mon.2) 246 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:58.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:58 smithi175 bash[29284]: cluster 2024-03-26T03:49:57.527611+0000 mon.a (mon.0) 1371 : cluster [WRN] Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:58.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:58 smithi175 bash[29284]: audit 2024-03-26T03:49:57.540203+0000 mon.a (mon.0) 1372 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi175-64406-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:58.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:58 smithi175 bash[29284]: audit 2024-03-26T03:49:57.540422+0000 mon.a (mon.0) 1373 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi175-64398-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:58.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:58 smithi175 bash[29284]: cluster 2024-03-26T03:49:57.547385+0000 mon.a (mon.0) 1374 : cluster [DBG] osdmap e141: 8 total, 8 up, 8 in 2024-03-26T03:49:58.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:58 smithi190 bash[23392]: audit 2024-03-26T03:49:57.503334+0000 mon.c (mon.2) 246 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:58.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:58 smithi190 bash[23392]: cluster 2024-03-26T03:49:57.527611+0000 mon.a (mon.0) 1371 : cluster [WRN] Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:49:58.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:58 smithi190 bash[23392]: audit 2024-03-26T03:49:57.540203+0000 mon.a (mon.0) 1372 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi175-64406-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:58.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:58 smithi190 bash[23392]: audit 2024-03-26T03:49:57.540422+0000 mon.a (mon.0) 1373 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi175-64398-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:49:58.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:58 smithi190 bash[23392]: cluster 2024-03-26T03:49:57.547385+0000 mon.a (mon.0) 1374 : cluster [DBG] osdmap e141: 8 total, 8 up, 8 in 2024-03-26T03:49:59.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:49:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:49:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:49:59.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:59 smithi190 bash[23392]: audit 2024-03-26T03:49:58.505143+0000 mon.c (mon.2) 247 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:59.749 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:59 smithi190 bash[23392]: cluster 2024-03-26T03:49:58.550512+0000 mon.a (mon.0) 1375 : cluster [DBG] osdmap e142: 8 total, 8 up, 8 in 2024-03-26T03:49:59.749 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:59 smithi190 bash[23392]: cluster 2024-03-26T03:49:58.603087+0000 mgr.y (mgr.24431) 237 : cluster [DBG] pgmap v210: 396 pgs: 8 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:49:59.749 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:59 smithi190 bash[23392]: audit 2024-03-26T03:49:59.044245+0000 mon.c (mon.2) 248 : audit [DBG] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-26T03:49:59.749 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:59 smithi190 bash[23392]: audit 2024-03-26T03:49:59.050334+0000 mon.c (mon.2) 249 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi175-64544-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:59.749 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:49:59 smithi190 bash[23392]: audit 2024-03-26T03:49:59.051135+0000 mon.a (mon.0) 1376 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi175-64544-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:59.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:59 smithi175 bash[29284]: audit 2024-03-26T03:49:58.505143+0000 mon.c (mon.2) 247 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:59.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:59 smithi175 bash[29284]: cluster 2024-03-26T03:49:58.550512+0000 mon.a (mon.0) 1375 : cluster [DBG] osdmap e142: 8 total, 8 up, 8 in 2024-03-26T03:49:59.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:59 smithi175 bash[29284]: cluster 2024-03-26T03:49:58.603087+0000 mgr.y (mgr.24431) 237 : cluster [DBG] pgmap v210: 396 pgs: 8 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:49:59.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:59 smithi175 bash[29284]: audit 2024-03-26T03:49:59.044245+0000 mon.c (mon.2) 248 : audit [DBG] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-26T03:49:59.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:59 smithi175 bash[29284]: audit 2024-03-26T03:49:59.050334+0000 mon.c (mon.2) 249 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi175-64544-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:59.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:49:59 smithi175 bash[29284]: audit 2024-03-26T03:49:59.051135+0000 mon.a (mon.0) 1376 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi175-64544-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:59.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:59 smithi175 bash[21527]: audit 2024-03-26T03:49:58.505143+0000 mon.c (mon.2) 247 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:49:59.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:59 smithi175 bash[21527]: cluster 2024-03-26T03:49:58.550512+0000 mon.a (mon.0) 1375 : cluster [DBG] osdmap e142: 8 total, 8 up, 8 in 2024-03-26T03:49:59.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:59 smithi175 bash[21527]: cluster 2024-03-26T03:49:58.603087+0000 mgr.y (mgr.24431) 237 : cluster [DBG] pgmap v210: 396 pgs: 8 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:49:59.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:59 smithi175 bash[21527]: audit 2024-03-26T03:49:59.044245+0000 mon.c (mon.2) 248 : audit [DBG] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-26T03:49:59.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:59 smithi175 bash[21527]: audit 2024-03-26T03:49:59.050334+0000 mon.c (mon.2) 249 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi175-64544-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:49:59.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:49:59 smithi175 bash[21527]: audit 2024-03-26T03:49:59.051135+0000 mon.a (mon.0) 1376 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi175-64544-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:00.240 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:49:59 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:50:00.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[21527]: audit 2024-03-26T03:49:59.506873+0000 mon.c (mon.2) 250 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:00.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[21527]: audit 2024-03-26T03:49:59.555376+0000 mon.a (mon.0) 1377 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi175-64544-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:00.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[21527]: cluster 2024-03-26T03:49:59.563707+0000 mon.a (mon.0) 1378 : cluster [DBG] osdmap e143: 8 total, 8 up, 8 in 2024-03-26T03:50:00.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[21527]: audit 2024-03-26T03:49:59.572982+0000 mon.c (mon.2) 251 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi175-64544-1","app": "app2"}]: dispatch 2024-03-26T03:50:00.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[21527]: audit 2024-03-26T03:49:59.573790+0000 mon.c (mon.2) 252 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi175-64544-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:00.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[21527]: audit 2024-03-26T03:49:59.576725+0000 mon.c (mon.2) 253 : audit [INF] from='client.? 172.21.15.175:0/3775829989' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi175-64406-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:00.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[21527]: audit 2024-03-26T03:49:59.576943+0000 mon.a (mon.0) 1379 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi175-64544-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:00.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[21527]: audit 2024-03-26T03:49:59.578738+0000 mon.a (mon.0) 1380 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi175-64406-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:00.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[21527]: audit 2024-03-26T03:49:59.579176+0000 mon.c (mon.2) 254 : audit [INF] from='client.? 172.21.15.175:0/2017369014' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi175-64398-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:00.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[21527]: audit 2024-03-26T03:49:59.582452+0000 mon.a (mon.0) 1381 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi175-64398-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:00.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[21527]: audit 2024-03-26T03:49:59.750604+0000 mgr.y (mgr.24431) 238 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:50:00.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[21527]: cluster 2024-03-26T03:50:00.000143+0000 mon.a (mon.0) 1382 : cluster [WRN] Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 5 pool(s) do not have an application enabled 2024-03-26T03:50:00.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[21527]: cluster 2024-03-26T03:50:00.000197+0000 mon.a (mon.0) 1383 : cluster [WRN] [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets 2024-03-26T03:50:00.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[21527]: cluster 2024-03-26T03:50:00.000246+0000 mon.a (mon.0) 1384 : cluster [WRN] pool 'test-rados-api-smithi175-64594-13' with cache_mode writeback needs hit_set_type to be set but it is not 2024-03-26T03:50:00.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[21527]: cluster 2024-03-26T03:50:00.000297+0000 mon.a (mon.0) 1385 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 5 pool(s) do not have an application enabled 2024-03-26T03:50:00.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[21527]: cluster 2024-03-26T03:50:00.000347+0000 mon.a (mon.0) 1386 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-26T03:50:00.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[29284]: audit 2024-03-26T03:49:59.506873+0000 mon.c (mon.2) 250 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:00.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[29284]: audit 2024-03-26T03:49:59.555376+0000 mon.a (mon.0) 1377 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi175-64544-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:00.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[29284]: cluster 2024-03-26T03:49:59.563707+0000 mon.a (mon.0) 1378 : cluster [DBG] osdmap e143: 8 total, 8 up, 8 in 2024-03-26T03:50:00.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[29284]: audit 2024-03-26T03:49:59.572982+0000 mon.c (mon.2) 251 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi175-64544-1","app": "app2"}]: dispatch 2024-03-26T03:50:00.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[29284]: audit 2024-03-26T03:49:59.573790+0000 mon.c (mon.2) 252 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi175-64544-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:00.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[29284]: audit 2024-03-26T03:49:59.576725+0000 mon.c (mon.2) 253 : audit [INF] from='client.? 172.21.15.175:0/3775829989' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi175-64406-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:00.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[29284]: audit 2024-03-26T03:49:59.576943+0000 mon.a (mon.0) 1379 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi175-64544-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:00.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[29284]: audit 2024-03-26T03:49:59.578738+0000 mon.a (mon.0) 1380 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi175-64406-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:00.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[29284]: audit 2024-03-26T03:49:59.579176+0000 mon.c (mon.2) 254 : audit [INF] from='client.? 172.21.15.175:0/2017369014' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi175-64398-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:00.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[29284]: audit 2024-03-26T03:49:59.582452+0000 mon.a (mon.0) 1381 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi175-64398-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:00.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[29284]: audit 2024-03-26T03:49:59.750604+0000 mgr.y (mgr.24431) 238 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:50:00.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[29284]: cluster 2024-03-26T03:50:00.000143+0000 mon.a (mon.0) 1382 : cluster [WRN] Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 5 pool(s) do not have an application enabled 2024-03-26T03:50:00.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[29284]: cluster 2024-03-26T03:50:00.000197+0000 mon.a (mon.0) 1383 : cluster [WRN] [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets 2024-03-26T03:50:00.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[29284]: cluster 2024-03-26T03:50:00.000246+0000 mon.a (mon.0) 1384 : cluster [WRN] pool 'test-rados-api-smithi175-64594-13' with cache_mode writeback needs hit_set_type to be set but it is not 2024-03-26T03:50:00.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[29284]: cluster 2024-03-26T03:50:00.000297+0000 mon.a (mon.0) 1385 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 5 pool(s) do not have an application enabled 2024-03-26T03:50:00.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[29284]: cluster 2024-03-26T03:50:00.000347+0000 mon.a (mon.0) 1386 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-26T03:50:00.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[21527]: cluster 2024-03-26T03:50:00.000385+0000 mon.a (mon.0) 1387 : cluster [WRN] application not enabled on pool 'WatchNotifysmithi175-65446-1' 2024-03-26T03:50:00.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[21527]: cluster 2024-03-26T03:50:00.000420+0000 mon.a (mon.0) 1388 : cluster [WRN] application not enabled on pool 'AssertExistssmithi175-65459-1' 2024-03-26T03:50:00.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[21527]: cluster 2024-03-26T03:50:00.000454+0000 mon.a (mon.0) 1389 : cluster [WRN] application not enabled on pool 'ReusePurgedSnapsmithi175-65381-11' 2024-03-26T03:50:00.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[21527]: cluster 2024-03-26T03:50:00.000488+0000 mon.a (mon.0) 1390 : cluster [WRN] application not enabled on pool 'LibRadosSnapshotsECPP_smithi175-64636-16' 2024-03-26T03:50:00.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[21527]: cluster 2024-03-26T03:50:00.000522+0000 mon.a (mon.0) 1391 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-26T03:50:00.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[29284]: cluster 2024-03-26T03:50:00.000385+0000 mon.a (mon.0) 1387 : cluster [WRN] application not enabled on pool 'WatchNotifysmithi175-65446-1' 2024-03-26T03:50:00.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[29284]: cluster 2024-03-26T03:50:00.000420+0000 mon.a (mon.0) 1388 : cluster [WRN] application not enabled on pool 'AssertExistssmithi175-65459-1' 2024-03-26T03:50:00.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[29284]: cluster 2024-03-26T03:50:00.000454+0000 mon.a (mon.0) 1389 : cluster [WRN] application not enabled on pool 'ReusePurgedSnapsmithi175-65381-11' 2024-03-26T03:50:00.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[29284]: cluster 2024-03-26T03:50:00.000488+0000 mon.a (mon.0) 1390 : cluster [WRN] application not enabled on pool 'LibRadosSnapshotsECPP_smithi175-64636-16' 2024-03-26T03:50:00.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:00 smithi175 bash[29284]: cluster 2024-03-26T03:50:00.000522+0000 mon.a (mon.0) 1391 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-26T03:50:01.037 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:00 smithi190 bash[23392]: audit 2024-03-26T03:49:59.506873+0000 mon.c (mon.2) 250 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:01.037 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:00 smithi190 bash[23392]: audit 2024-03-26T03:49:59.555376+0000 mon.a (mon.0) 1377 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi175-64544-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:01.037 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:00 smithi190 bash[23392]: cluster 2024-03-26T03:49:59.563707+0000 mon.a (mon.0) 1378 : cluster [DBG] osdmap e143: 8 total, 8 up, 8 in 2024-03-26T03:50:01.037 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:00 smithi190 bash[23392]: audit 2024-03-26T03:49:59.572982+0000 mon.c (mon.2) 251 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi175-64544-1","app": "app2"}]: dispatch 2024-03-26T03:50:01.037 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:00 smithi190 bash[23392]: audit 2024-03-26T03:49:59.573790+0000 mon.c (mon.2) 252 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi175-64544-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:01.037 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:00 smithi190 bash[23392]: audit 2024-03-26T03:49:59.576725+0000 mon.c (mon.2) 253 : audit [INF] from='client.? 172.21.15.175:0/3775829989' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi175-64406-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:01.037 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:00 smithi190 bash[23392]: audit 2024-03-26T03:49:59.576943+0000 mon.a (mon.0) 1379 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi175-64544-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:01.037 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:00 smithi190 bash[23392]: audit 2024-03-26T03:49:59.578738+0000 mon.a (mon.0) 1380 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi175-64406-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:01.038 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:00 smithi190 bash[23392]: audit 2024-03-26T03:49:59.579176+0000 mon.c (mon.2) 254 : audit [INF] from='client.? 172.21.15.175:0/2017369014' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi175-64398-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:01.038 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:00 smithi190 bash[23392]: audit 2024-03-26T03:49:59.582452+0000 mon.a (mon.0) 1381 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi175-64398-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:01.038 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:00 smithi190 bash[23392]: audit 2024-03-26T03:49:59.750604+0000 mgr.y (mgr.24431) 238 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:50:01.038 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:00 smithi190 bash[23392]: cluster 2024-03-26T03:50:00.000143+0000 mon.a (mon.0) 1382 : cluster [WRN] Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 5 pool(s) do not have an application enabled 2024-03-26T03:50:01.038 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:00 smithi190 bash[23392]: cluster 2024-03-26T03:50:00.000197+0000 mon.a (mon.0) 1383 : cluster [WRN] [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets 2024-03-26T03:50:01.038 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:00 smithi190 bash[23392]: cluster 2024-03-26T03:50:00.000246+0000 mon.a (mon.0) 1384 : cluster [WRN] pool 'test-rados-api-smithi175-64594-13' with cache_mode writeback needs hit_set_type to be set but it is not 2024-03-26T03:50:01.038 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:00 smithi190 bash[23392]: cluster 2024-03-26T03:50:00.000297+0000 mon.a (mon.0) 1385 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 5 pool(s) do not have an application enabled 2024-03-26T03:50:01.038 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:00 smithi190 bash[23392]: cluster 2024-03-26T03:50:00.000347+0000 mon.a (mon.0) 1386 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-26T03:50:01.038 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:00 smithi190 bash[23392]: cluster 2024-03-26T03:50:00.000385+0000 mon.a (mon.0) 1387 : cluster [WRN] application not enabled on pool 'WatchNotifysmithi175-65446-1' 2024-03-26T03:50:01.038 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:00 smithi190 bash[23392]: cluster 2024-03-26T03:50:00.000420+0000 mon.a (mon.0) 1388 : cluster [WRN] application not enabled on pool 'AssertExistssmithi175-65459-1' 2024-03-26T03:50:01.038 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:00 smithi190 bash[23392]: cluster 2024-03-26T03:50:00.000454+0000 mon.a (mon.0) 1389 : cluster [WRN] application not enabled on pool 'ReusePurgedSnapsmithi175-65381-11' 2024-03-26T03:50:01.038 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:00 smithi190 bash[23392]: cluster 2024-03-26T03:50:00.000488+0000 mon.a (mon.0) 1390 : cluster [WRN] application not enabled on pool 'LibRadosSnapshotsECPP_smithi175-64636-16' 2024-03-26T03:50:01.038 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:00 smithi190 bash[23392]: cluster 2024-03-26T03:50:00.000522+0000 mon.a (mon.0) 1391 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-26T03:50:01.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:01 smithi190 bash[23392]: audit 2024-03-26T03:50:00.508746+0000 mon.c (mon.2) 255 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:01.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:01 smithi190 bash[23392]: audit 2024-03-26T03:50:00.560914+0000 mon.a (mon.0) 1392 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi175-64544-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:01.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:01 smithi190 bash[23392]: audit 2024-03-26T03:50:00.561122+0000 mon.a (mon.0) 1393 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi175-64406-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:01.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:01 smithi190 bash[23392]: audit 2024-03-26T03:50:00.561239+0000 mon.a (mon.0) 1394 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi175-64398-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:01.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:01 smithi190 bash[23392]: cluster 2024-03-26T03:50:00.572888+0000 mon.a (mon.0) 1395 : cluster [DBG] osdmap e144: 8 total, 8 up, 8 in 2024-03-26T03:50:01.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:01 smithi190 bash[23392]: audit 2024-03-26T03:50:00.579382+0000 mon.c (mon.2) 256 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi175-64544-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-03-26T03:50:01.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:01 smithi190 bash[23392]: audit 2024-03-26T03:50:00.580951+0000 mon.c (mon.2) 257 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi175-64544-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-26T03:50:01.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:01 smithi190 bash[23392]: audit 2024-03-26T03:50:00.583972+0000 mon.a (mon.0) 1396 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi175-64544-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-26T03:50:01.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:01 smithi190 bash[23392]: cluster 2024-03-26T03:50:00.604286+0000 mgr.y (mgr.24431) 239 : cluster [DBG] pgmap v213: 460 pgs: 63 creating+peering, 1 unknown, 396 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-26T03:50:01.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:01 smithi190 bash[23392]: audit 2024-03-26T03:50:01.402508+0000 mon.a (mon.0) 1397 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi175-64544-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-03-26T03:50:01.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:01 smithi190 bash[23392]: cluster 2024-03-26T03:50:01.410079+0000 mon.a (mon.0) 1398 : cluster [DBG] osdmap e145: 8 total, 8 up, 8 in 2024-03-26T03:50:01.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:01 smithi190 bash[23392]: audit 2024-03-26T03:50:01.418976+0000 mon.c (mon.2) 258 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi175-64544-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-26T03:50:01.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:01 smithi190 bash[23392]: audit 2024-03-26T03:50:01.420474+0000 mon.a (mon.0) 1399 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi175-64544-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-26T03:50:01.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:01 smithi175 bash[21527]: audit 2024-03-26T03:50:00.508746+0000 mon.c (mon.2) 255 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:01.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:01 smithi175 bash[21527]: audit 2024-03-26T03:50:00.560914+0000 mon.a (mon.0) 1392 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi175-64544-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:01.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:01 smithi175 bash[21527]: audit 2024-03-26T03:50:00.561122+0000 mon.a (mon.0) 1393 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi175-64406-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:01.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:01 smithi175 bash[21527]: audit 2024-03-26T03:50:00.561239+0000 mon.a (mon.0) 1394 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi175-64398-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:01.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:01 smithi175 bash[21527]: cluster 2024-03-26T03:50:00.572888+0000 mon.a (mon.0) 1395 : cluster [DBG] osdmap e144: 8 total, 8 up, 8 in 2024-03-26T03:50:01.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:01 smithi175 bash[21527]: audit 2024-03-26T03:50:00.579382+0000 mon.c (mon.2) 256 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi175-64544-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-03-26T03:50:01.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:01 smithi175 bash[21527]: audit 2024-03-26T03:50:00.580951+0000 mon.c (mon.2) 257 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi175-64544-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-26T03:50:01.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:01 smithi175 bash[21527]: audit 2024-03-26T03:50:00.583972+0000 mon.a (mon.0) 1396 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi175-64544-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-26T03:50:01.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:01 smithi175 bash[21527]: cluster 2024-03-26T03:50:00.604286+0000 mgr.y (mgr.24431) 239 : cluster [DBG] pgmap v213: 460 pgs: 63 creating+peering, 1 unknown, 396 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-26T03:50:01.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:01 smithi175 bash[21527]: audit 2024-03-26T03:50:01.402508+0000 mon.a (mon.0) 1397 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi175-64544-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-03-26T03:50:01.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:01 smithi175 bash[21527]: cluster 2024-03-26T03:50:01.410079+0000 mon.a (mon.0) 1398 : cluster [DBG] osdmap e145: 8 total, 8 up, 8 in 2024-03-26T03:50:01.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:01 smithi175 bash[21527]: audit 2024-03-26T03:50:01.418976+0000 mon.c (mon.2) 258 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi175-64544-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-26T03:50:01.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:01 smithi175 bash[21527]: audit 2024-03-26T03:50:01.420474+0000 mon.a (mon.0) 1399 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi175-64544-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-26T03:50:01.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:01 smithi175 bash[29284]: audit 2024-03-26T03:50:00.508746+0000 mon.c (mon.2) 255 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:01.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:01 smithi175 bash[29284]: audit 2024-03-26T03:50:00.560914+0000 mon.a (mon.0) 1392 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi175-64544-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:01.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:01 smithi175 bash[29284]: audit 2024-03-26T03:50:00.561122+0000 mon.a (mon.0) 1393 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi175-64406-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:01.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:01 smithi175 bash[29284]: audit 2024-03-26T03:50:00.561239+0000 mon.a (mon.0) 1394 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi175-64398-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:01.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:01 smithi175 bash[29284]: cluster 2024-03-26T03:50:00.572888+0000 mon.a (mon.0) 1395 : cluster [DBG] osdmap e144: 8 total, 8 up, 8 in 2024-03-26T03:50:01.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:01 smithi175 bash[29284]: audit 2024-03-26T03:50:00.579382+0000 mon.c (mon.2) 256 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi175-64544-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-03-26T03:50:01.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:01 smithi175 bash[29284]: audit 2024-03-26T03:50:00.580951+0000 mon.c (mon.2) 257 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi175-64544-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-26T03:50:01.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:01 smithi175 bash[29284]: audit 2024-03-26T03:50:00.583972+0000 mon.a (mon.0) 1396 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi175-64544-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-26T03:50:01.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:01 smithi175 bash[29284]: cluster 2024-03-26T03:50:00.604286+0000 mgr.y (mgr.24431) 239 : cluster [DBG] pgmap v213: 460 pgs: 63 creating+peering, 1 unknown, 396 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-26T03:50:01.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:01 smithi175 bash[29284]: audit 2024-03-26T03:50:01.402508+0000 mon.a (mon.0) 1397 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi175-64544-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-03-26T03:50:01.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:01 smithi175 bash[29284]: cluster 2024-03-26T03:50:01.410079+0000 mon.a (mon.0) 1398 : cluster [DBG] osdmap e145: 8 total, 8 up, 8 in 2024-03-26T03:50:01.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:01 smithi175 bash[29284]: audit 2024-03-26T03:50:01.418976+0000 mon.c (mon.2) 258 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi175-64544-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-26T03:50:01.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:01 smithi175 bash[29284]: audit 2024-03-26T03:50:01.420474+0000 mon.a (mon.0) 1399 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi175-64544-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-26T03:50:02.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:02 smithi190 bash[23392]: audit 2024-03-26T03:50:01.510102+0000 mon.c (mon.2) 259 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:02.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:02 smithi190 bash[23392]: audit 2024-03-26T03:50:02.407474+0000 mon.a (mon.0) 1400 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi175-64544-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-03-26T03:50:02.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:02 smithi190 bash[23392]: cluster 2024-03-26T03:50:02.417105+0000 mon.a (mon.0) 1401 : cluster [DBG] osdmap e146: 8 total, 8 up, 8 in 2024-03-26T03:50:02.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:02 smithi190 bash[23392]: audit 2024-03-26T03:50:02.423230+0000 mon.c (mon.2) 260 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi175-64544-1","app":"app1","key":"key1"}]: dispatch 2024-03-26T03:50:02.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:02 smithi190 bash[23392]: audit 2024-03-26T03:50:02.425236+0000 mon.c (mon.2) 261 : audit [INF] from='client.? 172.21.15.175:0/2470251173' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi175-64398-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:02.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:02 smithi190 bash[23392]: audit 2024-03-26T03:50:02.429510+0000 mon.a (mon.0) 1402 : audit [INF] from='client.? 172.21.15.175:0/756353532' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi175-64406-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:02.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:02 smithi190 bash[23392]: audit 2024-03-26T03:50:02.430213+0000 mon.a (mon.0) 1403 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi175-64544-1","app":"app1","key":"key1"}]: dispatch 2024-03-26T03:50:02.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:02 smithi190 bash[23392]: audit 2024-03-26T03:50:02.431254+0000 mon.a (mon.0) 1404 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi175-64398-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:02.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:02 smithi175 bash[21527]: audit 2024-03-26T03:50:01.510102+0000 mon.c (mon.2) 259 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:02.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:02 smithi175 bash[21527]: audit 2024-03-26T03:50:02.407474+0000 mon.a (mon.0) 1400 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi175-64544-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-03-26T03:50:02.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:02 smithi175 bash[21527]: cluster 2024-03-26T03:50:02.417105+0000 mon.a (mon.0) 1401 : cluster [DBG] osdmap e146: 8 total, 8 up, 8 in 2024-03-26T03:50:02.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:02 smithi175 bash[21527]: audit 2024-03-26T03:50:02.423230+0000 mon.c (mon.2) 260 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi175-64544-1","app":"app1","key":"key1"}]: dispatch 2024-03-26T03:50:02.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:02 smithi175 bash[21527]: audit 2024-03-26T03:50:02.425236+0000 mon.c (mon.2) 261 : audit [INF] from='client.? 172.21.15.175:0/2470251173' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi175-64398-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:02.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:02 smithi175 bash[21527]: audit 2024-03-26T03:50:02.429510+0000 mon.a (mon.0) 1402 : audit [INF] from='client.? 172.21.15.175:0/756353532' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi175-64406-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:02.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:02 smithi175 bash[21527]: audit 2024-03-26T03:50:02.430213+0000 mon.a (mon.0) 1403 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi175-64544-1","app":"app1","key":"key1"}]: dispatch 2024-03-26T03:50:02.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:02 smithi175 bash[21527]: audit 2024-03-26T03:50:02.431254+0000 mon.a (mon.0) 1404 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi175-64398-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:02.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:02 smithi175 bash[29284]: audit 2024-03-26T03:50:01.510102+0000 mon.c (mon.2) 259 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:02.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:02 smithi175 bash[29284]: audit 2024-03-26T03:50:02.407474+0000 mon.a (mon.0) 1400 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi175-64544-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-03-26T03:50:02.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:02 smithi175 bash[29284]: cluster 2024-03-26T03:50:02.417105+0000 mon.a (mon.0) 1401 : cluster [DBG] osdmap e146: 8 total, 8 up, 8 in 2024-03-26T03:50:02.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:02 smithi175 bash[29284]: audit 2024-03-26T03:50:02.423230+0000 mon.c (mon.2) 260 : audit [INF] from='client.? 172.21.15.175:0/2667587751' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi175-64544-1","app":"app1","key":"key1"}]: dispatch 2024-03-26T03:50:02.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:02 smithi175 bash[29284]: audit 2024-03-26T03:50:02.425236+0000 mon.c (mon.2) 261 : audit [INF] from='client.? 172.21.15.175:0/2470251173' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi175-64398-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:02.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:02 smithi175 bash[29284]: audit 2024-03-26T03:50:02.429510+0000 mon.a (mon.0) 1402 : audit [INF] from='client.? 172.21.15.175:0/756353532' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi175-64406-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:02.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:02 smithi175 bash[29284]: audit 2024-03-26T03:50:02.430213+0000 mon.a (mon.0) 1403 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi175-64544-1","app":"app1","key":"key1"}]: dispatch 2024-03-26T03:50:02.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:02 smithi175 bash[29284]: audit 2024-03-26T03:50:02.431254+0000 mon.a (mon.0) 1404 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi175-64398-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:03.418 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: Running main() from gmock_main.cc 2024-03-26T03:50:03.418 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [==========] Running 21 tests from 5 test suites. 2024-03-26T03:50:03.418 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2024-03-26T03:50:03.418 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2024-03-26T03:50:03.418 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: seed 64636 2024-03-26T03:50:03.418 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-03-26T03:50:03.418 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (1999 ms) 2024-03-26T03:50:03.418 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-03-26T03:50:03.418 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (2047 ms) 2024-03-26T03:50:03.418 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-03-26T03:50:03.418 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (1999 ms) 2024-03-26T03:50:03.418 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-03-26T03:50:03.419 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (2025 ms) 2024-03-26T03:50:03.419 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-03-26T03:50:03.419 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (3992 ms) 2024-03-26T03:50:03.419 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (12063 ms total) 2024-03-26T03:50:03.419 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: 2024-03-26T03:50:03.419 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP 2024-03-26T03:50:03.419 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-03-26T03:50:03.419 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (4132 ms) 2024-03-26T03:50:03.419 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-03-26T03:50:03.419 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (3856 ms) 2024-03-26T03:50:03.419 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-03-26T03:50:03.419 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (6017 ms) 2024-03-26T03:50:03.419 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-03-26T03:50:03.419 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (3967 ms) 2024-03-26T03:50:03.419 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-03-26T03:50:03.420 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2097 ms) 2024-03-26T03:50:03.420 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-03-26T03:50:03.420 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: ./src/test/librados/snapshots_cxx.cc:463: Skipped 2024-03-26T03:50:03.420 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: 2024-03-26T03:50:03.420 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback (0 ms) 2024-03-26T03:50:03.420 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-03-26T03:50:03.420 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: deleting snap 14 in pool LibRadosSnapshotsSelfManagedPP_smithi175-64636-7 2024-03-26T03:50:03.420 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: waiting for snaps to purge 2024-03-26T03:50:03.420 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (17971 ms) 2024-03-26T03:50:03.420 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP (38040 ms total) 2024-03-26T03:50:03.420 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: 2024-03-26T03:50:03.420 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-03-26T03:50:03.420 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-03-26T03:50:03.420 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (27 ms) 2024-03-26T03:50:03.420 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-03-26T03:50:03.420 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5101 ms) 2024-03-26T03:50:03.421 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5128 ms total) 2024-03-26T03:50:03.421 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: 2024-03-26T03:50:03.421 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2024-03-26T03:50:03.421 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-03-26T03:50:03.421 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (3025 ms) 2024-03-26T03:50:03.421 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-03-26T03:50:03.421 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapRemovePP (1834 ms) 2024-03-26T03:50:03.421 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-03-26T03:50:03.421 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (2009 ms) 2024-03-26T03:50:03.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:03 smithi190 bash[23392]: audit 2024-03-26T03:50:02.511292+0000 mon.c (mon.2) 262 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:03.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:03 smithi190 bash[23392]: cluster 2024-03-26T03:50:02.605418+0000 mgr.y (mgr.24431) 240 : cluster [DBG] pgmap v216: 460 pgs: 64 unknown, 396 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-26T03:50:03.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:03 smithi190 bash[23392]: audit 2024-03-26T03:50:03.411102+0000 mon.a (mon.0) 1405 : audit [INF] from='client.? 172.21.15.175:0/756353532' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi175-64406-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:03.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:03 smithi190 bash[23392]: audit 2024-03-26T03:50:03.411340+0000 mon.a (mon.0) 1406 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi175-64544-1","app":"app1","key":"key1"}]': finished 2024-03-26T03:50:03.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:03 smithi190 bash[23392]: audit 2024-03-26T03:50:03.411572+0000 mon.a (mon.0) 1407 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi175-64398-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:03.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:03 smithi190 bash[23392]: cluster 2024-03-26T03:50:03.419462+0000 mon.a (mon.0) 1408 : cluster [DBG] osdmap e147: 8 total, 8 up, 8 in 2024-03-26T03:50:03.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:03 smithi175 bash[29284]: audit 2024-03-26T03:50:02.511292+0000 mon.c (mon.2) 262 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:03.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:03 smithi175 bash[29284]: cluster 2024-03-26T03:50:02.605418+0000 mgr.y (mgr.24431) 240 : cluster [DBG] pgmap v216: 460 pgs: 64 unknown, 396 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-26T03:50:03.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:03 smithi175 bash[29284]: audit 2024-03-26T03:50:03.411102+0000 mon.a (mon.0) 1405 : audit [INF] from='client.? 172.21.15.175:0/756353532' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi175-64406-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:03.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:03 smithi175 bash[29284]: audit 2024-03-26T03:50:03.411340+0000 mon.a (mon.0) 1406 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi175-64544-1","app":"app1","key":"key1"}]': finished 2024-03-26T03:50:03.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:03 smithi175 bash[29284]: audit 2024-03-26T03:50:03.411572+0000 mon.a (mon.0) 1407 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi175-64398-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:03.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:03 smithi175 bash[29284]: cluster 2024-03-26T03:50:03.419462+0000 mon.a (mon.0) 1408 : cluster [DBG] osdmap e147: 8 total, 8 up, 8 in 2024-03-26T03:50:03.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:03 smithi175 bash[21527]: audit 2024-03-26T03:50:02.511292+0000 mon.c (mon.2) 262 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:03.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:03 smithi175 bash[21527]: cluster 2024-03-26T03:50:02.605418+0000 mgr.y (mgr.24431) 240 : cluster [DBG] pgmap v216: 460 pgs: 64 unknown, 396 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-26T03:50:03.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:03 smithi175 bash[21527]: audit 2024-03-26T03:50:03.411102+0000 mon.a (mon.0) 1405 : audit [INF] from='client.? 172.21.15.175:0/756353532' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi175-64406-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:03.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:03 smithi175 bash[21527]: audit 2024-03-26T03:50:03.411340+0000 mon.a (mon.0) 1406 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi175-64544-1","app":"app1","key":"key1"}]': finished 2024-03-26T03:50:03.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:03 smithi175 bash[21527]: audit 2024-03-26T03:50:03.411572+0000 mon.a (mon.0) 1407 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi175-64398-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:03.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:03 smithi175 bash[21527]: cluster 2024-03-26T03:50:03.419462+0000 mon.a (mon.0) 1408 : cluster [DBG] osdmap e147: 8 total, 8 up, 8 in 2024-03-26T03:50:04.427 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_ api_misc_pp: [==========] Running 31 tests from 7 test suites. 2024-03-26T03:50:04.428 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [----------] Global test environment set-up. 2024-03-26T03:50:04.428 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2024-03-26T03:50:04.428 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2024-03-26T03:50:04.428 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-03-26T03:50:04.428 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-03-26T03:50:04.428 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: 2024-03-26T03:50:04.428 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2024-03-26T03:50:04.428 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: seed 64544 2024-03-26T03:50:04.428 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-03-26T03:50:04.428 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (4 ms) 2024-03-26T03:50:04.428 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2024-03-26T03:50:04.428 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (436 ms) 2024-03-26T03:50:04.428 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2024-03-26T03:50:04.428 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (19 ms) 2024-03-26T03:50:04.428 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2024-03-26T03:50:04.429 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (17 ms) 2024-03-26T03:50:04.429 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-03-26T03:50:04.429 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (19 ms) 2024-03-26T03:50:04.429 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2024-03-26T03:50:04.429 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (6 ms) 2024-03-26T03:50:04.429 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2024-03-26T03:50:04.429 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (3 ms) 2024-03-26T03:50:04.429 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2024-03-26T03:50:04.429 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (7 ms) 2024-03-26T03:50:04.429 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2024-03-26T03:50:04.429 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (2 ms) 2024-03-26T03:50:04.429 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2024-03-26T03:50:04.429 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (11 ms) 2024-03-26T03:50:04.429 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2024-03-26T03:50:04.429 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (3 ms) 2024-03-26T03:50:04.430 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2024-03-26T03:50:04.430 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (6 ms) 2024-03-26T03:50:04.430 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2024-03-26T03:50:04.430 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (9 ms) 2024-03-26T03:50:04.430 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2024-03-26T03:50:04.430 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: osd_max_attr_size = 0 2024-03-26T03:50:04.430 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2024-03-26T03:50:04.430 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (2424 ms) 2024-03-26T03:50:04.430 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2024-03-26T03:50:04.430 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (599 ms) 2024-03-26T03:50:04.430 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2024-03-26T03:50:04.430 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: waiting for initial deep scrubs... 2024-03-26T03:50:04.430 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: done waiting, doing copies 2024-03-26T03:50:04.430 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: waiting for final deep scrubs... 2024-03-26T03:50:04.430 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: done waiting 2024-03-26T03:50:04.431 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (61103 ms) 2024-03-26T03:50:04.431 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2024-03-26T03:50:04.431 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (28 ms) 2024-03-26T03:50:04.431 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2024-03-26T03:50:04.431 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (7 ms) 2024-03-26T03:50:04.431 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2024-03-26T03:50:04.431 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (4382 ms) 2024-03-26T03:50:04.431 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2024-03-26T03:50:04.431 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-03-26T03:50:04.431 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2024-03-26T03:50:04.431 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-03-26T03:50:04.431 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2024-03-26T03:50:04.431 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2024-03-26T03:50:04.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:04 smithi190 bash[23392]: cluster 2024-03-26T03:50:03.503195+0000 mon.a (mon.0) 1409 : cluster [WRN] Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:50:04.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:04 smithi190 bash[23392]: audit 2024-03-26T03:50:03.512669+0000 mon.c (mon.2) 263 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:04.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:04 smithi190 bash[23392]: audit 2024-03-26T03:50:04.181022+0000 mon.c (mon.2) 264 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:50:04.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:04 smithi190 bash[23392]: cluster 2024-03-26T03:50:04.422309+0000 mon.a (mon.0) 1410 : cluster [DBG] osdmap e148: 8 total, 8 up, 8 in 2024-03-26T03:50:04.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:04 smithi190 bash[23392]: audit 2024-03-26T03:50:04.476210+0000 mon.c (mon.2) 265 : audit [INF] from='client.? 172.21.15.175:0/489945439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64544-24"}]: dispatch 2024-03-26T03:50:04.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:04 smithi190 bash[23392]: audit 2024-03-26T03:50:04.476652+0000 mon.a (mon.0) 1411 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64544-24"}]: dispatch 2024-03-26T03:50:04.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:04 smithi190 bash[23392]: audit 2024-03-26T03:50:04.478402+0000 mon.c (mon.2) 266 : audit [INF] from='client.? 172.21.15.175:0/489945439' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64544-24"}]: dispatch 2024-03-26T03:50:04.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:04 smithi190 bash[23392]: audit 2024-03-26T03:50:04.485018+0000 mon.a (mon.0) 1412 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64544-24"}]: dispatch 2024-03-26T03:50:04.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:04 smithi190 bash[23392]: audit 2024-03-26T03:50:04.485781+0000 mon.c (mon.2) 267 : audit [INF] from='client.? 172.21.15.175:0/489945439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi175-64544-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:04.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:04 smithi190 bash[23392]: audit 2024-03-26T03:50:04.490101+0000 mon.a (mon.0) 1413 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi175-64544-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:04.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:04 smithi175 bash[21527]: cluster 2024-03-26T03:50:03.503195+0000 mon.a (mon.0) 1409 : cluster [WRN] Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:50:04.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:04 smithi175 bash[21527]: audit 2024-03-26T03:50:03.512669+0000 mon.c (mon.2) 263 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:04.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:04 smithi175 bash[21527]: audit 2024-03-26T03:50:04.181022+0000 mon.c (mon.2) 264 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:50:04.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:04 smithi175 bash[21527]: cluster 2024-03-26T03:50:04.422309+0000 mon.a (mon.0) 1410 : cluster [DBG] osdmap e148: 8 total, 8 up, 8 in 2024-03-26T03:50:04.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:04 smithi175 bash[21527]: audit 2024-03-26T03:50:04.476210+0000 mon.c (mon.2) 265 : audit [INF] from='client.? 172.21.15.175:0/489945439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64544-24"}]: dispatch 2024-03-26T03:50:04.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:04 smithi175 bash[21527]: audit 2024-03-26T03:50:04.476652+0000 mon.a (mon.0) 1411 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64544-24"}]: dispatch 2024-03-26T03:50:04.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:04 smithi175 bash[21527]: audit 2024-03-26T03:50:04.478402+0000 mon.c (mon.2) 266 : audit [INF] from='client.? 172.21.15.175:0/489945439' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64544-24"}]: dispatch 2024-03-26T03:50:04.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:04 smithi175 bash[21527]: audit 2024-03-26T03:50:04.485018+0000 mon.a (mon.0) 1412 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64544-24"}]: dispatch 2024-03-26T03:50:04.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:04 smithi175 bash[21527]: audit 2024-03-26T03:50:04.485781+0000 mon.c (mon.2) 267 : audit [INF] from='client.? 172.21.15.175:0/489945439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi175-64544-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:04.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:04 smithi175 bash[21527]: audit 2024-03-26T03:50:04.490101+0000 mon.a (mon.0) 1413 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi175-64544-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:04.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:04 smithi175 bash[29284]: cluster 2024-03-26T03:50:03.503195+0000 mon.a (mon.0) 1409 : cluster [WRN] Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:50:04.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:04 smithi175 bash[29284]: audit 2024-03-26T03:50:03.512669+0000 mon.c (mon.2) 263 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:04.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:04 smithi175 bash[29284]: audit 2024-03-26T03:50:04.181022+0000 mon.c (mon.2) 264 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:50:04.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:04 smithi175 bash[29284]: cluster 2024-03-26T03:50:04.422309+0000 mon.a (mon.0) 1410 : cluster [DBG] osdmap e148: 8 total, 8 up, 8 in 2024-03-26T03:50:04.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:04 smithi175 bash[29284]: audit 2024-03-26T03:50:04.476210+0000 mon.c (mon.2) 265 : audit [INF] from='client.? 172.21.15.175:0/489945439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64544-24"}]: dispatch 2024-03-26T03:50:04.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:04 smithi175 bash[29284]: audit 2024-03-26T03:50:04.476652+0000 mon.a (mon.0) 1411 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64544-24"}]: dispatch 2024-03-26T03:50:04.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:04 smithi175 bash[29284]: audit 2024-03-26T03:50:04.478402+0000 mon.c (mon.2) 266 : audit [INF] from='client.? 172.21.15.175:0/489945439' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64544-24"}]: dispatch 2024-03-26T03:50:04.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:04 smithi175 bash[29284]: audit 2024-03-26T03:50:04.485018+0000 mon.a (mon.0) 1412 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64544-24"}]: dispatch 2024-03-26T03:50:04.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:04 smithi175 bash[29284]: audit 2024-03-26T03:50:04.485781+0000 mon.c (mon.2) 267 : audit [INF] from='client.? 172.21.15.175:0/489945439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi175-64544-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:04.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:04 smithi175 bash[29284]: audit 2024-03-26T03:50:04.490101+0000 mon.a (mon.0) 1413 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi175-64544-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:05.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:05 smithi190 bash[23392]: audit 2024-03-26T03:50:04.513508+0000 mon.c (mon.2) 268 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:05.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:05 smithi190 bash[23392]: cluster 2024-03-26T03:50:04.607290+0000 mgr.y (mgr.24431) 241 : cluster [DBG] pgmap v219: 364 pgs: 364 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-26T03:50:05.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:05 smithi190 bash[23392]: audit 2024-03-26T03:50:04.607943+0000 mon.c (mon.2) 269 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi175-64438-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-03-26T03:50:05.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:05 smithi190 bash[23392]: audit 2024-03-26T03:50:04.611471+0000 mon.a (mon.0) 1414 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi175-64438-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-03-26T03:50:05.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:05 smithi190 bash[23392]: audit 2024-03-26T03:50:05.419244+0000 mon.a (mon.0) 1415 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi175-64544-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:05.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:05 smithi190 bash[23392]: audit 2024-03-26T03:50:05.419489+0000 mon.a (mon.0) 1416 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi175-64438-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-03-26T03:50:05.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:05 smithi190 bash[23392]: cluster 2024-03-26T03:50:05.427712+0000 mon.a (mon.0) 1417 : cluster [DBG] osdmap e149: 8 total, 8 up, 8 in 2024-03-26T03:50:05.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:05 smithi190 bash[23392]: audit 2024-03-26T03:50:05.433264+0000 mon.a (mon.0) 1418 : audit [INF] from='client.? 172.21.15.175:0/3232347027' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi175-64406-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:05.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:05 smithi190 bash[23392]: audit 2024-03-26T03:50:05.437539+0000 mon.c (mon.2) 270 : audit [INF] from='client.? 172.21.15.175:0/489945439' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64544-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi175-64544-24"}]: dispatch 2024-03-26T03:50:05.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:05 smithi190 bash[23392]: audit 2024-03-26T03:50:05.439637+0000 mon.a (mon.0) 1419 : audit [INF] from='client.? 172.21.15.175:0/667812707' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi175-64398-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:05.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:05 smithi190 bash[23392]: audit 2024-03-26T03:50:05.440235+0000 mon.a (mon.0) 1420 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64544-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi175-64544-24"}]: dispatch 2024-03-26T03:50:05.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:05 smithi175 bash[21527]: audit 2024-03-26T03:50:04.513508+0000 mon.c (mon.2) 268 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:05.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:05 smithi175 bash[21527]: cluster 2024-03-26T03:50:04.607290+0000 mgr.y (mgr.24431) 241 : cluster [DBG] pgmap v219: 364 pgs: 364 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-26T03:50:05.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:05 smithi175 bash[21527]: audit 2024-03-26T03:50:04.607943+0000 mon.c (mon.2) 269 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi175-64438-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-03-26T03:50:05.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:05 smithi175 bash[21527]: audit 2024-03-26T03:50:04.611471+0000 mon.a (mon.0) 1414 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi175-64438-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-03-26T03:50:05.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:05 smithi175 bash[21527]: audit 2024-03-26T03:50:05.419244+0000 mon.a (mon.0) 1415 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi175-64544-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:05.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:05 smithi175 bash[21527]: audit 2024-03-26T03:50:05.419489+0000 mon.a (mon.0) 1416 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi175-64438-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-03-26T03:50:05.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:05 smithi175 bash[21527]: cluster 2024-03-26T03:50:05.427712+0000 mon.a (mon.0) 1417 : cluster [DBG] osdmap e149: 8 total, 8 up, 8 in 2024-03-26T03:50:05.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:05 smithi175 bash[21527]: audit 2024-03-26T03:50:05.433264+0000 mon.a (mon.0) 1418 : audit [INF] from='client.? 172.21.15.175:0/3232347027' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi175-64406-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:05.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:05 smithi175 bash[21527]: audit 2024-03-26T03:50:05.437539+0000 mon.c (mon.2) 270 : audit [INF] from='client.? 172.21.15.175:0/489945439' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64544-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi175-64544-24"}]: dispatch 2024-03-26T03:50:05.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:05 smithi175 bash[21527]: audit 2024-03-26T03:50:05.439637+0000 mon.a (mon.0) 1419 : audit [INF] from='client.? 172.21.15.175:0/667812707' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi175-64398-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:05.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:05 smithi175 bash[21527]: audit 2024-03-26T03:50:05.440235+0000 mon.a (mon.0) 1420 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64544-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi175-64544-24"}]: dispatch 2024-03-26T03:50:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:05 smithi175 bash[29284]: audit 2024-03-26T03:50:04.513508+0000 mon.c (mon.2) 268 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:05 smithi175 bash[29284]: cluster 2024-03-26T03:50:04.607290+0000 mgr.y (mgr.24431) 241 : cluster [DBG] pgmap v219: 364 pgs: 364 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-26T03:50:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:05 smithi175 bash[29284]: audit 2024-03-26T03:50:04.607943+0000 mon.c (mon.2) 269 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi175-64438-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-03-26T03:50:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:05 smithi175 bash[29284]: audit 2024-03-26T03:50:04.611471+0000 mon.a (mon.0) 1414 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi175-64438-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-03-26T03:50:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:05 smithi175 bash[29284]: audit 2024-03-26T03:50:05.419244+0000 mon.a (mon.0) 1415 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi175-64544-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:05 smithi175 bash[29284]: audit 2024-03-26T03:50:05.419489+0000 mon.a (mon.0) 1416 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi175-64438-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-03-26T03:50:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:05 smithi175 bash[29284]: cluster 2024-03-26T03:50:05.427712+0000 mon.a (mon.0) 1417 : cluster [DBG] osdmap e149: 8 total, 8 up, 8 in 2024-03-26T03:50:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:05 smithi175 bash[29284]: audit 2024-03-26T03:50:05.433264+0000 mon.a (mon.0) 1418 : audit [INF] from='client.? 172.21.15.175:0/3232347027' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi175-64406-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:05 smithi175 bash[29284]: audit 2024-03-26T03:50:05.437539+0000 mon.c (mon.2) 270 : audit [INF] from='client.? 172.21.15.175:0/489945439' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64544-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi175-64544-24"}]: dispatch 2024-03-26T03:50:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:05 smithi175 bash[29284]: audit 2024-03-26T03:50:05.439637+0000 mon.a (mon.0) 1419 : audit [INF] from='client.? 172.21.15.175:0/667812707' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi175-64398-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:05 smithi175 bash[29284]: audit 2024-03-26T03:50:05.440235+0000 mon.a (mon.0) 1420 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64544-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi175-64544-24"}]: dispatch 2024-03-26T03:50:06.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:06 smithi190 bash[23392]: audit 2024-03-26T03:50:05.515754+0000 mon.c (mon.2) 271 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:06.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:06 smithi190 bash[23392]: audit 2024-03-26T03:50:06.406946+0000 mon.a (mon.0) 1421 : audit [INF] from='client.? 172.21.15.175:0/3232347027' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi175-64406-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:06.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:06 smithi190 bash[23392]: audit 2024-03-26T03:50:06.407115+0000 mon.a (mon.0) 1422 : audit [INF] from='client.? 172.21.15.175:0/667812707' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi175-64398-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:06.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:06 smithi190 bash[23392]: cluster 2024-03-26T03:50:06.426414+0000 mon.a (mon.0) 1423 : cluster [DBG] osdmap e150: 8 total, 8 up, 8 in 2024-03-26T03:50:06.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:06 smithi190 bash[23392]: audit 2024-03-26T03:50:06.431645+0000 mon.a (mon.0) 1424 : audit [INF] from='client.? 172.21.15.175:0/692307030' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi175-64636-16"}]: dispatch 2024-03-26T03:50:06.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:06 smithi175 bash[21527]: audit 2024-03-26T03:50:05.515754+0000 mon.c (mon.2) 271 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:06.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:06 smithi175 bash[21527]: audit 2024-03-26T03:50:06.406946+0000 mon.a (mon.0) 1421 : audit [INF] from='client.? 172.21.15.175:0/3232347027' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi175-64406-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:06.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:06 smithi175 bash[21527]: audit 2024-03-26T03:50:06.407115+0000 mon.a (mon.0) 1422 : audit [INF] from='client.? 172.21.15.175:0/667812707' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi175-64398-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:06.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:06 smithi175 bash[21527]: cluster 2024-03-26T03:50:06.426414+0000 mon.a (mon.0) 1423 : cluster [DBG] osdmap e150: 8 total, 8 up, 8 in 2024-03-26T03:50:06.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:06 smithi175 bash[21527]: audit 2024-03-26T03:50:06.431645+0000 mon.a (mon.0) 1424 : audit [INF] from='client.? 172.21.15.175:0/692307030' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi175-64636-16"}]: dispatch 2024-03-26T03:50:06.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:06 smithi175 bash[29284]: audit 2024-03-26T03:50:05.515754+0000 mon.c (mon.2) 271 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:06.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:06 smithi175 bash[29284]: audit 2024-03-26T03:50:06.406946+0000 mon.a (mon.0) 1421 : audit [INF] from='client.? 172.21.15.175:0/3232347027' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi175-64406-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:06.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:06 smithi175 bash[29284]: audit 2024-03-26T03:50:06.407115+0000 mon.a (mon.0) 1422 : audit [INF] from='client.? 172.21.15.175:0/667812707' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi175-64398-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:06.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:06 smithi175 bash[29284]: cluster 2024-03-26T03:50:06.426414+0000 mon.a (mon.0) 1423 : cluster [DBG] osdmap e150: 8 total, 8 up, 8 in 2024-03-26T03:50:06.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:06 smithi175 bash[29284]: audit 2024-03-26T03:50:06.431645+0000 mon.a (mon.0) 1424 : audit [INF] from='client.? 172.21.15.175:0/692307030' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi175-64636-16"}]: dispatch 2024-03-26T03:50:07.428 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscP snapshots: Running main() from gmock_main.cc 2024-03-26T03:50:07.429 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: [==========] Running 11 tests from 2 test suites. 2024-03-26T03:50:07.429 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: [----------] Global test environment set-up. 2024-03-26T03:50:07.429 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots 2024-03-26T03:50:07.429 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapList 2024-03-26T03:50:07.429 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapList (4763 ms) 2024-03-26T03:50:07.429 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapRemove 2024-03-26T03:50:07.429 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapRemove (5028 ms) 2024-03-26T03:50:07.430 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: [ RUN ] NeoRadosSnapshots.Rollback 2024-03-26T03:50:07.430 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: [ OK ] NeoRadosSnapshots.Rollback (4010 ms) 2024-03-26T03:50:07.430 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapGetName 2024-03-26T03:50:07.430 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapGetName (5008 ms) 2024-03-26T03:50:07.430 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapCreateRemove 2024-03-26T03:50:07.430 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapCreateRemove (7003 ms) 2024-03-26T03:50:07.430 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots (25812 ms total) 2024-03-26T03:50:07.431 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: 2024-03-26T03:50:07.431 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps 2024-03-26T03:50:07.431 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Snap 2024-03-26T03:50:07.431 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Snap (4990 ms) 2024-03-26T03:50:07.431 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Rollback 2024-03-26T03:50:07.431 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Rollback (6007 ms) 2024-03-26T03:50:07.431 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.SnapOverlap 2024-03-26T03:50:07.431 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.SnapOverlap (7996 ms) 2024-03-26T03:50:07.431 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Bug11677 2024-03-26T03:50:07.431 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Bug11677 (5997 ms) 2024-03-26T03:50:07.431 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.OrderSnap 2024-03-26T03:50:07.431 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.OrderSnap (4009 ms) 2024-03-26T03:50:07.431 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.ReusePurgedSnap 2024-03-26T03:50:07.431 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: Deleting snap 3 in pool ReusePurgedSnapsmithi175-65381-11. 2024-03-26T03:50:07.431 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: Waiting for snaps to purge. 2024-03-26T03:50:07.432 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.ReusePurgedSnap (20004 ms) 2024-03-26T03:50:07.432 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps (49003 ms total) 2024-03-26T03:50:07.432 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: 2024-03-26T03:50:07.432 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: [----------] Global test environment tear-down 2024-03-26T03:50:07.432 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: [==========] 11 tests from 2 test suites ran. (74815 ms total) 2024-03-26T03:50:07.432 INFO:tasks.workunit.client.0.smithi175.stdout: snapshots: [ PASSED ] 11 tests. 2024-03-26T03:50:07.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:07 smithi190 bash[23392]: audit 2024-03-26T03:50:06.517082+0000 mon.c (mon.2) 272 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:07.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:07 smithi190 bash[23392]: audit 2024-03-26T03:50:06.589973+0000 mon.c (mon.2) 273 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:07.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:07 smithi190 bash[23392]: audit 2024-03-26T03:50:06.590638+0000 mon.c (mon.2) 274 : audit [INF] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pg_num","val":"11"}]: dispatch 2024-03-26T03:50:07.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:07 smithi190 bash[23392]: audit 2024-03-26T03:50:06.590954+0000 mon.a (mon.0) 1425 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pg_num","val":"11"}]: dispatch 2024-03-26T03:50:07.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:07 smithi190 bash[23392]: cluster 2024-03-26T03:50:06.608009+0000 mgr.y (mgr.24431) 242 : cluster [DBG] pgmap v222: 420 pgs: 64 unknown, 356 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-26T03:50:07.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:07 smithi190 bash[23392]: audit 2024-03-26T03:50:07.410381+0000 mon.a (mon.0) 1426 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64544-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi175-64544-24"}]': finished 2024-03-26T03:50:07.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:07 smithi190 bash[23392]: audit 2024-03-26T03:50:07.410800+0000 mon.a (mon.0) 1427 : audit [INF] from='client.? 172.21.15.175:0/692307030' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi175-64636-16"}]': finished 2024-03-26T03:50:07.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:07 smithi190 bash[23392]: audit 2024-03-26T03:50:07.410997+0000 mon.a (mon.0) 1428 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pg_num","val":"11"}]': finished 2024-03-26T03:50:07.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:07 smithi190 bash[23392]: cluster 2024-03-26T03:50:07.419511+0000 mon.a (mon.0) 1429 : cluster [DBG] osdmap e151: 8 total, 8 up, 8 in 2024-03-26T03:50:07.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:07 smithi190 bash[23392]: audit 2024-03-26T03:50:07.421365+0000 mon.c (mon.2) 275 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:07.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:07 smithi190 bash[23392]: audit 2024-03-26T03:50:07.421990+0000 mon.a (mon.0) 1430 : audit [INF] from='client.? 172.21.15.175:0/692307030' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi175-64636-16"}]: dispatch 2024-03-26T03:50:07.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:07 smithi190 bash[23392]: audit 2024-03-26T03:50:07.423241+0000 mon.c (mon.2) 276 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:07.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:07 smithi190 bash[23392]: audit 2024-03-26T03:50:07.424555+0000 mon.c (mon.2) 277 : audit [INF] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-26T03:50:07.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:07 smithi190 bash[23392]: audit 2024-03-26T03:50:07.425184+0000 mon.a (mon.0) 1431 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-26T03:50:07.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:07 smithi175 bash[21527]: audit 2024-03-26T03:50:06.517082+0000 mon.c (mon.2) 272 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:07.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:07 smithi175 bash[21527]: audit 2024-03-26T03:50:06.589973+0000 mon.c (mon.2) 273 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:07.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:07 smithi175 bash[21527]: audit 2024-03-26T03:50:06.590638+0000 mon.c (mon.2) 274 : audit [INF] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pg_num","val":"11"}]: dispatch 2024-03-26T03:50:07.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:07 smithi175 bash[21527]: audit 2024-03-26T03:50:06.590954+0000 mon.a (mon.0) 1425 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pg_num","val":"11"}]: dispatch 2024-03-26T03:50:07.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:07 smithi175 bash[21527]: cluster 2024-03-26T03:50:06.608009+0000 mgr.y (mgr.24431) 242 : cluster [DBG] pgmap v222: 420 pgs: 64 unknown, 356 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-26T03:50:07.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:07 smithi175 bash[21527]: audit 2024-03-26T03:50:07.410381+0000 mon.a (mon.0) 1426 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64544-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi175-64544-24"}]': finished 2024-03-26T03:50:07.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:07 smithi175 bash[21527]: audit 2024-03-26T03:50:07.410800+0000 mon.a (mon.0) 1427 : audit [INF] from='client.? 172.21.15.175:0/692307030' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi175-64636-16"}]': finished 2024-03-26T03:50:07.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:07 smithi175 bash[21527]: audit 2024-03-26T03:50:07.410997+0000 mon.a (mon.0) 1428 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pg_num","val":"11"}]': finished 2024-03-26T03:50:07.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:07 smithi175 bash[21527]: cluster 2024-03-26T03:50:07.419511+0000 mon.a (mon.0) 1429 : cluster [DBG] osdmap e151: 8 total, 8 up, 8 in 2024-03-26T03:50:07.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:07 smithi175 bash[21527]: audit 2024-03-26T03:50:07.421365+0000 mon.c (mon.2) 275 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:07.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:07 smithi175 bash[21527]: audit 2024-03-26T03:50:07.421990+0000 mon.a (mon.0) 1430 : audit [INF] from='client.? 172.21.15.175:0/692307030' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi175-64636-16"}]: dispatch 2024-03-26T03:50:07.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:07 smithi175 bash[21527]: audit 2024-03-26T03:50:07.423241+0000 mon.c (mon.2) 276 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:07.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:07 smithi175 bash[21527]: audit 2024-03-26T03:50:07.424555+0000 mon.c (mon.2) 277 : audit [INF] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-26T03:50:07.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:07 smithi175 bash[21527]: audit 2024-03-26T03:50:07.425184+0000 mon.a (mon.0) 1431 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-26T03:50:07.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:07 smithi175 bash[29284]: audit 2024-03-26T03:50:06.517082+0000 mon.c (mon.2) 272 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:07.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:07 smithi175 bash[29284]: audit 2024-03-26T03:50:06.589973+0000 mon.c (mon.2) 273 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:07.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:07 smithi175 bash[29284]: audit 2024-03-26T03:50:06.590638+0000 mon.c (mon.2) 274 : audit [INF] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pg_num","val":"11"}]: dispatch 2024-03-26T03:50:07.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:07 smithi175 bash[29284]: audit 2024-03-26T03:50:06.590954+0000 mon.a (mon.0) 1425 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pg_num","val":"11"}]: dispatch 2024-03-26T03:50:07.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:07 smithi175 bash[29284]: cluster 2024-03-26T03:50:06.608009+0000 mgr.y (mgr.24431) 242 : cluster [DBG] pgmap v222: 420 pgs: 64 unknown, 356 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-26T03:50:07.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:07 smithi175 bash[29284]: audit 2024-03-26T03:50:07.410381+0000 mon.a (mon.0) 1426 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64544-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi175-64544-24"}]': finished 2024-03-26T03:50:07.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:07 smithi175 bash[29284]: audit 2024-03-26T03:50:07.410800+0000 mon.a (mon.0) 1427 : audit [INF] from='client.? 172.21.15.175:0/692307030' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi175-64636-16"}]': finished 2024-03-26T03:50:07.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:07 smithi175 bash[29284]: audit 2024-03-26T03:50:07.410997+0000 mon.a (mon.0) 1428 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pg_num","val":"11"}]': finished 2024-03-26T03:50:07.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:07 smithi175 bash[29284]: cluster 2024-03-26T03:50:07.419511+0000 mon.a (mon.0) 1429 : cluster [DBG] osdmap e151: 8 total, 8 up, 8 in 2024-03-26T03:50:07.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:07 smithi175 bash[29284]: audit 2024-03-26T03:50:07.421365+0000 mon.c (mon.2) 275 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:07.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:07 smithi175 bash[29284]: audit 2024-03-26T03:50:07.421990+0000 mon.a (mon.0) 1430 : audit [INF] from='client.? 172.21.15.175:0/692307030' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi175-64636-16"}]: dispatch 2024-03-26T03:50:07.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:07 smithi175 bash[29284]: audit 2024-03-26T03:50:07.423241+0000 mon.c (mon.2) 276 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:07.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:07 smithi175 bash[29284]: audit 2024-03-26T03:50:07.424555+0000 mon.c (mon.2) 277 : audit [INF] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-26T03:50:07.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:07 smithi175 bash[29284]: audit 2024-03-26T03:50:07.425184+0000 mon.a (mon.0) 1431 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-26T03:50:09.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:50:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:50:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:50:09.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:09 smithi190 bash[23392]: audit 2024-03-26T03:50:08.416530+0000 mon.a (mon.0) 1432 : audit [INF] from='client.? 172.21.15.175:0/692307030' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi175-64636-16"}]': finished 2024-03-26T03:50:09.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:09 smithi190 bash[23392]: audit 2024-03-26T03:50:08.416713+0000 mon.a (mon.0) 1433 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pgp_num","val":"11"}]': finished 2024-03-26T03:50:09.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:09 smithi190 bash[23392]: cluster 2024-03-26T03:50:08.425882+0000 mon.a (mon.0) 1434 : cluster [DBG] osdmap e152: 8 total, 8 up, 8 in 2024-03-26T03:50:09.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:09 smithi190 bash[23392]: audit 2024-03-26T03:50:08.428816+0000 mon.c (mon.2) 278 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:09.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:09 smithi190 bash[23392]: audit 2024-03-26T03:50:08.433775+0000 mon.a (mon.0) 1435 : audit [INF] from='client.? 172.21.15.175:0/1934680397' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi175-64398-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:09.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:09 smithi190 bash[23392]: audit 2024-03-26T03:50:08.437012+0000 mon.c (mon.2) 279 : audit [INF] from='client.? 172.21.15.175:0/489945439' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64544-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:09.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:09 smithi190 bash[23392]: audit 2024-03-26T03:50:08.437662+0000 mon.c (mon.2) 280 : audit [INF] from='client.? 172.21.15.175:0/3335881417' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi175-64406-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:09.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:09 smithi190 bash[23392]: audit 2024-03-26T03:50:08.438008+0000 mon.a (mon.0) 1436 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64544-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:09.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:09 smithi190 bash[23392]: audit 2024-03-26T03:50:08.441706+0000 mon.a (mon.0) 1437 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi175-64406-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:09.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:09 smithi190 bash[23392]: audit 2024-03-26T03:50:08.471642+0000 mon.a (mon.0) 1438 : audit [INF] from='client.? 172.21.15.175:0/4044186109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21"}]: dispatch 2024-03-26T03:50:09.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:09 smithi190 bash[23392]: audit 2024-03-26T03:50:08.473048+0000 mon.a (mon.0) 1439 : audit [INF] from='client.? 172.21.15.175:0/4044186109' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21"}]: dispatch 2024-03-26T03:50:09.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:09 smithi190 bash[23392]: audit 2024-03-26T03:50:08.473616+0000 mon.a (mon.0) 1440 : audit [INF] from='client.? 172.21.15.175:0/4044186109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:09.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:09 smithi175 bash[21527]: audit 2024-03-26T03:50:08.416530+0000 mon.a (mon.0) 1432 : audit [INF] from='client.? 172.21.15.175:0/692307030' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi175-64636-16"}]': finished 2024-03-26T03:50:09.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:09 smithi175 bash[21527]: audit 2024-03-26T03:50:08.416713+0000 mon.a (mon.0) 1433 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pgp_num","val":"11"}]': finished 2024-03-26T03:50:09.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:09 smithi175 bash[21527]: cluster 2024-03-26T03:50:08.425882+0000 mon.a (mon.0) 1434 : cluster [DBG] osdmap e152: 8 total, 8 up, 8 in 2024-03-26T03:50:09.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:09 smithi175 bash[21527]: audit 2024-03-26T03:50:08.428816+0000 mon.c (mon.2) 278 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:09.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:09 smithi175 bash[21527]: audit 2024-03-26T03:50:08.433775+0000 mon.a (mon.0) 1435 : audit [INF] from='client.? 172.21.15.175:0/1934680397' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi175-64398-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:09.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:09 smithi175 bash[21527]: audit 2024-03-26T03:50:08.437012+0000 mon.c (mon.2) 279 : audit [INF] from='client.? 172.21.15.175:0/489945439' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64544-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:09.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:09 smithi175 bash[21527]: audit 2024-03-26T03:50:08.437662+0000 mon.c (mon.2) 280 : audit [INF] from='client.? 172.21.15.175:0/3335881417' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi175-64406-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:09.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:09 smithi175 bash[21527]: audit 2024-03-26T03:50:08.438008+0000 mon.a (mon.0) 1436 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64544-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:09.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:09 smithi175 bash[21527]: audit 2024-03-26T03:50:08.441706+0000 mon.a (mon.0) 1437 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi175-64406-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:09.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:09 smithi175 bash[21527]: audit 2024-03-26T03:50:08.471642+0000 mon.a (mon.0) 1438 : audit [INF] from='client.? 172.21.15.175:0/4044186109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21"}]: dispatch 2024-03-26T03:50:09.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:09 smithi175 bash[21527]: audit 2024-03-26T03:50:08.473048+0000 mon.a (mon.0) 1439 : audit [INF] from='client.? 172.21.15.175:0/4044186109' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21"}]: dispatch 2024-03-26T03:50:09.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:09 smithi175 bash[21527]: audit 2024-03-26T03:50:08.473616+0000 mon.a (mon.0) 1440 : audit [INF] from='client.? 172.21.15.175:0/4044186109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:09.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:09 smithi175 bash[29284]: audit 2024-03-26T03:50:08.416530+0000 mon.a (mon.0) 1432 : audit [INF] from='client.? 172.21.15.175:0/692307030' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi175-64636-16"}]': finished 2024-03-26T03:50:09.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:09 smithi175 bash[29284]: audit 2024-03-26T03:50:08.416713+0000 mon.a (mon.0) 1433 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi175-64438-1","var":"pgp_num","val":"11"}]': finished 2024-03-26T03:50:09.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:09 smithi175 bash[29284]: cluster 2024-03-26T03:50:08.425882+0000 mon.a (mon.0) 1434 : cluster [DBG] osdmap e152: 8 total, 8 up, 8 in 2024-03-26T03:50:09.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:09 smithi175 bash[29284]: audit 2024-03-26T03:50:08.428816+0000 mon.c (mon.2) 278 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:09.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:09 smithi175 bash[29284]: audit 2024-03-26T03:50:08.433775+0000 mon.a (mon.0) 1435 : audit [INF] from='client.? 172.21.15.175:0/1934680397' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi175-64398-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:09.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:09 smithi175 bash[29284]: audit 2024-03-26T03:50:08.437012+0000 mon.c (mon.2) 279 : audit [INF] from='client.? 172.21.15.175:0/489945439' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64544-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:09.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:09 smithi175 bash[29284]: audit 2024-03-26T03:50:08.437662+0000 mon.c (mon.2) 280 : audit [INF] from='client.? 172.21.15.175:0/3335881417' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi175-64406-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:09.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:09 smithi175 bash[29284]: audit 2024-03-26T03:50:08.438008+0000 mon.a (mon.0) 1436 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64544-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:09.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:09 smithi175 bash[29284]: audit 2024-03-26T03:50:08.441706+0000 mon.a (mon.0) 1437 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi175-64406-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:09.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:09 smithi175 bash[29284]: audit 2024-03-26T03:50:08.471642+0000 mon.a (mon.0) 1438 : audit [INF] from='client.? 172.21.15.175:0/4044186109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21"}]: dispatch 2024-03-26T03:50:09.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:09 smithi175 bash[29284]: audit 2024-03-26T03:50:08.473048+0000 mon.a (mon.0) 1439 : audit [INF] from='client.? 172.21.15.175:0/4044186109' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21"}]: dispatch 2024-03-26T03:50:09.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:09 smithi175 bash[29284]: audit 2024-03-26T03:50:08.473616+0000 mon.a (mon.0) 1440 : audit [INF] from='client.? 172.21.15.175:0/4044186109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:10.240 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:50:09 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:50:10.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:10 smithi190 bash[23392]: cluster 2024-03-26T03:50:08.609009+0000 mgr.y (mgr.24431) 243 : cluster [DBG] pgmap v225: 428 pgs: 104 unknown, 324 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:50:10.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:10 smithi190 bash[23392]: cluster 2024-03-26T03:50:09.416554+0000 mon.a (mon.0) 1441 : cluster [WRN] Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:50:10.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:10 smithi190 bash[23392]: audit 2024-03-26T03:50:09.427221+0000 mon.a (mon.0) 1442 : audit [INF] from='client.? 172.21.15.175:0/1934680397' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi175-64398-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:10.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:10 smithi190 bash[23392]: audit 2024-03-26T03:50:09.427381+0000 mon.a (mon.0) 1443 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64544-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:10.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:10 smithi190 bash[23392]: audit 2024-03-26T03:50:09.427800+0000 mon.a (mon.0) 1444 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi175-64406-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:10.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:10 smithi190 bash[23392]: audit 2024-03-26T03:50:09.427975+0000 mon.a (mon.0) 1445 : audit [INF] from='client.? 172.21.15.175:0/4044186109' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:10.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:10 smithi190 bash[23392]: cluster 2024-03-26T03:50:09.436677+0000 mon.a (mon.0) 1446 : cluster [DBG] osdmap e153: 8 total, 8 up, 8 in 2024-03-26T03:50:10.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:10 smithi190 bash[23392]: audit 2024-03-26T03:50:09.440432+0000 mon.c (mon.2) 281 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:10.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:10 smithi190 bash[23392]: audit 2024-03-26T03:50:09.443559+0000 mon.a (mon.0) 1447 : audit [INF] from='client.? 172.21.15.175:0/4044186109' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21"}]: dispatch 2024-03-26T03:50:10.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:10 smithi190 bash[23392]: audit 2024-03-26T03:50:09.445602+0000 mon.c (mon.2) 282 : audit [INF] from='client.? 172.21.15.175:0/489945439' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64544-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:10.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:10 smithi190 bash[23392]: audit 2024-03-26T03:50:09.449612+0000 mon.a (mon.0) 1448 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64544-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:10.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:10 smithi175 bash[21527]: cluster 2024-03-26T03:50:08.609009+0000 mgr.y (mgr.24431) 243 : cluster [DBG] pgmap v225: 428 pgs: 104 unknown, 324 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:50:10.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:10 smithi175 bash[21527]: cluster 2024-03-26T03:50:09.416554+0000 mon.a (mon.0) 1441 : cluster [WRN] Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:50:10.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:10 smithi175 bash[21527]: audit 2024-03-26T03:50:09.427221+0000 mon.a (mon.0) 1442 : audit [INF] from='client.? 172.21.15.175:0/1934680397' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi175-64398-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:10.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:10 smithi175 bash[21527]: audit 2024-03-26T03:50:09.427381+0000 mon.a (mon.0) 1443 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64544-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:10.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:10 smithi175 bash[21527]: audit 2024-03-26T03:50:09.427800+0000 mon.a (mon.0) 1444 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi175-64406-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:10.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:10 smithi175 bash[21527]: audit 2024-03-26T03:50:09.427975+0000 mon.a (mon.0) 1445 : audit [INF] from='client.? 172.21.15.175:0/4044186109' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:10.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:10 smithi175 bash[21527]: cluster 2024-03-26T03:50:09.436677+0000 mon.a (mon.0) 1446 : cluster [DBG] osdmap e153: 8 total, 8 up, 8 in 2024-03-26T03:50:10.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:10 smithi175 bash[21527]: audit 2024-03-26T03:50:09.440432+0000 mon.c (mon.2) 281 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:10.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:10 smithi175 bash[21527]: audit 2024-03-26T03:50:09.443559+0000 mon.a (mon.0) 1447 : audit [INF] from='client.? 172.21.15.175:0/4044186109' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21"}]: dispatch 2024-03-26T03:50:10.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:10 smithi175 bash[21527]: audit 2024-03-26T03:50:09.445602+0000 mon.c (mon.2) 282 : audit [INF] from='client.? 172.21.15.175:0/489945439' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64544-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:10.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:10 smithi175 bash[21527]: audit 2024-03-26T03:50:09.449612+0000 mon.a (mon.0) 1448 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64544-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:10.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:10 smithi175 bash[29284]: cluster 2024-03-26T03:50:08.609009+0000 mgr.y (mgr.24431) 243 : cluster [DBG] pgmap v225: 428 pgs: 104 unknown, 324 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:50:10.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:10 smithi175 bash[29284]: cluster 2024-03-26T03:50:09.416554+0000 mon.a (mon.0) 1441 : cluster [WRN] Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:50:10.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:10 smithi175 bash[29284]: audit 2024-03-26T03:50:09.427221+0000 mon.a (mon.0) 1442 : audit [INF] from='client.? 172.21.15.175:0/1934680397' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi175-64398-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:10.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:10 smithi175 bash[29284]: audit 2024-03-26T03:50:09.427381+0000 mon.a (mon.0) 1443 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64544-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:10.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:10 smithi175 bash[29284]: audit 2024-03-26T03:50:09.427800+0000 mon.a (mon.0) 1444 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi175-64406-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:10.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:10 smithi175 bash[29284]: audit 2024-03-26T03:50:09.427975+0000 mon.a (mon.0) 1445 : audit [INF] from='client.? 172.21.15.175:0/4044186109' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:10.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:10 smithi175 bash[29284]: cluster 2024-03-26T03:50:09.436677+0000 mon.a (mon.0) 1446 : cluster [DBG] osdmap e153: 8 total, 8 up, 8 in 2024-03-26T03:50:10.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:10 smithi175 bash[29284]: audit 2024-03-26T03:50:09.440432+0000 mon.c (mon.2) 281 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:10.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:10 smithi175 bash[29284]: audit 2024-03-26T03:50:09.443559+0000 mon.a (mon.0) 1447 : audit [INF] from='client.? 172.21.15.175:0/4044186109' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21"}]: dispatch 2024-03-26T03:50:10.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:10 smithi175 bash[29284]: audit 2024-03-26T03:50:09.445602+0000 mon.c (mon.2) 282 : audit [INF] from='client.? 172.21.15.175:0/489945439' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64544-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:10.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:10 smithi175 bash[29284]: audit 2024-03-26T03:50:09.449612+0000 mon.a (mon.0) 1448 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64544-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:11.900 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[21527]: audit 2024-03-26T03:50:09.761955+0000 mgr.y (mgr.24431) 244 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:50:11.900 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[21527]: audit 2024-03-26T03:50:10.431285+0000 mon.a (mon.0) 1449 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64544-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:11.900 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[21527]: audit 2024-03-26T03:50:10.442625+0000 mon.c (mon.2) 283 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:11.900 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[21527]: cluster 2024-03-26T03:50:10.453484+0000 mon.a (mon.0) 1450 : cluster [DBG] osdmap e154: 8 total, 8 up, 8 in 2024-03-26T03:50:11.900 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[21527]: audit 2024-03-26T03:50:10.499497+0000 mon.c (mon.2) 284 : audit [INF] from='client.? 172.21.15.175:0/2412645833' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi175-64398-27"}]: dispatch 2024-03-26T03:50:11.900 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[21527]: audit 2024-03-26T03:50:10.502710+0000 mon.a (mon.0) 1451 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi175-64398-27"}]: dispatch 2024-03-26T03:50:11.900 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[21527]: audit 2024-03-26T03:50:10.503451+0000 mon.c (mon.2) 285 : audit [INF] from='client.? 172.21.15.175:0/2412645833' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi175-64398-27"}]: dispatch 2024-03-26T03:50:11.900 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[21527]: audit 2024-03-26T03:50:10.503964+0000 mon.a (mon.0) 1452 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi175-64398-27"}]: dispatch 2024-03-26T03:50:11.900 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[21527]: audit 2024-03-26T03:50:10.504467+0000 mon.c (mon.2) 286 : audit [INF] from='client.? 172.21.15.175:0/2412645833' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi175-64398-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:11.900 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[21527]: audit 2024-03-26T03:50:10.504816+0000 mon.a (mon.0) 1453 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi175-64398-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:11.901 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[21527]: audit 2024-03-26T03:50:10.610283+0000 mon.c (mon.2) 287 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi175-64438-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-03-26T03:50:11.901 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[21527]: audit 2024-03-26T03:50:10.611693+0000 mon.a (mon.0) 1454 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi175-64438-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-03-26T03:50:11.901 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[21527]: audit 2024-03-26T03:50:11.435841+0000 mon.a (mon.0) 1455 : audit [INF] from='client.? 172.21.15.175:0/4044186109' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21"}]': finished 2024-03-26T03:50:11.901 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[21527]: audit 2024-03-26T03:50:11.436076+0000 mon.a (mon.0) 1456 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi175-64398-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:11.901 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[21527]: audit 2024-03-26T03:50:11.436274+0000 mon.a (mon.0) 1457 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi175-64438-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-03-26T03:50:11.901 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[21527]: cluster 2024-03-26T03:50:11.448417+0000 mon.a (mon.0) 1458 : cluster [DBG] osdmap e155: 8 total, 8 up, 8 in 2024-03-26T03:50:11.901 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[21527]: audit 2024-03-26T03:50:11.455818+0000 mon.a (mon.0) 1459 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi175-64406-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:11.901 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[29284]: audit 2024-03-26T03:50:09.761955+0000 mgr.y (mgr.24431) 244 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:50:11.901 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[29284]: audit 2024-03-26T03:50:10.431285+0000 mon.a (mon.0) 1449 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64544-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:11.901 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[29284]: audit 2024-03-26T03:50:10.442625+0000 mon.c (mon.2) 283 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:11.901 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[29284]: cluster 2024-03-26T03:50:10.453484+0000 mon.a (mon.0) 1450 : cluster [DBG] osdmap e154: 8 total, 8 up, 8 in 2024-03-26T03:50:11.902 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[29284]: audit 2024-03-26T03:50:10.499497+0000 mon.c (mon.2) 284 : audit [INF] from='client.? 172.21.15.175:0/2412645833' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi175-64398-27"}]: dispatch 2024-03-26T03:50:11.902 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[29284]: audit 2024-03-26T03:50:10.502710+0000 mon.a (mon.0) 1451 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi175-64398-27"}]: dispatch 2024-03-26T03:50:11.902 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[29284]: audit 2024-03-26T03:50:10.503451+0000 mon.c (mon.2) 285 : audit [INF] from='client.? 172.21.15.175:0/2412645833' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi175-64398-27"}]: dispatch 2024-03-26T03:50:11.902 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[29284]: audit 2024-03-26T03:50:10.503964+0000 mon.a (mon.0) 1452 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi175-64398-27"}]: dispatch 2024-03-26T03:50:11.902 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[29284]: audit 2024-03-26T03:50:10.504467+0000 mon.c (mon.2) 286 : audit [INF] from='client.? 172.21.15.175:0/2412645833' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi175-64398-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:11.902 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[29284]: audit 2024-03-26T03:50:10.504816+0000 mon.a (mon.0) 1453 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi175-64398-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:11.902 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[29284]: audit 2024-03-26T03:50:10.610283+0000 mon.c (mon.2) 287 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi175-64438-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-03-26T03:50:11.902 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[29284]: audit 2024-03-26T03:50:10.611693+0000 mon.a (mon.0) 1454 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi175-64438-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-03-26T03:50:11.902 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[29284]: audit 2024-03-26T03:50:11.435841+0000 mon.a (mon.0) 1455 : audit [INF] from='client.? 172.21.15.175:0/4044186109' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21"}]': finished 2024-03-26T03:50:11.902 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[29284]: audit 2024-03-26T03:50:11.436076+0000 mon.a (mon.0) 1456 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi175-64398-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:11.902 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[29284]: audit 2024-03-26T03:50:11.436274+0000 mon.a (mon.0) 1457 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi175-64438-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-03-26T03:50:11.902 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[29284]: cluster 2024-03-26T03:50:11.448417+0000 mon.a (mon.0) 1458 : cluster [DBG] osdmap e155: 8 total, 8 up, 8 in 2024-03-26T03:50:11.902 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:11 smithi175 bash[29284]: audit 2024-03-26T03:50:11.455818+0000 mon.a (mon.0) 1459 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi175-64406-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:11.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:11 smithi190 bash[23392]: audit 2024-03-26T03:50:09.761955+0000 mgr.y (mgr.24431) 244 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:50:11.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:11 smithi190 bash[23392]: audit 2024-03-26T03:50:10.431285+0000 mon.a (mon.0) 1449 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64544-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:11.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:11 smithi190 bash[23392]: audit 2024-03-26T03:50:10.442625+0000 mon.c (mon.2) 283 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:11.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:11 smithi190 bash[23392]: cluster 2024-03-26T03:50:10.453484+0000 mon.a (mon.0) 1450 : cluster [DBG] osdmap e154: 8 total, 8 up, 8 in 2024-03-26T03:50:11.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:11 smithi190 bash[23392]: audit 2024-03-26T03:50:10.499497+0000 mon.c (mon.2) 284 : audit [INF] from='client.? 172.21.15.175:0/2412645833' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi175-64398-27"}]: dispatch 2024-03-26T03:50:11.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:11 smithi190 bash[23392]: audit 2024-03-26T03:50:10.502710+0000 mon.a (mon.0) 1451 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi175-64398-27"}]: dispatch 2024-03-26T03:50:11.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:11 smithi190 bash[23392]: audit 2024-03-26T03:50:10.503451+0000 mon.c (mon.2) 285 : audit [INF] from='client.? 172.21.15.175:0/2412645833' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi175-64398-27"}]: dispatch 2024-03-26T03:50:11.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:11 smithi190 bash[23392]: audit 2024-03-26T03:50:10.503964+0000 mon.a (mon.0) 1452 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi175-64398-27"}]: dispatch 2024-03-26T03:50:11.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:11 smithi190 bash[23392]: audit 2024-03-26T03:50:10.504467+0000 mon.c (mon.2) 286 : audit [INF] from='client.? 172.21.15.175:0/2412645833' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi175-64398-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:11.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:11 smithi190 bash[23392]: audit 2024-03-26T03:50:10.504816+0000 mon.a (mon.0) 1453 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi175-64398-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:11.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:11 smithi190 bash[23392]: audit 2024-03-26T03:50:10.610283+0000 mon.c (mon.2) 287 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi175-64438-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-03-26T03:50:11.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:11 smithi190 bash[23392]: audit 2024-03-26T03:50:10.611693+0000 mon.a (mon.0) 1454 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi175-64438-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-03-26T03:50:11.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:11 smithi190 bash[23392]: audit 2024-03-26T03:50:11.435841+0000 mon.a (mon.0) 1455 : audit [INF] from='client.? 172.21.15.175:0/4044186109' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21"}]': finished 2024-03-26T03:50:11.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:11 smithi190 bash[23392]: audit 2024-03-26T03:50:11.436076+0000 mon.a (mon.0) 1456 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi175-64398-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:11.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:11 smithi190 bash[23392]: audit 2024-03-26T03:50:11.436274+0000 mon.a (mon.0) 1457 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi175-64438-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-03-26T03:50:11.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:11 smithi190 bash[23392]: cluster 2024-03-26T03:50:11.448417+0000 mon.a (mon.0) 1458 : cluster [DBG] osdmap e155: 8 total, 8 up, 8 in 2024-03-26T03:50:11.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:11 smithi190 bash[23392]: audit 2024-03-26T03:50:11.455818+0000 mon.a (mon.0) 1459 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi175-64406-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:12.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:12 smithi175 bash[21527]: cluster 2024-03-26T03:50:10.609906+0000 mgr.y (mgr.24431) 245 : cluster [DBG] pgmap v228: 364 pgs: 364 active+clean; 592 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.7 KiB/s wr, 8 op/s; 31 B/s, 0 objects/s recovering 2024-03-26T03:50:12.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:12 smithi175 bash[21527]: audit 2024-03-26T03:50:11.451883+0000 mon.c (mon.2) 288 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:12.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:12 smithi175 bash[21527]: audit 2024-03-26T03:50:11.454029+0000 mon.c (mon.2) 289 : audit [INF] from='client.? 172.21.15.175:0/677313530' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi175-64406-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:12.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:12 smithi175 bash[21527]: audit 2024-03-26T03:50:11.462076+0000 mon.c (mon.2) 290 : audit [INF] from='client.? 172.21.15.175:0/2412645833' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi175-64398-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi175-64398-27"}]: dispatch 2024-03-26T03:50:12.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:12 smithi175 bash[21527]: audit 2024-03-26T03:50:11.487507+0000 mon.a (mon.0) 1460 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi175-64398-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi175-64398-27"}]: dispatch 2024-03-26T03:50:12.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:12 smithi175 bash[21527]: audit 2024-03-26T03:50:12.440005+0000 mon.a (mon.0) 1461 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi175-64406-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:12.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:12 smithi175 bash[21527]: audit 2024-03-26T03:50:12.457206+0000 mon.c (mon.2) 291 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:12.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:12 smithi175 bash[21527]: cluster 2024-03-26T03:50:12.461815+0000 mon.a (mon.0) 1462 : cluster [DBG] osdmap e156: 8 total, 8 up, 8 in 2024-03-26T03:50:12.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:12 smithi175 bash[21527]: audit 2024-03-26T03:50:12.468390+0000 mon.c (mon.2) 292 : audit [INF] from='client.? 172.21.15.175:0/489945439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64544-24"}]: dispatch 2024-03-26T03:50:12.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:12 smithi175 bash[21527]: audit 2024-03-26T03:50:12.470065+0000 mon.a (mon.0) 1463 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64544-24"}]: dispatch 2024-03-26T03:50:12.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:12 smithi175 bash[29284]: cluster 2024-03-26T03:50:10.609906+0000 mgr.y (mgr.24431) 245 : cluster [DBG] pgmap v228: 364 pgs: 364 active+clean; 592 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.7 KiB/s wr, 8 op/s; 31 B/s, 0 objects/s recovering 2024-03-26T03:50:12.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:12 smithi175 bash[29284]: audit 2024-03-26T03:50:11.451883+0000 mon.c (mon.2) 288 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:12.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:12 smithi175 bash[29284]: audit 2024-03-26T03:50:11.454029+0000 mon.c (mon.2) 289 : audit [INF] from='client.? 172.21.15.175:0/677313530' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi175-64406-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:12.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:12 smithi175 bash[29284]: audit 2024-03-26T03:50:11.462076+0000 mon.c (mon.2) 290 : audit [INF] from='client.? 172.21.15.175:0/2412645833' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi175-64398-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi175-64398-27"}]: dispatch 2024-03-26T03:50:12.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:12 smithi175 bash[29284]: audit 2024-03-26T03:50:11.487507+0000 mon.a (mon.0) 1460 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi175-64398-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi175-64398-27"}]: dispatch 2024-03-26T03:50:12.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:12 smithi175 bash[29284]: audit 2024-03-26T03:50:12.440005+0000 mon.a (mon.0) 1461 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi175-64406-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:12.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:12 smithi175 bash[29284]: audit 2024-03-26T03:50:12.457206+0000 mon.c (mon.2) 291 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:12.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:12 smithi175 bash[29284]: cluster 2024-03-26T03:50:12.461815+0000 mon.a (mon.0) 1462 : cluster [DBG] osdmap e156: 8 total, 8 up, 8 in 2024-03-26T03:50:12.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:12 smithi175 bash[29284]: audit 2024-03-26T03:50:12.468390+0000 mon.c (mon.2) 292 : audit [INF] from='client.? 172.21.15.175:0/489945439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64544-24"}]: dispatch 2024-03-26T03:50:12.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:12 smithi175 bash[29284]: audit 2024-03-26T03:50:12.470065+0000 mon.a (mon.0) 1463 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64544-24"}]: dispatch 2024-03-26T03:50:12.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:12 smithi190 bash[23392]: cluster 2024-03-26T03:50:10.609906+0000 mgr.y (mgr.24431) 245 : cluster [DBG] pgmap v228: 364 pgs: 364 active+clean; 592 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.7 KiB/s wr, 8 op/s; 31 B/s, 0 objects/s recovering 2024-03-26T03:50:12.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:12 smithi190 bash[23392]: audit 2024-03-26T03:50:11.451883+0000 mon.c (mon.2) 288 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:12.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:12 smithi190 bash[23392]: audit 2024-03-26T03:50:11.454029+0000 mon.c (mon.2) 289 : audit [INF] from='client.? 172.21.15.175:0/677313530' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi175-64406-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:12.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:12 smithi190 bash[23392]: audit 2024-03-26T03:50:11.462076+0000 mon.c (mon.2) 290 : audit [INF] from='client.? 172.21.15.175:0/2412645833' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi175-64398-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi175-64398-27"}]: dispatch 2024-03-26T03:50:12.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:12 smithi190 bash[23392]: audit 2024-03-26T03:50:11.487507+0000 mon.a (mon.0) 1460 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi175-64398-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi175-64398-27"}]: dispatch 2024-03-26T03:50:12.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:12 smithi190 bash[23392]: audit 2024-03-26T03:50:12.440005+0000 mon.a (mon.0) 1461 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi175-64406-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:12.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:12 smithi190 bash[23392]: audit 2024-03-26T03:50:12.457206+0000 mon.c (mon.2) 291 : audit [DBG] from='client.? 172.21.15.175:0/3945897686' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-26T03:50:12.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:12 smithi190 bash[23392]: cluster 2024-03-26T03:50:12.461815+0000 mon.a (mon.0) 1462 : cluster [DBG] osdmap e156: 8 total, 8 up, 8 in 2024-03-26T03:50:12.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:12 smithi190 bash[23392]: audit 2024-03-26T03:50:12.468390+0000 mon.c (mon.2) 292 : audit [INF] from='client.? 172.21.15.175:0/489945439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64544-24"}]: dispatch 2024-03-26T03:50:12.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:12 smithi190 bash[23392]: audit 2024-03-26T03:50:12.470065+0000 mon.a (mon.0) 1463 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64544-24"}]: dispatch 2024-03-26T03:50:14.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:14 smithi190 bash[23392]: cluster 2024-03-26T03:50:12.610979+0000 mgr.y (mgr.24431) 246 : cluster [DBG] pgmap v231: 364 pgs: 40 unknown, 324 active+clean; 592 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-03-26T03:50:14.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:14 smithi190 bash[23392]: audit 2024-03-26T03:50:13.443472+0000 mon.a (mon.0) 1464 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi175-64398-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi175-64398-27"}]': finished 2024-03-26T03:50:14.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:14 smithi190 bash[23392]: audit 2024-03-26T03:50:13.443693+0000 mon.a (mon.0) 1465 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64544-24"}]': finished 2024-03-26T03:50:14.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:14 smithi190 bash[23392]: cluster 2024-03-26T03:50:13.455282+0000 mon.a (mon.0) 1466 : cluster [DBG] osdmap e157: 8 total, 8 up, 8 in 2024-03-26T03:50:14.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:14 smithi190 bash[23392]: audit 2024-03-26T03:50:13.457272+0000 mon.c (mon.2) 293 : audit [INF] from='client.? 172.21.15.175:0/489945439' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64544-24"}]: dispatch 2024-03-26T03:50:14.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:14 smithi190 bash[23392]: audit 2024-03-26T03:50:13.458694+0000 mon.a (mon.0) 1467 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64544-24"}]: dispatch 2024-03-26T03:50:14.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:14 smithi175 bash[21527]: cluster 2024-03-26T03:50:12.610979+0000 mgr.y (mgr.24431) 246 : cluster [DBG] pgmap v231: 364 pgs: 40 unknown, 324 active+clean; 592 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-03-26T03:50:14.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:14 smithi175 bash[21527]: audit 2024-03-26T03:50:13.443472+0000 mon.a (mon.0) 1464 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi175-64398-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi175-64398-27"}]': finished 2024-03-26T03:50:14.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:14 smithi175 bash[21527]: audit 2024-03-26T03:50:13.443693+0000 mon.a (mon.0) 1465 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64544-24"}]': finished 2024-03-26T03:50:14.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:14 smithi175 bash[21527]: cluster 2024-03-26T03:50:13.455282+0000 mon.a (mon.0) 1466 : cluster [DBG] osdmap e157: 8 total, 8 up, 8 in 2024-03-26T03:50:14.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:14 smithi175 bash[21527]: audit 2024-03-26T03:50:13.457272+0000 mon.c (mon.2) 293 : audit [INF] from='client.? 172.21.15.175:0/489945439' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64544-24"}]: dispatch 2024-03-26T03:50:14.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:14 smithi175 bash[21527]: audit 2024-03-26T03:50:13.458694+0000 mon.a (mon.0) 1467 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64544-24"}]: dispatch 2024-03-26T03:50:14.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:14 smithi175 bash[29284]: cluster 2024-03-26T03:50:12.610979+0000 mgr.y (mgr.24431) 246 : cluster [DBG] pgmap v231: 364 pgs: 40 unknown, 324 active+clean; 592 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-03-26T03:50:14.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:14 smithi175 bash[29284]: audit 2024-03-26T03:50:13.443472+0000 mon.a (mon.0) 1464 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi175-64398-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi175-64398-27"}]': finished 2024-03-26T03:50:14.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:14 smithi175 bash[29284]: audit 2024-03-26T03:50:13.443693+0000 mon.a (mon.0) 1465 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64544-24"}]': finished 2024-03-26T03:50:14.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:14 smithi175 bash[29284]: cluster 2024-03-26T03:50:13.455282+0000 mon.a (mon.0) 1466 : cluster [DBG] osdmap e157: 8 total, 8 up, 8 in 2024-03-26T03:50:14.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:14 smithi175 bash[29284]: audit 2024-03-26T03:50:13.457272+0000 mon.c (mon.2) 293 : audit [INF] from='client.? 172.21.15.175:0/489945439' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64544-24"}]: dispatch 2024-03-26T03:50:14.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:14 smithi175 bash[29284]: audit 2024-03-26T03:50:13.458694+0000 mon.a (mon.0) 1467 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64544-24"}]: dispatch 2024-03-26T03:50:15.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:15 smithi190 bash[23392]: audit 2024-03-26T03:50:14.448982+0000 mon.a (mon.0) 1468 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64544-24"}]': finished 2024-03-26T03:50:15.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:15 smithi190 bash[23392]: cluster 2024-03-26T03:50:14.460829+0000 mon.a (mon.0) 1469 : cluster [DBG] osdmap e158: 8 total, 8 up, 8 in 2024-03-26T03:50:15.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:15 smithi190 bash[23392]: audit 2024-03-26T03:50:14.474715+0000 mon.c (mon.2) 294 : audit [INF] from='client.? 172.21.15.175:0/2117170821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi175-64406-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:15.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:15 smithi190 bash[23392]: audit 2024-03-26T03:50:14.501348+0000 mon.a (mon.0) 1470 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi175-64406-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:15.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:15 smithi190 bash[23392]: audit 2024-03-26T03:50:15.455093+0000 mon.a (mon.0) 1471 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi175-64406-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:15.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:15 smithi190 bash[23392]: cluster 2024-03-26T03:50:15.469950+0000 mon.a (mon.0) 1472 : cluster [DBG] osdmap e159: 8 total, 8 up, 8 in 2024-03-26T03:50:15.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:15 smithi190 bash[23392]: audit 2024-03-26T03:50:15.477591+0000 mon.c (mon.2) 295 : audit [INF] from='client.? 172.21.15.175:0/2412645833' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi175-64398-27"}]: dispatch 2024-03-26T03:50:15.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:15 smithi175 bash[21527]: audit 2024-03-26T03:50:14.448982+0000 mon.a (mon.0) 1468 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64544-24"}]': finished 2024-03-26T03:50:15.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:15 smithi175 bash[21527]: cluster 2024-03-26T03:50:14.460829+0000 mon.a (mon.0) 1469 : cluster [DBG] osdmap e158: 8 total, 8 up, 8 in 2024-03-26T03:50:15.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:15 smithi175 bash[21527]: audit 2024-03-26T03:50:14.474715+0000 mon.c (mon.2) 294 : audit [INF] from='client.? 172.21.15.175:0/2117170821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi175-64406-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:15.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:15 smithi175 bash[21527]: audit 2024-03-26T03:50:14.501348+0000 mon.a (mon.0) 1470 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi175-64406-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:15.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:15 smithi175 bash[21527]: audit 2024-03-26T03:50:15.455093+0000 mon.a (mon.0) 1471 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi175-64406-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:15.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:15 smithi175 bash[21527]: cluster 2024-03-26T03:50:15.469950+0000 mon.a (mon.0) 1472 : cluster [DBG] osdmap e159: 8 total, 8 up, 8 in 2024-03-26T03:50:15.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:15 smithi175 bash[21527]: audit 2024-03-26T03:50:15.477591+0000 mon.c (mon.2) 295 : audit [INF] from='client.? 172.21.15.175:0/2412645833' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi175-64398-27"}]: dispatch 2024-03-26T03:50:15.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:15 smithi175 bash[29284]: audit 2024-03-26T03:50:14.448982+0000 mon.a (mon.0) 1468 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64544-24"}]': finished 2024-03-26T03:50:15.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:15 smithi175 bash[29284]: cluster 2024-03-26T03:50:14.460829+0000 mon.a (mon.0) 1469 : cluster [DBG] osdmap e158: 8 total, 8 up, 8 in 2024-03-26T03:50:15.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:15 smithi175 bash[29284]: audit 2024-03-26T03:50:14.474715+0000 mon.c (mon.2) 294 : audit [INF] from='client.? 172.21.15.175:0/2117170821' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi175-64406-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:15.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:15 smithi175 bash[29284]: audit 2024-03-26T03:50:14.501348+0000 mon.a (mon.0) 1470 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi175-64406-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:15.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:15 smithi175 bash[29284]: audit 2024-03-26T03:50:15.455093+0000 mon.a (mon.0) 1471 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi175-64406-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:15.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:15 smithi175 bash[29284]: cluster 2024-03-26T03:50:15.469950+0000 mon.a (mon.0) 1472 : cluster [DBG] osdmap e159: 8 total, 8 up, 8 in 2024-03-26T03:50:15.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:15 smithi175 bash[29284]: audit 2024-03-26T03:50:15.477591+0000 mon.c (mon.2) 295 : audit [INF] from='client.? 172.21.15.175:0/2412645833' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi175-64398-27"}]: dispatch 2024-03-26T03:50:16.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:16 smithi190 bash[23392]: cluster 2024-03-26T03:50:14.612067+0000 mgr.y (mgr.24431) 247 : cluster [DBG] pgmap v234: 371 pgs: 13 creating+peering, 29 unknown, 329 active+clean; 590 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 2 op/s 2024-03-26T03:50:16.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:16 smithi190 bash[23392]: cluster 2024-03-26T03:50:15.479672+0000 mon.a (mon.0) 1473 : cluster [WRN] Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:50:16.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:16 smithi190 bash[23392]: audit 2024-03-26T03:50:15.482084+0000 mon.a (mon.0) 1474 : audit [INF] from='client.? 172.21.15.175:0/1295298618' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi175-64544-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:16.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:16 smithi190 bash[23392]: audit 2024-03-26T03:50:15.482794+0000 mon.a (mon.0) 1475 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi175-64398-27"}]: dispatch 2024-03-26T03:50:16.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:16 smithi190 bash[23392]: audit 2024-03-26T03:50:15.535289+0000 mon.c (mon.2) 296 : audit [INF] from='client.? 172.21.15.175:0/7681524' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi175-64438-2"}]: dispatch 2024-03-26T03:50:16.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:16 smithi190 bash[23392]: audit 2024-03-26T03:50:15.536401+0000 mon.a (mon.0) 1476 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi175-64438-2"}]: dispatch 2024-03-26T03:50:16.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:16 smithi190 bash[23392]: audit 2024-03-26T03:50:15.538824+0000 mon.c (mon.2) 297 : audit [INF] from='client.? 172.21.15.175:0/7681524' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi175-64438-2"}]: dispatch 2024-03-26T03:50:16.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:16 smithi190 bash[23392]: audit 2024-03-26T03:50:15.539376+0000 mon.a (mon.0) 1477 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi175-64438-2"}]: dispatch 2024-03-26T03:50:16.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:16 smithi190 bash[23392]: audit 2024-03-26T03:50:15.540426+0000 mon.c (mon.2) 298 : audit [INF] from='client.? 172.21.15.175:0/7681524' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi175-64438-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:16.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:16 smithi190 bash[23392]: audit 2024-03-26T03:50:15.541034+0000 mon.a (mon.0) 1478 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi175-64438-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:16.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:16 smithi190 bash[23392]: audit 2024-03-26T03:50:16.458181+0000 mon.a (mon.0) 1479 : audit [INF] from='client.? 172.21.15.175:0/1295298618' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi175-64544-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:16.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:16 smithi190 bash[23392]: audit 2024-03-26T03:50:16.458351+0000 mon.a (mon.0) 1480 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi175-64398-27"}]': finished 2024-03-26T03:50:16.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:16 smithi190 bash[23392]: audit 2024-03-26T03:50:16.458499+0000 mon.a (mon.0) 1481 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi175-64438-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:16.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:16 smithi190 bash[23392]: cluster 2024-03-26T03:50:16.466736+0000 mon.a (mon.0) 1482 : cluster [DBG] osdmap e160: 8 total, 8 up, 8 in 2024-03-26T03:50:16.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:16 smithi190 bash[23392]: audit 2024-03-26T03:50:16.469132+0000 mon.c (mon.2) 299 : audit [INF] from='client.? 172.21.15.175:0/7681524' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi175-64438-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi175-64438-2"}]: dispatch 2024-03-26T03:50:16.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:16 smithi190 bash[23392]: audit 2024-03-26T03:50:16.470325+0000 mon.c (mon.2) 300 : audit [INF] from='client.? 172.21.15.175:0/2412645833' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi175-64398-27"}]: dispatch 2024-03-26T03:50:16.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:16 smithi190 bash[23392]: audit 2024-03-26T03:50:16.474122+0000 mon.a (mon.0) 1483 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi175-64438-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi175-64438-2"}]: dispatch 2024-03-26T03:50:16.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:16 smithi190 bash[23392]: audit 2024-03-26T03:50:16.475776+0000 mon.a (mon.0) 1484 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi175-64398-27"}]: dispatch 2024-03-26T03:50:16.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[21527]: cluster 2024-03-26T03:50:14.612067+0000 mgr.y (mgr.24431) 247 : cluster [DBG] pgmap v234: 371 pgs: 13 creating+peering, 29 unknown, 329 active+clean; 590 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 2 op/s 2024-03-26T03:50:16.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[21527]: cluster 2024-03-26T03:50:15.479672+0000 mon.a (mon.0) 1473 : cluster [WRN] Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:50:16.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[21527]: audit 2024-03-26T03:50:15.482084+0000 mon.a (mon.0) 1474 : audit [INF] from='client.? 172.21.15.175:0/1295298618' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi175-64544-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:16.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[21527]: audit 2024-03-26T03:50:15.482794+0000 mon.a (mon.0) 1475 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi175-64398-27"}]: dispatch 2024-03-26T03:50:16.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[21527]: audit 2024-03-26T03:50:15.535289+0000 mon.c (mon.2) 296 : audit [INF] from='client.? 172.21.15.175:0/7681524' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi175-64438-2"}]: dispatch 2024-03-26T03:50:16.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[21527]: audit 2024-03-26T03:50:15.536401+0000 mon.a (mon.0) 1476 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi175-64438-2"}]: dispatch 2024-03-26T03:50:16.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[21527]: audit 2024-03-26T03:50:15.538824+0000 mon.c (mon.2) 297 : audit [INF] from='client.? 172.21.15.175:0/7681524' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi175-64438-2"}]: dispatch 2024-03-26T03:50:16.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[21527]: audit 2024-03-26T03:50:15.539376+0000 mon.a (mon.0) 1477 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi175-64438-2"}]: dispatch 2024-03-26T03:50:16.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[21527]: audit 2024-03-26T03:50:15.540426+0000 mon.c (mon.2) 298 : audit [INF] from='client.? 172.21.15.175:0/7681524' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi175-64438-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:16.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[21527]: audit 2024-03-26T03:50:15.541034+0000 mon.a (mon.0) 1478 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi175-64438-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:16.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[21527]: audit 2024-03-26T03:50:16.458181+0000 mon.a (mon.0) 1479 : audit [INF] from='client.? 172.21.15.175:0/1295298618' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi175-64544-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:16.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[21527]: audit 2024-03-26T03:50:16.458351+0000 mon.a (mon.0) 1480 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi175-64398-27"}]': finished 2024-03-26T03:50:16.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[21527]: audit 2024-03-26T03:50:16.458499+0000 mon.a (mon.0) 1481 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi175-64438-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:16.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[21527]: cluster 2024-03-26T03:50:16.466736+0000 mon.a (mon.0) 1482 : cluster [DBG] osdmap e160: 8 total, 8 up, 8 in 2024-03-26T03:50:16.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[29284]: cluster 2024-03-26T03:50:14.612067+0000 mgr.y (mgr.24431) 247 : cluster [DBG] pgmap v234: 371 pgs: 13 creating+peering, 29 unknown, 329 active+clean; 590 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 2 op/s 2024-03-26T03:50:16.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[29284]: cluster 2024-03-26T03:50:15.479672+0000 mon.a (mon.0) 1473 : cluster [WRN] Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:50:16.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[29284]: audit 2024-03-26T03:50:15.482084+0000 mon.a (mon.0) 1474 : audit [INF] from='client.? 172.21.15.175:0/1295298618' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi175-64544-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:16.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[29284]: audit 2024-03-26T03:50:15.482794+0000 mon.a (mon.0) 1475 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi175-64398-27"}]: dispatch 2024-03-26T03:50:16.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[29284]: audit 2024-03-26T03:50:15.535289+0000 mon.c (mon.2) 296 : audit [INF] from='client.? 172.21.15.175:0/7681524' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi175-64438-2"}]: dispatch 2024-03-26T03:50:16.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[29284]: audit 2024-03-26T03:50:15.536401+0000 mon.a (mon.0) 1476 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi175-64438-2"}]: dispatch 2024-03-26T03:50:16.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[29284]: audit 2024-03-26T03:50:15.538824+0000 mon.c (mon.2) 297 : audit [INF] from='client.? 172.21.15.175:0/7681524' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi175-64438-2"}]: dispatch 2024-03-26T03:50:16.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[29284]: audit 2024-03-26T03:50:15.539376+0000 mon.a (mon.0) 1477 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi175-64438-2"}]: dispatch 2024-03-26T03:50:16.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[29284]: audit 2024-03-26T03:50:15.540426+0000 mon.c (mon.2) 298 : audit [INF] from='client.? 172.21.15.175:0/7681524' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi175-64438-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:16.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[29284]: audit 2024-03-26T03:50:15.541034+0000 mon.a (mon.0) 1478 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi175-64438-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:16.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[29284]: audit 2024-03-26T03:50:16.458181+0000 mon.a (mon.0) 1479 : audit [INF] from='client.? 172.21.15.175:0/1295298618' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi175-64544-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:16.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[29284]: audit 2024-03-26T03:50:16.458351+0000 mon.a (mon.0) 1480 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi175-64398-27"}]': finished 2024-03-26T03:50:16.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[29284]: audit 2024-03-26T03:50:16.458499+0000 mon.a (mon.0) 1481 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi175-64438-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:16.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[29284]: cluster 2024-03-26T03:50:16.466736+0000 mon.a (mon.0) 1482 : cluster [DBG] osdmap e160: 8 total, 8 up, 8 in 2024-03-26T03:50:16.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[21527]: audit 2024-03-26T03:50:16.469132+0000 mon.c (mon.2) 299 : audit [INF] from='client.? 172.21.15.175:0/7681524' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi175-64438-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi175-64438-2"}]: dispatch 2024-03-26T03:50:16.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[21527]: audit 2024-03-26T03:50:16.470325+0000 mon.c (mon.2) 300 : audit [INF] from='client.? 172.21.15.175:0/2412645833' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi175-64398-27"}]: dispatch 2024-03-26T03:50:16.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[21527]: audit 2024-03-26T03:50:16.474122+0000 mon.a (mon.0) 1483 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi175-64438-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi175-64438-2"}]: dispatch 2024-03-26T03:50:16.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[21527]: audit 2024-03-26T03:50:16.475776+0000 mon.a (mon.0) 1484 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi175-64398-27"}]: dispatch 2024-03-26T03:50:16.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[29284]: audit 2024-03-26T03:50:16.469132+0000 mon.c (mon.2) 299 : audit [INF] from='client.? 172.21.15.175:0/7681524' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi175-64438-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi175-64438-2"}]: dispatch 2024-03-26T03:50:16.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[29284]: audit 2024-03-26T03:50:16.470325+0000 mon.c (mon.2) 300 : audit [INF] from='client.? 172.21.15.175:0/2412645833' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi175-64398-27"}]: dispatch 2024-03-26T03:50:16.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[29284]: audit 2024-03-26T03:50:16.474122+0000 mon.a (mon.0) 1483 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi175-64438-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi175-64438-2"}]: dispatch 2024-03-26T03:50:16.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:16 smithi175 bash[29284]: audit 2024-03-26T03:50:16.475776+0000 mon.a (mon.0) 1484 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi175-64398-27"}]: dispatch 2024-03-26T03:50:17.487 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: Running main() from gmock_main.cc 2024-03-26T03:50:17.488 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [==========] Running 42 tests from 2 test suites. 2024-03-26T03:50:17.488 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [----------] Global test environment set-up. 2024-03-26T03:50:17.488 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [----------] 26 tests from LibRadosAio 2024-03-26T03:50:17.488 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2024-03-26T03:50:17.488 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAio.TooBig (3140 ms) 2024-03-26T03:50:17.488 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2024-03-26T03:50:17.488 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (3068 ms) 2024-03-26T03:50:17.488 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2024-03-26T03:50:17.488 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (2980 ms) 2024-03-26T03:50:17.488 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2024-03-26T03:50:17.488 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (3028 ms) 2024-03-26T03:50:17.488 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2024-03-26T03:50:17.488 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (3022 ms) 2024-03-26T03:50:17.488 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2024-03-26T03:50:17.489 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (2990 ms) 2024-03-26T03:50:17.489 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2024-03-26T03:50:17.489 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (3056 ms) 2024-03-26T03:50:17.489 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2024-03-26T03:50:17.489 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (2948 ms) 2024-03-26T03:50:17.489 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2024-03-26T03:50:17.489 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (2992 ms) 2024-03-26T03:50:17.489 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2024-03-26T03:50:17.489 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (3011 ms) 2024-03-26T03:50:17.489 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2024-03-26T03:50:17.489 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (3002 ms) 2024-03-26T03:50:17.489 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2024-03-26T03:50:17.489 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (2969 ms) 2024-03-26T03:50:17.489 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2024-03-26T03:50:17.490 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (3022 ms) 2024-03-26T03:50:17.490 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2024-03-26T03:50:17.490 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (3027 ms) 2024-03-26T03:50:17.490 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2024-03-26T03:50:17.490 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAio.Flush (2961 ms) 2024-03-26T03:50:17.490 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2024-03-26T03:50:17.490 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (3016 ms) 2024-03-26T03:50:17.490 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2024-03-26T03:50:17.490 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (2983 ms) 2024-03-26T03:50:17.490 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2024-03-26T03:50:17.490 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (2996 ms) 2024-03-26T03:50:17.490 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2024-03-26T03:50:17.490 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (3027 ms) 2024-03-26T03:50:17.490 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2024-03-26T03:50:17.491 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (3004 ms) 2024-03-26T03:50:17.491 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAio.Operate2Mtime 2024-03-26T03:50:17.491 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAio.Operate2Mtime (3109 ms) 2024-03-26T03:50:17.491 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2024-03-26T03:50:17.491 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (3018 ms) 2024-03-26T03:50:17.491 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2024-03-26T03:50:17.491 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (2851 ms) 2024-03-26T03:50:17.491 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2024-03-26T03:50:17.491 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (3028 ms) 2024-03-26T03:50:17.491 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2024-03-26T03:50:17.491 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (2979 ms) 2024-03-26T03:50:17.491 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2024-03-26T03:50:17.491 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (3031 ms) 2024-03-26T03:50:17.491 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [----------] 26 tests from LibRadosAio (78258 ms total) 2024-03-26T03:50:17.492 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: 2024-03-26T03:50:17.492 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2024-03-26T03:50:17.492 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleWrite 2024-03-26T03:50:18.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:18 smithi190 bash[23392]: cluster 2024-03-26T03:50:16.612768+0000 mgr.y (mgr.24431) 248 : cluster [DBG] pgmap v237: 332 pgs: 32 unknown, 300 active+clean; 605 KiB data, 598 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1 op/s 2024-03-26T03:50:18.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:18 smithi190 bash[23392]: audit 2024-03-26T03:50:17.464028+0000 mon.a (mon.0) 1485 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi175-64398-27"}]': finished 2024-03-26T03:50:18.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:18 smithi190 bash[23392]: cluster 2024-03-26T03:50:17.490392+0000 mon.a (mon.0) 1486 : cluster [DBG] osdmap e161: 8 total, 8 up, 8 in 2024-03-26T03:50:18.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:18 smithi190 bash[23392]: audit 2024-03-26T03:50:17.494735+0000 mon.a (mon.0) 1487 : audit [INF] from='client.? 172.21.15.175:0/3205161221' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi175-64406-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:18.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:18 smithi190 bash[23392]: audit 2024-03-26T03:50:17.531091+0000 mon.c (mon.2) 301 : audit [INF] from='client.? 172.21.15.175:0/300870566' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi175-64398-28"}]: dispatch 2024-03-26T03:50:18.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:18 smithi190 bash[23392]: audit 2024-03-26T03:50:17.531976+0000 mon.a (mon.0) 1488 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi175-64398-28"}]: dispatch 2024-03-26T03:50:18.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:18 smithi190 bash[23392]: audit 2024-03-26T03:50:17.533043+0000 mon.c (mon.2) 302 : audit [INF] from='client.? 172.21.15.175:0/300870566' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi175-64398-28"}]: dispatch 2024-03-26T03:50:18.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:18 smithi190 bash[23392]: audit 2024-03-26T03:50:17.533557+0000 mon.a (mon.0) 1489 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi175-64398-28"}]: dispatch 2024-03-26T03:50:18.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:18 smithi190 bash[23392]: audit 2024-03-26T03:50:17.536644+0000 mon.c (mon.2) 303 : audit [INF] from='client.? 172.21.15.175:0/300870566' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi175-64398-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:18.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:18 smithi190 bash[23392]: audit 2024-03-26T03:50:17.537116+0000 mon.a (mon.0) 1490 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi175-64398-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:18.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:18 smithi190 bash[23392]: audit 2024-03-26T03:50:17.703633+0000 mon.a (mon.0) 1491 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:50:18.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:18 smithi175 bash[21527]: cluster 2024-03-26T03:50:16.612768+0000 mgr.y (mgr.24431) 248 : cluster [DBG] pgmap v237: 332 pgs: 32 unknown, 300 active+clean; 605 KiB data, 598 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1 op/s 2024-03-26T03:50:18.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:18 smithi175 bash[21527]: audit 2024-03-26T03:50:17.464028+0000 mon.a (mon.0) 1485 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi175-64398-27"}]': finished 2024-03-26T03:50:18.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:18 smithi175 bash[21527]: cluster 2024-03-26T03:50:17.490392+0000 mon.a (mon.0) 1486 : cluster [DBG] osdmap e161: 8 total, 8 up, 8 in 2024-03-26T03:50:18.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:18 smithi175 bash[21527]: audit 2024-03-26T03:50:17.494735+0000 mon.a (mon.0) 1487 : audit [INF] from='client.? 172.21.15.175:0/3205161221' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi175-64406-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:18.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:18 smithi175 bash[21527]: audit 2024-03-26T03:50:17.531091+0000 mon.c (mon.2) 301 : audit [INF] from='client.? 172.21.15.175:0/300870566' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi175-64398-28"}]: dispatch 2024-03-26T03:50:18.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:18 smithi175 bash[21527]: audit 2024-03-26T03:50:17.531976+0000 mon.a (mon.0) 1488 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi175-64398-28"}]: dispatch 2024-03-26T03:50:18.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:18 smithi175 bash[21527]: audit 2024-03-26T03:50:17.533043+0000 mon.c (mon.2) 302 : audit [INF] from='client.? 172.21.15.175:0/300870566' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi175-64398-28"}]: dispatch 2024-03-26T03:50:18.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:18 smithi175 bash[21527]: audit 2024-03-26T03:50:17.533557+0000 mon.a (mon.0) 1489 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi175-64398-28"}]: dispatch 2024-03-26T03:50:18.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:18 smithi175 bash[21527]: audit 2024-03-26T03:50:17.536644+0000 mon.c (mon.2) 303 : audit [INF] from='client.? 172.21.15.175:0/300870566' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi175-64398-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:18.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:18 smithi175 bash[21527]: audit 2024-03-26T03:50:17.537116+0000 mon.a (mon.0) 1490 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi175-64398-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:18.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:18 smithi175 bash[21527]: audit 2024-03-26T03:50:17.703633+0000 mon.a (mon.0) 1491 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:50:18.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:18 smithi175 bash[29284]: cluster 2024-03-26T03:50:16.612768+0000 mgr.y (mgr.24431) 248 : cluster [DBG] pgmap v237: 332 pgs: 32 unknown, 300 active+clean; 605 KiB data, 598 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1 op/s 2024-03-26T03:50:18.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:18 smithi175 bash[29284]: audit 2024-03-26T03:50:17.464028+0000 mon.a (mon.0) 1485 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi175-64398-27"}]': finished 2024-03-26T03:50:18.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:18 smithi175 bash[29284]: cluster 2024-03-26T03:50:17.490392+0000 mon.a (mon.0) 1486 : cluster [DBG] osdmap e161: 8 total, 8 up, 8 in 2024-03-26T03:50:18.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:18 smithi175 bash[29284]: audit 2024-03-26T03:50:17.494735+0000 mon.a (mon.0) 1487 : audit [INF] from='client.? 172.21.15.175:0/3205161221' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi175-64406-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:18.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:18 smithi175 bash[29284]: audit 2024-03-26T03:50:17.531091+0000 mon.c (mon.2) 301 : audit [INF] from='client.? 172.21.15.175:0/300870566' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi175-64398-28"}]: dispatch 2024-03-26T03:50:18.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:18 smithi175 bash[29284]: audit 2024-03-26T03:50:17.531976+0000 mon.a (mon.0) 1488 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi175-64398-28"}]: dispatch 2024-03-26T03:50:18.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:18 smithi175 bash[29284]: audit 2024-03-26T03:50:17.533043+0000 mon.c (mon.2) 302 : audit [INF] from='client.? 172.21.15.175:0/300870566' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi175-64398-28"}]: dispatch 2024-03-26T03:50:18.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:18 smithi175 bash[29284]: audit 2024-03-26T03:50:17.533557+0000 mon.a (mon.0) 1489 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi175-64398-28"}]: dispatch 2024-03-26T03:50:18.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:18 smithi175 bash[29284]: audit 2024-03-26T03:50:17.536644+0000 mon.c (mon.2) 303 : audit [INF] from='client.? 172.21.15.175:0/300870566' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi175-64398-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:18.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:18 smithi175 bash[29284]: audit 2024-03-26T03:50:17.537116+0000 mon.a (mon.0) 1490 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi175-64398-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:18.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:18 smithi175 bash[29284]: audit 2024-03-26T03:50:17.703633+0000 mon.a (mon.0) 1491 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:50:19.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:50:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:50:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:50:19.632 INFO:tasks.workunit.client.0.smithi175.stdout: expected=8 2024-03-26T03:50:19.632 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : seek to 13:b29083e3:::5:head 2024-03-26T03:50:19.632 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : cursor()=13:b29083e3:::5:head expected=13:b29083e3:::5:head 2024-03-26T03:50:19.632 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > 13:b29083e3:::5:head -> 5 2024-03-26T03:50:19.632 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : entry=5 expected=5 2024-03-26T03:50:19.632 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : seek to 13:f905c69b:::2:head 2024-03-26T03:50:19.632 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : cursor()=13:f905c69b:::2:head expected=13:f905c69b:::2:head 2024-03-26T03:50:19.632 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > 13:f905c69b:::2:head -> 2 2024-03-26T03:50:19.632 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : entry=2 expected=2 2024-03-26T03:50:19.632 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : seek to 13:89d3ae78:::11:head 2024-03-26T03:50:19.632 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : cursor()=13:89d3ae78:::11:head expected=13:89d3ae78:::11:head 2024-03-26T03:50:19.632 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > 13:89d3ae78:::11:head -> 11 2024-03-26T03:50:19.632 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : entry=11 expected=11 2024-03-26T03:50:19.633 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : seek to 13:62a1935d:::14:head 2024-03-26T03:50:19.633 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : cursor()=13:62a1935d:::14:head expected=13:62a1935d:::14:head 2024-03-26T03:50:19.633 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > 13:62a1935d:::14:head -> 14 2024-03-26T03:50:19.633 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : entry=14 expected=14 2024-03-26T03:50:19.633 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : seek to 13:5c6b0b28:::7:head 2024-03-26T03:50:19.633 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : cursor()=13:5c6b0b28:::7:head expected=13:5c6b0b28:::7:head 2024-03-26T03:50:19.633 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > 13:5c6b0b28:::7:head -> 7 2024-03-26T03:50:19.633 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : entry=7 expected=7 2024-03-26T03:50:19.633 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : seek to 13:566253c9:::13:head 2024-03-26T03:50:19.633 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : cursor()=13:566253c9:::13:head expected=13:566253c9:::13:head 2024-03-26T03:50:19.633 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > 13:566253c9:::13:head -> 13 2024-03-26T03:50:19.633 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : entry=13 expected=13 2024-03-26T03:50:19.633 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : seek to 13:52ea6a34:::10:head 2024-03-26T03:50:19.634 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : cursor()=13:52ea6a34:::10:head expected=13:52ea6a34:::10:head 2024-03-26T03:50:19.634 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > 13:52ea6a34:::10:head -> 10 2024-03-26T03:50:19.634 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : entry=10 expected=10 2024-03-26T03:50:19.634 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : seek to 13:02547ec2:::1:head 2024-03-26T03:50:19.634 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : cursor()=13:02547ec2:::1:head expected=13:02547ec2:::1:head 2024-03-26T03:50:19.634 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: > 13:02547ec2:::1:head -> 1 2024-03-26T03:50:19.634 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: : entry=1 expected=1 2024-03-26T03:50:19.634 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (227 ms) 2024-03-26T03:50:19.634 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2024-03-26T03:50:19.634 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (71268 ms) 2024-03-26T03:50:19.634 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2024-03-26T03:50:19.634 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: split 0/5 -> MIN 13:33333333::::head 2024-03-26T03:50:19.634 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: split 1/5 -> 13:33333333::::head 13:66666666::::head 2024-03-26T03:50:19.634 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: split 2/5 -> 13:66666666::::head 13:99999999::::head 2024-03-26T03:50:19.634 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: split 3/5 -> 13:99999999::::head 13:cccccccc::::head 2024-03-26T03:50:19.635 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: split 4/5 -> 13:cccccccc::::head MAX 2024-03-26T03:50:19.635 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (8005 ms) 2024-03-26T03:50:19.635 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [----------] 7 tests from LibRadosList (80233 ms total) 2024-03-26T03:50:19.635 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 2024-03-26T03:50:19.635 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [----------] 3 tests from LibRadosListEC 2024-03-26T03:50:19.635 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2024-03-26T03:50:19.635 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (1096 ms) 2024-03-26T03:50:19.635 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2024-03-26T03:50:19.635 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: myset foo1,foo2,foo3 2024-03-26T03:50:19.635 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: foo1 2024-03-26T03:50:19.635 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: foo2 2024-03-26T03:50:19.635 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: foo3 2024-03-26T03:50:19.635 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: myset foo1,foo4,foo5 2024-03-26T03:50:19.635 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: foo4 2024-03-26T03:50:19.635 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: foo5 2024-03-26T03:50:19.636 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: foo1 2024-03-26T03:50:19.636 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: myset foo6,foo7 2024-03-26T03:50:19.636 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: foo7 2024-03-26T03:50:19.636 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: foo6 2024-03-26T03:50:19.636 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-03-26T03:50:19.636 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: ns1:foo4 2024-03-26T03:50:19.636 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: ns1:foo5 2024-03-26T03:50:19.636 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: ns2:foo7 2024-03-26T03:50:19.636 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: ns2:foo6 2024-03-26T03:50:19.636 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: ns1:foo1 2024-03-26T03:50:19.636 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: :foo1 2024-03-26T03:50:19.636 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: :foo2 2024-03-26T03:50:19.636 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: :foo3 2024-03-26T03:50:19.636 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (38 ms) 2024-03-26T03:50:19.768 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:19 smithi190 bash[23392]: audit 2024-03-26T03:50:18.482258+0000 mon.a (mon.0) 1492 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi175-64438-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi175-64438-2"}]': finished 2024-03-26T03:50:19.768 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:19 smithi190 bash[23392]: audit 2024-03-26T03:50:18.482463+0000 mon.a (mon.0) 1493 : audit [INF] from='client.? 172.21.15.175:0/3205161221' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi175-64406-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:19.768 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:19 smithi190 bash[23392]: audit 2024-03-26T03:50:18.482614+0000 mon.a (mon.0) 1494 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi175-64398-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:19.768 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:19 smithi190 bash[23392]: audit 2024-03-26T03:50:18.482762+0000 mon.a (mon.0) 1495 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:50:19.768 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:19 smithi190 bash[23392]: cluster 2024-03-26T03:50:18.495120+0000 mon.a (mon.0) 1496 : cluster [DBG] osdmap e162: 8 total, 8 up, 8 in 2024-03-26T03:50:19.768 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:19 smithi190 bash[23392]: audit 2024-03-26T03:50:18.498464+0000 mon.a (mon.0) 1497 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-13"}]: dispatch 2024-03-26T03:50:19.768 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:19 smithi190 bash[23392]: audit 2024-03-26T03:50:18.501391+0000 mon.c (mon.2) 304 : audit [INF] from='client.? 172.21.15.175:0/300870566' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi175-64398-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi175-64398-28"}]: dispatch 2024-03-26T03:50:19.769 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:19 smithi190 bash[23392]: audit 2024-03-26T03:50:18.501754+0000 mon.c (mon.2) 305 : audit [INF] from='client.? 172.21.15.175:0/827906644' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi175-64544-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:19.769 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:19 smithi190 bash[23392]: audit 2024-03-26T03:50:18.506665+0000 mon.a (mon.0) 1498 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi175-64398-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi175-64398-28"}]: dispatch 2024-03-26T03:50:19.769 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:19 smithi190 bash[23392]: audit 2024-03-26T03:50:18.507211+0000 mon.a (mon.0) 1499 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi175-64544-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:19.769 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:19 smithi190 bash[23392]: cluster 2024-03-26T03:50:18.613873+0000 mgr.y (mgr.24431) 249 : cluster [DBG] pgmap v240: 372 pgs: 72 unknown, 300 active+clean; 605 KiB data, 598 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1 op/s 2024-03-26T03:50:19.769 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:19 smithi190 bash[23392]: audit 2024-03-26T03:50:19.219031+0000 mon.c (mon.2) 306 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:50:19.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:19 smithi175 bash[21527]: audit 2024-03-26T03:50:18.482258+0000 mon.a (mon.0) 1492 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi175-64438-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi175-64438-2"}]': finished 2024-03-26T03:50:19.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:19 smithi175 bash[21527]: audit 2024-03-26T03:50:18.482463+0000 mon.a (mon.0) 1493 : audit [INF] from='client.? 172.21.15.175:0/3205161221' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi175-64406-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:19.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:19 smithi175 bash[21527]: audit 2024-03-26T03:50:18.482614+0000 mon.a (mon.0) 1494 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi175-64398-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:19.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:19 smithi175 bash[21527]: audit 2024-03-26T03:50:18.482762+0000 mon.a (mon.0) 1495 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:50:19.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:19 smithi175 bash[21527]: cluster 2024-03-26T03:50:18.495120+0000 mon.a (mon.0) 1496 : cluster [DBG] osdmap e162: 8 total, 8 up, 8 in 2024-03-26T03:50:19.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:19 smithi175 bash[21527]: audit 2024-03-26T03:50:18.498464+0000 mon.a (mon.0) 1497 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-13"}]: dispatch 2024-03-26T03:50:19.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:19 smithi175 bash[21527]: audit 2024-03-26T03:50:18.501391+0000 mon.c (mon.2) 304 : audit [INF] from='client.? 172.21.15.175:0/300870566' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi175-64398-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi175-64398-28"}]: dispatch 2024-03-26T03:50:19.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:19 smithi175 bash[21527]: audit 2024-03-26T03:50:18.501754+0000 mon.c (mon.2) 305 : audit [INF] from='client.? 172.21.15.175:0/827906644' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi175-64544-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:19.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:19 smithi175 bash[21527]: audit 2024-03-26T03:50:18.506665+0000 mon.a (mon.0) 1498 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi175-64398-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi175-64398-28"}]: dispatch 2024-03-26T03:50:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:19 smithi175 bash[21527]: audit 2024-03-26T03:50:18.507211+0000 mon.a (mon.0) 1499 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi175-64544-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:19 smithi175 bash[21527]: cluster 2024-03-26T03:50:18.613873+0000 mgr.y (mgr.24431) 249 : cluster [DBG] pgmap v240: 372 pgs: 72 unknown, 300 active+clean; 605 KiB data, 598 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1 op/s 2024-03-26T03:50:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:19 smithi175 bash[21527]: audit 2024-03-26T03:50:19.219031+0000 mon.c (mon.2) 306 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:50:19.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:19 smithi175 bash[29284]: audit 2024-03-26T03:50:18.482258+0000 mon.a (mon.0) 1492 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi175-64438-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi175-64438-2"}]': finished 2024-03-26T03:50:19.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:19 smithi175 bash[29284]: audit 2024-03-26T03:50:18.482463+0000 mon.a (mon.0) 1493 : audit [INF] from='client.? 172.21.15.175:0/3205161221' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi175-64406-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:19.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:19 smithi175 bash[29284]: audit 2024-03-26T03:50:18.482614+0000 mon.a (mon.0) 1494 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi175-64398-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:19.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:19 smithi175 bash[29284]: audit 2024-03-26T03:50:18.482762+0000 mon.a (mon.0) 1495 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:50:19.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:19 smithi175 bash[29284]: cluster 2024-03-26T03:50:18.495120+0000 mon.a (mon.0) 1496 : cluster [DBG] osdmap e162: 8 total, 8 up, 8 in 2024-03-26T03:50:19.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:19 smithi175 bash[29284]: audit 2024-03-26T03:50:18.498464+0000 mon.a (mon.0) 1497 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-13"}]: dispatch 2024-03-26T03:50:19.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:19 smithi175 bash[29284]: audit 2024-03-26T03:50:18.501391+0000 mon.c (mon.2) 304 : audit [INF] from='client.? 172.21.15.175:0/300870566' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi175-64398-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi175-64398-28"}]: dispatch 2024-03-26T03:50:19.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:19 smithi175 bash[29284]: audit 2024-03-26T03:50:18.501754+0000 mon.c (mon.2) 305 : audit [INF] from='client.? 172.21.15.175:0/827906644' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi175-64544-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:19.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:19 smithi175 bash[29284]: audit 2024-03-26T03:50:18.506665+0000 mon.a (mon.0) 1498 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi175-64398-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi175-64398-28"}]: dispatch 2024-03-26T03:50:19.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:19 smithi175 bash[29284]: audit 2024-03-26T03:50:18.507211+0000 mon.a (mon.0) 1499 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi175-64544-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:19.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:19 smithi175 bash[29284]: cluster 2024-03-26T03:50:18.613873+0000 mgr.y (mgr.24431) 249 : cluster [DBG] pgmap v240: 372 pgs: 72 unknown, 300 active+clean; 605 KiB data, 598 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1 op/s 2024-03-26T03:50:19.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:19 smithi175 bash[29284]: audit 2024-03-26T03:50:19.219031+0000 mon.c (mon.2) 306 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:50:20.240 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:50:19 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:50:20.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:20 smithi190 bash[23392]: cluster 2024-03-26T03:50:19.484826+0000 mon.a (mon.0) 1500 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:50:20.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:20 smithi190 bash[23392]: audit 2024-03-26T03:50:19.495717+0000 mon.a (mon.0) 1501 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-13"}]': finished 2024-03-26T03:50:20.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:20 smithi190 bash[23392]: audit 2024-03-26T03:50:19.495978+0000 mon.a (mon.0) 1502 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi175-64544-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:20.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:20 smithi190 bash[23392]: cluster 2024-03-26T03:50:19.520711+0000 mon.a (mon.0) 1503 : cluster [DBG] osdmap e163: 8 total, 8 up, 8 in 2024-03-26T03:50:20.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:20 smithi190 bash[23392]: audit 2024-03-26T03:50:19.770411+0000 mgr.y (mgr.24431) 250 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:50:20.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:20 smithi175 bash[29284]: cluster 2024-03-26T03:50:19.484826+0000 mon.a (mon.0) 1500 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:50:20.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:20 smithi175 bash[29284]: audit 2024-03-26T03:50:19.495717+0000 mon.a (mon.0) 1501 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-13"}]': finished 2024-03-26T03:50:20.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:20 smithi175 bash[29284]: audit 2024-03-26T03:50:19.495978+0000 mon.a (mon.0) 1502 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi175-64544-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:20.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:20 smithi175 bash[29284]: cluster 2024-03-26T03:50:19.520711+0000 mon.a (mon.0) 1503 : cluster [DBG] osdmap e163: 8 total, 8 up, 8 in 2024-03-26T03:50:20.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:20 smithi175 bash[29284]: audit 2024-03-26T03:50:19.770411+0000 mgr.y (mgr.24431) 250 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:50:20.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:20 smithi175 bash[21527]: cluster 2024-03-26T03:50:19.484826+0000 mon.a (mon.0) 1500 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:50:20.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:20 smithi175 bash[21527]: audit 2024-03-26T03:50:19.495717+0000 mon.a (mon.0) 1501 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-13"}]': finished 2024-03-26T03:50:20.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:20 smithi175 bash[21527]: audit 2024-03-26T03:50:19.495978+0000 mon.a (mon.0) 1502 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi175-64544-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:20.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:20 smithi175 bash[21527]: cluster 2024-03-26T03:50:19.520711+0000 mon.a (mon.0) 1503 : cluster [DBG] osdmap e163: 8 total, 8 up, 8 in 2024-03-26T03:50:20.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:20 smithi175 bash[21527]: audit 2024-03-26T03:50:19.770411+0000 mgr.y (mgr.24431) 250 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:50:21.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:21 smithi190 bash[23392]: audit 2024-03-26T03:50:20.512193+0000 mon.a (mon.0) 1504 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi175-64398-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi175-64398-28"}]': finished 2024-03-26T03:50:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:21 smithi190 bash[23392]: cluster 2024-03-26T03:50:20.520256+0000 mon.a (mon.0) 1505 : cluster [DBG] osdmap e164: 8 total, 8 up, 8 in 2024-03-26T03:50:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:21 smithi190 bash[23392]: audit 2024-03-26T03:50:20.525015+0000 mon.c (mon.2) 307 : audit [INF] from='client.? 172.21.15.175:0/7681524' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi175-64438-2"}]: dispatch 2024-03-26T03:50:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:21 smithi190 bash[23392]: audit 2024-03-26T03:50:20.526232+0000 mon.c (mon.2) 308 : audit [INF] from='client.? 172.21.15.175:0/2681852920' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi175-64406-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:21 smithi190 bash[23392]: audit 2024-03-26T03:50:20.538660+0000 mon.a (mon.0) 1506 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi175-64438-2"}]: dispatch 2024-03-26T03:50:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:21 smithi190 bash[23392]: audit 2024-03-26T03:50:20.539116+0000 mon.a (mon.0) 1507 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi175-64406-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:21 smithi190 bash[23392]: cluster 2024-03-26T03:50:20.614607+0000 mgr.y (mgr.24431) 251 : cluster [DBG] pgmap v243: 308 pgs: 40 unknown, 268 active+clean; 608 KiB data, 596 MiB used, 715 GiB / 715 GiB avail; 8.0 KiB/s wr, 1 op/s 2024-03-26T03:50:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:21 smithi190 bash[23392]: audit 2024-03-26T03:50:21.415410+0000 mon.a (mon.0) 1508 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi175-64438-2"}]': finished 2024-03-26T03:50:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:21 smithi190 bash[23392]: audit 2024-03-26T03:50:21.415682+0000 mon.a (mon.0) 1509 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi175-64406-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:21 smithi190 bash[23392]: cluster 2024-03-26T03:50:21.426788+0000 mon.a (mon.0) 1510 : cluster [DBG] osdmap e165: 8 total, 8 up, 8 in 2024-03-26T03:50:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:21 smithi190 bash[23392]: audit 2024-03-26T03:50:21.427416+0000 mon.c (mon.2) 309 : audit [INF] from='client.? 172.21.15.175:0/7681524' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi175-64438-2"}]: dispatch 2024-03-26T03:50:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:21 smithi190 bash[23392]: audit 2024-03-26T03:50:21.430398+0000 mon.a (mon.0) 1511 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi175-64438-2"}]: dispatch 2024-03-26T03:50:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:21 smithi190 bash[23392]: audit 2024-03-26T03:50:21.432237+0000 mon.a (mon.0) 1512 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:21.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:21 smithi190 bash[23392]: audit 2024-03-26T03:50:21.433867+0000 mon.c (mon.2) 310 : audit [INF] from='client.? 172.21.15.175:0/1007132050' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi175-64544-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:21.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:21 smithi190 bash[23392]: audit 2024-03-26T03:50:21.434703+0000 mon.a (mon.0) 1513 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi175-64544-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:21.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:21 smithi175 bash[21527]: audit 2024-03-26T03:50:20.512193+0000 mon.a (mon.0) 1504 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi175-64398-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi175-64398-28"}]': finished 2024-03-26T03:50:21.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:21 smithi175 bash[21527]: cluster 2024-03-26T03:50:20.520256+0000 mon.a (mon.0) 1505 : cluster [DBG] osdmap e164: 8 total, 8 up, 8 in 2024-03-26T03:50:21.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:21 smithi175 bash[21527]: audit 2024-03-26T03:50:20.525015+0000 mon.c (mon.2) 307 : audit [INF] from='client.? 172.21.15.175:0/7681524' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi175-64438-2"}]: dispatch 2024-03-26T03:50:21.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:21 smithi175 bash[21527]: audit 2024-03-26T03:50:20.526232+0000 mon.c (mon.2) 308 : audit [INF] from='client.? 172.21.15.175:0/2681852920' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi175-64406-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:21.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:21 smithi175 bash[21527]: audit 2024-03-26T03:50:20.538660+0000 mon.a (mon.0) 1506 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi175-64438-2"}]: dispatch 2024-03-26T03:50:21.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:21 smithi175 bash[21527]: audit 2024-03-26T03:50:20.539116+0000 mon.a (mon.0) 1507 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi175-64406-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:21.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:21 smithi175 bash[21527]: cluster 2024-03-26T03:50:20.614607+0000 mgr.y (mgr.24431) 251 : cluster [DBG] pgmap v243: 308 pgs: 40 unknown, 268 active+clean; 608 KiB data, 596 MiB used, 715 GiB / 715 GiB avail; 8.0 KiB/s wr, 1 op/s 2024-03-26T03:50:21.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:21 smithi175 bash[21527]: audit 2024-03-26T03:50:21.415410+0000 mon.a (mon.0) 1508 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi175-64438-2"}]': finished 2024-03-26T03:50:21.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:21 smithi175 bash[21527]: audit 2024-03-26T03:50:21.415682+0000 mon.a (mon.0) 1509 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi175-64406-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:21.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:21 smithi175 bash[21527]: cluster 2024-03-26T03:50:21.426788+0000 mon.a (mon.0) 1510 : cluster [DBG] osdmap e165: 8 total, 8 up, 8 in 2024-03-26T03:50:21.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:21 smithi175 bash[21527]: audit 2024-03-26T03:50:21.427416+0000 mon.c (mon.2) 309 : audit [INF] from='client.? 172.21.15.175:0/7681524' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi175-64438-2"}]: dispatch 2024-03-26T03:50:21.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:21 smithi175 bash[21527]: audit 2024-03-26T03:50:21.430398+0000 mon.a (mon.0) 1511 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi175-64438-2"}]: dispatch 2024-03-26T03:50:21.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:21 smithi175 bash[21527]: audit 2024-03-26T03:50:21.432237+0000 mon.a (mon.0) 1512 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:21.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:21 smithi175 bash[21527]: audit 2024-03-26T03:50:21.433867+0000 mon.c (mon.2) 310 : audit [INF] from='client.? 172.21.15.175:0/1007132050' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi175-64544-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:21.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:21 smithi175 bash[21527]: audit 2024-03-26T03:50:21.434703+0000 mon.a (mon.0) 1513 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi175-64544-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:21.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:21 smithi175 bash[29284]: audit 2024-03-26T03:50:20.512193+0000 mon.a (mon.0) 1504 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi175-64398-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi175-64398-28"}]': finished 2024-03-26T03:50:21.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:21 smithi175 bash[29284]: cluster 2024-03-26T03:50:20.520256+0000 mon.a (mon.0) 1505 : cluster [DBG] osdmap e164: 8 total, 8 up, 8 in 2024-03-26T03:50:21.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:21 smithi175 bash[29284]: audit 2024-03-26T03:50:20.525015+0000 mon.c (mon.2) 307 : audit [INF] from='client.? 172.21.15.175:0/7681524' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi175-64438-2"}]: dispatch 2024-03-26T03:50:21.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:21 smithi175 bash[29284]: audit 2024-03-26T03:50:20.526232+0000 mon.c (mon.2) 308 : audit [INF] from='client.? 172.21.15.175:0/2681852920' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi175-64406-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:21.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:21 smithi175 bash[29284]: audit 2024-03-26T03:50:20.538660+0000 mon.a (mon.0) 1506 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi175-64438-2"}]: dispatch 2024-03-26T03:50:21.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:21 smithi175 bash[29284]: audit 2024-03-26T03:50:20.539116+0000 mon.a (mon.0) 1507 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi175-64406-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:21.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:21 smithi175 bash[29284]: cluster 2024-03-26T03:50:20.614607+0000 mgr.y (mgr.24431) 251 : cluster [DBG] pgmap v243: 308 pgs: 40 unknown, 268 active+clean; 608 KiB data, 596 MiB used, 715 GiB / 715 GiB avail; 8.0 KiB/s wr, 1 op/s 2024-03-26T03:50:21.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:21 smithi175 bash[29284]: audit 2024-03-26T03:50:21.415410+0000 mon.a (mon.0) 1508 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi175-64438-2"}]': finished 2024-03-26T03:50:21.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:21 smithi175 bash[29284]: audit 2024-03-26T03:50:21.415682+0000 mon.a (mon.0) 1509 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi175-64406-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:21.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:21 smithi175 bash[29284]: cluster 2024-03-26T03:50:21.426788+0000 mon.a (mon.0) 1510 : cluster [DBG] osdmap e165: 8 total, 8 up, 8 in 2024-03-26T03:50:21.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:21 smithi175 bash[29284]: audit 2024-03-26T03:50:21.427416+0000 mon.c (mon.2) 309 : audit [INF] from='client.? 172.21.15.175:0/7681524' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi175-64438-2"}]: dispatch 2024-03-26T03:50:21.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:21 smithi175 bash[29284]: audit 2024-03-26T03:50:21.430398+0000 mon.a (mon.0) 1511 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi175-64438-2"}]: dispatch 2024-03-26T03:50:21.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:21 smithi175 bash[29284]: audit 2024-03-26T03:50:21.432237+0000 mon.a (mon.0) 1512 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:21.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:21 smithi175 bash[29284]: audit 2024-03-26T03:50:21.433867+0000 mon.c (mon.2) 310 : audit [INF] from='client.? 172.21.15.175:0/1007132050' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi175-64544-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:21.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:21 smithi175 bash[29284]: audit 2024-03-26T03:50:21.434703+0000 mon.a (mon.0) 1513 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi175-64544-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:23.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:23 smithi190 bash[23392]: audit 2024-03-26T03:50:22.420943+0000 mon.a (mon.0) 1514 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi175-64438-2"}]': finished 2024-03-26T03:50:23.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:23 smithi190 bash[23392]: audit 2024-03-26T03:50:22.421169+0000 mon.a (mon.0) 1515 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:23.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:23 smithi190 bash[23392]: audit 2024-03-26T03:50:22.421337+0000 mon.a (mon.0) 1516 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi175-64544-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:23.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:23 smithi190 bash[23392]: cluster 2024-03-26T03:50:22.429790+0000 mon.a (mon.0) 1517 : cluster [DBG] osdmap e166: 8 total, 8 up, 8 in 2024-03-26T03:50:23.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:23 smithi190 bash[23392]: audit 2024-03-26T03:50:22.435327+0000 mon.c (mon.2) 311 : audit [INF] from='client.? 172.21.15.175:0/300870566' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi175-64398-28"}]: dispatch 2024-03-26T03:50:23.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:23 smithi190 bash[23392]: audit 2024-03-26T03:50:22.437872+0000 mon.a (mon.0) 1518 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi175-64398-28"}]: dispatch 2024-03-26T03:50:23.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:23 smithi175 bash[21527]: audit 2024-03-26T03:50:22.420943+0000 mon.a (mon.0) 1514 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi175-64438-2"}]': finished 2024-03-26T03:50:23.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:23 smithi175 bash[21527]: audit 2024-03-26T03:50:22.421169+0000 mon.a (mon.0) 1515 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:23.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:23 smithi175 bash[21527]: audit 2024-03-26T03:50:22.421337+0000 mon.a (mon.0) 1516 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi175-64544-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:23.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:23 smithi175 bash[21527]: cluster 2024-03-26T03:50:22.429790+0000 mon.a (mon.0) 1517 : cluster [DBG] osdmap e166: 8 total, 8 up, 8 in 2024-03-26T03:50:23.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:23 smithi175 bash[21527]: audit 2024-03-26T03:50:22.435327+0000 mon.c (mon.2) 311 : audit [INF] from='client.? 172.21.15.175:0/300870566' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi175-64398-28"}]: dispatch 2024-03-26T03:50:23.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:23 smithi175 bash[21527]: audit 2024-03-26T03:50:22.437872+0000 mon.a (mon.0) 1518 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi175-64398-28"}]: dispatch 2024-03-26T03:50:23.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:23 smithi175 bash[29284]: audit 2024-03-26T03:50:22.420943+0000 mon.a (mon.0) 1514 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi175-64438-2"}]': finished 2024-03-26T03:50:23.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:23 smithi175 bash[29284]: audit 2024-03-26T03:50:22.421169+0000 mon.a (mon.0) 1515 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:23.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:23 smithi175 bash[29284]: audit 2024-03-26T03:50:22.421337+0000 mon.a (mon.0) 1516 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi175-64544-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:23.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:23 smithi175 bash[29284]: cluster 2024-03-26T03:50:22.429790+0000 mon.a (mon.0) 1517 : cluster [DBG] osdmap e166: 8 total, 8 up, 8 in 2024-03-26T03:50:23.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:23 smithi175 bash[29284]: audit 2024-03-26T03:50:22.435327+0000 mon.c (mon.2) 311 : audit [INF] from='client.? 172.21.15.175:0/300870566' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi175-64398-28"}]: dispatch 2024-03-26T03:50:23.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:23 smithi175 bash[29284]: audit 2024-03-26T03:50:22.437872+0000 mon.a (mon.0) 1518 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi175-64398-28"}]: dispatch 2024-03-26T03:50:24.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:24 smithi190 bash[23392]: cluster 2024-03-26T03:50:22.615502+0000 mgr.y (mgr.24431) 252 : cluster [DBG] pgmap v246: 332 pgs: 64 unknown, 268 active+clean; 608 KiB data, 596 MiB used, 715 GiB / 715 GiB avail; 8.0 KiB/s wr, 1 op/s 2024-03-26T03:50:24.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:24 smithi190 bash[23392]: cluster 2024-03-26T03:50:23.423590+0000 mon.a (mon.0) 1519 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:50:24.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:24 smithi190 bash[23392]: audit 2024-03-26T03:50:23.425651+0000 mon.a (mon.0) 1520 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi175-64398-28"}]': finished 2024-03-26T03:50:24.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:24 smithi190 bash[23392]: cluster 2024-03-26T03:50:23.436179+0000 mon.a (mon.0) 1521 : cluster [DBG] osdmap e167: 8 total, 8 up, 8 in 2024-03-26T03:50:24.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:24 smithi190 bash[23392]: audit 2024-03-26T03:50:23.439284+0000 mon.c (mon.2) 312 : audit [INF] from='client.? 172.21.15.175:0/300870566' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi175-64398-28"}]: dispatch 2024-03-26T03:50:24.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:24 smithi190 bash[23392]: audit 2024-03-26T03:50:23.440795+0000 mon.c (mon.2) 313 : audit [INF] from='client.? 172.21.15.175:0/2468284571' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi175-64406-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:24.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:24 smithi190 bash[23392]: audit 2024-03-26T03:50:23.441746+0000 mon.c (mon.2) 314 : audit [INF] from='client.? 172.21.15.175:0/2266041579' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64438-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:24.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:24 smithi190 bash[23392]: audit 2024-03-26T03:50:23.467641+0000 mon.a (mon.0) 1522 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi175-64398-28"}]: dispatch 2024-03-26T03:50:24.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:24 smithi190 bash[23392]: audit 2024-03-26T03:50:23.468187+0000 mon.a (mon.0) 1523 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi175-64406-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:24.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:24 smithi190 bash[23392]: audit 2024-03-26T03:50:23.468665+0000 mon.a (mon.0) 1524 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64438-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:24.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:24 smithi190 bash[23392]: audit 2024-03-26T03:50:23.484473+0000 mon.c (mon.2) 315 : audit [INF] from='client.? 172.21.15.175:0/4035522258' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi175-64544-36"}]: dispatch 2024-03-26T03:50:24.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:24 smithi190 bash[23392]: audit 2024-03-26T03:50:23.484924+0000 mon.a (mon.0) 1525 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi175-64544-36"}]: dispatch 2024-03-26T03:50:24.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:24 smithi190 bash[23392]: audit 2024-03-26T03:50:23.488623+0000 mon.c (mon.2) 316 : audit [INF] from='client.? 172.21.15.175:0/4035522258' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi175-64544-36"}]: dispatch 2024-03-26T03:50:24.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:24 smithi190 bash[23392]: audit 2024-03-26T03:50:23.489199+0000 mon.a (mon.0) 1526 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi175-64544-36"}]: dispatch 2024-03-26T03:50:24.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:24 smithi190 bash[23392]: audit 2024-03-26T03:50:23.489486+0000 mon.a (mon.0) 1527 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:50:24.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:24 smithi190 bash[23392]: audit 2024-03-26T03:50:23.489810+0000 mon.c (mon.2) 317 : audit [INF] from='client.? 172.21.15.175:0/4035522258' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi175-64544-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:24.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:24 smithi190 bash[23392]: audit 2024-03-26T03:50:23.490310+0000 mon.a (mon.0) 1528 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi175-64544-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:24.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:24 smithi190 bash[23392]: audit 2024-03-26T03:50:24.430409+0000 mon.a (mon.0) 1529 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi175-64398-28"}]': finished 2024-03-26T03:50:24.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:24 smithi190 bash[23392]: audit 2024-03-26T03:50:24.430571+0000 mon.a (mon.0) 1530 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi175-64406-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:24.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:24 smithi190 bash[23392]: audit 2024-03-26T03:50:24.430694+0000 mon.a (mon.0) 1531 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64438-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:24.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:24 smithi190 bash[23392]: audit 2024-03-26T03:50:24.430811+0000 mon.a (mon.0) 1532 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-15", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:50:24.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:24 smithi190 bash[23392]: audit 2024-03-26T03:50:24.430918+0000 mon.a (mon.0) 1533 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi175-64544-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:24.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:24 smithi190 bash[23392]: cluster 2024-03-26T03:50:24.439144+0000 mon.a (mon.0) 1534 : cluster [DBG] osdmap e168: 8 total, 8 up, 8 in 2024-03-26T03:50:24.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:24 smithi190 bash[23392]: audit 2024-03-26T03:50:24.443297+0000 mon.c (mon.2) 318 : audit [INF] from='client.? 172.21.15.175:0/4035522258' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi175-64544-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi175-64544-36"}]: dispatch 2024-03-26T03:50:24.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:24 smithi190 bash[23392]: audit 2024-03-26T03:50:24.444171+0000 mon.a (mon.0) 1535 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-15"}]: dispatch 2024-03-26T03:50:24.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:24 smithi190 bash[23392]: audit 2024-03-26T03:50:24.445131+0000 mon.a (mon.0) 1536 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi175-64544-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi175-64544-36"}]: dispatch 2024-03-26T03:50:24.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:24 smithi190 bash[23392]: audit 2024-03-26T03:50:24.451125+0000 mon.a (mon.0) 1537 : audit [INF] from='client.? 172.21.15.175:0/4044186109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21"}]: dispatch 2024-03-26T03:50:24.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[21527]: cluster 2024-03-26T03:50:22.615502+0000 mgr.y (mgr.24431) 252 : cluster [DBG] pgmap v246: 332 pgs: 64 unknown, 268 active+clean; 608 KiB data, 596 MiB used, 715 GiB / 715 GiB avail; 8.0 KiB/s wr, 1 op/s 2024-03-26T03:50:24.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[21527]: cluster 2024-03-26T03:50:23.423590+0000 mon.a (mon.0) 1519 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:50:24.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[21527]: audit 2024-03-26T03:50:23.425651+0000 mon.a (mon.0) 1520 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi175-64398-28"}]': finished 2024-03-26T03:50:24.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[21527]: cluster 2024-03-26T03:50:23.436179+0000 mon.a (mon.0) 1521 : cluster [DBG] osdmap e167: 8 total, 8 up, 8 in 2024-03-26T03:50:24.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[21527]: audit 2024-03-26T03:50:23.439284+0000 mon.c (mon.2) 312 : audit [INF] from='client.? 172.21.15.175:0/300870566' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi175-64398-28"}]: dispatch 2024-03-26T03:50:24.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[21527]: audit 2024-03-26T03:50:23.440795+0000 mon.c (mon.2) 313 : audit [INF] from='client.? 172.21.15.175:0/2468284571' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi175-64406-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:24.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[21527]: audit 2024-03-26T03:50:23.441746+0000 mon.c (mon.2) 314 : audit [INF] from='client.? 172.21.15.175:0/2266041579' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64438-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:24.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[21527]: audit 2024-03-26T03:50:23.467641+0000 mon.a (mon.0) 1522 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi175-64398-28"}]: dispatch 2024-03-26T03:50:24.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[21527]: audit 2024-03-26T03:50:23.468187+0000 mon.a (mon.0) 1523 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi175-64406-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:24.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[21527]: audit 2024-03-26T03:50:23.468665+0000 mon.a (mon.0) 1524 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64438-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:24.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[21527]: audit 2024-03-26T03:50:23.484473+0000 mon.c (mon.2) 315 : audit [INF] from='client.? 172.21.15.175:0/4035522258' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi175-64544-36"}]: dispatch 2024-03-26T03:50:24.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[21527]: audit 2024-03-26T03:50:23.484924+0000 mon.a (mon.0) 1525 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi175-64544-36"}]: dispatch 2024-03-26T03:50:24.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[21527]: audit 2024-03-26T03:50:23.488623+0000 mon.c (mon.2) 316 : audit [INF] from='client.? 172.21.15.175:0/4035522258' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi175-64544-36"}]: dispatch 2024-03-26T03:50:24.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[21527]: audit 2024-03-26T03:50:23.489199+0000 mon.a (mon.0) 1526 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi175-64544-36"}]: dispatch 2024-03-26T03:50:24.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[21527]: audit 2024-03-26T03:50:23.489486+0000 mon.a (mon.0) 1527 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:50:24.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[29284]: cluster 2024-03-26T03:50:22.615502+0000 mgr.y (mgr.24431) 252 : cluster [DBG] pgmap v246: 332 pgs: 64 unknown, 268 active+clean; 608 KiB data, 596 MiB used, 715 GiB / 715 GiB avail; 8.0 KiB/s wr, 1 op/s 2024-03-26T03:50:24.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[29284]: cluster 2024-03-26T03:50:23.423590+0000 mon.a (mon.0) 1519 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:50:24.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[29284]: audit 2024-03-26T03:50:23.425651+0000 mon.a (mon.0) 1520 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi175-64398-28"}]': finished 2024-03-26T03:50:24.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[29284]: cluster 2024-03-26T03:50:23.436179+0000 mon.a (mon.0) 1521 : cluster [DBG] osdmap e167: 8 total, 8 up, 8 in 2024-03-26T03:50:24.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[29284]: audit 2024-03-26T03:50:23.439284+0000 mon.c (mon.2) 312 : audit [INF] from='client.? 172.21.15.175:0/300870566' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi175-64398-28"}]: dispatch 2024-03-26T03:50:24.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[29284]: audit 2024-03-26T03:50:23.440795+0000 mon.c (mon.2) 313 : audit [INF] from='client.? 172.21.15.175:0/2468284571' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi175-64406-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:24.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[29284]: audit 2024-03-26T03:50:23.441746+0000 mon.c (mon.2) 314 : audit [INF] from='client.? 172.21.15.175:0/2266041579' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64438-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:24.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[29284]: audit 2024-03-26T03:50:23.467641+0000 mon.a (mon.0) 1522 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi175-64398-28"}]: dispatch 2024-03-26T03:50:24.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[29284]: audit 2024-03-26T03:50:23.468187+0000 mon.a (mon.0) 1523 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi175-64406-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:24.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[29284]: audit 2024-03-26T03:50:23.468665+0000 mon.a (mon.0) 1524 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64438-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:24.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[29284]: audit 2024-03-26T03:50:23.484473+0000 mon.c (mon.2) 315 : audit [INF] from='client.? 172.21.15.175:0/4035522258' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi175-64544-36"}]: dispatch 2024-03-26T03:50:24.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[29284]: audit 2024-03-26T03:50:23.484924+0000 mon.a (mon.0) 1525 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi175-64544-36"}]: dispatch 2024-03-26T03:50:24.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[29284]: audit 2024-03-26T03:50:23.488623+0000 mon.c (mon.2) 316 : audit [INF] from='client.? 172.21.15.175:0/4035522258' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi175-64544-36"}]: dispatch 2024-03-26T03:50:24.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[29284]: audit 2024-03-26T03:50:23.489199+0000 mon.a (mon.0) 1526 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi175-64544-36"}]: dispatch 2024-03-26T03:50:24.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[29284]: audit 2024-03-26T03:50:23.489486+0000 mon.a (mon.0) 1527 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:50:24.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[21527]: audit 2024-03-26T03:50:23.489810+0000 mon.c (mon.2) 317 : audit [INF] from='client.? 172.21.15.175:0/4035522258' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi175-64544-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:24.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[21527]: audit 2024-03-26T03:50:23.490310+0000 mon.a (mon.0) 1528 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi175-64544-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:24.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[21527]: audit 2024-03-26T03:50:24.430409+0000 mon.a (mon.0) 1529 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi175-64398-28"}]': finished 2024-03-26T03:50:24.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[21527]: audit 2024-03-26T03:50:24.430571+0000 mon.a (mon.0) 1530 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi175-64406-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:24.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[21527]: audit 2024-03-26T03:50:24.430694+0000 mon.a (mon.0) 1531 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64438-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:24.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[21527]: audit 2024-03-26T03:50:24.430811+0000 mon.a (mon.0) 1532 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-15", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:50:24.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[21527]: audit 2024-03-26T03:50:24.430918+0000 mon.a (mon.0) 1533 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi175-64544-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:24.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[21527]: cluster 2024-03-26T03:50:24.439144+0000 mon.a (mon.0) 1534 : cluster [DBG] osdmap e168: 8 total, 8 up, 8 in 2024-03-26T03:50:24.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[21527]: audit 2024-03-26T03:50:24.443297+0000 mon.c (mon.2) 318 : audit [INF] from='client.? 172.21.15.175:0/4035522258' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi175-64544-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi175-64544-36"}]: dispatch 2024-03-26T03:50:24.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[21527]: audit 2024-03-26T03:50:24.444171+0000 mon.a (mon.0) 1535 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-15"}]: dispatch 2024-03-26T03:50:24.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[21527]: audit 2024-03-26T03:50:24.445131+0000 mon.a (mon.0) 1536 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi175-64544-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi175-64544-36"}]: dispatch 2024-03-26T03:50:24.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[21527]: audit 2024-03-26T03:50:24.451125+0000 mon.a (mon.0) 1537 : audit [INF] from='client.? 172.21.15.175:0/4044186109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21"}]: dispatch 2024-03-26T03:50:24.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[29284]: audit 2024-03-26T03:50:23.489810+0000 mon.c (mon.2) 317 : audit [INF] from='client.? 172.21.15.175:0/4035522258' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi175-64544-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:24.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[29284]: audit 2024-03-26T03:50:23.490310+0000 mon.a (mon.0) 1528 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi175-64544-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:24.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[29284]: audit 2024-03-26T03:50:24.430409+0000 mon.a (mon.0) 1529 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi175-64398-28"}]': finished 2024-03-26T03:50:24.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[29284]: audit 2024-03-26T03:50:24.430571+0000 mon.a (mon.0) 1530 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi175-64406-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:24.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[29284]: audit 2024-03-26T03:50:24.430694+0000 mon.a (mon.0) 1531 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64438-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:24.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[29284]: audit 2024-03-26T03:50:24.430811+0000 mon.a (mon.0) 1532 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-15", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:50:24.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[29284]: audit 2024-03-26T03:50:24.430918+0000 mon.a (mon.0) 1533 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi175-64544-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:24.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[29284]: cluster 2024-03-26T03:50:24.439144+0000 mon.a (mon.0) 1534 : cluster [DBG] osdmap e168: 8 total, 8 up, 8 in 2024-03-26T03:50:24.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[29284]: audit 2024-03-26T03:50:24.443297+0000 mon.c (mon.2) 318 : audit [INF] from='client.? 172.21.15.175:0/4035522258' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi175-64544-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi175-64544-36"}]: dispatch 2024-03-26T03:50:24.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[29284]: audit 2024-03-26T03:50:24.444171+0000 mon.a (mon.0) 1535 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-15"}]: dispatch 2024-03-26T03:50:24.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[29284]: audit 2024-03-26T03:50:24.445131+0000 mon.a (mon.0) 1536 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi175-64544-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi175-64544-36"}]: dispatch 2024-03-26T03:50:24.747 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:24 smithi175 bash[29284]: audit 2024-03-26T03:50:24.451125+0000 mon.a (mon.0) 1537 : audit [INF] from='client.? 172.21.15.175:0/4044186109' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21"}]: dispatch 2024-03-26T03:50:25.458 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsStart 2024-03-26T03:50:25.458 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 1 0 2024-03-26T03:50:25.458 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 10 0 2024-03-26T03:50:25.458 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 13 0 2024-03-26T03:50:25.458 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 7 0 2024-03-26T03:50:25.459 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 14 0 2024-03-26T03:50:25.459 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 0 0 2024-03-26T03:50:25.459 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 15 0 2024-03-26T03:50:25.459 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 11 0 2024-03-26T03:50:25.459 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 5 0 2024-03-26T03:50:25.459 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 8 0 2024-03-26T03:50:25.459 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 6 0 2024-03-26T03:50:25.459 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 3 0 2024-03-26T03:50:25.459 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 4 0 2024-03-26T03:50:25.459 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 12 0 2024-03-26T03:50:25.459 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 9 0 2024-03-26T03:50:25.459 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 2 0 2024-03-26T03:50:25.459 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-03-26T03:50:25.459 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (58 ms) 2024-03-26T03:50:25.460 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [----------] 3 tests from LibRadosListEC (1192 ms total) 2024-03-26T03:50:25.460 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 2024-03-26T03:50:25.460 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [----------] 1 test from LibRadosListNP 2024-03-26T03:50:25.460 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2024-03-26T03:50:25.460 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (3008 ms) 2024-03-26T03:50:25.460 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [----------] 1 test from LibRadosListNP (3008 ms total) 2024-03-26T03:50:25.460 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: 2024-03-26T03:50:25.460 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [----------] Global test environment tear-down 2024-03-26T03:50:25.460 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [==========] 11 tests from 3 test suites ran. (93217 ms total) 2024-03-26T03:50:25.460 INFO:tasks.workunit.client.0.smithi175.stdout: api_list: [ PASSED ] 11 tests. 2024-03-26T03:50:25.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:25 smithi190 bash[23392]: audit 2024-03-26T03:50:24.494799+0000 mon.c (mon.2) 319 : audit [INF] from='client.? 172.21.15.175:0/3797846564' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi175-64398-29"}]: dispatch 2024-03-26T03:50:25.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:25 smithi190 bash[23392]: audit 2024-03-26T03:50:24.495249+0000 mon.a (mon.0) 1538 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi175-64398-29"}]: dispatch 2024-03-26T03:50:25.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:25 smithi190 bash[23392]: audit 2024-03-26T03:50:24.498172+0000 mon.c (mon.2) 320 : audit [INF] from='client.? 172.21.15.175:0/2266041579' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi175-64438-3","pool2":"test-rados-api-smithi175-64438-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-03-26T03:50:25.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:25 smithi190 bash[23392]: audit 2024-03-26T03:50:24.498526+0000 mon.c (mon.2) 321 : audit [INF] from='client.? 172.21.15.175:0/3797846564' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi175-64398-29"}]: dispatch 2024-03-26T03:50:25.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:25 smithi190 bash[23392]: audit 2024-03-26T03:50:24.499454+0000 mon.a (mon.0) 1539 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi175-64438-3","pool2":"test-rados-api-smithi175-64438-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-03-26T03:50:25.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:25 smithi190 bash[23392]: audit 2024-03-26T03:50:24.499724+0000 mon.a (mon.0) 1540 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi175-64398-29"}]: dispatch 2024-03-26T03:50:25.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:25 smithi190 bash[23392]: audit 2024-03-26T03:50:24.500616+0000 mon.c (mon.2) 322 : audit [INF] from='client.? 172.21.15.175:0/3797846564' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi175-64398-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:25.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:25 smithi190 bash[23392]: audit 2024-03-26T03:50:24.501045+0000 mon.a (mon.0) 1541 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi175-64398-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:25.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:25 smithi190 bash[23392]: audit 2024-03-26T03:50:25.434240+0000 mon.a (mon.0) 1542 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-15"}]': finished 2024-03-26T03:50:25.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:25 smithi190 bash[23392]: audit 2024-03-26T03:50:25.434449+0000 mon.a (mon.0) 1543 : audit [INF] from='client.? 172.21.15.175:0/4044186109' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21"}]': finished 2024-03-26T03:50:25.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:25 smithi190 bash[23392]: audit 2024-03-26T03:50:25.434631+0000 mon.a (mon.0) 1544 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi175-64438-3","pool2":"test-rados-api-smithi175-64438-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-03-26T03:50:25.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:25 smithi175 bash[21527]: audit 2024-03-26T03:50:24.494799+0000 mon.c (mon.2) 319 : audit [INF] from='client.? 172.21.15.175:0/3797846564' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi175-64398-29"}]: dispatch 2024-03-26T03:50:25.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:25 smithi175 bash[21527]: audit 2024-03-26T03:50:24.495249+0000 mon.a (mon.0) 1538 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi175-64398-29"}]: dispatch 2024-03-26T03:50:25.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:25 smithi175 bash[21527]: audit 2024-03-26T03:50:24.498172+0000 mon.c (mon.2) 320 : audit [INF] from='client.? 172.21.15.175:0/2266041579' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi175-64438-3","pool2":"test-rados-api-smithi175-64438-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-03-26T03:50:25.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:25 smithi175 bash[21527]: audit 2024-03-26T03:50:24.498526+0000 mon.c (mon.2) 321 : audit [INF] from='client.? 172.21.15.175:0/3797846564' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi175-64398-29"}]: dispatch 2024-03-26T03:50:25.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:25 smithi175 bash[21527]: audit 2024-03-26T03:50:24.499454+0000 mon.a (mon.0) 1539 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi175-64438-3","pool2":"test-rados-api-smithi175-64438-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-03-26T03:50:25.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:25 smithi175 bash[21527]: audit 2024-03-26T03:50:24.499724+0000 mon.a (mon.0) 1540 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi175-64398-29"}]: dispatch 2024-03-26T03:50:25.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:25 smithi175 bash[21527]: audit 2024-03-26T03:50:24.500616+0000 mon.c (mon.2) 322 : audit [INF] from='client.? 172.21.15.175:0/3797846564' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi175-64398-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:25.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:25 smithi175 bash[21527]: audit 2024-03-26T03:50:24.501045+0000 mon.a (mon.0) 1541 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi175-64398-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:25.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:25 smithi175 bash[21527]: audit 2024-03-26T03:50:25.434240+0000 mon.a (mon.0) 1542 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-15"}]': finished 2024-03-26T03:50:25.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:25 smithi175 bash[21527]: audit 2024-03-26T03:50:25.434449+0000 mon.a (mon.0) 1543 : audit [INF] from='client.? 172.21.15.175:0/4044186109' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21"}]': finished 2024-03-26T03:50:25.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:25 smithi175 bash[21527]: audit 2024-03-26T03:50:25.434631+0000 mon.a (mon.0) 1544 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi175-64438-3","pool2":"test-rados-api-smithi175-64438-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-03-26T03:50:25.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:25 smithi175 bash[29284]: audit 2024-03-26T03:50:24.494799+0000 mon.c (mon.2) 319 : audit [INF] from='client.? 172.21.15.175:0/3797846564' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi175-64398-29"}]: dispatch 2024-03-26T03:50:25.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:25 smithi175 bash[29284]: audit 2024-03-26T03:50:24.495249+0000 mon.a (mon.0) 1538 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi175-64398-29"}]: dispatch 2024-03-26T03:50:25.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:25 smithi175 bash[29284]: audit 2024-03-26T03:50:24.498172+0000 mon.c (mon.2) 320 : audit [INF] from='client.? 172.21.15.175:0/2266041579' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi175-64438-3","pool2":"test-rados-api-smithi175-64438-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-03-26T03:50:25.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:25 smithi175 bash[29284]: audit 2024-03-26T03:50:24.498526+0000 mon.c (mon.2) 321 : audit [INF] from='client.? 172.21.15.175:0/3797846564' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi175-64398-29"}]: dispatch 2024-03-26T03:50:25.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:25 smithi175 bash[29284]: audit 2024-03-26T03:50:24.499454+0000 mon.a (mon.0) 1539 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi175-64438-3","pool2":"test-rados-api-smithi175-64438-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-03-26T03:50:25.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:25 smithi175 bash[29284]: audit 2024-03-26T03:50:24.499724+0000 mon.a (mon.0) 1540 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi175-64398-29"}]: dispatch 2024-03-26T03:50:25.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:25 smithi175 bash[29284]: audit 2024-03-26T03:50:24.500616+0000 mon.c (mon.2) 322 : audit [INF] from='client.? 172.21.15.175:0/3797846564' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi175-64398-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:25.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:25 smithi175 bash[29284]: audit 2024-03-26T03:50:24.501045+0000 mon.a (mon.0) 1541 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi175-64398-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:25.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:25 smithi175 bash[29284]: audit 2024-03-26T03:50:25.434240+0000 mon.a (mon.0) 1542 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-15"}]': finished 2024-03-26T03:50:25.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:25 smithi175 bash[29284]: audit 2024-03-26T03:50:25.434449+0000 mon.a (mon.0) 1543 : audit [INF] from='client.? 172.21.15.175:0/4044186109' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21"}]': finished 2024-03-26T03:50:25.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:25 smithi175 bash[29284]: audit 2024-03-26T03:50:25.434631+0000 mon.a (mon.0) 1544 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi175-64438-3","pool2":"test-rados-api-smithi175-64438-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-03-26T03:50:26.460 INFO:tasks.workunit.client.0.smithi175.stdout:pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-03-26T03:50:26.460 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (2011 ms) 2024-03-26T03:50:26.460 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (8880 ms total) 2024-03-26T03:50:26.461 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: 2024-03-26T03:50:26.461 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-03-26T03:50:26.461 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-03-26T03:50:26.461 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (4113 ms) 2024-03-26T03:50:26.461 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-03-26T03:50:26.461 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (3957 ms) 2024-03-26T03:50:26.461 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-03-26T03:50:26.461 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (3919 ms) 2024-03-26T03:50:26.461 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (11989 ms total) 2024-03-26T03:50:26.461 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: 2024-03-26T03:50:26.461 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2024-03-26T03:50:26.461 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [==========] 21 tests from 5 test suites ran. (94142 ms total) 2024-03-26T03:50:26.461 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ PASSED ] 20 tests. 2024-03-26T03:50:26.461 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ SKIPPED ] 1 test, listed below: 2024-03-26T03:50:26.462 INFO:tasks.workunit.client.0.smithi175.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-03-26T03:50:26.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:26 smithi190 bash[23392]: cluster 2024-03-26T03:50:24.617303+0000 mgr.y (mgr.24431) 253 : cluster [DBG] pgmap v249: 356 pgs: 42 unknown, 314 active+clean; 584 KiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:50:26.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:26 smithi190 bash[23392]: audit 2024-03-26T03:50:25.434832+0000 mon.a (mon.0) 1545 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi175-64398-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:26.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:26 smithi190 bash[23392]: audit 2024-03-26T03:50:25.448228+0000 mon.c (mon.2) 323 : audit [INF] from='client.? 172.21.15.175:0/3797846564' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi175-64398-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi175-64398-29"}]: dispatch 2024-03-26T03:50:26.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:26 smithi190 bash[23392]: cluster 2024-03-26T03:50:25.456489+0000 mon.a (mon.0) 1546 : cluster [DBG] osdmap e169: 8 total, 8 up, 8 in 2024-03-26T03:50:26.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:26 smithi190 bash[23392]: audit 2024-03-26T03:50:25.465297+0000 mon.a (mon.0) 1547 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-15", "mode": "writeback"}]: dispatch 2024-03-26T03:50:26.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:26 smithi190 bash[23392]: audit 2024-03-26T03:50:25.473063+0000 mon.a (mon.0) 1548 : audit [INF] from='client.? 172.21.15.175:0/4044186109' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21"}]: dispatch 2024-03-26T03:50:26.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:26 smithi190 bash[23392]: audit 2024-03-26T03:50:25.474103+0000 mon.a (mon.0) 1549 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi175-64398-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi175-64398-29"}]: dispatch 2024-03-26T03:50:26.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:26 smithi190 bash[23392]: cluster 2024-03-26T03:50:26.417516+0000 mon.a (mon.0) 1550 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:50:26.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:26 smithi190 bash[23392]: audit 2024-03-26T03:50:26.422361+0000 mon.a (mon.0) 1551 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi175-64544-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi175-64544-36"}]': finished 2024-03-26T03:50:26.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:26 smithi190 bash[23392]: audit 2024-03-26T03:50:26.422652+0000 mon.a (mon.0) 1552 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-15", "mode": "writeback"}]': finished 2024-03-26T03:50:26.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:26 smithi190 bash[23392]: audit 2024-03-26T03:50:26.422804+0000 mon.a (mon.0) 1553 : audit [INF] from='client.? 172.21.15.175:0/4044186109' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21"}]': finished 2024-03-26T03:50:26.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:26 smithi190 bash[23392]: cluster 2024-03-26T03:50:26.453400+0000 mon.a (mon.0) 1554 : cluster [DBG] osdmap e170: 8 total, 8 up, 8 in 2024-03-26T03:50:26.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:26 smithi190 bash[23392]: audit 2024-03-26T03:50:26.454910+0000 mon.c (mon.2) 324 : audit [INF] from='client.? 172.21.15.175:0/1834855509' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi175-64406-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:26.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:26 smithi190 bash[23392]: audit 2024-03-26T03:50:26.458270+0000 mon.a (mon.0) 1555 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi175-64406-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:26.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:26 smithi175 bash[21527]: cluster 2024-03-26T03:50:24.617303+0000 mgr.y (mgr.24431) 253 : cluster [DBG] pgmap v249: 356 pgs: 42 unknown, 314 active+clean; 584 KiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:50:26.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:26 smithi175 bash[21527]: audit 2024-03-26T03:50:25.434832+0000 mon.a (mon.0) 1545 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi175-64398-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:26.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:26 smithi175 bash[21527]: audit 2024-03-26T03:50:25.448228+0000 mon.c (mon.2) 323 : audit [INF] from='client.? 172.21.15.175:0/3797846564' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi175-64398-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi175-64398-29"}]: dispatch 2024-03-26T03:50:26.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:26 smithi175 bash[21527]: cluster 2024-03-26T03:50:25.456489+0000 mon.a (mon.0) 1546 : cluster [DBG] osdmap e169: 8 total, 8 up, 8 in 2024-03-26T03:50:26.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:26 smithi175 bash[21527]: audit 2024-03-26T03:50:25.465297+0000 mon.a (mon.0) 1547 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-15", "mode": "writeback"}]: dispatch 2024-03-26T03:50:26.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:26 smithi175 bash[21527]: audit 2024-03-26T03:50:25.473063+0000 mon.a (mon.0) 1548 : audit [INF] from='client.? 172.21.15.175:0/4044186109' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21"}]: dispatch 2024-03-26T03:50:26.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:26 smithi175 bash[21527]: audit 2024-03-26T03:50:25.474103+0000 mon.a (mon.0) 1549 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi175-64398-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi175-64398-29"}]: dispatch 2024-03-26T03:50:26.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:26 smithi175 bash[21527]: cluster 2024-03-26T03:50:26.417516+0000 mon.a (mon.0) 1550 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:50:26.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:26 smithi175 bash[21527]: audit 2024-03-26T03:50:26.422361+0000 mon.a (mon.0) 1551 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi175-64544-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi175-64544-36"}]': finished 2024-03-26T03:50:26.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:26 smithi175 bash[21527]: audit 2024-03-26T03:50:26.422652+0000 mon.a (mon.0) 1552 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-15", "mode": "writeback"}]': finished 2024-03-26T03:50:26.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:26 smithi175 bash[21527]: audit 2024-03-26T03:50:26.422804+0000 mon.a (mon.0) 1553 : audit [INF] from='client.? 172.21.15.175:0/4044186109' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21"}]': finished 2024-03-26T03:50:26.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:26 smithi175 bash[21527]: cluster 2024-03-26T03:50:26.453400+0000 mon.a (mon.0) 1554 : cluster [DBG] osdmap e170: 8 total, 8 up, 8 in 2024-03-26T03:50:26.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:26 smithi175 bash[21527]: audit 2024-03-26T03:50:26.454910+0000 mon.c (mon.2) 324 : audit [INF] from='client.? 172.21.15.175:0/1834855509' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi175-64406-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:26.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:26 smithi175 bash[21527]: audit 2024-03-26T03:50:26.458270+0000 mon.a (mon.0) 1555 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi175-64406-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:26.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:26 smithi175 bash[29284]: cluster 2024-03-26T03:50:24.617303+0000 mgr.y (mgr.24431) 253 : cluster [DBG] pgmap v249: 356 pgs: 42 unknown, 314 active+clean; 584 KiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:50:26.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:26 smithi175 bash[29284]: audit 2024-03-26T03:50:25.434832+0000 mon.a (mon.0) 1545 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi175-64398-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:26.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:26 smithi175 bash[29284]: audit 2024-03-26T03:50:25.448228+0000 mon.c (mon.2) 323 : audit [INF] from='client.? 172.21.15.175:0/3797846564' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi175-64398-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi175-64398-29"}]: dispatch 2024-03-26T03:50:26.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:26 smithi175 bash[29284]: cluster 2024-03-26T03:50:25.456489+0000 mon.a (mon.0) 1546 : cluster [DBG] osdmap e169: 8 total, 8 up, 8 in 2024-03-26T03:50:26.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:26 smithi175 bash[29284]: audit 2024-03-26T03:50:25.465297+0000 mon.a (mon.0) 1547 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-15", "mode": "writeback"}]: dispatch 2024-03-26T03:50:26.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:26 smithi175 bash[29284]: audit 2024-03-26T03:50:25.473063+0000 mon.a (mon.0) 1548 : audit [INF] from='client.? 172.21.15.175:0/4044186109' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21"}]: dispatch 2024-03-26T03:50:26.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:26 smithi175 bash[29284]: audit 2024-03-26T03:50:25.474103+0000 mon.a (mon.0) 1549 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi175-64398-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi175-64398-29"}]: dispatch 2024-03-26T03:50:26.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:26 smithi175 bash[29284]: cluster 2024-03-26T03:50:26.417516+0000 mon.a (mon.0) 1550 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:50:26.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:26 smithi175 bash[29284]: audit 2024-03-26T03:50:26.422361+0000 mon.a (mon.0) 1551 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi175-64544-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi175-64544-36"}]': finished 2024-03-26T03:50:26.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:26 smithi175 bash[29284]: audit 2024-03-26T03:50:26.422652+0000 mon.a (mon.0) 1552 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-15", "mode": "writeback"}]': finished 2024-03-26T03:50:26.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:26 smithi175 bash[29284]: audit 2024-03-26T03:50:26.422804+0000 mon.a (mon.0) 1553 : audit [INF] from='client.? 172.21.15.175:0/4044186109' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi175-64636-21"}]': finished 2024-03-26T03:50:26.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:26 smithi175 bash[29284]: cluster 2024-03-26T03:50:26.453400+0000 mon.a (mon.0) 1554 : cluster [DBG] osdmap e170: 8 total, 8 up, 8 in 2024-03-26T03:50:26.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:26 smithi175 bash[29284]: audit 2024-03-26T03:50:26.454910+0000 mon.c (mon.2) 324 : audit [INF] from='client.? 172.21.15.175:0/1834855509' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi175-64406-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:26.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:26 smithi175 bash[29284]: audit 2024-03-26T03:50:26.458270+0000 mon.a (mon.0) 1555 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi175-64406-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:28.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:28 smithi190 bash[23392]: cluster 2024-03-26T03:50:26.618230+0000 mgr.y (mgr.24431) 254 : cluster [DBG] pgmap v252: 332 pgs: 40 unknown, 292 active+clean; 584 KiB data, 597 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:50:28.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:28 smithi190 bash[23392]: audit 2024-03-26T03:50:27.426754+0000 mon.a (mon.0) 1556 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi175-64398-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi175-64398-29"}]': finished 2024-03-26T03:50:28.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:28 smithi190 bash[23392]: audit 2024-03-26T03:50:27.426976+0000 mon.a (mon.0) 1557 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi175-64406-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:28.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:28 smithi190 bash[23392]: cluster 2024-03-26T03:50:27.436467+0000 mon.a (mon.0) 1558 : cluster [DBG] osdmap e171: 8 total, 8 up, 8 in 2024-03-26T03:50:28.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:28 smithi190 bash[23392]: audit 2024-03-26T03:50:27.520659+0000 mon.a (mon.0) 1559 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:50:28.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:28 smithi175 bash[21527]: cluster 2024-03-26T03:50:26.618230+0000 mgr.y (mgr.24431) 254 : cluster [DBG] pgmap v252: 332 pgs: 40 unknown, 292 active+clean; 584 KiB data, 597 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:50:28.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:28 smithi175 bash[21527]: audit 2024-03-26T03:50:27.426754+0000 mon.a (mon.0) 1556 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi175-64398-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi175-64398-29"}]': finished 2024-03-26T03:50:28.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:28 smithi175 bash[21527]: audit 2024-03-26T03:50:27.426976+0000 mon.a (mon.0) 1557 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi175-64406-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:28.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:28 smithi175 bash[21527]: cluster 2024-03-26T03:50:27.436467+0000 mon.a (mon.0) 1558 : cluster [DBG] osdmap e171: 8 total, 8 up, 8 in 2024-03-26T03:50:28.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:28 smithi175 bash[21527]: audit 2024-03-26T03:50:27.520659+0000 mon.a (mon.0) 1559 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:50:28.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:28 smithi175 bash[29284]: cluster 2024-03-26T03:50:26.618230+0000 mgr.y (mgr.24431) 254 : cluster [DBG] pgmap v252: 332 pgs: 40 unknown, 292 active+clean; 584 KiB data, 597 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:50:28.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:28 smithi175 bash[29284]: audit 2024-03-26T03:50:27.426754+0000 mon.a (mon.0) 1556 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi175-64398-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi175-64398-29"}]': finished 2024-03-26T03:50:28.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:28 smithi175 bash[29284]: audit 2024-03-26T03:50:27.426976+0000 mon.a (mon.0) 1557 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi175-64406-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:28.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:28 smithi175 bash[29284]: cluster 2024-03-26T03:50:27.436467+0000 mon.a (mon.0) 1558 : cluster [DBG] osdmap e171: 8 total, 8 up, 8 in 2024-03-26T03:50:28.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:28 smithi175 bash[29284]: audit 2024-03-26T03:50:27.520659+0000 mon.a (mon.0) 1559 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:50:29.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:50:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:50:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:50:29.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:29 smithi190 bash[23392]: audit 2024-03-26T03:50:28.439704+0000 mon.a (mon.0) 1560 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:50:29.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:29 smithi190 bash[23392]: cluster 2024-03-26T03:50:28.448227+0000 mon.a (mon.0) 1561 : cluster [DBG] osdmap e172: 8 total, 8 up, 8 in 2024-03-26T03:50:29.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:29 smithi190 bash[23392]: audit 2024-03-26T03:50:28.449068+0000 mon.a (mon.0) 1562 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-15"}]: dispatch 2024-03-26T03:50:29.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:29 smithi190 bash[23392]: audit 2024-03-26T03:50:28.483766+0000 mon.c (mon.2) 325 : audit [INF] from='client.? 172.21.15.175:0/4035522258' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi175-64544-36"}]: dispatch 2024-03-26T03:50:29.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:29 smithi190 bash[23392]: audit 2024-03-26T03:50:28.484536+0000 mon.a (mon.0) 1563 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi175-64544-36"}]: dispatch 2024-03-26T03:50:29.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:29 smithi175 bash[21527]: audit 2024-03-26T03:50:28.439704+0000 mon.a (mon.0) 1560 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:50:29.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:29 smithi175 bash[21527]: cluster 2024-03-26T03:50:28.448227+0000 mon.a (mon.0) 1561 : cluster [DBG] osdmap e172: 8 total, 8 up, 8 in 2024-03-26T03:50:29.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:29 smithi175 bash[21527]: audit 2024-03-26T03:50:28.449068+0000 mon.a (mon.0) 1562 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-15"}]: dispatch 2024-03-26T03:50:29.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:29 smithi175 bash[21527]: audit 2024-03-26T03:50:28.483766+0000 mon.c (mon.2) 325 : audit [INF] from='client.? 172.21.15.175:0/4035522258' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi175-64544-36"}]: dispatch 2024-03-26T03:50:29.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:29 smithi175 bash[21527]: audit 2024-03-26T03:50:28.484536+0000 mon.a (mon.0) 1563 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi175-64544-36"}]: dispatch 2024-03-26T03:50:29.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:29 smithi175 bash[29284]: audit 2024-03-26T03:50:28.439704+0000 mon.a (mon.0) 1560 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:50:29.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:29 smithi175 bash[29284]: cluster 2024-03-26T03:50:28.448227+0000 mon.a (mon.0) 1561 : cluster [DBG] osdmap e172: 8 total, 8 up, 8 in 2024-03-26T03:50:29.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:29 smithi175 bash[29284]: audit 2024-03-26T03:50:28.449068+0000 mon.a (mon.0) 1562 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-15"}]: dispatch 2024-03-26T03:50:29.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:29 smithi175 bash[29284]: audit 2024-03-26T03:50:28.483766+0000 mon.c (mon.2) 325 : audit [INF] from='client.? 172.21.15.175:0/4035522258' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi175-64544-36"}]: dispatch 2024-03-26T03:50:29.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:29 smithi175 bash[29284]: audit 2024-03-26T03:50:28.484536+0000 mon.a (mon.0) 1563 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi175-64544-36"}]: dispatch 2024-03-26T03:50:30.240 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:50:29 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:50:30.521 INFO:tasks.workunit.client.0.smithi175.stdout:P (69085 ms total) 2024-03-26T03:50:30.521 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: 2024-03-26T03:50:30.521 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2024-03-26T03:50:30.521 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-03-26T03:50:30.521 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (211 ms) 2024-03-26T03:50:30.521 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (211 ms total) 2024-03-26T03:50:30.521 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: 2024-03-26T03:50:30.521 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-03-26T03:50:30.521 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2024-03-26T03:50:30.521 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (67 ms) 2024-03-26T03:50:30.521 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2024-03-26T03:50:30.522 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (14 ms) 2024-03-26T03:50:30.522 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (81 ms total) 2024-03-26T03:50:30.522 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: 2024-03-26T03:50:30.522 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-03-26T03:50:30.522 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2024-03-26T03:50:30.522 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (64 ms) 2024-03-26T03:50:30.522 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2024-03-26T03:50:30.522 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (3 ms) 2024-03-26T03:50:30.522 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (67 ms total) 2024-03-26T03:50:30.522 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: 2024-03-26T03:50:30.522 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-03-26T03:50:30.522 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2024-03-26T03:50:30.522 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (48 ms) 2024-03-26T03:50:30.523 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2024-03-26T03:50:30.523 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (9 ms) 2024-03-26T03:50:30.523 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (57 ms total) 2024-03-26T03:50:30.523 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: 2024-03-26T03:50:30.523 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2024-03-26T03:50:30.523 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-03-26T03:50:30.523 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (1061 ms) 2024-03-26T03:50:30.523 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (1061 ms total) 2024-03-26T03:50:30.523 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: 2024-03-26T03:50:30.523 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [----------] Global test environment tear-down 2024-03-26T03:50:30.523 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (98209 ms total) 2024-03-26T03:50:30.523 INFO:tasks.workunit.client.0.smithi175.stdout: api_misc_pp: [ PASSED ] 31 tests. 2024-03-26T03:50:30.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:30 smithi190 bash[23392]: cluster 2024-03-26T03:50:28.619201+0000 mgr.y (mgr.24431) 255 : cluster [DBG] pgmap v255: 300 pgs: 8 unknown, 292 active+clean; 584 KiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T03:50:30.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:30 smithi190 bash[23392]: cluster 2024-03-26T03:50:29.439805+0000 mon.a (mon.0) 1564 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:50:30.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:30 smithi190 bash[23392]: cluster 2024-03-26T03:50:29.441924+0000 mon.a (mon.0) 1565 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:50:30.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:30 smithi190 bash[23392]: audit 2024-03-26T03:50:29.451574+0000 mon.a (mon.0) 1566 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-15"}]': finished 2024-03-26T03:50:30.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:30 smithi190 bash[23392]: audit 2024-03-26T03:50:29.451793+0000 mon.a (mon.0) 1567 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi175-64544-36"}]': finished 2024-03-26T03:50:30.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:30 smithi190 bash[23392]: cluster 2024-03-26T03:50:29.463604+0000 mon.a (mon.0) 1568 : cluster [DBG] osdmap e173: 8 total, 8 up, 8 in 2024-03-26T03:50:30.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:30 smithi190 bash[23392]: audit 2024-03-26T03:50:29.469068+0000 mon.c (mon.2) 326 : audit [INF] from='client.? 172.21.15.175:0/3797846564' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi175-64398-29"}]: dispatch 2024-03-26T03:50:30.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:30 smithi190 bash[23392]: audit 2024-03-26T03:50:29.469708+0000 mon.c (mon.2) 327 : audit [INF] from='client.? 172.21.15.175:0/4035522258' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi175-64544-36"}]: dispatch 2024-03-26T03:50:30.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:30 smithi190 bash[23392]: audit 2024-03-26T03:50:29.470990+0000 mon.a (mon.0) 1569 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi175-64398-29"}]: dispatch 2024-03-26T03:50:30.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:30 smithi190 bash[23392]: audit 2024-03-26T03:50:29.471421+0000 mon.a (mon.0) 1570 : audit [INF] from='client.? 172.21.15.175:0/870801209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi175-64406-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:30.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:30 smithi190 bash[23392]: audit 2024-03-26T03:50:29.471811+0000 mon.a (mon.0) 1571 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi175-64544-36"}]: dispatch 2024-03-26T03:50:30.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:30 smithi175 bash[29284]: cluster 2024-03-26T03:50:28.619201+0000 mgr.y (mgr.24431) 255 : cluster [DBG] pgmap v255: 300 pgs: 8 unknown, 292 active+clean; 584 KiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T03:50:30.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:30 smithi175 bash[29284]: cluster 2024-03-26T03:50:29.439805+0000 mon.a (mon.0) 1564 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:50:30.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:30 smithi175 bash[29284]: cluster 2024-03-26T03:50:29.441924+0000 mon.a (mon.0) 1565 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:50:30.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:30 smithi175 bash[29284]: audit 2024-03-26T03:50:29.451574+0000 mon.a (mon.0) 1566 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-15"}]': finished 2024-03-26T03:50:30.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:30 smithi175 bash[29284]: audit 2024-03-26T03:50:29.451793+0000 mon.a (mon.0) 1567 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi175-64544-36"}]': finished 2024-03-26T03:50:30.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:30 smithi175 bash[29284]: cluster 2024-03-26T03:50:29.463604+0000 mon.a (mon.0) 1568 : cluster [DBG] osdmap e173: 8 total, 8 up, 8 in 2024-03-26T03:50:30.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:30 smithi175 bash[29284]: audit 2024-03-26T03:50:29.469068+0000 mon.c (mon.2) 326 : audit [INF] from='client.? 172.21.15.175:0/3797846564' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi175-64398-29"}]: dispatch 2024-03-26T03:50:30.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:30 smithi175 bash[29284]: audit 2024-03-26T03:50:29.469708+0000 mon.c (mon.2) 327 : audit [INF] from='client.? 172.21.15.175:0/4035522258' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi175-64544-36"}]: dispatch 2024-03-26T03:50:30.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:30 smithi175 bash[29284]: audit 2024-03-26T03:50:29.470990+0000 mon.a (mon.0) 1569 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi175-64398-29"}]: dispatch 2024-03-26T03:50:30.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:30 smithi175 bash[29284]: audit 2024-03-26T03:50:29.471421+0000 mon.a (mon.0) 1570 : audit [INF] from='client.? 172.21.15.175:0/870801209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi175-64406-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:30.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:30 smithi175 bash[29284]: audit 2024-03-26T03:50:29.471811+0000 mon.a (mon.0) 1571 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi175-64544-36"}]: dispatch 2024-03-26T03:50:30.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:30 smithi175 bash[21527]: cluster 2024-03-26T03:50:28.619201+0000 mgr.y (mgr.24431) 255 : cluster [DBG] pgmap v255: 300 pgs: 8 unknown, 292 active+clean; 584 KiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T03:50:30.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:30 smithi175 bash[21527]: cluster 2024-03-26T03:50:29.439805+0000 mon.a (mon.0) 1564 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:50:30.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:30 smithi175 bash[21527]: cluster 2024-03-26T03:50:29.441924+0000 mon.a (mon.0) 1565 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:50:30.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:30 smithi175 bash[21527]: audit 2024-03-26T03:50:29.451574+0000 mon.a (mon.0) 1566 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-15"}]': finished 2024-03-26T03:50:30.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:30 smithi175 bash[21527]: audit 2024-03-26T03:50:29.451793+0000 mon.a (mon.0) 1567 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi175-64544-36"}]': finished 2024-03-26T03:50:30.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:30 smithi175 bash[21527]: cluster 2024-03-26T03:50:29.463604+0000 mon.a (mon.0) 1568 : cluster [DBG] osdmap e173: 8 total, 8 up, 8 in 2024-03-26T03:50:30.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:30 smithi175 bash[21527]: audit 2024-03-26T03:50:29.469068+0000 mon.c (mon.2) 326 : audit [INF] from='client.? 172.21.15.175:0/3797846564' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi175-64398-29"}]: dispatch 2024-03-26T03:50:30.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:30 smithi175 bash[21527]: audit 2024-03-26T03:50:29.469708+0000 mon.c (mon.2) 327 : audit [INF] from='client.? 172.21.15.175:0/4035522258' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi175-64544-36"}]: dispatch 2024-03-26T03:50:30.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:30 smithi175 bash[21527]: audit 2024-03-26T03:50:29.470990+0000 mon.a (mon.0) 1569 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi175-64398-29"}]: dispatch 2024-03-26T03:50:30.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:30 smithi175 bash[21527]: audit 2024-03-26T03:50:29.471421+0000 mon.a (mon.0) 1570 : audit [INF] from='client.? 172.21.15.175:0/870801209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi175-64406-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:30.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:30 smithi175 bash[21527]: audit 2024-03-26T03:50:29.471811+0000 mon.a (mon.0) 1571 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi175-64544-36"}]: dispatch 2024-03-26T03:50:31.482 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: Running main() from gmock_main.cc 2024-03-26T03:50:31.483 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [==========] Running 57 tests from 4 test suites. 2024-03-26T03:50:31.483 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [----------] Global test environment set-up. 2024-03-26T03:50:31.483 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio 2024-03-26T03:50:31.483 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2024-03-26T03:50:31.483 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (2158 ms) 2024-03-26T03:50:31.483 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2024-03-26T03:50:31.483 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (19096 ms) 2024-03-26T03:50:31.483 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2024-03-26T03:50:31.483 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (5996 ms) 2024-03-26T03:50:31.483 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2024-03-26T03:50:31.483 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (2969 ms) 2024-03-26T03:50:31.483 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2024-03-26T03:50:31.484 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (3012 ms) 2024-03-26T03:50:31.484 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2024-03-26T03:50:31.484 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (2959 ms) 2024-03-26T03:50:31.484 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2024-03-26T03:50:31.484 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (3054 ms) 2024-03-26T03:50:31.484 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-03-26T03:50:31.484 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (2985 ms) 2024-03-26T03:50:31.484 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2024-03-26T03:50:31.484 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (2977 ms) 2024-03-26T03:50:31.484 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2024-03-26T03:50:31.484 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (3027 ms) 2024-03-26T03:50:31.484 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2024-03-26T03:50:31.484 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (2979 ms) 2024-03-26T03:50:31.484 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2024-03-26T03:50:31.485 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (2985 ms) 2024-03-26T03:50:31.485 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2024-03-26T03:50:31.485 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (3026 ms) 2024-03-26T03:50:31.485 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-03-26T03:50:31.485 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (3010 ms) 2024-03-26T03:50:31.485 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-03-26T03:50:31.485 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (3099 ms) 2024-03-26T03:50:31.485 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-03-26T03:50:31.485 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (3009 ms) 2024-03-26T03:50:31.485 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-03-26T03:50:31.485 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (2862 ms) 2024-03-26T03:50:31.485 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2024-03-26T03:50:31.485 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (3012 ms) 2024-03-26T03:50:31.486 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2024-03-26T03:50:31.486 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (2996 ms) 2024-03-26T03:50:31.486 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2024-03-26T03:50:31.486 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (3030 ms) 2024-03-26T03:50:31.486 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2024-03-26T03:50:31.486 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (3004 ms) 2024-03-26T03:50:31.486 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2024-03-26T03:50:31.486 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (3012 ms) 2024-03-26T03:50:31.486 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2024-03-26T03:50:31.486 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (3052 ms) 2024-03-26T03:50:31.486 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2024-03-26T03:50:31.486 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (2913 ms) 2024-03-26T03:50:31.486 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2024-03-26T03:50:31.486 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (3024 ms) 2024-03-26T03:50:31.487 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2024-03-26T03:50:31.487 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (3005 ms) 2024-03-26T03:50:31.487 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2024-03-26T03:50:31.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:31 smithi175 bash[21527]: audit 2024-03-26T03:50:29.781775+0000 mgr.y (mgr.24431) 256 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:50:31.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:31 smithi175 bash[21527]: audit 2024-03-26T03:50:30.455630+0000 mon.a (mon.0) 1572 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi175-64398-29"}]': finished 2024-03-26T03:50:31.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:31 smithi175 bash[21527]: audit 2024-03-26T03:50:30.455785+0000 mon.a (mon.0) 1573 : audit [INF] from='client.? 172.21.15.175:0/870801209' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi175-64406-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:31.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:31 smithi175 bash[21527]: audit 2024-03-26T03:50:30.455989+0000 mon.a (mon.0) 1574 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi175-64544-36"}]': finished 2024-03-26T03:50:31.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:31 smithi175 bash[21527]: cluster 2024-03-26T03:50:30.463248+0000 mon.a (mon.0) 1575 : cluster [DBG] osdmap e174: 8 total, 8 up, 8 in 2024-03-26T03:50:31.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:31 smithi175 bash[21527]: audit 2024-03-26T03:50:30.469174+0000 mon.c (mon.2) 328 : audit [INF] from='client.? 172.21.15.175:0/3797846564' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi175-64398-29"}]: dispatch 2024-03-26T03:50:31.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:31 smithi175 bash[21527]: audit 2024-03-26T03:50:30.511387+0000 mon.a (mon.0) 1576 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi175-64398-29"}]: dispatch 2024-03-26T03:50:31.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:31 smithi175 bash[21527]: cluster 2024-03-26T03:50:30.620134+0000 mgr.y (mgr.24431) 257 : cluster [DBG] pgmap v258: 292 pgs: 32 creating+peering, 260 active+clean; 584 KiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:50:31.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:31 smithi175 bash[21527]: audit 2024-03-26T03:50:31.459840+0000 mon.a (mon.0) 1577 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi175-64398-29"}]': finished 2024-03-26T03:50:31.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:31 smithi175 bash[21527]: cluster 2024-03-26T03:50:31.469354+0000 mon.a (mon.0) 1578 : cluster [DBG] osdmap e175: 8 total, 8 up, 8 in 2024-03-26T03:50:31.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:31 smithi175 bash[21527]: audit 2024-03-26T03:50:31.479074+0000 mon.a (mon.0) 1579 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:31.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:31 smithi175 bash[29284]: audit 2024-03-26T03:50:29.781775+0000 mgr.y (mgr.24431) 256 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:50:31.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:31 smithi175 bash[29284]: audit 2024-03-26T03:50:30.455630+0000 mon.a (mon.0) 1572 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi175-64398-29"}]': finished 2024-03-26T03:50:31.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:31 smithi175 bash[29284]: audit 2024-03-26T03:50:30.455785+0000 mon.a (mon.0) 1573 : audit [INF] from='client.? 172.21.15.175:0/870801209' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi175-64406-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:31.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:31 smithi175 bash[29284]: audit 2024-03-26T03:50:30.455989+0000 mon.a (mon.0) 1574 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi175-64544-36"}]': finished 2024-03-26T03:50:31.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:31 smithi175 bash[29284]: cluster 2024-03-26T03:50:30.463248+0000 mon.a (mon.0) 1575 : cluster [DBG] osdmap e174: 8 total, 8 up, 8 in 2024-03-26T03:50:31.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:31 smithi175 bash[29284]: audit 2024-03-26T03:50:30.469174+0000 mon.c (mon.2) 328 : audit [INF] from='client.? 172.21.15.175:0/3797846564' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi175-64398-29"}]: dispatch 2024-03-26T03:50:31.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:31 smithi175 bash[29284]: audit 2024-03-26T03:50:30.511387+0000 mon.a (mon.0) 1576 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi175-64398-29"}]: dispatch 2024-03-26T03:50:31.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:31 smithi175 bash[29284]: cluster 2024-03-26T03:50:30.620134+0000 mgr.y (mgr.24431) 257 : cluster [DBG] pgmap v258: 292 pgs: 32 creating+peering, 260 active+clean; 584 KiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:50:31.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:31 smithi175 bash[29284]: audit 2024-03-26T03:50:31.459840+0000 mon.a (mon.0) 1577 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi175-64398-29"}]': finished 2024-03-26T03:50:31.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:31 smithi175 bash[29284]: cluster 2024-03-26T03:50:31.469354+0000 mon.a (mon.0) 1578 : cluster [DBG] osdmap e175: 8 total, 8 up, 8 in 2024-03-26T03:50:31.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:31 smithi175 bash[29284]: audit 2024-03-26T03:50:31.479074+0000 mon.a (mon.0) 1579 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:31.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:31 smithi190 bash[23392]: audit 2024-03-26T03:50:29.781775+0000 mgr.y (mgr.24431) 256 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:50:31.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:31 smithi190 bash[23392]: audit 2024-03-26T03:50:30.455630+0000 mon.a (mon.0) 1572 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi175-64398-29"}]': finished 2024-03-26T03:50:31.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:31 smithi190 bash[23392]: audit 2024-03-26T03:50:30.455785+0000 mon.a (mon.0) 1573 : audit [INF] from='client.? 172.21.15.175:0/870801209' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi175-64406-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:31.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:31 smithi190 bash[23392]: audit 2024-03-26T03:50:30.455989+0000 mon.a (mon.0) 1574 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi175-64544-36"}]': finished 2024-03-26T03:50:31.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:31 smithi190 bash[23392]: cluster 2024-03-26T03:50:30.463248+0000 mon.a (mon.0) 1575 : cluster [DBG] osdmap e174: 8 total, 8 up, 8 in 2024-03-26T03:50:31.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:31 smithi190 bash[23392]: audit 2024-03-26T03:50:30.469174+0000 mon.c (mon.2) 328 : audit [INF] from='client.? 172.21.15.175:0/3797846564' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi175-64398-29"}]: dispatch 2024-03-26T03:50:31.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:31 smithi190 bash[23392]: audit 2024-03-26T03:50:30.511387+0000 mon.a (mon.0) 1576 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi175-64398-29"}]: dispatch 2024-03-26T03:50:31.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:31 smithi190 bash[23392]: cluster 2024-03-26T03:50:30.620134+0000 mgr.y (mgr.24431) 257 : cluster [DBG] pgmap v258: 292 pgs: 32 creating+peering, 260 active+clean; 584 KiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:50:31.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:31 smithi190 bash[23392]: audit 2024-03-26T03:50:31.459840+0000 mon.a (mon.0) 1577 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi175-64398-29"}]': finished 2024-03-26T03:50:31.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:31 smithi190 bash[23392]: cluster 2024-03-26T03:50:31.469354+0000 mon.a (mon.0) 1578 : cluster [DBG] osdmap e175: 8 total, 8 up, 8 in 2024-03-26T03:50:31.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:31 smithi190 bash[23392]: audit 2024-03-26T03:50:31.479074+0000 mon.a (mon.0) 1579 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:32.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:32 smithi190 bash[23392]: audit 2024-03-26T03:50:31.520356+0000 mon.a (mon.0) 1580 : audit [INF] from='client.? 172.21.15.175:0/3653366758' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi175-64398-30"}]: dispatch 2024-03-26T03:50:32.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:32 smithi190 bash[23392]: audit 2024-03-26T03:50:31.522429+0000 mon.a (mon.0) 1581 : audit [INF] from='client.? 172.21.15.175:0/3653366758' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi175-64398-30"}]: dispatch 2024-03-26T03:50:32.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:32 smithi190 bash[23392]: audit 2024-03-26T03:50:31.522914+0000 mon.a (mon.0) 1582 : audit [INF] from='client.? 172.21.15.175:0/3653366758' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi175-64398-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:32.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:32 smithi190 bash[23392]: audit 2024-03-26T03:50:32.464349+0000 mon.a (mon.0) 1583 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:32.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:32 smithi190 bash[23392]: audit 2024-03-26T03:50:32.464623+0000 mon.a (mon.0) 1584 : audit [INF] from='client.? 172.21.15.175:0/3653366758' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi175-64398-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:32.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:32 smithi190 bash[23392]: cluster 2024-03-26T03:50:32.477029+0000 mon.a (mon.0) 1585 : cluster [DBG] osdmap e176: 8 total, 8 up, 8 in 2024-03-26T03:50:32.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:32 smithi190 bash[23392]: audit 2024-03-26T03:50:32.484727+0000 mon.a (mon.0) 1586 : audit [INF] from='client.? 172.21.15.175:0/3653366758' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi175-64398-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi175-64398-30"}]: dispatch 2024-03-26T03:50:32.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:32 smithi190 bash[23392]: audit 2024-03-26T03:50:32.485906+0000 mon.c (mon.2) 329 : audit [INF] from='client.? 172.21.15.175:0/707999435' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi175-64406-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:32.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:32 smithi190 bash[23392]: audit 2024-03-26T03:50:32.488476+0000 mon.a (mon.0) 1587 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi175-64406-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:32 smithi175 bash[21527]: audit 2024-03-26T03:50:31.520356+0000 mon.a (mon.0) 1580 : audit [INF] from='client.? 172.21.15.175:0/3653366758' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi175-64398-30"}]: dispatch 2024-03-26T03:50:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:32 smithi175 bash[21527]: audit 2024-03-26T03:50:31.522429+0000 mon.a (mon.0) 1581 : audit [INF] from='client.? 172.21.15.175:0/3653366758' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi175-64398-30"}]: dispatch 2024-03-26T03:50:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:32 smithi175 bash[21527]: audit 2024-03-26T03:50:31.522914+0000 mon.a (mon.0) 1582 : audit [INF] from='client.? 172.21.15.175:0/3653366758' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi175-64398-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:32 smithi175 bash[21527]: audit 2024-03-26T03:50:32.464349+0000 mon.a (mon.0) 1583 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:32 smithi175 bash[21527]: audit 2024-03-26T03:50:32.464623+0000 mon.a (mon.0) 1584 : audit [INF] from='client.? 172.21.15.175:0/3653366758' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi175-64398-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:32 smithi175 bash[21527]: cluster 2024-03-26T03:50:32.477029+0000 mon.a (mon.0) 1585 : cluster [DBG] osdmap e176: 8 total, 8 up, 8 in 2024-03-26T03:50:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:32 smithi175 bash[21527]: audit 2024-03-26T03:50:32.484727+0000 mon.a (mon.0) 1586 : audit [INF] from='client.? 172.21.15.175:0/3653366758' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi175-64398-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi175-64398-30"}]: dispatch 2024-03-26T03:50:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:32 smithi175 bash[21527]: audit 2024-03-26T03:50:32.485906+0000 mon.c (mon.2) 329 : audit [INF] from='client.? 172.21.15.175:0/707999435' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi175-64406-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:32 smithi175 bash[21527]: audit 2024-03-26T03:50:32.488476+0000 mon.a (mon.0) 1587 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi175-64406-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:32.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:32 smithi175 bash[29284]: audit 2024-03-26T03:50:31.520356+0000 mon.a (mon.0) 1580 : audit [INF] from='client.? 172.21.15.175:0/3653366758' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi175-64398-30"}]: dispatch 2024-03-26T03:50:32.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:32 smithi175 bash[29284]: audit 2024-03-26T03:50:31.522429+0000 mon.a (mon.0) 1581 : audit [INF] from='client.? 172.21.15.175:0/3653366758' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi175-64398-30"}]: dispatch 2024-03-26T03:50:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:32 smithi175 bash[29284]: audit 2024-03-26T03:50:31.522914+0000 mon.a (mon.0) 1582 : audit [INF] from='client.? 172.21.15.175:0/3653366758' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi175-64398-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:32 smithi175 bash[29284]: audit 2024-03-26T03:50:32.464349+0000 mon.a (mon.0) 1583 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:32 smithi175 bash[29284]: audit 2024-03-26T03:50:32.464623+0000 mon.a (mon.0) 1584 : audit [INF] from='client.? 172.21.15.175:0/3653366758' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi175-64398-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:32 smithi175 bash[29284]: cluster 2024-03-26T03:50:32.477029+0000 mon.a (mon.0) 1585 : cluster [DBG] osdmap e176: 8 total, 8 up, 8 in 2024-03-26T03:50:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:32 smithi175 bash[29284]: audit 2024-03-26T03:50:32.484727+0000 mon.a (mon.0) 1586 : audit [INF] from='client.? 172.21.15.175:0/3653366758' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi175-64398-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi175-64398-30"}]: dispatch 2024-03-26T03:50:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:32 smithi175 bash[29284]: audit 2024-03-26T03:50:32.485906+0000 mon.c (mon.2) 329 : audit [INF] from='client.? 172.21.15.175:0/707999435' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi175-64406-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:32 smithi175 bash[29284]: audit 2024-03-26T03:50:32.488476+0000 mon.a (mon.0) 1587 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi175-64406-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:33.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:33 smithi190 bash[23392]: audit 2024-03-26T03:50:32.520808+0000 mon.a (mon.0) 1588 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:50:33.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:33 smithi190 bash[23392]: cluster 2024-03-26T03:50:32.621082+0000 mgr.y (mgr.24431) 258 : cluster [DBG] pgmap v261: 324 pgs: 64 unknown, 260 active+clean; 584 KiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:50:33.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:33 smithi190 bash[23392]: audit 2024-03-26T03:50:33.468615+0000 mon.a (mon.0) 1589 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi175-64406-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:33.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:33 smithi190 bash[23392]: audit 2024-03-26T03:50:33.468843+0000 mon.a (mon.0) 1590 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-17", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:50:33.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:33 smithi190 bash[23392]: cluster 2024-03-26T03:50:33.493365+0000 mon.a (mon.0) 1591 : cluster [DBG] osdmap e177: 8 total, 8 up, 8 in 2024-03-26T03:50:33.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:33 smithi190 bash[23392]: audit 2024-03-26T03:50:33.494359+0000 mon.a (mon.0) 1592 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-17"}]: dispatch 2024-03-26T03:50:33.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:33 smithi175 bash[21527]: audit 2024-03-26T03:50:32.520808+0000 mon.a (mon.0) 1588 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:50:33.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:33 smithi175 bash[21527]: cluster 2024-03-26T03:50:32.621082+0000 mgr.y (mgr.24431) 258 : cluster [DBG] pgmap v261: 324 pgs: 64 unknown, 260 active+clean; 584 KiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:50:33.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:33 smithi175 bash[21527]: audit 2024-03-26T03:50:33.468615+0000 mon.a (mon.0) 1589 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi175-64406-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:33.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:33 smithi175 bash[21527]: audit 2024-03-26T03:50:33.468843+0000 mon.a (mon.0) 1590 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-17", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:50:33.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:33 smithi175 bash[21527]: cluster 2024-03-26T03:50:33.493365+0000 mon.a (mon.0) 1591 : cluster [DBG] osdmap e177: 8 total, 8 up, 8 in 2024-03-26T03:50:33.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:33 smithi175 bash[21527]: audit 2024-03-26T03:50:33.494359+0000 mon.a (mon.0) 1592 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-17"}]: dispatch 2024-03-26T03:50:33.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:33 smithi175 bash[29284]: audit 2024-03-26T03:50:32.520808+0000 mon.a (mon.0) 1588 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:50:33.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:33 smithi175 bash[29284]: cluster 2024-03-26T03:50:32.621082+0000 mgr.y (mgr.24431) 258 : cluster [DBG] pgmap v261: 324 pgs: 64 unknown, 260 active+clean; 584 KiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:50:33.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:33 smithi175 bash[29284]: audit 2024-03-26T03:50:33.468615+0000 mon.a (mon.0) 1589 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi175-64406-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:33.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:33 smithi175 bash[29284]: audit 2024-03-26T03:50:33.468843+0000 mon.a (mon.0) 1590 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-17", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:50:33.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:33 smithi175 bash[29284]: cluster 2024-03-26T03:50:33.493365+0000 mon.a (mon.0) 1591 : cluster [DBG] osdmap e177: 8 total, 8 up, 8 in 2024-03-26T03:50:33.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:33 smithi175 bash[29284]: audit 2024-03-26T03:50:33.494359+0000 mon.a (mon.0) 1592 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-17"}]: dispatch 2024-03-26T03:50:35.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:35 smithi190 bash[23392]: audit 2024-03-26T03:50:34.262523+0000 mon.a (mon.0) 1593 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:50:35.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:35 smithi190 bash[23392]: audit 2024-03-26T03:50:34.265681+0000 mon.c (mon.2) 330 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:50:35.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:35 smithi190 bash[23392]: audit 2024-03-26T03:50:34.471948+0000 mon.a (mon.0) 1594 : audit [INF] from='client.? 172.21.15.175:0/3653366758' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi175-64398-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi175-64398-30"}]': finished 2024-03-26T03:50:35.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:35 smithi190 bash[23392]: audit 2024-03-26T03:50:34.472185+0000 mon.a (mon.0) 1595 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-17"}]': finished 2024-03-26T03:50:35.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:35 smithi190 bash[23392]: cluster 2024-03-26T03:50:34.480068+0000 mon.a (mon.0) 1596 : cluster [DBG] osdmap e178: 8 total, 8 up, 8 in 2024-03-26T03:50:35.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:35 smithi190 bash[23392]: audit 2024-03-26T03:50:34.481075+0000 mon.a (mon.0) 1597 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-17", "mode": "writeback"}]: dispatch 2024-03-26T03:50:35.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:35 smithi175 bash[21527]: audit 2024-03-26T03:50:34.262523+0000 mon.a (mon.0) 1593 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:50:35.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:35 smithi175 bash[21527]: audit 2024-03-26T03:50:34.265681+0000 mon.c (mon.2) 330 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:50:35.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:35 smithi175 bash[21527]: audit 2024-03-26T03:50:34.471948+0000 mon.a (mon.0) 1594 : audit [INF] from='client.? 172.21.15.175:0/3653366758' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi175-64398-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi175-64398-30"}]': finished 2024-03-26T03:50:35.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:35 smithi175 bash[21527]: audit 2024-03-26T03:50:34.472185+0000 mon.a (mon.0) 1595 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-17"}]': finished 2024-03-26T03:50:35.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:35 smithi175 bash[21527]: cluster 2024-03-26T03:50:34.480068+0000 mon.a (mon.0) 1596 : cluster [DBG] osdmap e178: 8 total, 8 up, 8 in 2024-03-26T03:50:35.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:35 smithi175 bash[21527]: audit 2024-03-26T03:50:34.481075+0000 mon.a (mon.0) 1597 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-17", "mode": "writeback"}]: dispatch 2024-03-26T03:50:35.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:35 smithi175 bash[29284]: audit 2024-03-26T03:50:34.262523+0000 mon.a (mon.0) 1593 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:50:35.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:35 smithi175 bash[29284]: audit 2024-03-26T03:50:34.265681+0000 mon.c (mon.2) 330 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:50:35.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:35 smithi175 bash[29284]: audit 2024-03-26T03:50:34.471948+0000 mon.a (mon.0) 1594 : audit [INF] from='client.? 172.21.15.175:0/3653366758' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi175-64398-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi175-64398-30"}]': finished 2024-03-26T03:50:35.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:35 smithi175 bash[29284]: audit 2024-03-26T03:50:34.472185+0000 mon.a (mon.0) 1595 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-17"}]': finished 2024-03-26T03:50:35.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:35 smithi175 bash[29284]: cluster 2024-03-26T03:50:34.480068+0000 mon.a (mon.0) 1596 : cluster [DBG] osdmap e178: 8 total, 8 up, 8 in 2024-03-26T03:50:35.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:35 smithi175 bash[29284]: audit 2024-03-26T03:50:34.481075+0000 mon.a (mon.0) 1597 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-17", "mode": "writeback"}]: dispatch 2024-03-26T03:50:36.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:36 smithi190 bash[23392]: cluster 2024-03-26T03:50:34.622793+0000 mgr.y (mgr.24431) 259 : cluster [DBG] pgmap v264: 300 pgs: 26 unknown, 274 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:50:36.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:36 smithi190 bash[23392]: cluster 2024-03-26T03:50:35.474200+0000 mon.a (mon.0) 1598 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:50:36.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:36 smithi190 bash[23392]: audit 2024-03-26T03:50:35.482789+0000 mon.a (mon.0) 1599 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-17", "mode": "writeback"}]': finished 2024-03-26T03:50:36.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:36 smithi190 bash[23392]: cluster 2024-03-26T03:50:35.491184+0000 mon.a (mon.0) 1600 : cluster [DBG] osdmap e179: 8 total, 8 up, 8 in 2024-03-26T03:50:36.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:36 smithi190 bash[23392]: audit 2024-03-26T03:50:35.497950+0000 mon.c (mon.2) 331 : audit [INF] from='client.? 172.21.15.175:0/40277844' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi175-64406-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:36.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:36 smithi190 bash[23392]: audit 2024-03-26T03:50:35.500396+0000 mon.a (mon.0) 1601 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi175-64406-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:36.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:36 smithi190 bash[23392]: audit 2024-03-26T03:50:35.610733+0000 mon.a (mon.0) 1602 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:50:36.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:36 smithi175 bash[29284]: cluster 2024-03-26T03:50:34.622793+0000 mgr.y (mgr.24431) 259 : cluster [DBG] pgmap v264: 300 pgs: 26 unknown, 274 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:50:36.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:36 smithi175 bash[29284]: cluster 2024-03-26T03:50:35.474200+0000 mon.a (mon.0) 1598 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:50:36.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:36 smithi175 bash[29284]: audit 2024-03-26T03:50:35.482789+0000 mon.a (mon.0) 1599 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-17", "mode": "writeback"}]': finished 2024-03-26T03:50:36.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:36 smithi175 bash[29284]: cluster 2024-03-26T03:50:35.491184+0000 mon.a (mon.0) 1600 : cluster [DBG] osdmap e179: 8 total, 8 up, 8 in 2024-03-26T03:50:36.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:36 smithi175 bash[29284]: audit 2024-03-26T03:50:35.497950+0000 mon.c (mon.2) 331 : audit [INF] from='client.? 172.21.15.175:0/40277844' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi175-64406-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:36.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:36 smithi175 bash[29284]: audit 2024-03-26T03:50:35.500396+0000 mon.a (mon.0) 1601 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi175-64406-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:36.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:36 smithi175 bash[29284]: audit 2024-03-26T03:50:35.610733+0000 mon.a (mon.0) 1602 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:50:36.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:36 smithi175 bash[21527]: cluster 2024-03-26T03:50:34.622793+0000 mgr.y (mgr.24431) 259 : cluster [DBG] pgmap v264: 300 pgs: 26 unknown, 274 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:50:36.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:36 smithi175 bash[21527]: cluster 2024-03-26T03:50:35.474200+0000 mon.a (mon.0) 1598 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:50:36.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:36 smithi175 bash[21527]: audit 2024-03-26T03:50:35.482789+0000 mon.a (mon.0) 1599 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-17", "mode": "writeback"}]': finished 2024-03-26T03:50:36.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:36 smithi175 bash[21527]: cluster 2024-03-26T03:50:35.491184+0000 mon.a (mon.0) 1600 : cluster [DBG] osdmap e179: 8 total, 8 up, 8 in 2024-03-26T03:50:36.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:36 smithi175 bash[21527]: audit 2024-03-26T03:50:35.497950+0000 mon.c (mon.2) 331 : audit [INF] from='client.? 172.21.15.175:0/40277844' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi175-64406-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:36.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:36 smithi175 bash[21527]: audit 2024-03-26T03:50:35.500396+0000 mon.a (mon.0) 1601 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi175-64406-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:36.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:36 smithi175 bash[21527]: audit 2024-03-26T03:50:35.610733+0000 mon.a (mon.0) 1602 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:50:37.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:37 smithi190 bash[23392]: audit 2024-03-26T03:50:36.425798+0000 mon.a (mon.0) 1603 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi175-64406-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:37.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:37 smithi190 bash[23392]: audit 2024-03-26T03:50:36.425979+0000 mon.a (mon.0) 1604 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:50:37.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:37 smithi190 bash[23392]: cluster 2024-03-26T03:50:36.436524+0000 mon.a (mon.0) 1605 : cluster [DBG] osdmap e180: 8 total, 8 up, 8 in 2024-03-26T03:50:37.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:37 smithi190 bash[23392]: audit 2024-03-26T03:50:36.443780+0000 mon.a (mon.0) 1606 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-17"}]: dispatch 2024-03-26T03:50:37.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:37 smithi190 bash[23392]: audit 2024-03-26T03:50:36.445573+0000 mon.a (mon.0) 1607 : audit [INF] from='client.? 172.21.15.175:0/3653366758' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi175-64398-30"}]: dispatch 2024-03-26T03:50:37.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:37 smithi175 bash[21527]: audit 2024-03-26T03:50:36.425798+0000 mon.a (mon.0) 1603 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi175-64406-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:37.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:37 smithi175 bash[21527]: audit 2024-03-26T03:50:36.425979+0000 mon.a (mon.0) 1604 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:50:37.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:37 smithi175 bash[21527]: cluster 2024-03-26T03:50:36.436524+0000 mon.a (mon.0) 1605 : cluster [DBG] osdmap e180: 8 total, 8 up, 8 in 2024-03-26T03:50:37.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:37 smithi175 bash[21527]: audit 2024-03-26T03:50:36.443780+0000 mon.a (mon.0) 1606 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-17"}]: dispatch 2024-03-26T03:50:37.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:37 smithi175 bash[21527]: audit 2024-03-26T03:50:36.445573+0000 mon.a (mon.0) 1607 : audit [INF] from='client.? 172.21.15.175:0/3653366758' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi175-64398-30"}]: dispatch 2024-03-26T03:50:37.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:37 smithi175 bash[29284]: audit 2024-03-26T03:50:36.425798+0000 mon.a (mon.0) 1603 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi175-64406-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:37.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:37 smithi175 bash[29284]: audit 2024-03-26T03:50:36.425979+0000 mon.a (mon.0) 1604 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:50:37.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:37 smithi175 bash[29284]: cluster 2024-03-26T03:50:36.436524+0000 mon.a (mon.0) 1605 : cluster [DBG] osdmap e180: 8 total, 8 up, 8 in 2024-03-26T03:50:37.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:37 smithi175 bash[29284]: audit 2024-03-26T03:50:36.443780+0000 mon.a (mon.0) 1606 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-17"}]: dispatch 2024-03-26T03:50:37.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:37 smithi175 bash[29284]: audit 2024-03-26T03:50:36.445573+0000 mon.a (mon.0) 1607 : audit [INF] from='client.? 172.21.15.175:0/3653366758' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi175-64398-30"}]: dispatch 2024-03-26T03:50:38.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:38 smithi190 bash[23392]: cluster 2024-03-26T03:50:36.623743+0000 mgr.y (mgr.24431) 260 : cluster [DBG] pgmap v267: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:50:38.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:38 smithi190 bash[23392]: cluster 2024-03-26T03:50:37.425841+0000 mon.a (mon.0) 1608 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:50:38.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:38 smithi190 bash[23392]: cluster 2024-03-26T03:50:37.427064+0000 mon.a (mon.0) 1609 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:50:38.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:38 smithi190 bash[23392]: audit 2024-03-26T03:50:37.435152+0000 mon.a (mon.0) 1610 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-17"}]': finished 2024-03-26T03:50:38.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:38 smithi190 bash[23392]: audit 2024-03-26T03:50:37.435323+0000 mon.a (mon.0) 1611 : audit [INF] from='client.? 172.21.15.175:0/3653366758' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi175-64398-30"}]': finished 2024-03-26T03:50:38.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:38 smithi190 bash[23392]: cluster 2024-03-26T03:50:37.444434+0000 mon.a (mon.0) 1612 : cluster [DBG] osdmap e181: 8 total, 8 up, 8 in 2024-03-26T03:50:38.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:38 smithi190 bash[23392]: audit 2024-03-26T03:50:37.449213+0000 mon.a (mon.0) 1613 : audit [INF] from='client.? 172.21.15.175:0/3653366758' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi175-64398-30"}]: dispatch 2024-03-26T03:50:38.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:38 smithi175 bash[21527]: cluster 2024-03-26T03:50:36.623743+0000 mgr.y (mgr.24431) 260 : cluster [DBG] pgmap v267: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:50:38.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:38 smithi175 bash[21527]: cluster 2024-03-26T03:50:37.425841+0000 mon.a (mon.0) 1608 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:50:38.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:38 smithi175 bash[21527]: cluster 2024-03-26T03:50:37.427064+0000 mon.a (mon.0) 1609 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:50:38.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:38 smithi175 bash[21527]: audit 2024-03-26T03:50:37.435152+0000 mon.a (mon.0) 1610 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-17"}]': finished 2024-03-26T03:50:38.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:38 smithi175 bash[21527]: audit 2024-03-26T03:50:37.435323+0000 mon.a (mon.0) 1611 : audit [INF] from='client.? 172.21.15.175:0/3653366758' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi175-64398-30"}]': finished 2024-03-26T03:50:38.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:38 smithi175 bash[21527]: cluster 2024-03-26T03:50:37.444434+0000 mon.a (mon.0) 1612 : cluster [DBG] osdmap e181: 8 total, 8 up, 8 in 2024-03-26T03:50:38.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:38 smithi175 bash[21527]: audit 2024-03-26T03:50:37.449213+0000 mon.a (mon.0) 1613 : audit [INF] from='client.? 172.21.15.175:0/3653366758' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi175-64398-30"}]: dispatch 2024-03-26T03:50:38.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:38 smithi175 bash[29284]: cluster 2024-03-26T03:50:36.623743+0000 mgr.y (mgr.24431) 260 : cluster [DBG] pgmap v267: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:50:38.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:38 smithi175 bash[29284]: cluster 2024-03-26T03:50:37.425841+0000 mon.a (mon.0) 1608 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:50:38.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:38 smithi175 bash[29284]: cluster 2024-03-26T03:50:37.427064+0000 mon.a (mon.0) 1609 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:50:38.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:38 smithi175 bash[29284]: audit 2024-03-26T03:50:37.435152+0000 mon.a (mon.0) 1610 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-17"}]': finished 2024-03-26T03:50:38.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:38 smithi175 bash[29284]: audit 2024-03-26T03:50:37.435323+0000 mon.a (mon.0) 1611 : audit [INF] from='client.? 172.21.15.175:0/3653366758' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi175-64398-30"}]': finished 2024-03-26T03:50:38.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:38 smithi175 bash[29284]: cluster 2024-03-26T03:50:37.444434+0000 mon.a (mon.0) 1612 : cluster [DBG] osdmap e181: 8 total, 8 up, 8 in 2024-03-26T03:50:38.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:38 smithi175 bash[29284]: audit 2024-03-26T03:50:37.449213+0000 mon.a (mon.0) 1613 : audit [INF] from='client.? 172.21.15.175:0/3653366758' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi175-64398-30"}]: dispatch 2024-03-26T03:50:39.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:50:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:50:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:50:39.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:39 smithi190 bash[23392]: audit 2024-03-26T03:50:38.452062+0000 mon.a (mon.0) 1614 : audit [INF] from='client.? 172.21.15.175:0/3653366758' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi175-64398-30"}]': finished 2024-03-26T03:50:39.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:39 smithi190 bash[23392]: cluster 2024-03-26T03:50:38.460029+0000 mon.a (mon.0) 1615 : cluster [DBG] osdmap e182: 8 total, 8 up, 8 in 2024-03-26T03:50:39.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:39 smithi190 bash[23392]: audit 2024-03-26T03:50:38.463644+0000 mon.c (mon.2) 332 : audit [INF] from='client.? 172.21.15.175:0/1631438816' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi175-64406-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:39.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:39 smithi190 bash[23392]: audit 2024-03-26T03:50:38.466424+0000 mon.a (mon.0) 1616 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi175-64406-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:39.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:39 smithi190 bash[23392]: audit 2024-03-26T03:50:38.511074+0000 mon.c (mon.2) 333 : audit [INF] from='client.? 172.21.15.175:0/3221615236' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi175-64398-31"}]: dispatch 2024-03-26T03:50:39.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:39 smithi190 bash[23392]: audit 2024-03-26T03:50:38.511445+0000 mon.a (mon.0) 1617 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi175-64398-31"}]: dispatch 2024-03-26T03:50:39.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:39 smithi190 bash[23392]: audit 2024-03-26T03:50:38.512136+0000 mon.c (mon.2) 334 : audit [INF] from='client.? 172.21.15.175:0/3221615236' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi175-64398-31"}]: dispatch 2024-03-26T03:50:39.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:39 smithi190 bash[23392]: audit 2024-03-26T03:50:38.512434+0000 mon.a (mon.0) 1618 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi175-64398-31"}]: dispatch 2024-03-26T03:50:39.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:39 smithi190 bash[23392]: audit 2024-03-26T03:50:38.512962+0000 mon.c (mon.2) 335 : audit [INF] from='client.? 172.21.15.175:0/3221615236' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi175-64398-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:39.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:39 smithi190 bash[23392]: audit 2024-03-26T03:50:38.513254+0000 mon.a (mon.0) 1619 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi175-64398-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:39.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:39 smithi175 bash[21527]: audit 2024-03-26T03:50:38.452062+0000 mon.a (mon.0) 1614 : audit [INF] from='client.? 172.21.15.175:0/3653366758' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi175-64398-30"}]': finished 2024-03-26T03:50:39.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:39 smithi175 bash[21527]: cluster 2024-03-26T03:50:38.460029+0000 mon.a (mon.0) 1615 : cluster [DBG] osdmap e182: 8 total, 8 up, 8 in 2024-03-26T03:50:39.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:39 smithi175 bash[21527]: audit 2024-03-26T03:50:38.463644+0000 mon.c (mon.2) 332 : audit [INF] from='client.? 172.21.15.175:0/1631438816' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi175-64406-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:39.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:39 smithi175 bash[21527]: audit 2024-03-26T03:50:38.466424+0000 mon.a (mon.0) 1616 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi175-64406-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:39.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:39 smithi175 bash[21527]: audit 2024-03-26T03:50:38.511074+0000 mon.c (mon.2) 333 : audit [INF] from='client.? 172.21.15.175:0/3221615236' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi175-64398-31"}]: dispatch 2024-03-26T03:50:39.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:39 smithi175 bash[21527]: audit 2024-03-26T03:50:38.511445+0000 mon.a (mon.0) 1617 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi175-64398-31"}]: dispatch 2024-03-26T03:50:39.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:39 smithi175 bash[21527]: audit 2024-03-26T03:50:38.512136+0000 mon.c (mon.2) 334 : audit [INF] from='client.? 172.21.15.175:0/3221615236' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi175-64398-31"}]: dispatch 2024-03-26T03:50:39.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:39 smithi175 bash[21527]: audit 2024-03-26T03:50:38.512434+0000 mon.a (mon.0) 1618 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi175-64398-31"}]: dispatch 2024-03-26T03:50:39.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:39 smithi175 bash[21527]: audit 2024-03-26T03:50:38.512962+0000 mon.c (mon.2) 335 : audit [INF] from='client.? 172.21.15.175:0/3221615236' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi175-64398-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:39.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:39 smithi175 bash[21527]: audit 2024-03-26T03:50:38.513254+0000 mon.a (mon.0) 1619 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi175-64398-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:39.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:39 smithi175 bash[29284]: audit 2024-03-26T03:50:38.452062+0000 mon.a (mon.0) 1614 : audit [INF] from='client.? 172.21.15.175:0/3653366758' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi175-64398-30"}]': finished 2024-03-26T03:50:39.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:39 smithi175 bash[29284]: cluster 2024-03-26T03:50:38.460029+0000 mon.a (mon.0) 1615 : cluster [DBG] osdmap e182: 8 total, 8 up, 8 in 2024-03-26T03:50:39.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:39 smithi175 bash[29284]: audit 2024-03-26T03:50:38.463644+0000 mon.c (mon.2) 332 : audit [INF] from='client.? 172.21.15.175:0/1631438816' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi175-64406-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:39.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:39 smithi175 bash[29284]: audit 2024-03-26T03:50:38.466424+0000 mon.a (mon.0) 1616 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi175-64406-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:39.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:39 smithi175 bash[29284]: audit 2024-03-26T03:50:38.511074+0000 mon.c (mon.2) 333 : audit [INF] from='client.? 172.21.15.175:0/3221615236' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi175-64398-31"}]: dispatch 2024-03-26T03:50:39.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:39 smithi175 bash[29284]: audit 2024-03-26T03:50:38.511445+0000 mon.a (mon.0) 1617 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi175-64398-31"}]: dispatch 2024-03-26T03:50:39.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:39 smithi175 bash[29284]: audit 2024-03-26T03:50:38.512136+0000 mon.c (mon.2) 334 : audit [INF] from='client.? 172.21.15.175:0/3221615236' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi175-64398-31"}]: dispatch 2024-03-26T03:50:39.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:39 smithi175 bash[29284]: audit 2024-03-26T03:50:38.512434+0000 mon.a (mon.0) 1618 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi175-64398-31"}]: dispatch 2024-03-26T03:50:39.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:39 smithi175 bash[29284]: audit 2024-03-26T03:50:38.512962+0000 mon.c (mon.2) 335 : audit [INF] from='client.? 172.21.15.175:0/3221615236' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi175-64398-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:39.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:39 smithi175 bash[29284]: audit 2024-03-26T03:50:38.513254+0000 mon.a (mon.0) 1619 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi175-64398-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:40.240 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:50:39 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:50:40.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:40 smithi190 bash[23392]: cluster 2024-03-26T03:50:38.624629+0000 mgr.y (mgr.24431) 261 : cluster [DBG] pgmap v270: 292 pgs: 32 unknown, 260 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:50:40.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:40 smithi190 bash[23392]: audit 2024-03-26T03:50:39.456448+0000 mon.a (mon.0) 1620 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi175-64406-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:40.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:40 smithi190 bash[23392]: audit 2024-03-26T03:50:39.456748+0000 mon.a (mon.0) 1621 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi175-64398-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:40.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:40 smithi190 bash[23392]: audit 2024-03-26T03:50:39.466830+0000 mon.c (mon.2) 336 : audit [INF] from='client.? 172.21.15.175:0/3221615236' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi175-64398-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi175-64398-31"}]: dispatch 2024-03-26T03:50:40.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:40 smithi190 bash[23392]: cluster 2024-03-26T03:50:39.468880+0000 mon.a (mon.0) 1622 : cluster [DBG] osdmap e183: 8 total, 8 up, 8 in 2024-03-26T03:50:40.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:40 smithi190 bash[23392]: audit 2024-03-26T03:50:39.474245+0000 mon.a (mon.0) 1623 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi175-64398-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi175-64398-31"}]: dispatch 2024-03-26T03:50:40.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:40 smithi190 bash[23392]: audit 2024-03-26T03:50:39.479340+0000 mon.a (mon.0) 1624 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:40.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:40 smithi190 bash[23392]: audit 2024-03-26T03:50:40.460549+0000 mon.a (mon.0) 1625 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:40.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:40 smithi175 bash[21527]: cluster 2024-03-26T03:50:38.624629+0000 mgr.y (mgr.24431) 261 : cluster [DBG] pgmap v270: 292 pgs: 32 unknown, 260 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:50:40.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:40 smithi175 bash[21527]: audit 2024-03-26T03:50:39.456448+0000 mon.a (mon.0) 1620 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi175-64406-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:40.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:40 smithi175 bash[21527]: audit 2024-03-26T03:50:39.456748+0000 mon.a (mon.0) 1621 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi175-64398-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:40.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:40 smithi175 bash[21527]: audit 2024-03-26T03:50:39.466830+0000 mon.c (mon.2) 336 : audit [INF] from='client.? 172.21.15.175:0/3221615236' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi175-64398-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi175-64398-31"}]: dispatch 2024-03-26T03:50:40.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:40 smithi175 bash[21527]: cluster 2024-03-26T03:50:39.468880+0000 mon.a (mon.0) 1622 : cluster [DBG] osdmap e183: 8 total, 8 up, 8 in 2024-03-26T03:50:40.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:40 smithi175 bash[21527]: audit 2024-03-26T03:50:39.474245+0000 mon.a (mon.0) 1623 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi175-64398-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi175-64398-31"}]: dispatch 2024-03-26T03:50:40.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:40 smithi175 bash[21527]: audit 2024-03-26T03:50:39.479340+0000 mon.a (mon.0) 1624 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:40.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:40 smithi175 bash[21527]: audit 2024-03-26T03:50:40.460549+0000 mon.a (mon.0) 1625 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:40.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:40 smithi175 bash[29284]: cluster 2024-03-26T03:50:38.624629+0000 mgr.y (mgr.24431) 261 : cluster [DBG] pgmap v270: 292 pgs: 32 unknown, 260 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:50:40.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:40 smithi175 bash[29284]: audit 2024-03-26T03:50:39.456448+0000 mon.a (mon.0) 1620 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi175-64406-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:40.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:40 smithi175 bash[29284]: audit 2024-03-26T03:50:39.456748+0000 mon.a (mon.0) 1621 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi175-64398-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:40.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:40 smithi175 bash[29284]: audit 2024-03-26T03:50:39.466830+0000 mon.c (mon.2) 336 : audit [INF] from='client.? 172.21.15.175:0/3221615236' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi175-64398-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi175-64398-31"}]: dispatch 2024-03-26T03:50:40.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:40 smithi175 bash[29284]: cluster 2024-03-26T03:50:39.468880+0000 mon.a (mon.0) 1622 : cluster [DBG] osdmap e183: 8 total, 8 up, 8 in 2024-03-26T03:50:40.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:40 smithi175 bash[29284]: audit 2024-03-26T03:50:39.474245+0000 mon.a (mon.0) 1623 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi175-64398-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi175-64398-31"}]: dispatch 2024-03-26T03:50:40.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:40 smithi175 bash[29284]: audit 2024-03-26T03:50:39.479340+0000 mon.a (mon.0) 1624 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:40.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:40 smithi175 bash[29284]: audit 2024-03-26T03:50:40.460549+0000 mon.a (mon.0) 1625 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:41.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:41 smithi190 bash[23392]: audit 2024-03-26T03:50:39.793032+0000 mgr.y (mgr.24431) 262 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:50:41.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:41 smithi190 bash[23392]: cluster 2024-03-26T03:50:40.486376+0000 mon.a (mon.0) 1626 : cluster [DBG] osdmap e184: 8 total, 8 up, 8 in 2024-03-26T03:50:41.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:41 smithi190 bash[23392]: audit 2024-03-26T03:50:40.494401+0000 mon.a (mon.0) 1627 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:50:41.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:41 smithi190 bash[23392]: cluster 2024-03-26T03:50:40.625636+0000 mgr.y (mgr.24431) 263 : cluster [DBG] pgmap v273: 292 pgs: 32 creating+peering, 260 active+clean; 584 KiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:50:41.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:41 smithi190 bash[23392]: audit 2024-03-26T03:50:41.464581+0000 mon.a (mon.0) 1628 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi175-64398-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi175-64398-31"}]': finished 2024-03-26T03:50:41.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:41 smithi190 bash[23392]: audit 2024-03-26T03:50:41.464865+0000 mon.a (mon.0) 1629 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-19", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:50:41.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:41 smithi190 bash[23392]: cluster 2024-03-26T03:50:41.474848+0000 mon.a (mon.0) 1630 : cluster [DBG] osdmap e185: 8 total, 8 up, 8 in 2024-03-26T03:50:41.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:41 smithi190 bash[23392]: audit 2024-03-26T03:50:41.477854+0000 mon.a (mon.0) 1631 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-19"}]: dispatch 2024-03-26T03:50:41.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:41 smithi190 bash[23392]: audit 2024-03-26T03:50:41.482580+0000 mon.c (mon.2) 337 : audit [INF] from='client.? 172.21.15.175:0/3030175265' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi175-64406-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:41.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:41 smithi190 bash[23392]: audit 2024-03-26T03:50:41.483309+0000 mon.a (mon.0) 1632 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi175-64406-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:41.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:41 smithi175 bash[21527]: audit 2024-03-26T03:50:39.793032+0000 mgr.y (mgr.24431) 262 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:50:41.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:41 smithi175 bash[21527]: cluster 2024-03-26T03:50:40.486376+0000 mon.a (mon.0) 1626 : cluster [DBG] osdmap e184: 8 total, 8 up, 8 in 2024-03-26T03:50:41.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:41 smithi175 bash[21527]: audit 2024-03-26T03:50:40.494401+0000 mon.a (mon.0) 1627 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:50:41.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:41 smithi175 bash[21527]: cluster 2024-03-26T03:50:40.625636+0000 mgr.y (mgr.24431) 263 : cluster [DBG] pgmap v273: 292 pgs: 32 creating+peering, 260 active+clean; 584 KiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:50:41.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:41 smithi175 bash[21527]: audit 2024-03-26T03:50:41.464581+0000 mon.a (mon.0) 1628 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi175-64398-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi175-64398-31"}]': finished 2024-03-26T03:50:41.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:41 smithi175 bash[21527]: audit 2024-03-26T03:50:41.464865+0000 mon.a (mon.0) 1629 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-19", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:50:41.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:41 smithi175 bash[21527]: cluster 2024-03-26T03:50:41.474848+0000 mon.a (mon.0) 1630 : cluster [DBG] osdmap e185: 8 total, 8 up, 8 in 2024-03-26T03:50:41.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:41 smithi175 bash[21527]: audit 2024-03-26T03:50:41.477854+0000 mon.a (mon.0) 1631 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-19"}]: dispatch 2024-03-26T03:50:41.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:41 smithi175 bash[21527]: audit 2024-03-26T03:50:41.482580+0000 mon.c (mon.2) 337 : audit [INF] from='client.? 172.21.15.175:0/3030175265' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi175-64406-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:41.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:41 smithi175 bash[21527]: audit 2024-03-26T03:50:41.483309+0000 mon.a (mon.0) 1632 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi175-64406-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:41 smithi175 bash[29284]: audit 2024-03-26T03:50:39.793032+0000 mgr.y (mgr.24431) 262 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:50:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:41 smithi175 bash[29284]: cluster 2024-03-26T03:50:40.486376+0000 mon.a (mon.0) 1626 : cluster [DBG] osdmap e184: 8 total, 8 up, 8 in 2024-03-26T03:50:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:41 smithi175 bash[29284]: audit 2024-03-26T03:50:40.494401+0000 mon.a (mon.0) 1627 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:50:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:41 smithi175 bash[29284]: cluster 2024-03-26T03:50:40.625636+0000 mgr.y (mgr.24431) 263 : cluster [DBG] pgmap v273: 292 pgs: 32 creating+peering, 260 active+clean; 584 KiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:50:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:41 smithi175 bash[29284]: audit 2024-03-26T03:50:41.464581+0000 mon.a (mon.0) 1628 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi175-64398-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi175-64398-31"}]': finished 2024-03-26T03:50:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:41 smithi175 bash[29284]: audit 2024-03-26T03:50:41.464865+0000 mon.a (mon.0) 1629 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-19", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:50:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:41 smithi175 bash[29284]: cluster 2024-03-26T03:50:41.474848+0000 mon.a (mon.0) 1630 : cluster [DBG] osdmap e185: 8 total, 8 up, 8 in 2024-03-26T03:50:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:41 smithi175 bash[29284]: audit 2024-03-26T03:50:41.477854+0000 mon.a (mon.0) 1631 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-19"}]: dispatch 2024-03-26T03:50:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:41 smithi175 bash[29284]: audit 2024-03-26T03:50:41.482580+0000 mon.c (mon.2) 337 : audit [INF] from='client.? 172.21.15.175:0/3030175265' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi175-64406-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:41 smithi175 bash[29284]: audit 2024-03-26T03:50:41.483309+0000 mon.a (mon.0) 1632 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi175-64406-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:43.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:43 smithi190 bash[23392]: audit 2024-03-26T03:50:42.468000+0000 mon.a (mon.0) 1633 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-19"}]': finished 2024-03-26T03:50:43.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:43 smithi190 bash[23392]: audit 2024-03-26T03:50:42.468230+0000 mon.a (mon.0) 1634 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi175-64406-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:43.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:43 smithi190 bash[23392]: cluster 2024-03-26T03:50:42.476218+0000 mon.a (mon.0) 1635 : cluster [DBG] osdmap e186: 8 total, 8 up, 8 in 2024-03-26T03:50:43.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:43 smithi190 bash[23392]: audit 2024-03-26T03:50:42.478636+0000 mon.a (mon.0) 1636 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-19", "mode": "writeback"}]: dispatch 2024-03-26T03:50:43.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:43 smithi190 bash[23392]: audit 2024-03-26T03:50:42.978316+0000 mon.c (mon.2) 338 : audit [INF] from='client.? 172.21.15.175:0/3030175265' entity='client.admin' cmd=[{ 2024-03-26T03:50:43.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:43 smithi190 bash[23392]: audit 2024-03-26T03:50:42.978371+0000 mon.c (mon.2) 339 : audit [INF] "prefix": "osd pool set", 2024-03-26T03:50:43.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:43 smithi190 bash[23392]: audit 2024-03-26T03:50:42.978419+0000 mon.c (mon.2) 340 : audit [INF] "pool": "PoolEIOFlag_smithi175-64406-33", 2024-03-26T03:50:43.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:43 smithi190 bash[23392]: audit 2024-03-26T03:50:42.978457+0000 mon.c (mon.2) 341 : audit [INF] "var": "eio", 2024-03-26T03:50:43.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:43 smithi190 bash[23392]: audit 2024-03-26T03:50:42.978494+0000 mon.c (mon.2) 342 : audit [INF] "val": "true" 2024-03-26T03:50:43.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:43 smithi190 bash[23392]: audit 2024-03-26T03:50:42.978529+0000 mon.c (mon.2) 343 : audit [INF] }]: dispatch 2024-03-26T03:50:43.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:43 smithi190 bash[23392]: audit 2024-03-26T03:50:42.979561+0000 mon.a (mon.0) 1637 : audit [INF] from='client.? ' entity='client.admin' cmd=[{ 2024-03-26T03:50:43.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:43 smithi190 bash[23392]: audit 2024-03-26T03:50:42.979633+0000 mon.a (mon.0) 1638 : audit [INF] "prefix": "osd pool set", 2024-03-26T03:50:43.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:43 smithi190 bash[23392]: audit 2024-03-26T03:50:42.979680+0000 mon.a (mon.0) 1639 : audit [INF] "pool": "PoolEIOFlag_smithi175-64406-33", 2024-03-26T03:50:43.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:43 smithi190 bash[23392]: audit 2024-03-26T03:50:42.979727+0000 mon.a (mon.0) 1640 : audit [INF] "var": "eio", 2024-03-26T03:50:43.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:43 smithi190 bash[23392]: audit 2024-03-26T03:50:42.979808+0000 mon.a (mon.0) 1641 : audit [INF] "val": "true" 2024-03-26T03:50:43.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:43 smithi190 bash[23392]: audit 2024-03-26T03:50:42.979854+0000 mon.a (mon.0) 1642 : audit [INF] }]: dispatch 2024-03-26T03:50:43.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:43 smithi175 bash[21527]: audit 2024-03-26T03:50:42.468000+0000 mon.a (mon.0) 1633 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-19"}]': finished 2024-03-26T03:50:43.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:43 smithi175 bash[21527]: audit 2024-03-26T03:50:42.468230+0000 mon.a (mon.0) 1634 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi175-64406-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:43.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:43 smithi175 bash[21527]: cluster 2024-03-26T03:50:42.476218+0000 mon.a (mon.0) 1635 : cluster [DBG] osdmap e186: 8 total, 8 up, 8 in 2024-03-26T03:50:43.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:43 smithi175 bash[21527]: audit 2024-03-26T03:50:42.478636+0000 mon.a (mon.0) 1636 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-19", "mode": "writeback"}]: dispatch 2024-03-26T03:50:43.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:43 smithi175 bash[21527]: audit 2024-03-26T03:50:42.978316+0000 mon.c (mon.2) 338 : audit [INF] from='client.? 172.21.15.175:0/3030175265' entity='client.admin' cmd=[{ 2024-03-26T03:50:43.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:43 smithi175 bash[21527]: audit 2024-03-26T03:50:42.978371+0000 mon.c (mon.2) 339 : audit [INF] "prefix": "osd pool set", 2024-03-26T03:50:43.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:43 smithi175 bash[21527]: audit 2024-03-26T03:50:42.978419+0000 mon.c (mon.2) 340 : audit [INF] "pool": "PoolEIOFlag_smithi175-64406-33", 2024-03-26T03:50:43.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:43 smithi175 bash[21527]: audit 2024-03-26T03:50:42.978457+0000 mon.c (mon.2) 341 : audit [INF] "var": "eio", 2024-03-26T03:50:43.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:43 smithi175 bash[21527]: audit 2024-03-26T03:50:42.978494+0000 mon.c (mon.2) 342 : audit [INF] "val": "true" 2024-03-26T03:50:43.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:43 smithi175 bash[21527]: audit 2024-03-26T03:50:42.978529+0000 mon.c (mon.2) 343 : audit [INF] }]: dispatch 2024-03-26T03:50:43.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:43 smithi175 bash[21527]: audit 2024-03-26T03:50:42.979561+0000 mon.a (mon.0) 1637 : audit [INF] from='client.? ' entity='client.admin' cmd=[{ 2024-03-26T03:50:43.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:43 smithi175 bash[21527]: audit 2024-03-26T03:50:42.979633+0000 mon.a (mon.0) 1638 : audit [INF] "prefix": "osd pool set", 2024-03-26T03:50:43.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:43 smithi175 bash[21527]: audit 2024-03-26T03:50:42.979680+0000 mon.a (mon.0) 1639 : audit [INF] "pool": "PoolEIOFlag_smithi175-64406-33", 2024-03-26T03:50:43.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:43 smithi175 bash[21527]: audit 2024-03-26T03:50:42.979727+0000 mon.a (mon.0) 1640 : audit [INF] "var": "eio", 2024-03-26T03:50:43.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:43 smithi175 bash[21527]: audit 2024-03-26T03:50:42.979808+0000 mon.a (mon.0) 1641 : audit [INF] "val": "true" 2024-03-26T03:50:43.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:43 smithi175 bash[21527]: audit 2024-03-26T03:50:42.979854+0000 mon.a (mon.0) 1642 : audit [INF] }]: dispatch 2024-03-26T03:50:43.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:43 smithi175 bash[29284]: audit 2024-03-26T03:50:42.468000+0000 mon.a (mon.0) 1633 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-19"}]': finished 2024-03-26T03:50:43.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:43 smithi175 bash[29284]: audit 2024-03-26T03:50:42.468230+0000 mon.a (mon.0) 1634 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi175-64406-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:43.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:43 smithi175 bash[29284]: cluster 2024-03-26T03:50:42.476218+0000 mon.a (mon.0) 1635 : cluster [DBG] osdmap e186: 8 total, 8 up, 8 in 2024-03-26T03:50:43.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:43 smithi175 bash[29284]: audit 2024-03-26T03:50:42.478636+0000 mon.a (mon.0) 1636 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-19", "mode": "writeback"}]: dispatch 2024-03-26T03:50:43.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:43 smithi175 bash[29284]: audit 2024-03-26T03:50:42.978316+0000 mon.c (mon.2) 338 : audit [INF] from='client.? 172.21.15.175:0/3030175265' entity='client.admin' cmd=[{ 2024-03-26T03:50:43.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:43 smithi175 bash[29284]: audit 2024-03-26T03:50:42.978371+0000 mon.c (mon.2) 339 : audit [INF] "prefix": "osd pool set", 2024-03-26T03:50:43.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:43 smithi175 bash[29284]: audit 2024-03-26T03:50:42.978419+0000 mon.c (mon.2) 340 : audit [INF] "pool": "PoolEIOFlag_smithi175-64406-33", 2024-03-26T03:50:43.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:43 smithi175 bash[29284]: audit 2024-03-26T03:50:42.978457+0000 mon.c (mon.2) 341 : audit [INF] "var": "eio", 2024-03-26T03:50:43.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:43 smithi175 bash[29284]: audit 2024-03-26T03:50:42.978494+0000 mon.c (mon.2) 342 : audit [INF] "val": "true" 2024-03-26T03:50:43.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:43 smithi175 bash[29284]: audit 2024-03-26T03:50:42.978529+0000 mon.c (mon.2) 343 : audit [INF] }]: dispatch 2024-03-26T03:50:43.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:43 smithi175 bash[29284]: audit 2024-03-26T03:50:42.979561+0000 mon.a (mon.0) 1637 : audit [INF] from='client.? ' entity='client.admin' cmd=[{ 2024-03-26T03:50:43.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:43 smithi175 bash[29284]: audit 2024-03-26T03:50:42.979633+0000 mon.a (mon.0) 1638 : audit [INF] "prefix": "osd pool set", 2024-03-26T03:50:43.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:43 smithi175 bash[29284]: audit 2024-03-26T03:50:42.979680+0000 mon.a (mon.0) 1639 : audit [INF] "pool": "PoolEIOFlag_smithi175-64406-33", 2024-03-26T03:50:43.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:43 smithi175 bash[29284]: audit 2024-03-26T03:50:42.979727+0000 mon.a (mon.0) 1640 : audit [INF] "var": "eio", 2024-03-26T03:50:43.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:43 smithi175 bash[29284]: audit 2024-03-26T03:50:42.979808+0000 mon.a (mon.0) 1641 : audit [INF] "val": "true" 2024-03-26T03:50:43.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:43 smithi175 bash[29284]: audit 2024-03-26T03:50:42.979854+0000 mon.a (mon.0) 1642 : audit [INF] }]: dispatch 2024-03-26T03:50:44.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:44 smithi190 bash[23392]: cluster 2024-03-26T03:50:42.626342+0000 mgr.y (mgr.24431) 264 : cluster [DBG] pgmap v276: 332 pgs: 40 unknown, 32 creating+peering, 260 active+clean; 584 KiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:50:44.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:44 smithi190 bash[23392]: cluster 2024-03-26T03:50:43.467851+0000 mon.a (mon.0) 1643 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:50:44.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:44 smithi190 bash[23392]: cluster 2024-03-26T03:50:43.469236+0000 mon.a (mon.0) 1644 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:50:44.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:44 smithi190 bash[23392]: audit 2024-03-26T03:50:43.478088+0000 mon.a (mon.0) 1645 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-19", "mode": "writeback"}]': finished 2024-03-26T03:50:44.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:44 smithi190 bash[23392]: audit 2024-03-26T03:50:43.478254+0000 mon.a (mon.0) 1646 : audit [INF] from='client.? ' entity='client.admin' cmd='[{ 2024-03-26T03:50:44.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:44 smithi190 bash[23392]: audit 2024-03-26T03:50:43.478286+0000 mon.a (mon.0) 1647 : audit [INF] "prefix": "osd pool set", 2024-03-26T03:50:44.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:44 smithi190 bash[23392]: audit 2024-03-26T03:50:43.478323+0000 mon.a (mon.0) 1648 : audit [INF] "pool": "PoolEIOFlag_smithi175-64406-33", 2024-03-26T03:50:44.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:44 smithi190 bash[23392]: audit 2024-03-26T03:50:43.478355+0000 mon.a (mon.0) 1649 : audit [INF] "var": "eio", 2024-03-26T03:50:44.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:44 smithi190 bash[23392]: audit 2024-03-26T03:50:43.478388+0000 mon.a (mon.0) 1650 : audit [INF] "val": "true" 2024-03-26T03:50:44.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:44 smithi190 bash[23392]: audit 2024-03-26T03:50:43.478417+0000 mon.a (mon.0) 1651 : audit [INF] }]': finished 2024-03-26T03:50:44.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:44 smithi190 bash[23392]: cluster 2024-03-26T03:50:43.493425+0000 mon.a (mon.0) 1652 : cluster [DBG] osdmap e187: 8 total, 8 up, 8 in 2024-03-26T03:50:44.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:44 smithi190 bash[23392]: audit 2024-03-26T03:50:43.502551+0000 mon.c (mon.2) 344 : audit [INF] from='client.? 172.21.15.175:0/3221615236' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi175-64398-31"}]: dispatch 2024-03-26T03:50:44.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:44 smithi190 bash[23392]: audit 2024-03-26T03:50:43.503463+0000 mon.a (mon.0) 1653 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi175-64398-31"}]: dispatch 2024-03-26T03:50:44.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:44 smithi190 bash[23392]: audit 2024-03-26T03:50:43.591829+0000 mon.a (mon.0) 1654 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:50:44.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:44 smithi175 bash[29284]: cluster 2024-03-26T03:50:42.626342+0000 mgr.y (mgr.24431) 264 : cluster [DBG] pgmap v276: 332 pgs: 40 unknown, 32 creating+peering, 260 active+clean; 584 KiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:50:44.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:44 smithi175 bash[29284]: cluster 2024-03-26T03:50:43.467851+0000 mon.a (mon.0) 1643 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:50:44.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:44 smithi175 bash[29284]: cluster 2024-03-26T03:50:43.469236+0000 mon.a (mon.0) 1644 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:50:44.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:44 smithi175 bash[29284]: audit 2024-03-26T03:50:43.478088+0000 mon.a (mon.0) 1645 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-19", "mode": "writeback"}]': finished 2024-03-26T03:50:44.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:44 smithi175 bash[29284]: audit 2024-03-26T03:50:43.478254+0000 mon.a (mon.0) 1646 : audit [INF] from='client.? ' entity='client.admin' cmd='[{ 2024-03-26T03:50:44.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:44 smithi175 bash[29284]: audit 2024-03-26T03:50:43.478286+0000 mon.a (mon.0) 1647 : audit [INF] "prefix": "osd pool set", 2024-03-26T03:50:44.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:44 smithi175 bash[29284]: audit 2024-03-26T03:50:43.478323+0000 mon.a (mon.0) 1648 : audit [INF] "pool": "PoolEIOFlag_smithi175-64406-33", 2024-03-26T03:50:44.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:44 smithi175 bash[29284]: audit 2024-03-26T03:50:43.478355+0000 mon.a (mon.0) 1649 : audit [INF] "var": "eio", 2024-03-26T03:50:44.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:44 smithi175 bash[29284]: audit 2024-03-26T03:50:43.478388+0000 mon.a (mon.0) 1650 : audit [INF] "val": "true" 2024-03-26T03:50:44.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:44 smithi175 bash[29284]: audit 2024-03-26T03:50:43.478417+0000 mon.a (mon.0) 1651 : audit [INF] }]': finished 2024-03-26T03:50:44.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:44 smithi175 bash[29284]: cluster 2024-03-26T03:50:43.493425+0000 mon.a (mon.0) 1652 : cluster [DBG] osdmap e187: 8 total, 8 up, 8 in 2024-03-26T03:50:44.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:44 smithi175 bash[29284]: audit 2024-03-26T03:50:43.502551+0000 mon.c (mon.2) 344 : audit [INF] from='client.? 172.21.15.175:0/3221615236' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi175-64398-31"}]: dispatch 2024-03-26T03:50:44.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:44 smithi175 bash[29284]: audit 2024-03-26T03:50:43.503463+0000 mon.a (mon.0) 1653 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi175-64398-31"}]: dispatch 2024-03-26T03:50:44.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:44 smithi175 bash[29284]: audit 2024-03-26T03:50:43.591829+0000 mon.a (mon.0) 1654 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:50:44.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:44 smithi175 bash[21527]: cluster 2024-03-26T03:50:42.626342+0000 mgr.y (mgr.24431) 264 : cluster [DBG] pgmap v276: 332 pgs: 40 unknown, 32 creating+peering, 260 active+clean; 584 KiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:50:44.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:44 smithi175 bash[21527]: cluster 2024-03-26T03:50:43.467851+0000 mon.a (mon.0) 1643 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:50:44.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:44 smithi175 bash[21527]: cluster 2024-03-26T03:50:43.469236+0000 mon.a (mon.0) 1644 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:50:44.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:44 smithi175 bash[21527]: audit 2024-03-26T03:50:43.478088+0000 mon.a (mon.0) 1645 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-19", "mode": "writeback"}]': finished 2024-03-26T03:50:44.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:44 smithi175 bash[21527]: audit 2024-03-26T03:50:43.478254+0000 mon.a (mon.0) 1646 : audit [INF] from='client.? ' entity='client.admin' cmd='[{ 2024-03-26T03:50:44.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:44 smithi175 bash[21527]: audit 2024-03-26T03:50:43.478286+0000 mon.a (mon.0) 1647 : audit [INF] "prefix": "osd pool set", 2024-03-26T03:50:44.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:44 smithi175 bash[21527]: audit 2024-03-26T03:50:43.478323+0000 mon.a (mon.0) 1648 : audit [INF] "pool": "PoolEIOFlag_smithi175-64406-33", 2024-03-26T03:50:44.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:44 smithi175 bash[21527]: audit 2024-03-26T03:50:43.478355+0000 mon.a (mon.0) 1649 : audit [INF] "var": "eio", 2024-03-26T03:50:44.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:44 smithi175 bash[21527]: audit 2024-03-26T03:50:43.478388+0000 mon.a (mon.0) 1650 : audit [INF] "val": "true" 2024-03-26T03:50:44.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:44 smithi175 bash[21527]: audit 2024-03-26T03:50:43.478417+0000 mon.a (mon.0) 1651 : audit [INF] }]': finished 2024-03-26T03:50:44.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:44 smithi175 bash[21527]: cluster 2024-03-26T03:50:43.493425+0000 mon.a (mon.0) 1652 : cluster [DBG] osdmap e187: 8 total, 8 up, 8 in 2024-03-26T03:50:44.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:44 smithi175 bash[21527]: audit 2024-03-26T03:50:43.502551+0000 mon.c (mon.2) 344 : audit [INF] from='client.? 172.21.15.175:0/3221615236' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi175-64398-31"}]: dispatch 2024-03-26T03:50:44.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:44 smithi175 bash[21527]: audit 2024-03-26T03:50:43.503463+0000 mon.a (mon.0) 1653 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi175-64398-31"}]: dispatch 2024-03-26T03:50:44.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:44 smithi175 bash[21527]: audit 2024-03-26T03:50:43.591829+0000 mon.a (mon.0) 1654 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:50:45.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:45 smithi190 bash[23392]: audit 2024-03-26T03:50:44.480932+0000 mon.a (mon.0) 1655 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi175-64398-31"}]': finished 2024-03-26T03:50:45.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:45 smithi190 bash[23392]: audit 2024-03-26T03:50:44.481136+0000 mon.a (mon.0) 1656 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:50:45.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:45 smithi190 bash[23392]: cluster 2024-03-26T03:50:44.490133+0000 mon.a (mon.0) 1657 : cluster [DBG] osdmap e188: 8 total, 8 up, 8 in 2024-03-26T03:50:45.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:45 smithi190 bash[23392]: audit 2024-03-26T03:50:44.493888+0000 mon.c (mon.2) 345 : audit [INF] from='client.? 172.21.15.175:0/3221615236' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi175-64398-31"}]: dispatch 2024-03-26T03:50:45.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:45 smithi190 bash[23392]: audit 2024-03-26T03:50:44.495779+0000 mon.a (mon.0) 1658 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-19"}]: dispatch 2024-03-26T03:50:45.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:45 smithi190 bash[23392]: audit 2024-03-26T03:50:44.496561+0000 mon.a (mon.0) 1659 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi175-64398-31"}]: dispatch 2024-03-26T03:50:45.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:45 smithi190 bash[23392]: cluster 2024-03-26T03:50:45.483652+0000 mon.a (mon.0) 1660 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:50:45.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:45 smithi190 bash[23392]: audit 2024-03-26T03:50:45.486400+0000 mon.a (mon.0) 1661 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-19"}]': finished 2024-03-26T03:50:45.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:45 smithi175 bash[29284]: audit 2024-03-26T03:50:44.480932+0000 mon.a (mon.0) 1655 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi175-64398-31"}]': finished 2024-03-26T03:50:45.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:45 smithi175 bash[29284]: audit 2024-03-26T03:50:44.481136+0000 mon.a (mon.0) 1656 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:50:45.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:45 smithi175 bash[29284]: cluster 2024-03-26T03:50:44.490133+0000 mon.a (mon.0) 1657 : cluster [DBG] osdmap e188: 8 total, 8 up, 8 in 2024-03-26T03:50:45.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:45 smithi175 bash[29284]: audit 2024-03-26T03:50:44.493888+0000 mon.c (mon.2) 345 : audit [INF] from='client.? 172.21.15.175:0/3221615236' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi175-64398-31"}]: dispatch 2024-03-26T03:50:45.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:45 smithi175 bash[29284]: audit 2024-03-26T03:50:44.495779+0000 mon.a (mon.0) 1658 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-19"}]: dispatch 2024-03-26T03:50:45.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:45 smithi175 bash[29284]: audit 2024-03-26T03:50:44.496561+0000 mon.a (mon.0) 1659 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi175-64398-31"}]: dispatch 2024-03-26T03:50:45.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:45 smithi175 bash[29284]: cluster 2024-03-26T03:50:45.483652+0000 mon.a (mon.0) 1660 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:50:45.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:45 smithi175 bash[29284]: audit 2024-03-26T03:50:45.486400+0000 mon.a (mon.0) 1661 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-19"}]': finished 2024-03-26T03:50:45.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:45 smithi175 bash[21527]: audit 2024-03-26T03:50:44.480932+0000 mon.a (mon.0) 1655 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi175-64398-31"}]': finished 2024-03-26T03:50:45.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:45 smithi175 bash[21527]: audit 2024-03-26T03:50:44.481136+0000 mon.a (mon.0) 1656 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:50:45.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:45 smithi175 bash[21527]: cluster 2024-03-26T03:50:44.490133+0000 mon.a (mon.0) 1657 : cluster [DBG] osdmap e188: 8 total, 8 up, 8 in 2024-03-26T03:50:45.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:45 smithi175 bash[21527]: audit 2024-03-26T03:50:44.493888+0000 mon.c (mon.2) 345 : audit [INF] from='client.? 172.21.15.175:0/3221615236' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi175-64398-31"}]: dispatch 2024-03-26T03:50:45.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:45 smithi175 bash[21527]: audit 2024-03-26T03:50:44.495779+0000 mon.a (mon.0) 1658 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-19"}]: dispatch 2024-03-26T03:50:45.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:45 smithi175 bash[21527]: audit 2024-03-26T03:50:44.496561+0000 mon.a (mon.0) 1659 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi175-64398-31"}]: dispatch 2024-03-26T03:50:45.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:45 smithi175 bash[21527]: cluster 2024-03-26T03:50:45.483652+0000 mon.a (mon.0) 1660 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:50:45.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:45 smithi175 bash[21527]: audit 2024-03-26T03:50:45.486400+0000 mon.a (mon.0) 1661 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-19"}]': finished 2024-03-26T03:50:46.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:46 smithi190 bash[23392]: cluster 2024-03-26T03:50:44.627487+0000 mgr.y (mgr.24431) 265 : cluster [DBG] pgmap v279: 292 pgs: 18 creating+peering, 274 active+clean; 584 KiB data, 559 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-26T03:50:46.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:46 smithi190 bash[23392]: audit 2024-03-26T03:50:45.486624+0000 mon.a (mon.0) 1662 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi175-64398-31"}]': finished 2024-03-26T03:50:46.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:46 smithi190 bash[23392]: cluster 2024-03-26T03:50:45.501451+0000 mon.a (mon.0) 1663 : cluster [DBG] osdmap e189: 8 total, 8 up, 8 in 2024-03-26T03:50:46.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:46 smithi190 bash[23392]: audit 2024-03-26T03:50:45.506808+0000 mon.c (mon.2) 346 : audit [INF] from='client.? 172.21.15.175:0/2719387937' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi175-64406-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:46.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:46 smithi190 bash[23392]: audit 2024-03-26T03:50:45.519981+0000 mon.a (mon.0) 1664 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi175-64406-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:46.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:46 smithi190 bash[23392]: audit 2024-03-26T03:50:45.542326+0000 mon.c (mon.2) 347 : audit [INF] from='client.? 172.21.15.175:0/171500640' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi175-64398-32"}]: dispatch 2024-03-26T03:50:46.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:46 smithi190 bash[23392]: audit 2024-03-26T03:50:45.542736+0000 mon.a (mon.0) 1665 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi175-64398-32"}]: dispatch 2024-03-26T03:50:46.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:46 smithi190 bash[23392]: audit 2024-03-26T03:50:45.543330+0000 mon.c (mon.2) 348 : audit [INF] from='client.? 172.21.15.175:0/171500640' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi175-64398-32"}]: dispatch 2024-03-26T03:50:46.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:46 smithi190 bash[23392]: audit 2024-03-26T03:50:45.543645+0000 mon.a (mon.0) 1666 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi175-64398-32"}]: dispatch 2024-03-26T03:50:46.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:46 smithi190 bash[23392]: audit 2024-03-26T03:50:45.544200+0000 mon.c (mon.2) 349 : audit [INF] from='client.? 172.21.15.175:0/171500640' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi175-64398-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:46.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:46 smithi190 bash[23392]: audit 2024-03-26T03:50:45.544626+0000 mon.a (mon.0) 1667 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi175-64398-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:46.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:46 smithi190 bash[23392]: audit 2024-03-26T03:50:46.489583+0000 mon.a (mon.0) 1668 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi175-64406-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:46.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:46 smithi190 bash[23392]: audit 2024-03-26T03:50:46.489816+0000 mon.a (mon.0) 1669 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi175-64398-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:46.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:46 smithi190 bash[23392]: cluster 2024-03-26T03:50:46.498722+0000 mon.a (mon.0) 1670 : cluster [DBG] osdmap e190: 8 total, 8 up, 8 in 2024-03-26T03:50:46.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:46 smithi190 bash[23392]: audit 2024-03-26T03:50:46.506540+0000 mon.c (mon.2) 350 : audit [INF] from='client.? 172.21.15.175:0/171500640' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi175-64398-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi175-64398-32"}]: dispatch 2024-03-26T03:50:46.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:46 smithi190 bash[23392]: audit 2024-03-26T03:50:46.515432+0000 mon.a (mon.0) 1671 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi175-64398-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi175-64398-32"}]: dispatch 2024-03-26T03:50:46.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:46 smithi175 bash[21527]: cluster 2024-03-26T03:50:44.627487+0000 mgr.y (mgr.24431) 265 : cluster [DBG] pgmap v279: 292 pgs: 18 creating+peering, 274 active+clean; 584 KiB data, 559 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-26T03:50:46.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:46 smithi175 bash[21527]: audit 2024-03-26T03:50:45.486624+0000 mon.a (mon.0) 1662 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi175-64398-31"}]': finished 2024-03-26T03:50:46.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:46 smithi175 bash[21527]: cluster 2024-03-26T03:50:45.501451+0000 mon.a (mon.0) 1663 : cluster [DBG] osdmap e189: 8 total, 8 up, 8 in 2024-03-26T03:50:46.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:46 smithi175 bash[21527]: audit 2024-03-26T03:50:45.506808+0000 mon.c (mon.2) 346 : audit [INF] from='client.? 172.21.15.175:0/2719387937' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi175-64406-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:46.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:46 smithi175 bash[21527]: audit 2024-03-26T03:50:45.519981+0000 mon.a (mon.0) 1664 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi175-64406-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:46.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:46 smithi175 bash[21527]: audit 2024-03-26T03:50:45.542326+0000 mon.c (mon.2) 347 : audit [INF] from='client.? 172.21.15.175:0/171500640' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi175-64398-32"}]: dispatch 2024-03-26T03:50:46.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:46 smithi175 bash[21527]: audit 2024-03-26T03:50:45.542736+0000 mon.a (mon.0) 1665 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi175-64398-32"}]: dispatch 2024-03-26T03:50:46.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:46 smithi175 bash[21527]: audit 2024-03-26T03:50:45.543330+0000 mon.c (mon.2) 348 : audit [INF] from='client.? 172.21.15.175:0/171500640' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi175-64398-32"}]: dispatch 2024-03-26T03:50:46.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:46 smithi175 bash[21527]: audit 2024-03-26T03:50:45.543645+0000 mon.a (mon.0) 1666 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi175-64398-32"}]: dispatch 2024-03-26T03:50:46.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:46 smithi175 bash[21527]: audit 2024-03-26T03:50:45.544200+0000 mon.c (mon.2) 349 : audit [INF] from='client.? 172.21.15.175:0/171500640' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi175-64398-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:46.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:46 smithi175 bash[21527]: audit 2024-03-26T03:50:45.544626+0000 mon.a (mon.0) 1667 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi175-64398-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:46.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:46 smithi175 bash[21527]: audit 2024-03-26T03:50:46.489583+0000 mon.a (mon.0) 1668 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi175-64406-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:46.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:46 smithi175 bash[21527]: audit 2024-03-26T03:50:46.489816+0000 mon.a (mon.0) 1669 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi175-64398-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:46.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:46 smithi175 bash[21527]: cluster 2024-03-26T03:50:46.498722+0000 mon.a (mon.0) 1670 : cluster [DBG] osdmap e190: 8 total, 8 up, 8 in 2024-03-26T03:50:46.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:46 smithi175 bash[21527]: audit 2024-03-26T03:50:46.506540+0000 mon.c (mon.2) 350 : audit [INF] from='client.? 172.21.15.175:0/171500640' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi175-64398-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi175-64398-32"}]: dispatch 2024-03-26T03:50:46.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:46 smithi175 bash[21527]: audit 2024-03-26T03:50:46.515432+0000 mon.a (mon.0) 1671 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi175-64398-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi175-64398-32"}]: dispatch 2024-03-26T03:50:46.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:46 smithi175 bash[29284]: cluster 2024-03-26T03:50:44.627487+0000 mgr.y (mgr.24431) 265 : cluster [DBG] pgmap v279: 292 pgs: 18 creating+peering, 274 active+clean; 584 KiB data, 559 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-26T03:50:46.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:46 smithi175 bash[29284]: audit 2024-03-26T03:50:45.486624+0000 mon.a (mon.0) 1662 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi175-64398-31"}]': finished 2024-03-26T03:50:46.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:46 smithi175 bash[29284]: cluster 2024-03-26T03:50:45.501451+0000 mon.a (mon.0) 1663 : cluster [DBG] osdmap e189: 8 total, 8 up, 8 in 2024-03-26T03:50:46.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:46 smithi175 bash[29284]: audit 2024-03-26T03:50:45.506808+0000 mon.c (mon.2) 346 : audit [INF] from='client.? 172.21.15.175:0/2719387937' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi175-64406-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:46.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:46 smithi175 bash[29284]: audit 2024-03-26T03:50:45.519981+0000 mon.a (mon.0) 1664 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi175-64406-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:46.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:46 smithi175 bash[29284]: audit 2024-03-26T03:50:45.542326+0000 mon.c (mon.2) 347 : audit [INF] from='client.? 172.21.15.175:0/171500640' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi175-64398-32"}]: dispatch 2024-03-26T03:50:46.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:46 smithi175 bash[29284]: audit 2024-03-26T03:50:45.542736+0000 mon.a (mon.0) 1665 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi175-64398-32"}]: dispatch 2024-03-26T03:50:46.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:46 smithi175 bash[29284]: audit 2024-03-26T03:50:45.543330+0000 mon.c (mon.2) 348 : audit [INF] from='client.? 172.21.15.175:0/171500640' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi175-64398-32"}]: dispatch 2024-03-26T03:50:46.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:46 smithi175 bash[29284]: audit 2024-03-26T03:50:45.543645+0000 mon.a (mon.0) 1666 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi175-64398-32"}]: dispatch 2024-03-26T03:50:46.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:46 smithi175 bash[29284]: audit 2024-03-26T03:50:45.544200+0000 mon.c (mon.2) 349 : audit [INF] from='client.? 172.21.15.175:0/171500640' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi175-64398-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:46.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:46 smithi175 bash[29284]: audit 2024-03-26T03:50:45.544626+0000 mon.a (mon.0) 1667 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi175-64398-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:46.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:46 smithi175 bash[29284]: audit 2024-03-26T03:50:46.489583+0000 mon.a (mon.0) 1668 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi175-64406-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:46.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:46 smithi175 bash[29284]: audit 2024-03-26T03:50:46.489816+0000 mon.a (mon.0) 1669 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi175-64398-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:46.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:46 smithi175 bash[29284]: cluster 2024-03-26T03:50:46.498722+0000 mon.a (mon.0) 1670 : cluster [DBG] osdmap e190: 8 total, 8 up, 8 in 2024-03-26T03:50:46.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:46 smithi175 bash[29284]: audit 2024-03-26T03:50:46.506540+0000 mon.c (mon.2) 350 : audit [INF] from='client.? 172.21.15.175:0/171500640' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi175-64398-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi175-64398-32"}]: dispatch 2024-03-26T03:50:46.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:46 smithi175 bash[29284]: audit 2024-03-26T03:50:46.515432+0000 mon.a (mon.0) 1671 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi175-64398-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi175-64398-32"}]: dispatch 2024-03-26T03:50:47.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:47 smithi190 bash[23392]: cluster 2024-03-26T03:50:46.628274+0000 mgr.y (mgr.24431) 266 : cluster [DBG] pgmap v282: 292 pgs: 32 unknown, 260 active+clean; 584 KiB data, 559 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:50:47.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:47 smithi190 bash[23392]: cluster 2024-03-26T03:50:47.514981+0000 mon.a (mon.0) 1672 : cluster [DBG] osdmap e191: 8 total, 8 up, 8 in 2024-03-26T03:50:47.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:47 smithi190 bash[23392]: audit 2024-03-26T03:50:47.517640+0000 mon.a (mon.0) 1673 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:47.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:47 smithi175 bash[29284]: cluster 2024-03-26T03:50:46.628274+0000 mgr.y (mgr.24431) 266 : cluster [DBG] pgmap v282: 292 pgs: 32 unknown, 260 active+clean; 584 KiB data, 559 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:50:47.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:47 smithi175 bash[29284]: cluster 2024-03-26T03:50:47.514981+0000 mon.a (mon.0) 1672 : cluster [DBG] osdmap e191: 8 total, 8 up, 8 in 2024-03-26T03:50:47.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:47 smithi175 bash[29284]: audit 2024-03-26T03:50:47.517640+0000 mon.a (mon.0) 1673 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:47.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:47 smithi175 bash[21527]: cluster 2024-03-26T03:50:46.628274+0000 mgr.y (mgr.24431) 266 : cluster [DBG] pgmap v282: 292 pgs: 32 unknown, 260 active+clean; 584 KiB data, 559 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:50:47.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:47 smithi175 bash[21527]: cluster 2024-03-26T03:50:47.514981+0000 mon.a (mon.0) 1672 : cluster [DBG] osdmap e191: 8 total, 8 up, 8 in 2024-03-26T03:50:47.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:47 smithi175 bash[21527]: audit 2024-03-26T03:50:47.517640+0000 mon.a (mon.0) 1673 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:49.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:50:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:50:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:50:49.802 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:49 smithi190 bash[23392]: audit 2024-03-26T03:50:48.498899+0000 mon.a (mon.0) 1674 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi175-64398-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi175-64398-32"}]': finished 2024-03-26T03:50:49.802 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:49 smithi190 bash[23392]: audit 2024-03-26T03:50:48.499203+0000 mon.a (mon.0) 1675 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:49.802 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:49 smithi190 bash[23392]: cluster 2024-03-26T03:50:48.512251+0000 mon.a (mon.0) 1676 : cluster [DBG] osdmap e192: 8 total, 8 up, 8 in 2024-03-26T03:50:49.802 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:49 smithi190 bash[23392]: audit 2024-03-26T03:50:48.519711+0000 mon.c (mon.2) 351 : audit [INF] from='client.? 172.21.15.175:0/1789795922' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi175-64406-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:49.803 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:49 smithi190 bash[23392]: audit 2024-03-26T03:50:48.520661+0000 mon.a (mon.0) 1677 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi175-64406-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:49.803 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:49 smithi190 bash[23392]: audit 2024-03-26T03:50:48.536911+0000 mon.a (mon.0) 1678 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:50:49.803 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:49 smithi190 bash[23392]: cluster 2024-03-26T03:50:48.629222+0000 mgr.y (mgr.24431) 267 : cluster [DBG] pgmap v285: 332 pgs: 72 unknown, 260 active+clean; 584 KiB data, 559 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:50:49.803 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:49 smithi190 bash[23392]: audit 2024-03-26T03:50:49.301065+0000 mon.c (mon.2) 352 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:50:49.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:49 smithi175 bash[29284]: audit 2024-03-26T03:50:48.498899+0000 mon.a (mon.0) 1674 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi175-64398-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi175-64398-32"}]': finished 2024-03-26T03:50:49.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:49 smithi175 bash[29284]: audit 2024-03-26T03:50:48.499203+0000 mon.a (mon.0) 1675 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:49.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:49 smithi175 bash[29284]: cluster 2024-03-26T03:50:48.512251+0000 mon.a (mon.0) 1676 : cluster [DBG] osdmap e192: 8 total, 8 up, 8 in 2024-03-26T03:50:49.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:49 smithi175 bash[29284]: audit 2024-03-26T03:50:48.519711+0000 mon.c (mon.2) 351 : audit [INF] from='client.? 172.21.15.175:0/1789795922' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi175-64406-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:49.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:49 smithi175 bash[29284]: audit 2024-03-26T03:50:48.520661+0000 mon.a (mon.0) 1677 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi175-64406-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:49.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:49 smithi175 bash[29284]: audit 2024-03-26T03:50:48.536911+0000 mon.a (mon.0) 1678 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:50:49.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:49 smithi175 bash[29284]: cluster 2024-03-26T03:50:48.629222+0000 mgr.y (mgr.24431) 267 : cluster [DBG] pgmap v285: 332 pgs: 72 unknown, 260 active+clean; 584 KiB data, 559 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:50:49.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:49 smithi175 bash[29284]: audit 2024-03-26T03:50:49.301065+0000 mon.c (mon.2) 352 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:50:49.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:49 smithi175 bash[21527]: audit 2024-03-26T03:50:48.498899+0000 mon.a (mon.0) 1674 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi175-64398-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi175-64398-32"}]': finished 2024-03-26T03:50:49.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:49 smithi175 bash[21527]: audit 2024-03-26T03:50:48.499203+0000 mon.a (mon.0) 1675 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:49.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:49 smithi175 bash[21527]: cluster 2024-03-26T03:50:48.512251+0000 mon.a (mon.0) 1676 : cluster [DBG] osdmap e192: 8 total, 8 up, 8 in 2024-03-26T03:50:49.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:49 smithi175 bash[21527]: audit 2024-03-26T03:50:48.519711+0000 mon.c (mon.2) 351 : audit [INF] from='client.? 172.21.15.175:0/1789795922' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi175-64406-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:49.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:49 smithi175 bash[21527]: audit 2024-03-26T03:50:48.520661+0000 mon.a (mon.0) 1677 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi175-64406-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:49.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:49 smithi175 bash[21527]: audit 2024-03-26T03:50:48.536911+0000 mon.a (mon.0) 1678 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:50:49.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:49 smithi175 bash[21527]: cluster 2024-03-26T03:50:48.629222+0000 mgr.y (mgr.24431) 267 : cluster [DBG] pgmap v285: 332 pgs: 72 unknown, 260 active+clean; 584 KiB data, 559 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:50:49.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:49 smithi175 bash[21527]: audit 2024-03-26T03:50:49.301065+0000 mon.c (mon.2) 352 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:50:50.240 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:50:49 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:50:50.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:50 smithi190 bash[23392]: cluster 2024-03-26T03:50:49.500332+0000 mon.a (mon.0) 1679 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:50:50.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:50 smithi190 bash[23392]: audit 2024-03-26T03:50:49.502231+0000 mon.a (mon.0) 1680 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi175-64406-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:50.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:50 smithi190 bash[23392]: audit 2024-03-26T03:50:49.502501+0000 mon.a (mon.0) 1681 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-21", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:50:50.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:50 smithi190 bash[23392]: cluster 2024-03-26T03:50:49.513618+0000 mon.a (mon.0) 1682 : cluster [DBG] osdmap e193: 8 total, 8 up, 8 in 2024-03-26T03:50:50.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:50 smithi190 bash[23392]: audit 2024-03-26T03:50:49.517034+0000 mon.a (mon.0) 1683 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-21"}]: dispatch 2024-03-26T03:50:50.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:50 smithi190 bash[23392]: audit 2024-03-26T03:50:49.804309+0000 mgr.y (mgr.24431) 268 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:50:50.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:50 smithi190 bash[23392]: audit 2024-03-26T03:50:50.505356+0000 mon.a (mon.0) 1684 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-21"}]': finished 2024-03-26T03:50:50.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:50 smithi190 bash[23392]: cluster 2024-03-26T03:50:50.515182+0000 mon.a (mon.0) 1685 : cluster [DBG] osdmap e194: 8 total, 8 up, 8 in 2024-03-26T03:50:50.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:50 smithi190 bash[23392]: audit 2024-03-26T03:50:50.516099+0000 mon.a (mon.0) 1686 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-21", "mode": "writeback"}]: dispatch 2024-03-26T03:50:50.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:50 smithi190 bash[23392]: audit 2024-03-26T03:50:50.518739+0000 mon.c (mon.2) 353 : audit [INF] from='client.? 172.21.15.175:0/171500640' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi175-64398-32"}]: dispatch 2024-03-26T03:50:50.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:50 smithi190 bash[23392]: audit 2024-03-26T03:50:50.519946+0000 mon.a (mon.0) 1687 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi175-64398-32"}]: dispatch 2024-03-26T03:50:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:50 smithi175 bash[21527]: cluster 2024-03-26T03:50:49.500332+0000 mon.a (mon.0) 1679 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:50:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:50 smithi175 bash[21527]: audit 2024-03-26T03:50:49.502231+0000 mon.a (mon.0) 1680 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi175-64406-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:50 smithi175 bash[21527]: audit 2024-03-26T03:50:49.502501+0000 mon.a (mon.0) 1681 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-21", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:50:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:50 smithi175 bash[21527]: cluster 2024-03-26T03:50:49.513618+0000 mon.a (mon.0) 1682 : cluster [DBG] osdmap e193: 8 total, 8 up, 8 in 2024-03-26T03:50:50.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:50 smithi175 bash[21527]: audit 2024-03-26T03:50:49.517034+0000 mon.a (mon.0) 1683 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-21"}]: dispatch 2024-03-26T03:50:50.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:50 smithi175 bash[21527]: audit 2024-03-26T03:50:49.804309+0000 mgr.y (mgr.24431) 268 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:50:50.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:50 smithi175 bash[21527]: audit 2024-03-26T03:50:50.505356+0000 mon.a (mon.0) 1684 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-21"}]': finished 2024-03-26T03:50:50.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:50 smithi175 bash[21527]: cluster 2024-03-26T03:50:50.515182+0000 mon.a (mon.0) 1685 : cluster [DBG] osdmap e194: 8 total, 8 up, 8 in 2024-03-26T03:50:50.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:50 smithi175 bash[21527]: audit 2024-03-26T03:50:50.516099+0000 mon.a (mon.0) 1686 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-21", "mode": "writeback"}]: dispatch 2024-03-26T03:50:50.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:50 smithi175 bash[21527]: audit 2024-03-26T03:50:50.518739+0000 mon.c (mon.2) 353 : audit [INF] from='client.? 172.21.15.175:0/171500640' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi175-64398-32"}]: dispatch 2024-03-26T03:50:50.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:50 smithi175 bash[21527]: audit 2024-03-26T03:50:50.519946+0000 mon.a (mon.0) 1687 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi175-64398-32"}]: dispatch 2024-03-26T03:50:50.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:50 smithi175 bash[29284]: cluster 2024-03-26T03:50:49.500332+0000 mon.a (mon.0) 1679 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:50:50.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:50 smithi175 bash[29284]: audit 2024-03-26T03:50:49.502231+0000 mon.a (mon.0) 1680 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi175-64406-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:50.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:50 smithi175 bash[29284]: audit 2024-03-26T03:50:49.502501+0000 mon.a (mon.0) 1681 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-21", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:50:50.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:50 smithi175 bash[29284]: cluster 2024-03-26T03:50:49.513618+0000 mon.a (mon.0) 1682 : cluster [DBG] osdmap e193: 8 total, 8 up, 8 in 2024-03-26T03:50:50.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:50 smithi175 bash[29284]: audit 2024-03-26T03:50:49.517034+0000 mon.a (mon.0) 1683 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-21"}]: dispatch 2024-03-26T03:50:50.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:50 smithi175 bash[29284]: audit 2024-03-26T03:50:49.804309+0000 mgr.y (mgr.24431) 268 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:50:50.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:50 smithi175 bash[29284]: audit 2024-03-26T03:50:50.505356+0000 mon.a (mon.0) 1684 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-21"}]': finished 2024-03-26T03:50:50.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:50 smithi175 bash[29284]: cluster 2024-03-26T03:50:50.515182+0000 mon.a (mon.0) 1685 : cluster [DBG] osdmap e194: 8 total, 8 up, 8 in 2024-03-26T03:50:50.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:50 smithi175 bash[29284]: audit 2024-03-26T03:50:50.516099+0000 mon.a (mon.0) 1686 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-21", "mode": "writeback"}]: dispatch 2024-03-26T03:50:50.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:50 smithi175 bash[29284]: audit 2024-03-26T03:50:50.518739+0000 mon.c (mon.2) 353 : audit [INF] from='client.? 172.21.15.175:0/171500640' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi175-64398-32"}]: dispatch 2024-03-26T03:50:50.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:50 smithi175 bash[29284]: audit 2024-03-26T03:50:50.519946+0000 mon.a (mon.0) 1687 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi175-64398-32"}]: dispatch 2024-03-26T03:50:51.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:51 smithi190 bash[23392]: cluster 2024-03-26T03:50:50.630227+0000 mgr.y (mgr.24431) 269 : cluster [DBG] pgmap v288: 292 pgs: 292 active+clean; 584 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:50:51.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:51 smithi190 bash[23392]: audit 2024-03-26T03:50:51.136421+0000 mon.c (mon.2) 354 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:50:51.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:51 smithi190 bash[23392]: cluster 2024-03-26T03:50:51.506311+0000 mon.a (mon.0) 1688 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:50:51.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:51 smithi190 bash[23392]: audit 2024-03-26T03:50:51.508768+0000 mon.a (mon.0) 1689 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-21", "mode": "writeback"}]': finished 2024-03-26T03:50:51.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:51 smithi190 bash[23392]: audit 2024-03-26T03:50:51.508947+0000 mon.a (mon.0) 1690 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi175-64398-32"}]': finished 2024-03-26T03:50:51.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:51 smithi190 bash[23392]: cluster 2024-03-26T03:50:51.520310+0000 mon.a (mon.0) 1691 : cluster [DBG] osdmap e195: 8 total, 8 up, 8 in 2024-03-26T03:50:51.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:51 smithi190 bash[23392]: audit 2024-03-26T03:50:51.520965+0000 mon.c (mon.2) 355 : audit [INF] from='client.? 172.21.15.175:0/171500640' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi175-64398-32"}]: dispatch 2024-03-26T03:50:51.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:51 smithi190 bash[23392]: audit 2024-03-26T03:50:51.523005+0000 mon.a (mon.0) 1692 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi175-64398-32"}]: dispatch 2024-03-26T03:50:51.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:51 smithi190 bash[23392]: audit 2024-03-26T03:50:51.523441+0000 mon.a (mon.0) 1693 : audit [INF] from='client.? 172.21.15.175:0/2399977692' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi175-64406-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:51.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:51 smithi175 bash[21527]: cluster 2024-03-26T03:50:50.630227+0000 mgr.y (mgr.24431) 269 : cluster [DBG] pgmap v288: 292 pgs: 292 active+clean; 584 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:50:51.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:51 smithi175 bash[21527]: audit 2024-03-26T03:50:51.136421+0000 mon.c (mon.2) 354 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:50:51.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:51 smithi175 bash[21527]: cluster 2024-03-26T03:50:51.506311+0000 mon.a (mon.0) 1688 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:50:51.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:51 smithi175 bash[21527]: audit 2024-03-26T03:50:51.508768+0000 mon.a (mon.0) 1689 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-21", "mode": "writeback"}]': finished 2024-03-26T03:50:51.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:51 smithi175 bash[21527]: audit 2024-03-26T03:50:51.508947+0000 mon.a (mon.0) 1690 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi175-64398-32"}]': finished 2024-03-26T03:50:51.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:51 smithi175 bash[21527]: cluster 2024-03-26T03:50:51.520310+0000 mon.a (mon.0) 1691 : cluster [DBG] osdmap e195: 8 total, 8 up, 8 in 2024-03-26T03:50:51.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:51 smithi175 bash[21527]: audit 2024-03-26T03:50:51.520965+0000 mon.c (mon.2) 355 : audit [INF] from='client.? 172.21.15.175:0/171500640' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi175-64398-32"}]: dispatch 2024-03-26T03:50:51.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:51 smithi175 bash[21527]: audit 2024-03-26T03:50:51.523005+0000 mon.a (mon.0) 1692 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi175-64398-32"}]: dispatch 2024-03-26T03:50:51.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:51 smithi175 bash[21527]: audit 2024-03-26T03:50:51.523441+0000 mon.a (mon.0) 1693 : audit [INF] from='client.? 172.21.15.175:0/2399977692' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi175-64406-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:51.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:51 smithi175 bash[29284]: cluster 2024-03-26T03:50:50.630227+0000 mgr.y (mgr.24431) 269 : cluster [DBG] pgmap v288: 292 pgs: 292 active+clean; 584 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:50:51.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:51 smithi175 bash[29284]: audit 2024-03-26T03:50:51.136421+0000 mon.c (mon.2) 354 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:50:51.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:51 smithi175 bash[29284]: cluster 2024-03-26T03:50:51.506311+0000 mon.a (mon.0) 1688 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:50:51.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:51 smithi175 bash[29284]: audit 2024-03-26T03:50:51.508768+0000 mon.a (mon.0) 1689 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-21", "mode": "writeback"}]': finished 2024-03-26T03:50:51.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:51 smithi175 bash[29284]: audit 2024-03-26T03:50:51.508947+0000 mon.a (mon.0) 1690 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi175-64398-32"}]': finished 2024-03-26T03:50:51.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:51 smithi175 bash[29284]: cluster 2024-03-26T03:50:51.520310+0000 mon.a (mon.0) 1691 : cluster [DBG] osdmap e195: 8 total, 8 up, 8 in 2024-03-26T03:50:51.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:51 smithi175 bash[29284]: audit 2024-03-26T03:50:51.520965+0000 mon.c (mon.2) 355 : audit [INF] from='client.? 172.21.15.175:0/171500640' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi175-64398-32"}]: dispatch 2024-03-26T03:50:51.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:51 smithi175 bash[29284]: audit 2024-03-26T03:50:51.523005+0000 mon.a (mon.0) 1692 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi175-64398-32"}]: dispatch 2024-03-26T03:50:51.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:51 smithi175 bash[29284]: audit 2024-03-26T03:50:51.523441+0000 mon.a (mon.0) 1693 : audit [INF] from='client.? 172.21.15.175:0/2399977692' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi175-64406-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:52.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:52 smithi190 bash[23392]: audit 2024-03-26T03:50:51.605760+0000 mon.a (mon.0) 1694 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:50:52.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:52 smithi190 bash[23392]: audit 2024-03-26T03:50:51.878708+0000 mon.c (mon.2) 356 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:50:52.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:52 smithi190 bash[23392]: audit 2024-03-26T03:50:51.880236+0000 mon.c (mon.2) 357 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:50:52.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:52 smithi190 bash[23392]: audit 2024-03-26T03:50:51.890641+0000 mon.a (mon.0) 1695 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:50:52.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:52 smithi190 bash[23392]: audit 2024-03-26T03:50:52.513007+0000 mon.a (mon.0) 1696 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi175-64398-32"}]': finished 2024-03-26T03:50:52.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:52 smithi190 bash[23392]: audit 2024-03-26T03:50:52.513190+0000 mon.a (mon.0) 1697 : audit [INF] from='client.? 172.21.15.175:0/2399977692' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi175-64406-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:52.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:52 smithi190 bash[23392]: audit 2024-03-26T03:50:52.513335+0000 mon.a (mon.0) 1698 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:50:52.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:52 smithi190 bash[23392]: cluster 2024-03-26T03:50:52.521081+0000 mon.a (mon.0) 1699 : cluster [DBG] osdmap e196: 8 total, 8 up, 8 in 2024-03-26T03:50:52.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:52 smithi190 bash[23392]: audit 2024-03-26T03:50:52.522333+0000 mon.a (mon.0) 1700 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-21"}]: dispatch 2024-03-26T03:50:52.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:52 smithi175 bash[29284]: audit 2024-03-26T03:50:51.605760+0000 mon.a (mon.0) 1694 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:50:52.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:52 smithi175 bash[29284]: audit 2024-03-26T03:50:51.878708+0000 mon.c (mon.2) 356 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:50:52.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:52 smithi175 bash[29284]: audit 2024-03-26T03:50:51.880236+0000 mon.c (mon.2) 357 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:50:52.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:52 smithi175 bash[29284]: audit 2024-03-26T03:50:51.890641+0000 mon.a (mon.0) 1695 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:50:52.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:52 smithi175 bash[29284]: audit 2024-03-26T03:50:52.513007+0000 mon.a (mon.0) 1696 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi175-64398-32"}]': finished 2024-03-26T03:50:52.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:52 smithi175 bash[29284]: audit 2024-03-26T03:50:52.513190+0000 mon.a (mon.0) 1697 : audit [INF] from='client.? 172.21.15.175:0/2399977692' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi175-64406-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:52.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:52 smithi175 bash[29284]: audit 2024-03-26T03:50:52.513335+0000 mon.a (mon.0) 1698 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:50:52.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:52 smithi175 bash[29284]: cluster 2024-03-26T03:50:52.521081+0000 mon.a (mon.0) 1699 : cluster [DBG] osdmap e196: 8 total, 8 up, 8 in 2024-03-26T03:50:52.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:52 smithi175 bash[29284]: audit 2024-03-26T03:50:52.522333+0000 mon.a (mon.0) 1700 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-21"}]: dispatch 2024-03-26T03:50:52.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:52 smithi175 bash[21527]: audit 2024-03-26T03:50:51.605760+0000 mon.a (mon.0) 1694 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:50:52.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:52 smithi175 bash[21527]: audit 2024-03-26T03:50:51.878708+0000 mon.c (mon.2) 356 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:50:52.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:52 smithi175 bash[21527]: audit 2024-03-26T03:50:51.880236+0000 mon.c (mon.2) 357 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:50:52.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:52 smithi175 bash[21527]: audit 2024-03-26T03:50:51.890641+0000 mon.a (mon.0) 1695 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:50:52.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:52 smithi175 bash[21527]: audit 2024-03-26T03:50:52.513007+0000 mon.a (mon.0) 1696 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi175-64398-32"}]': finished 2024-03-26T03:50:52.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:52 smithi175 bash[21527]: audit 2024-03-26T03:50:52.513190+0000 mon.a (mon.0) 1697 : audit [INF] from='client.? 172.21.15.175:0/2399977692' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi175-64406-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:52.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:52 smithi175 bash[21527]: audit 2024-03-26T03:50:52.513335+0000 mon.a (mon.0) 1698 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:50:52.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:52 smithi175 bash[21527]: cluster 2024-03-26T03:50:52.521081+0000 mon.a (mon.0) 1699 : cluster [DBG] osdmap e196: 8 total, 8 up, 8 in 2024-03-26T03:50:52.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:52 smithi175 bash[21527]: audit 2024-03-26T03:50:52.522333+0000 mon.a (mon.0) 1700 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-21"}]: dispatch 2024-03-26T03:50:53.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:53 smithi190 bash[23392]: audit 2024-03-26T03:50:52.578923+0000 mon.a (mon.0) 1701 : audit [INF] from='client.? 172.21.15.175:0/3272159784' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi175-64398-33"}]: dispatch 2024-03-26T03:50:53.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:53 smithi190 bash[23392]: audit 2024-03-26T03:50:52.580128+0000 mon.a (mon.0) 1702 : audit [INF] from='client.? 172.21.15.175:0/3272159784' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi175-64398-33"}]: dispatch 2024-03-26T03:50:53.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:53 smithi190 bash[23392]: audit 2024-03-26T03:50:52.580512+0000 mon.a (mon.0) 1703 : audit [INF] from='client.? 172.21.15.175:0/3272159784' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi175-64398-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:53.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:53 smithi190 bash[23392]: cluster 2024-03-26T03:50:52.630898+0000 mgr.y (mgr.24431) 270 : cluster [DBG] pgmap v291: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:50:53.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:53 smithi190 bash[23392]: cluster 2024-03-26T03:50:53.514291+0000 mon.a (mon.0) 1704 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:50:53.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:53 smithi190 bash[23392]: audit 2024-03-26T03:50:53.517029+0000 mon.a (mon.0) 1705 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-21"}]': finished 2024-03-26T03:50:53.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:53 smithi190 bash[23392]: audit 2024-03-26T03:50:53.517215+0000 mon.a (mon.0) 1706 : audit [INF] from='client.? 172.21.15.175:0/3272159784' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi175-64398-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:53.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:53 smithi190 bash[23392]: cluster 2024-03-26T03:50:53.525199+0000 mon.a (mon.0) 1707 : cluster [DBG] osdmap e197: 8 total, 8 up, 8 in 2024-03-26T03:50:53.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:53 smithi190 bash[23392]: audit 2024-03-26T03:50:53.527222+0000 mon.a (mon.0) 1708 : audit [INF] from='client.? 172.21.15.175:0/3272159784' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi175-64398-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi175-64398-33"}]: dispatch 2024-03-26T03:50:53.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:53 smithi190 bash[23392]: audit 2024-03-26T03:50:53.529274+0000 mon.a (mon.0) 1709 : audit [INF] from='client.? 172.21.15.175:0/715686378' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi175-64406-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:53.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:53 smithi175 bash[21527]: audit 2024-03-26T03:50:52.578923+0000 mon.a (mon.0) 1701 : audit [INF] from='client.? 172.21.15.175:0/3272159784' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi175-64398-33"}]: dispatch 2024-03-26T03:50:53.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:53 smithi175 bash[21527]: audit 2024-03-26T03:50:52.580128+0000 mon.a (mon.0) 1702 : audit [INF] from='client.? 172.21.15.175:0/3272159784' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi175-64398-33"}]: dispatch 2024-03-26T03:50:53.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:53 smithi175 bash[21527]: audit 2024-03-26T03:50:52.580512+0000 mon.a (mon.0) 1703 : audit [INF] from='client.? 172.21.15.175:0/3272159784' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi175-64398-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:53.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:53 smithi175 bash[21527]: cluster 2024-03-26T03:50:52.630898+0000 mgr.y (mgr.24431) 270 : cluster [DBG] pgmap v291: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:50:53.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:53 smithi175 bash[21527]: cluster 2024-03-26T03:50:53.514291+0000 mon.a (mon.0) 1704 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:50:53.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:53 smithi175 bash[21527]: audit 2024-03-26T03:50:53.517029+0000 mon.a (mon.0) 1705 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-21"}]': finished 2024-03-26T03:50:53.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:53 smithi175 bash[21527]: audit 2024-03-26T03:50:53.517215+0000 mon.a (mon.0) 1706 : audit [INF] from='client.? 172.21.15.175:0/3272159784' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi175-64398-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:53.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:53 smithi175 bash[21527]: cluster 2024-03-26T03:50:53.525199+0000 mon.a (mon.0) 1707 : cluster [DBG] osdmap e197: 8 total, 8 up, 8 in 2024-03-26T03:50:53.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:53 smithi175 bash[21527]: audit 2024-03-26T03:50:53.527222+0000 mon.a (mon.0) 1708 : audit [INF] from='client.? 172.21.15.175:0/3272159784' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi175-64398-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi175-64398-33"}]: dispatch 2024-03-26T03:50:53.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:53 smithi175 bash[21527]: audit 2024-03-26T03:50:53.529274+0000 mon.a (mon.0) 1709 : audit [INF] from='client.? 172.21.15.175:0/715686378' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi175-64406-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:53.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:53 smithi175 bash[29284]: audit 2024-03-26T03:50:52.578923+0000 mon.a (mon.0) 1701 : audit [INF] from='client.? 172.21.15.175:0/3272159784' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi175-64398-33"}]: dispatch 2024-03-26T03:50:53.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:53 smithi175 bash[29284]: audit 2024-03-26T03:50:52.580128+0000 mon.a (mon.0) 1702 : audit [INF] from='client.? 172.21.15.175:0/3272159784' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi175-64398-33"}]: dispatch 2024-03-26T03:50:53.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:53 smithi175 bash[29284]: audit 2024-03-26T03:50:52.580512+0000 mon.a (mon.0) 1703 : audit [INF] from='client.? 172.21.15.175:0/3272159784' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi175-64398-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:50:53.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:53 smithi175 bash[29284]: cluster 2024-03-26T03:50:52.630898+0000 mgr.y (mgr.24431) 270 : cluster [DBG] pgmap v291: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:50:53.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:53 smithi175 bash[29284]: cluster 2024-03-26T03:50:53.514291+0000 mon.a (mon.0) 1704 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:50:53.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:53 smithi175 bash[29284]: audit 2024-03-26T03:50:53.517029+0000 mon.a (mon.0) 1705 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-21"}]': finished 2024-03-26T03:50:53.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:53 smithi175 bash[29284]: audit 2024-03-26T03:50:53.517215+0000 mon.a (mon.0) 1706 : audit [INF] from='client.? 172.21.15.175:0/3272159784' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi175-64398-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:50:53.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:53 smithi175 bash[29284]: cluster 2024-03-26T03:50:53.525199+0000 mon.a (mon.0) 1707 : cluster [DBG] osdmap e197: 8 total, 8 up, 8 in 2024-03-26T03:50:53.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:53 smithi175 bash[29284]: audit 2024-03-26T03:50:53.527222+0000 mon.a (mon.0) 1708 : audit [INF] from='client.? 172.21.15.175:0/3272159784' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi175-64398-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi175-64398-33"}]: dispatch 2024-03-26T03:50:53.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:53 smithi175 bash[29284]: audit 2024-03-26T03:50:53.529274+0000 mon.a (mon.0) 1709 : audit [INF] from='client.? 172.21.15.175:0/715686378' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi175-64406-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:55.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:55 smithi190 bash[23392]: audit 2024-03-26T03:50:54.521261+0000 mon.a (mon.0) 1710 : audit [INF] from='client.? 172.21.15.175:0/715686378' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi175-64406-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:55.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:55 smithi190 bash[23392]: cluster 2024-03-26T03:50:54.547294+0000 mon.a (mon.0) 1711 : cluster [DBG] osdmap e198: 8 total, 8 up, 8 in 2024-03-26T03:50:55.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:55 smithi190 bash[23392]: cluster 2024-03-26T03:50:54.631861+0000 mgr.y (mgr.24431) 271 : cluster [DBG] pgmap v294: 324 pgs: 2 creating+activating, 31 unknown, 291 active+clean; 584 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-26T03:50:55.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:55 smithi175 bash[29284]: audit 2024-03-26T03:50:54.521261+0000 mon.a (mon.0) 1710 : audit [INF] from='client.? 172.21.15.175:0/715686378' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi175-64406-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:55.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:55 smithi175 bash[29284]: cluster 2024-03-26T03:50:54.547294+0000 mon.a (mon.0) 1711 : cluster [DBG] osdmap e198: 8 total, 8 up, 8 in 2024-03-26T03:50:55.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:55 smithi175 bash[29284]: cluster 2024-03-26T03:50:54.631861+0000 mgr.y (mgr.24431) 271 : cluster [DBG] pgmap v294: 324 pgs: 2 creating+activating, 31 unknown, 291 active+clean; 584 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-26T03:50:55.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:55 smithi175 bash[21527]: audit 2024-03-26T03:50:54.521261+0000 mon.a (mon.0) 1710 : audit [INF] from='client.? 172.21.15.175:0/715686378' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi175-64406-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:55.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:55 smithi175 bash[21527]: cluster 2024-03-26T03:50:54.547294+0000 mon.a (mon.0) 1711 : cluster [DBG] osdmap e198: 8 total, 8 up, 8 in 2024-03-26T03:50:55.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:55 smithi175 bash[21527]: cluster 2024-03-26T03:50:54.631861+0000 mgr.y (mgr.24431) 271 : cluster [DBG] pgmap v294: 324 pgs: 2 creating+activating, 31 unknown, 291 active+clean; 584 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-26T03:50:56.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:56 smithi190 bash[23392]: audit 2024-03-26T03:50:55.537062+0000 mon.a (mon.0) 1712 : audit [INF] from='client.? 172.21.15.175:0/3272159784' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi175-64398-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi175-64398-33"}]': finished 2024-03-26T03:50:56.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:56 smithi190 bash[23392]: cluster 2024-03-26T03:50:55.547048+0000 mon.a (mon.0) 1713 : cluster [DBG] osdmap e199: 8 total, 8 up, 8 in 2024-03-26T03:50:56.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:56 smithi190 bash[23392]: audit 2024-03-26T03:50:55.550815+0000 mon.a (mon.0) 1714 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:56.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:56 smithi190 bash[23392]: audit 2024-03-26T03:50:55.556203+0000 mon.c (mon.2) 358 : audit [INF] from='client.? 172.21.15.175:0/1991897381' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi175-64406-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:56.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:56 smithi190 bash[23392]: audit 2024-03-26T03:50:55.557007+0000 mon.a (mon.0) 1715 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi175-64406-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:56.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:56 smithi190 bash[23392]: cluster 2024-03-26T03:50:56.429510+0000 mon.a (mon.0) 1716 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:50:56.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:56 smithi175 bash[21527]: audit 2024-03-26T03:50:55.537062+0000 mon.a (mon.0) 1712 : audit [INF] from='client.? 172.21.15.175:0/3272159784' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi175-64398-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi175-64398-33"}]': finished 2024-03-26T03:50:56.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:56 smithi175 bash[21527]: cluster 2024-03-26T03:50:55.547048+0000 mon.a (mon.0) 1713 : cluster [DBG] osdmap e199: 8 total, 8 up, 8 in 2024-03-26T03:50:56.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:56 smithi175 bash[21527]: audit 2024-03-26T03:50:55.550815+0000 mon.a (mon.0) 1714 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:56.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:56 smithi175 bash[21527]: audit 2024-03-26T03:50:55.556203+0000 mon.c (mon.2) 358 : audit [INF] from='client.? 172.21.15.175:0/1991897381' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi175-64406-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:56.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:56 smithi175 bash[21527]: audit 2024-03-26T03:50:55.557007+0000 mon.a (mon.0) 1715 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi175-64406-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:56.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:56 smithi175 bash[21527]: cluster 2024-03-26T03:50:56.429510+0000 mon.a (mon.0) 1716 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:50:56.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:56 smithi175 bash[29284]: audit 2024-03-26T03:50:55.537062+0000 mon.a (mon.0) 1712 : audit [INF] from='client.? 172.21.15.175:0/3272159784' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi175-64398-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi175-64398-33"}]': finished 2024-03-26T03:50:56.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:56 smithi175 bash[29284]: cluster 2024-03-26T03:50:55.547048+0000 mon.a (mon.0) 1713 : cluster [DBG] osdmap e199: 8 total, 8 up, 8 in 2024-03-26T03:50:56.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:56 smithi175 bash[29284]: audit 2024-03-26T03:50:55.550815+0000 mon.a (mon.0) 1714 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:56.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:56 smithi175 bash[29284]: audit 2024-03-26T03:50:55.556203+0000 mon.c (mon.2) 358 : audit [INF] from='client.? 172.21.15.175:0/1991897381' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi175-64406-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:56.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:56 smithi175 bash[29284]: audit 2024-03-26T03:50:55.557007+0000 mon.a (mon.0) 1715 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi175-64406-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:50:56.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:56 smithi175 bash[29284]: cluster 2024-03-26T03:50:56.429510+0000 mon.a (mon.0) 1716 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:50:57.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:57 smithi190 bash[23392]: audit 2024-03-26T03:50:56.539974+0000 mon.a (mon.0) 1717 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:57.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:57 smithi190 bash[23392]: audit 2024-03-26T03:50:56.540111+0000 mon.a (mon.0) 1718 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi175-64406-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:57.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:57 smithi190 bash[23392]: cluster 2024-03-26T03:50:56.547407+0000 mon.a (mon.0) 1719 : cluster [DBG] osdmap e200: 8 total, 8 up, 8 in 2024-03-26T03:50:57.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:57 smithi190 bash[23392]: cluster 2024-03-26T03:50:56.632505+0000 mgr.y (mgr.24431) 272 : cluster [DBG] pgmap v297: 396 pgs: 72 unknown, 2 creating+activating, 322 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-26T03:50:57.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:57 smithi175 bash[29284]: audit 2024-03-26T03:50:56.539974+0000 mon.a (mon.0) 1717 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:57.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:57 smithi175 bash[29284]: audit 2024-03-26T03:50:56.540111+0000 mon.a (mon.0) 1718 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi175-64406-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:57.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:57 smithi175 bash[29284]: cluster 2024-03-26T03:50:56.547407+0000 mon.a (mon.0) 1719 : cluster [DBG] osdmap e200: 8 total, 8 up, 8 in 2024-03-26T03:50:57.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:57 smithi175 bash[29284]: cluster 2024-03-26T03:50:56.632505+0000 mgr.y (mgr.24431) 272 : cluster [DBG] pgmap v297: 396 pgs: 72 unknown, 2 creating+activating, 322 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-26T03:50:57.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:57 smithi175 bash[21527]: audit 2024-03-26T03:50:56.539974+0000 mon.a (mon.0) 1717 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:57.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:57 smithi175 bash[21527]: audit 2024-03-26T03:50:56.540111+0000 mon.a (mon.0) 1718 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi175-64406-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:50:57.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:57 smithi175 bash[21527]: cluster 2024-03-26T03:50:56.547407+0000 mon.a (mon.0) 1719 : cluster [DBG] osdmap e200: 8 total, 8 up, 8 in 2024-03-26T03:50:57.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:57 smithi175 bash[21527]: cluster 2024-03-26T03:50:56.632505+0000 mgr.y (mgr.24431) 272 : cluster [DBG] pgmap v297: 396 pgs: 72 unknown, 2 creating+activating, 322 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-26T03:50:58.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:58 smithi190 bash[23392]: cluster 2024-03-26T03:50:57.582630+0000 mon.a (mon.0) 1720 : cluster [DBG] osdmap e201: 8 total, 8 up, 8 in 2024-03-26T03:50:58.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:58 smithi190 bash[23392]: audit 2024-03-26T03:50:57.585603+0000 mon.a (mon.0) 1721 : audit [INF] from='client.? 172.21.15.175:0/3272159784' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi175-64398-33"}]: dispatch 2024-03-26T03:50:58.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:58 smithi190 bash[23392]: audit 2024-03-26T03:50:57.659944+0000 mon.a (mon.0) 1722 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:50:58.991 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:50:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:50:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:50:58.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:58 smithi175 bash[29284]: cluster 2024-03-26T03:50:57.582630+0000 mon.a (mon.0) 1720 : cluster [DBG] osdmap e201: 8 total, 8 up, 8 in 2024-03-26T03:50:58.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:58 smithi175 bash[29284]: audit 2024-03-26T03:50:57.585603+0000 mon.a (mon.0) 1721 : audit [INF] from='client.? 172.21.15.175:0/3272159784' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi175-64398-33"}]: dispatch 2024-03-26T03:50:58.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:58 smithi175 bash[29284]: audit 2024-03-26T03:50:57.659944+0000 mon.a (mon.0) 1722 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:50:58.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:58 smithi175 bash[21527]: cluster 2024-03-26T03:50:57.582630+0000 mon.a (mon.0) 1720 : cluster [DBG] osdmap e201: 8 total, 8 up, 8 in 2024-03-26T03:50:58.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:58 smithi175 bash[21527]: audit 2024-03-26T03:50:57.585603+0000 mon.a (mon.0) 1721 : audit [INF] from='client.? 172.21.15.175:0/3272159784' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi175-64398-33"}]: dispatch 2024-03-26T03:50:58.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:58 smithi175 bash[21527]: audit 2024-03-26T03:50:57.659944+0000 mon.a (mon.0) 1722 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:50:59.990 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:50:59 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:50:59.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:59 smithi190 bash[23392]: audit 2024-03-26T03:50:58.577745+0000 mon.a (mon.0) 1723 : audit [INF] from='client.? 172.21.15.175:0/3272159784' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi175-64398-33"}]': finished 2024-03-26T03:50:59.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:59 smithi190 bash[23392]: audit 2024-03-26T03:50:58.577950+0000 mon.a (mon.0) 1724 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-23", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:50:59.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:59 smithi190 bash[23392]: cluster 2024-03-26T03:50:58.585797+0000 mon.a (mon.0) 1725 : cluster [DBG] osdmap e202: 8 total, 8 up, 8 in 2024-03-26T03:50:59.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:59 smithi190 bash[23392]: audit 2024-03-26T03:50:58.587640+0000 mon.a (mon.0) 1726 : audit [INF] from='client.? 172.21.15.175:0/3272159784' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi175-64398-33"}]: dispatch 2024-03-26T03:50:59.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:59 smithi190 bash[23392]: audit 2024-03-26T03:50:58.588648+0000 mon.a (mon.0) 1727 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-23"}]: dispatch 2024-03-26T03:50:59.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:59 smithi190 bash[23392]: cluster 2024-03-26T03:50:58.633385+0000 mgr.y (mgr.24431) 273 : cluster [DBG] pgmap v300: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:50:59.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:50:59 smithi190 bash[23392]: audit 2024-03-26T03:50:59.581119+0000 mon.a (mon.0) 1728 : audit [INF] from='client.? 172.21.15.175:0/3272159784' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi175-64398-33"}]': finished 2024-03-26T03:50:59.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:59 smithi175 bash[21527]: audit 2024-03-26T03:50:58.577745+0000 mon.a (mon.0) 1723 : audit [INF] from='client.? 172.21.15.175:0/3272159784' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi175-64398-33"}]': finished 2024-03-26T03:50:59.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:59 smithi175 bash[21527]: audit 2024-03-26T03:50:58.577950+0000 mon.a (mon.0) 1724 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-23", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:50:59.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:59 smithi175 bash[21527]: cluster 2024-03-26T03:50:58.585797+0000 mon.a (mon.0) 1725 : cluster [DBG] osdmap e202: 8 total, 8 up, 8 in 2024-03-26T03:50:59.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:59 smithi175 bash[21527]: audit 2024-03-26T03:50:58.587640+0000 mon.a (mon.0) 1726 : audit [INF] from='client.? 172.21.15.175:0/3272159784' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi175-64398-33"}]: dispatch 2024-03-26T03:50:59.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:59 smithi175 bash[21527]: audit 2024-03-26T03:50:58.588648+0000 mon.a (mon.0) 1727 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-23"}]: dispatch 2024-03-26T03:50:59.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:59 smithi175 bash[21527]: cluster 2024-03-26T03:50:58.633385+0000 mgr.y (mgr.24431) 273 : cluster [DBG] pgmap v300: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:50:59.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:50:59 smithi175 bash[21527]: audit 2024-03-26T03:50:59.581119+0000 mon.a (mon.0) 1728 : audit [INF] from='client.? 172.21.15.175:0/3272159784' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi175-64398-33"}]': finished 2024-03-26T03:50:59.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:59 smithi175 bash[29284]: audit 2024-03-26T03:50:58.577745+0000 mon.a (mon.0) 1723 : audit [INF] from='client.? 172.21.15.175:0/3272159784' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi175-64398-33"}]': finished 2024-03-26T03:50:59.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:59 smithi175 bash[29284]: audit 2024-03-26T03:50:58.577950+0000 mon.a (mon.0) 1724 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-23", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:50:59.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:59 smithi175 bash[29284]: cluster 2024-03-26T03:50:58.585797+0000 mon.a (mon.0) 1725 : cluster [DBG] osdmap e202: 8 total, 8 up, 8 in 2024-03-26T03:50:59.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:59 smithi175 bash[29284]: audit 2024-03-26T03:50:58.587640+0000 mon.a (mon.0) 1726 : audit [INF] from='client.? 172.21.15.175:0/3272159784' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi175-64398-33"}]: dispatch 2024-03-26T03:50:59.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:59 smithi175 bash[29284]: audit 2024-03-26T03:50:58.588648+0000 mon.a (mon.0) 1727 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-23"}]: dispatch 2024-03-26T03:50:59.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:59 smithi175 bash[29284]: cluster 2024-03-26T03:50:58.633385+0000 mgr.y (mgr.24431) 273 : cluster [DBG] pgmap v300: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:50:59.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:50:59 smithi175 bash[29284]: audit 2024-03-26T03:50:59.581119+0000 mon.a (mon.0) 1728 : audit [INF] from='client.? 172.21.15.175:0/3272159784' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi175-64398-33"}]': finished 2024-03-26T03:51:00.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:00 smithi190 bash[23392]: audit 2024-03-26T03:50:59.581296+0000 mon.a (mon.0) 1729 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-23"}]': finished 2024-03-26T03:51:00.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:00 smithi190 bash[23392]: cluster 2024-03-26T03:50:59.588944+0000 mon.a (mon.0) 1730 : cluster [DBG] osdmap e203: 8 total, 8 up, 8 in 2024-03-26T03:51:00.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:00 smithi190 bash[23392]: audit 2024-03-26T03:50:59.593592+0000 mon.a (mon.0) 1731 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-23", "mode": "writeback"}]: dispatch 2024-03-26T03:51:00.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:00 smithi190 bash[23392]: audit 2024-03-26T03:50:59.641599+0000 mon.c (mon.2) 359 : audit [INF] from='client.? 172.21.15.175:0/3430821660' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi175-64398-34"}]: dispatch 2024-03-26T03:51:00.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:00 smithi190 bash[23392]: audit 2024-03-26T03:50:59.642045+0000 mon.a (mon.0) 1732 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi175-64398-34"}]: dispatch 2024-03-26T03:51:00.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:00 smithi190 bash[23392]: audit 2024-03-26T03:50:59.642742+0000 mon.c (mon.2) 360 : audit [INF] from='client.? 172.21.15.175:0/3430821660' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi175-64398-34"}]: dispatch 2024-03-26T03:51:00.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:00 smithi190 bash[23392]: audit 2024-03-26T03:50:59.648223+0000 mon.a (mon.0) 1733 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi175-64398-34"}]: dispatch 2024-03-26T03:51:00.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:00 smithi190 bash[23392]: audit 2024-03-26T03:50:59.648870+0000 mon.c (mon.2) 361 : audit [INF] from='client.? 172.21.15.175:0/3430821660' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi175-64398-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:00.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:00 smithi190 bash[23392]: audit 2024-03-26T03:50:59.649214+0000 mon.a (mon.0) 1734 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi175-64398-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:00.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:00 smithi190 bash[23392]: audit 2024-03-26T03:50:59.813908+0000 mgr.y (mgr.24431) 274 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:51:00.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:00 smithi190 bash[23392]: cluster 2024-03-26T03:51:00.582474+0000 mon.a (mon.0) 1735 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:51:00.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:00 smithi190 bash[23392]: audit 2024-03-26T03:51:00.585478+0000 mon.a (mon.0) 1736 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-23", "mode": "writeback"}]': finished 2024-03-26T03:51:00.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:00 smithi190 bash[23392]: audit 2024-03-26T03:51:00.585742+0000 mon.a (mon.0) 1737 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi175-64398-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:00.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:00 smithi190 bash[23392]: cluster 2024-03-26T03:51:00.599059+0000 mon.a (mon.0) 1738 : cluster [DBG] osdmap e204: 8 total, 8 up, 8 in 2024-03-26T03:51:00.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:00 smithi175 bash[29284]: audit 2024-03-26T03:50:59.581296+0000 mon.a (mon.0) 1729 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-23"}]': finished 2024-03-26T03:51:00.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:00 smithi175 bash[29284]: cluster 2024-03-26T03:50:59.588944+0000 mon.a (mon.0) 1730 : cluster [DBG] osdmap e203: 8 total, 8 up, 8 in 2024-03-26T03:51:00.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:00 smithi175 bash[29284]: audit 2024-03-26T03:50:59.593592+0000 mon.a (mon.0) 1731 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-23", "mode": "writeback"}]: dispatch 2024-03-26T03:51:00.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:00 smithi175 bash[29284]: audit 2024-03-26T03:50:59.641599+0000 mon.c (mon.2) 359 : audit [INF] from='client.? 172.21.15.175:0/3430821660' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi175-64398-34"}]: dispatch 2024-03-26T03:51:00.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:00 smithi175 bash[29284]: audit 2024-03-26T03:50:59.642045+0000 mon.a (mon.0) 1732 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi175-64398-34"}]: dispatch 2024-03-26T03:51:00.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:00 smithi175 bash[29284]: audit 2024-03-26T03:50:59.642742+0000 mon.c (mon.2) 360 : audit [INF] from='client.? 172.21.15.175:0/3430821660' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi175-64398-34"}]: dispatch 2024-03-26T03:51:00.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:00 smithi175 bash[29284]: audit 2024-03-26T03:50:59.648223+0000 mon.a (mon.0) 1733 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi175-64398-34"}]: dispatch 2024-03-26T03:51:00.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:00 smithi175 bash[29284]: audit 2024-03-26T03:50:59.648870+0000 mon.c (mon.2) 361 : audit [INF] from='client.? 172.21.15.175:0/3430821660' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi175-64398-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:00.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:00 smithi175 bash[29284]: audit 2024-03-26T03:50:59.649214+0000 mon.a (mon.0) 1734 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi175-64398-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:00.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:00 smithi175 bash[29284]: audit 2024-03-26T03:50:59.813908+0000 mgr.y (mgr.24431) 274 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:51:00.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:00 smithi175 bash[29284]: cluster 2024-03-26T03:51:00.582474+0000 mon.a (mon.0) 1735 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:51:00.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:00 smithi175 bash[29284]: audit 2024-03-26T03:51:00.585478+0000 mon.a (mon.0) 1736 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-23", "mode": "writeback"}]': finished 2024-03-26T03:51:00.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:00 smithi175 bash[29284]: audit 2024-03-26T03:51:00.585742+0000 mon.a (mon.0) 1737 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi175-64398-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:00.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:00 smithi175 bash[29284]: cluster 2024-03-26T03:51:00.599059+0000 mon.a (mon.0) 1738 : cluster [DBG] osdmap e204: 8 total, 8 up, 8 in 2024-03-26T03:51:00.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:00 smithi175 bash[21527]: audit 2024-03-26T03:50:59.581296+0000 mon.a (mon.0) 1729 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-23"}]': finished 2024-03-26T03:51:00.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:00 smithi175 bash[21527]: cluster 2024-03-26T03:50:59.588944+0000 mon.a (mon.0) 1730 : cluster [DBG] osdmap e203: 8 total, 8 up, 8 in 2024-03-26T03:51:00.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:00 smithi175 bash[21527]: audit 2024-03-26T03:50:59.593592+0000 mon.a (mon.0) 1731 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-23", "mode": "writeback"}]: dispatch 2024-03-26T03:51:00.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:00 smithi175 bash[21527]: audit 2024-03-26T03:50:59.641599+0000 mon.c (mon.2) 359 : audit [INF] from='client.? 172.21.15.175:0/3430821660' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi175-64398-34"}]: dispatch 2024-03-26T03:51:00.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:00 smithi175 bash[21527]: audit 2024-03-26T03:50:59.642045+0000 mon.a (mon.0) 1732 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi175-64398-34"}]: dispatch 2024-03-26T03:51:00.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:00 smithi175 bash[21527]: audit 2024-03-26T03:50:59.642742+0000 mon.c (mon.2) 360 : audit [INF] from='client.? 172.21.15.175:0/3430821660' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi175-64398-34"}]: dispatch 2024-03-26T03:51:00.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:00 smithi175 bash[21527]: audit 2024-03-26T03:50:59.648223+0000 mon.a (mon.0) 1733 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi175-64398-34"}]: dispatch 2024-03-26T03:51:00.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:00 smithi175 bash[21527]: audit 2024-03-26T03:50:59.648870+0000 mon.c (mon.2) 361 : audit [INF] from='client.? 172.21.15.175:0/3430821660' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi175-64398-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:00.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:00 smithi175 bash[21527]: audit 2024-03-26T03:50:59.649214+0000 mon.a (mon.0) 1734 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi175-64398-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:00.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:00 smithi175 bash[21527]: audit 2024-03-26T03:50:59.813908+0000 mgr.y (mgr.24431) 274 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:51:00.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:00 smithi175 bash[21527]: cluster 2024-03-26T03:51:00.582474+0000 mon.a (mon.0) 1735 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:51:00.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:00 smithi175 bash[21527]: audit 2024-03-26T03:51:00.585478+0000 mon.a (mon.0) 1736 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-23", "mode": "writeback"}]': finished 2024-03-26T03:51:00.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:00 smithi175 bash[21527]: audit 2024-03-26T03:51:00.585742+0000 mon.a (mon.0) 1737 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi175-64398-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:00.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:00 smithi175 bash[21527]: cluster 2024-03-26T03:51:00.599059+0000 mon.a (mon.0) 1738 : cluster [DBG] osdmap e204: 8 total, 8 up, 8 in 2024-03-26T03:51:01.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:01 smithi190 bash[23392]: audit 2024-03-26T03:51:00.597979+0000 mon.c (mon.2) 362 : audit [INF] from='client.? 172.21.15.175:0/3430821660' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi175-64398-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi175-64398-34"}]: dispatch 2024-03-26T03:51:01.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:01 smithi190 bash[23392]: audit 2024-03-26T03:51:00.603713+0000 mon.a (mon.0) 1739 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi175-64398-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi175-64398-34"}]: dispatch 2024-03-26T03:51:01.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:01 smithi190 bash[23392]: audit 2024-03-26T03:51:00.619377+0000 mon.c (mon.2) 363 : audit [INF] from='client.? 172.21.15.175:0/3709532255' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:01.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:01 smithi190 bash[23392]: audit 2024-03-26T03:51:00.622187+0000 mon.a (mon.0) 1740 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:01.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:01 smithi190 bash[23392]: cluster 2024-03-26T03:51:00.634028+0000 mgr.y (mgr.24431) 275 : cluster [DBG] pgmap v303: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 561 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-26T03:51:01.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:01 smithi190 bash[23392]: audit 2024-03-26T03:51:01.589567+0000 mon.a (mon.0) 1741 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:01.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:01 smithi190 bash[23392]: cluster 2024-03-26T03:51:01.611346+0000 mon.a (mon.0) 1742 : cluster [DBG] osdmap e205: 8 total, 8 up, 8 in 2024-03-26T03:51:01.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:01 smithi175 bash[21527]: audit 2024-03-26T03:51:00.597979+0000 mon.c (mon.2) 362 : audit [INF] from='client.? 172.21.15.175:0/3430821660' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi175-64398-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi175-64398-34"}]: dispatch 2024-03-26T03:51:01.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:01 smithi175 bash[21527]: audit 2024-03-26T03:51:00.603713+0000 mon.a (mon.0) 1739 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi175-64398-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi175-64398-34"}]: dispatch 2024-03-26T03:51:01.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:01 smithi175 bash[21527]: audit 2024-03-26T03:51:00.619377+0000 mon.c (mon.2) 363 : audit [INF] from='client.? 172.21.15.175:0/3709532255' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:01.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:01 smithi175 bash[21527]: audit 2024-03-26T03:51:00.622187+0000 mon.a (mon.0) 1740 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:01.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:01 smithi175 bash[21527]: cluster 2024-03-26T03:51:00.634028+0000 mgr.y (mgr.24431) 275 : cluster [DBG] pgmap v303: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 561 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-26T03:51:01.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:01 smithi175 bash[21527]: audit 2024-03-26T03:51:01.589567+0000 mon.a (mon.0) 1741 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:01.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:01 smithi175 bash[21527]: cluster 2024-03-26T03:51:01.611346+0000 mon.a (mon.0) 1742 : cluster [DBG] osdmap e205: 8 total, 8 up, 8 in 2024-03-26T03:51:01.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:01 smithi175 bash[29284]: audit 2024-03-26T03:51:00.597979+0000 mon.c (mon.2) 362 : audit [INF] from='client.? 172.21.15.175:0/3430821660' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi175-64398-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi175-64398-34"}]: dispatch 2024-03-26T03:51:01.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:01 smithi175 bash[29284]: audit 2024-03-26T03:51:00.603713+0000 mon.a (mon.0) 1739 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi175-64398-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi175-64398-34"}]: dispatch 2024-03-26T03:51:01.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:01 smithi175 bash[29284]: audit 2024-03-26T03:51:00.619377+0000 mon.c (mon.2) 363 : audit [INF] from='client.? 172.21.15.175:0/3709532255' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:01.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:01 smithi175 bash[29284]: audit 2024-03-26T03:51:00.622187+0000 mon.a (mon.0) 1740 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:01.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:01 smithi175 bash[29284]: cluster 2024-03-26T03:51:00.634028+0000 mgr.y (mgr.24431) 275 : cluster [DBG] pgmap v303: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 561 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-26T03:51:01.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:01 smithi175 bash[29284]: audit 2024-03-26T03:51:01.589567+0000 mon.a (mon.0) 1741 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:01.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:01 smithi175 bash[29284]: cluster 2024-03-26T03:51:01.611346+0000 mon.a (mon.0) 1742 : cluster [DBG] osdmap e205: 8 total, 8 up, 8 in 2024-03-26T03:51:02.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:02 smithi190 bash[23392]: cluster 2024-03-26T03:51:01.624518+0000 mon.a (mon.0) 1743 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:51:02.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:02 smithi190 bash[23392]: audit 2024-03-26T03:51:01.684740+0000 mon.a (mon.0) 1744 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:51:02.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:02 smithi190 bash[23392]: audit 2024-03-26T03:51:02.594135+0000 mon.a (mon.0) 1745 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi175-64398-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi175-64398-34"}]': finished 2024-03-26T03:51:02.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:02 smithi190 bash[23392]: audit 2024-03-26T03:51:02.594458+0000 mon.a (mon.0) 1746 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:51:02.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:02 smithi190 bash[23392]: cluster 2024-03-26T03:51:02.605618+0000 mon.a (mon.0) 1747 : cluster [DBG] osdmap e206: 8 total, 8 up, 8 in 2024-03-26T03:51:02.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:02 smithi190 bash[23392]: audit 2024-03-26T03:51:02.607921+0000 mon.a (mon.0) 1748 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-23"}]: dispatch 2024-03-26T03:51:02.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:02 smithi190 bash[23392]: audit 2024-03-26T03:51:02.608015+0000 mon.c (mon.2) 364 : audit [INF] from='client.? 172.21.15.175:0/1930267929' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:02.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:02 smithi190 bash[23392]: audit 2024-03-26T03:51:02.609492+0000 mon.a (mon.0) 1749 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:02.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:02 smithi175 bash[21527]: cluster 2024-03-26T03:51:01.624518+0000 mon.a (mon.0) 1743 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:51:02.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:02 smithi175 bash[21527]: audit 2024-03-26T03:51:01.684740+0000 mon.a (mon.0) 1744 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:51:02.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:02 smithi175 bash[21527]: audit 2024-03-26T03:51:02.594135+0000 mon.a (mon.0) 1745 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi175-64398-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi175-64398-34"}]': finished 2024-03-26T03:51:02.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:02 smithi175 bash[21527]: audit 2024-03-26T03:51:02.594458+0000 mon.a (mon.0) 1746 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:51:02.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:02 smithi175 bash[21527]: cluster 2024-03-26T03:51:02.605618+0000 mon.a (mon.0) 1747 : cluster [DBG] osdmap e206: 8 total, 8 up, 8 in 2024-03-26T03:51:02.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:02 smithi175 bash[21527]: audit 2024-03-26T03:51:02.607921+0000 mon.a (mon.0) 1748 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-23"}]: dispatch 2024-03-26T03:51:02.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:02 smithi175 bash[21527]: audit 2024-03-26T03:51:02.608015+0000 mon.c (mon.2) 364 : audit [INF] from='client.? 172.21.15.175:0/1930267929' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:02.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:02 smithi175 bash[21527]: audit 2024-03-26T03:51:02.609492+0000 mon.a (mon.0) 1749 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:02.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:02 smithi175 bash[29284]: cluster 2024-03-26T03:51:01.624518+0000 mon.a (mon.0) 1743 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:51:02.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:02 smithi175 bash[29284]: audit 2024-03-26T03:51:01.684740+0000 mon.a (mon.0) 1744 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:51:02.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:02 smithi175 bash[29284]: audit 2024-03-26T03:51:02.594135+0000 mon.a (mon.0) 1745 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi175-64398-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi175-64398-34"}]': finished 2024-03-26T03:51:02.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:02 smithi175 bash[29284]: audit 2024-03-26T03:51:02.594458+0000 mon.a (mon.0) 1746 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:51:02.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:02 smithi175 bash[29284]: cluster 2024-03-26T03:51:02.605618+0000 mon.a (mon.0) 1747 : cluster [DBG] osdmap e206: 8 total, 8 up, 8 in 2024-03-26T03:51:02.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:02 smithi175 bash[29284]: audit 2024-03-26T03:51:02.607921+0000 mon.a (mon.0) 1748 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-23"}]: dispatch 2024-03-26T03:51:02.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:02 smithi175 bash[29284]: audit 2024-03-26T03:51:02.608015+0000 mon.c (mon.2) 364 : audit [INF] from='client.? 172.21.15.175:0/1930267929' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:02.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:02 smithi175 bash[29284]: audit 2024-03-26T03:51:02.609492+0000 mon.a (mon.0) 1749 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:03.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:03 smithi190 bash[23392]: cluster 2024-03-26T03:51:02.634771+0000 mgr.y (mgr.24431) 276 : cluster [DBG] pgmap v306: 364 pgs: 72 unknown, 292 active+clean; 584 KiB data, 561 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-26T03:51:03.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:03 smithi190 bash[23392]: cluster 2024-03-26T03:51:03.595991+0000 mon.a (mon.0) 1750 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:51:03.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:03 smithi190 bash[23392]: audit 2024-03-26T03:51:03.608044+0000 mon.a (mon.0) 1751 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-23"}]': finished 2024-03-26T03:51:03.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:03 smithi190 bash[23392]: audit 2024-03-26T03:51:03.608237+0000 mon.a (mon.0) 1752 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:03.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:03 smithi190 bash[23392]: cluster 2024-03-26T03:51:03.614934+0000 mon.a (mon.0) 1753 : cluster [DBG] osdmap e207: 8 total, 8 up, 8 in 2024-03-26T03:51:03.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:03 smithi175 bash[21527]: cluster 2024-03-26T03:51:02.634771+0000 mgr.y (mgr.24431) 276 : cluster [DBG] pgmap v306: 364 pgs: 72 unknown, 292 active+clean; 584 KiB data, 561 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-26T03:51:03.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:03 smithi175 bash[21527]: cluster 2024-03-26T03:51:03.595991+0000 mon.a (mon.0) 1750 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:51:03.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:03 smithi175 bash[21527]: audit 2024-03-26T03:51:03.608044+0000 mon.a (mon.0) 1751 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-23"}]': finished 2024-03-26T03:51:03.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:03 smithi175 bash[21527]: audit 2024-03-26T03:51:03.608237+0000 mon.a (mon.0) 1752 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:03.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:03 smithi175 bash[21527]: cluster 2024-03-26T03:51:03.614934+0000 mon.a (mon.0) 1753 : cluster [DBG] osdmap e207: 8 total, 8 up, 8 in 2024-03-26T03:51:03.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:03 smithi175 bash[29284]: cluster 2024-03-26T03:51:02.634771+0000 mgr.y (mgr.24431) 276 : cluster [DBG] pgmap v306: 364 pgs: 72 unknown, 292 active+clean; 584 KiB data, 561 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-26T03:51:03.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:03 smithi175 bash[29284]: cluster 2024-03-26T03:51:03.595991+0000 mon.a (mon.0) 1750 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:51:03.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:03 smithi175 bash[29284]: audit 2024-03-26T03:51:03.608044+0000 mon.a (mon.0) 1751 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-23"}]': finished 2024-03-26T03:51:03.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:03 smithi175 bash[29284]: audit 2024-03-26T03:51:03.608237+0000 mon.a (mon.0) 1752 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:03.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:03 smithi175 bash[29284]: cluster 2024-03-26T03:51:03.614934+0000 mon.a (mon.0) 1753 : cluster [DBG] osdmap e207: 8 total, 8 up, 8 in 2024-03-26T03:51:04.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:04 smithi190 bash[23392]: audit 2024-03-26T03:51:04.336996+0000 mon.c (mon.2) 365 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:51:04.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:04 smithi190 bash[23392]: cluster 2024-03-26T03:51:04.647825+0000 mon.a (mon.0) 1754 : cluster [DBG] osdmap e208: 8 total, 8 up, 8 in 2024-03-26T03:51:04.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:04 smithi190 bash[23392]: audit 2024-03-26T03:51:04.649862+0000 mon.c (mon.2) 366 : audit [INF] from='client.? 172.21.15.175:0/3430821660' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi175-64398-34"}]: dispatch 2024-03-26T03:51:04.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:04 smithi190 bash[23392]: audit 2024-03-26T03:51:04.662860+0000 mon.b (mon.1) 110 : audit [INF] from='client.? 172.21.15.175:0/3961377789' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:04.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:04 smithi190 bash[23392]: audit 2024-03-26T03:51:04.676433+0000 mon.a (mon.0) 1755 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi175-64398-34"}]: dispatch 2024-03-26T03:51:04.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:04 smithi175 bash[29284]: audit 2024-03-26T03:51:04.336996+0000 mon.c (mon.2) 365 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:51:04.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:04 smithi175 bash[29284]: cluster 2024-03-26T03:51:04.647825+0000 mon.a (mon.0) 1754 : cluster [DBG] osdmap e208: 8 total, 8 up, 8 in 2024-03-26T03:51:04.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:04 smithi175 bash[29284]: audit 2024-03-26T03:51:04.649862+0000 mon.c (mon.2) 366 : audit [INF] from='client.? 172.21.15.175:0/3430821660' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi175-64398-34"}]: dispatch 2024-03-26T03:51:04.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:04 smithi175 bash[29284]: audit 2024-03-26T03:51:04.662860+0000 mon.b (mon.1) 110 : audit [INF] from='client.? 172.21.15.175:0/3961377789' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:04.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:04 smithi175 bash[29284]: audit 2024-03-26T03:51:04.676433+0000 mon.a (mon.0) 1755 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi175-64398-34"}]: dispatch 2024-03-26T03:51:04.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:04 smithi175 bash[21527]: audit 2024-03-26T03:51:04.336996+0000 mon.c (mon.2) 365 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:51:04.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:04 smithi175 bash[21527]: cluster 2024-03-26T03:51:04.647825+0000 mon.a (mon.0) 1754 : cluster [DBG] osdmap e208: 8 total, 8 up, 8 in 2024-03-26T03:51:04.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:04 smithi175 bash[21527]: audit 2024-03-26T03:51:04.649862+0000 mon.c (mon.2) 366 : audit [INF] from='client.? 172.21.15.175:0/3430821660' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi175-64398-34"}]: dispatch 2024-03-26T03:51:04.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:04 smithi175 bash[21527]: audit 2024-03-26T03:51:04.662860+0000 mon.b (mon.1) 110 : audit [INF] from='client.? 172.21.15.175:0/3961377789' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:04.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:04 smithi175 bash[21527]: audit 2024-03-26T03:51:04.676433+0000 mon.a (mon.0) 1755 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi175-64398-34"}]: dispatch 2024-03-26T03:51:05.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:05 smithi190 bash[23392]: cluster 2024-03-26T03:51:04.641220+0000 mgr.y (mgr.24431) 277 : cluster [DBG] pgmap v309: 356 pgs: 64 unknown, 292 active+clean; 584 KiB data, 562 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 255 B/s wr, 0 op/s 2024-03-26T03:51:05.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:05 smithi190 bash[23392]: audit 2024-03-26T03:51:04.676933+0000 mon.a (mon.0) 1756 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:05.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:05 smithi190 bash[23392]: audit 2024-03-26T03:51:05.617035+0000 mon.a (mon.0) 1757 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi175-64398-34"}]': finished 2024-03-26T03:51:05.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:05 smithi190 bash[23392]: audit 2024-03-26T03:51:05.617301+0000 mon.a (mon.0) 1758 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:05.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:05 smithi190 bash[23392]: cluster 2024-03-26T03:51:05.626796+0000 mon.a (mon.0) 1759 : cluster [DBG] osdmap e209: 8 total, 8 up, 8 in 2024-03-26T03:51:05.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:05 smithi190 bash[23392]: audit 2024-03-26T03:51:05.627335+0000 mon.c (mon.2) 367 : audit [INF] from='client.? 172.21.15.175:0/3430821660' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi175-64398-34"}]: dispatch 2024-03-26T03:51:05.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:05 smithi190 bash[23392]: audit 2024-03-26T03:51:05.637921+0000 mon.a (mon.0) 1760 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi175-64398-34"}]: dispatch 2024-03-26T03:51:05.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:05 smithi190 bash[23392]: audit 2024-03-26T03:51:05.648349+0000 mon.a (mon.0) 1761 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:05.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:05 smithi175 bash[21527]: cluster 2024-03-26T03:51:04.641220+0000 mgr.y (mgr.24431) 277 : cluster [DBG] pgmap v309: 356 pgs: 64 unknown, 292 active+clean; 584 KiB data, 562 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 255 B/s wr, 0 op/s 2024-03-26T03:51:05.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:05 smithi175 bash[21527]: audit 2024-03-26T03:51:04.676933+0000 mon.a (mon.0) 1756 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:05.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:05 smithi175 bash[21527]: audit 2024-03-26T03:51:05.617035+0000 mon.a (mon.0) 1757 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi175-64398-34"}]': finished 2024-03-26T03:51:05.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:05 smithi175 bash[21527]: audit 2024-03-26T03:51:05.617301+0000 mon.a (mon.0) 1758 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:05.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:05 smithi175 bash[21527]: cluster 2024-03-26T03:51:05.626796+0000 mon.a (mon.0) 1759 : cluster [DBG] osdmap e209: 8 total, 8 up, 8 in 2024-03-26T03:51:05.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:05 smithi175 bash[21527]: audit 2024-03-26T03:51:05.627335+0000 mon.c (mon.2) 367 : audit [INF] from='client.? 172.21.15.175:0/3430821660' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi175-64398-34"}]: dispatch 2024-03-26T03:51:05.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:05 smithi175 bash[21527]: audit 2024-03-26T03:51:05.637921+0000 mon.a (mon.0) 1760 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi175-64398-34"}]: dispatch 2024-03-26T03:51:05.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:05 smithi175 bash[21527]: audit 2024-03-26T03:51:05.648349+0000 mon.a (mon.0) 1761 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:05 smithi175 bash[29284]: cluster 2024-03-26T03:51:04.641220+0000 mgr.y (mgr.24431) 277 : cluster [DBG] pgmap v309: 356 pgs: 64 unknown, 292 active+clean; 584 KiB data, 562 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 255 B/s wr, 0 op/s 2024-03-26T03:51:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:05 smithi175 bash[29284]: audit 2024-03-26T03:51:04.676933+0000 mon.a (mon.0) 1756 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:05 smithi175 bash[29284]: audit 2024-03-26T03:51:05.617035+0000 mon.a (mon.0) 1757 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi175-64398-34"}]': finished 2024-03-26T03:51:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:05 smithi175 bash[29284]: audit 2024-03-26T03:51:05.617301+0000 mon.a (mon.0) 1758 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:05 smithi175 bash[29284]: cluster 2024-03-26T03:51:05.626796+0000 mon.a (mon.0) 1759 : cluster [DBG] osdmap e209: 8 total, 8 up, 8 in 2024-03-26T03:51:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:05 smithi175 bash[29284]: audit 2024-03-26T03:51:05.627335+0000 mon.c (mon.2) 367 : audit [INF] from='client.? 172.21.15.175:0/3430821660' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi175-64398-34"}]: dispatch 2024-03-26T03:51:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:05 smithi175 bash[29284]: audit 2024-03-26T03:51:05.637921+0000 mon.a (mon.0) 1760 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi175-64398-34"}]: dispatch 2024-03-26T03:51:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:05 smithi175 bash[29284]: audit 2024-03-26T03:51:05.648349+0000 mon.a (mon.0) 1761 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:07.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:07 smithi190 bash[23392]: audit 2024-03-26T03:51:06.622569+0000 mon.a (mon.0) 1762 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi175-64398-34"}]': finished 2024-03-26T03:51:07.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:07 smithi190 bash[23392]: audit 2024-03-26T03:51:06.622788+0000 mon.a (mon.0) 1763 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:07.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:07 smithi190 bash[23392]: cluster 2024-03-26T03:51:06.634004+0000 mon.a (mon.0) 1764 : cluster [DBG] osdmap e210: 8 total, 8 up, 8 in 2024-03-26T03:51:07.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:07 smithi190 bash[23392]: cluster 2024-03-26T03:51:06.641989+0000 mgr.y (mgr.24431) 278 : cluster [DBG] pgmap v312: 420 pgs: 10 creating+peering, 86 unknown, 324 active+clean; 584 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T03:51:07.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:07 smithi190 bash[23392]: audit 2024-03-26T03:51:06.642916+0000 mon.c (mon.2) 368 : audit [INF] from='client.? 172.21.15.175:0/2166936688' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:07.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:07 smithi190 bash[23392]: audit 2024-03-26T03:51:06.645831+0000 mon.a (mon.0) 1765 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:07.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:07 smithi190 bash[23392]: audit 2024-03-26T03:51:06.682789+0000 mon.a (mon.0) 1766 : audit [INF] from='client.? 172.21.15.175:0/2339775793' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi175-64398-35"}]: dispatch 2024-03-26T03:51:07.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:07 smithi190 bash[23392]: audit 2024-03-26T03:51:06.687184+0000 mon.a (mon.0) 1767 : audit [INF] from='client.? 172.21.15.175:0/2339775793' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi175-64398-35"}]: dispatch 2024-03-26T03:51:07.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:07 smithi190 bash[23392]: audit 2024-03-26T03:51:06.689558+0000 mon.a (mon.0) 1768 : audit [INF] from='client.? 172.21.15.175:0/2339775793' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi175-64398-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:07.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:07 smithi175 bash[29284]: audit 2024-03-26T03:51:06.622569+0000 mon.a (mon.0) 1762 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi175-64398-34"}]': finished 2024-03-26T03:51:07.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:07 smithi175 bash[29284]: audit 2024-03-26T03:51:06.622788+0000 mon.a (mon.0) 1763 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:07.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:07 smithi175 bash[29284]: cluster 2024-03-26T03:51:06.634004+0000 mon.a (mon.0) 1764 : cluster [DBG] osdmap e210: 8 total, 8 up, 8 in 2024-03-26T03:51:07.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:07 smithi175 bash[29284]: cluster 2024-03-26T03:51:06.641989+0000 mgr.y (mgr.24431) 278 : cluster [DBG] pgmap v312: 420 pgs: 10 creating+peering, 86 unknown, 324 active+clean; 584 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T03:51:07.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:07 smithi175 bash[29284]: audit 2024-03-26T03:51:06.642916+0000 mon.c (mon.2) 368 : audit [INF] from='client.? 172.21.15.175:0/2166936688' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:07.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:07 smithi175 bash[29284]: audit 2024-03-26T03:51:06.645831+0000 mon.a (mon.0) 1765 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:07.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:07 smithi175 bash[29284]: audit 2024-03-26T03:51:06.682789+0000 mon.a (mon.0) 1766 : audit [INF] from='client.? 172.21.15.175:0/2339775793' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi175-64398-35"}]: dispatch 2024-03-26T03:51:07.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:07 smithi175 bash[29284]: audit 2024-03-26T03:51:06.687184+0000 mon.a (mon.0) 1767 : audit [INF] from='client.? 172.21.15.175:0/2339775793' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi175-64398-35"}]: dispatch 2024-03-26T03:51:07.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:07 smithi175 bash[29284]: audit 2024-03-26T03:51:06.689558+0000 mon.a (mon.0) 1768 : audit [INF] from='client.? 172.21.15.175:0/2339775793' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi175-64398-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:07.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:07 smithi175 bash[21527]: audit 2024-03-26T03:51:06.622569+0000 mon.a (mon.0) 1762 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi175-64398-34"}]': finished 2024-03-26T03:51:07.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:07 smithi175 bash[21527]: audit 2024-03-26T03:51:06.622788+0000 mon.a (mon.0) 1763 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:07.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:07 smithi175 bash[21527]: cluster 2024-03-26T03:51:06.634004+0000 mon.a (mon.0) 1764 : cluster [DBG] osdmap e210: 8 total, 8 up, 8 in 2024-03-26T03:51:07.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:07 smithi175 bash[21527]: cluster 2024-03-26T03:51:06.641989+0000 mgr.y (mgr.24431) 278 : cluster [DBG] pgmap v312: 420 pgs: 10 creating+peering, 86 unknown, 324 active+clean; 584 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T03:51:07.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:07 smithi175 bash[21527]: audit 2024-03-26T03:51:06.642916+0000 mon.c (mon.2) 368 : audit [INF] from='client.? 172.21.15.175:0/2166936688' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:07.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:07 smithi175 bash[21527]: audit 2024-03-26T03:51:06.645831+0000 mon.a (mon.0) 1765 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:07.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:07 smithi175 bash[21527]: audit 2024-03-26T03:51:06.682789+0000 mon.a (mon.0) 1766 : audit [INF] from='client.? 172.21.15.175:0/2339775793' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi175-64398-35"}]: dispatch 2024-03-26T03:51:07.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:07 smithi175 bash[21527]: audit 2024-03-26T03:51:06.687184+0000 mon.a (mon.0) 1767 : audit [INF] from='client.? 172.21.15.175:0/2339775793' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi175-64398-35"}]: dispatch 2024-03-26T03:51:07.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:07 smithi175 bash[21527]: audit 2024-03-26T03:51:06.689558+0000 mon.a (mon.0) 1768 : audit [INF] from='client.? 172.21.15.175:0/2339775793' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi175-64398-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:08.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:08 smithi190 bash[23392]: audit 2024-03-26T03:51:07.625989+0000 mon.a (mon.0) 1769 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:08.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:08 smithi190 bash[23392]: audit 2024-03-26T03:51:07.626295+0000 mon.a (mon.0) 1770 : audit [INF] from='client.? 172.21.15.175:0/2339775793' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi175-64398-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:08.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:08 smithi190 bash[23392]: cluster 2024-03-26T03:51:07.634400+0000 mon.a (mon.0) 1771 : cluster [DBG] osdmap e211: 8 total, 8 up, 8 in 2024-03-26T03:51:08.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:08 smithi190 bash[23392]: audit 2024-03-26T03:51:07.637776+0000 mon.a (mon.0) 1772 : audit [INF] from='client.? 172.21.15.175:0/2339775793' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi175-64398-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi175-64398-35"}]: dispatch 2024-03-26T03:51:08.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:08 smithi190 bash[23392]: audit 2024-03-26T03:51:07.682071+0000 mon.a (mon.0) 1773 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:51:08.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:08 smithi190 bash[23392]: audit 2024-03-26T03:51:08.633023+0000 mon.a (mon.0) 1774 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-25", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:51:08.991 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:51:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:51:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:51:08.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:08 smithi175 bash[29284]: audit 2024-03-26T03:51:07.625989+0000 mon.a (mon.0) 1769 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:08.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:08 smithi175 bash[29284]: audit 2024-03-26T03:51:07.626295+0000 mon.a (mon.0) 1770 : audit [INF] from='client.? 172.21.15.175:0/2339775793' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi175-64398-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:08.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:08 smithi175 bash[29284]: cluster 2024-03-26T03:51:07.634400+0000 mon.a (mon.0) 1771 : cluster [DBG] osdmap e211: 8 total, 8 up, 8 in 2024-03-26T03:51:08.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:08 smithi175 bash[29284]: audit 2024-03-26T03:51:07.637776+0000 mon.a (mon.0) 1772 : audit [INF] from='client.? 172.21.15.175:0/2339775793' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi175-64398-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi175-64398-35"}]: dispatch 2024-03-26T03:51:08.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:08 smithi175 bash[29284]: audit 2024-03-26T03:51:07.682071+0000 mon.a (mon.0) 1773 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:51:08.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:08 smithi175 bash[29284]: audit 2024-03-26T03:51:08.633023+0000 mon.a (mon.0) 1774 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-25", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:51:08.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:08 smithi175 bash[21527]: audit 2024-03-26T03:51:07.625989+0000 mon.a (mon.0) 1769 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:08.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:08 smithi175 bash[21527]: audit 2024-03-26T03:51:07.626295+0000 mon.a (mon.0) 1770 : audit [INF] from='client.? 172.21.15.175:0/2339775793' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi175-64398-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:08.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:08 smithi175 bash[21527]: cluster 2024-03-26T03:51:07.634400+0000 mon.a (mon.0) 1771 : cluster [DBG] osdmap e211: 8 total, 8 up, 8 in 2024-03-26T03:51:08.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:08 smithi175 bash[21527]: audit 2024-03-26T03:51:07.637776+0000 mon.a (mon.0) 1772 : audit [INF] from='client.? 172.21.15.175:0/2339775793' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi175-64398-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi175-64398-35"}]: dispatch 2024-03-26T03:51:08.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:08 smithi175 bash[21527]: audit 2024-03-26T03:51:07.682071+0000 mon.a (mon.0) 1773 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:51:08.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:08 smithi175 bash[21527]: audit 2024-03-26T03:51:08.633023+0000 mon.a (mon.0) 1774 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-25", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:51:09.673 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:09 smithi175 bash[29284]: cluster 2024-03-26T03:51:08.643311+0000 mgr.y (mgr.24431) 279 : cluster [DBG] pgmap v315: 452 pgs: 10 creating+peering, 118 unknown, 324 active+clean; 584 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:51:09.673 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:09 smithi175 bash[29284]: cluster 2024-03-26T03:51:08.659320+0000 mon.a (mon.0) 1775 : cluster [DBG] osdmap e212: 8 total, 8 up, 8 in 2024-03-26T03:51:09.673 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:09 smithi175 bash[29284]: audit 2024-03-26T03:51:08.662374+0000 mon.a (mon.0) 1776 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-25"}]: dispatch 2024-03-26T03:51:09.673 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:09 smithi175 bash[29284]: audit 2024-03-26T03:51:08.665482+0000 mon.c (mon.2) 369 : audit [INF] from='client.? 172.21.15.175:0/1802544349' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:09.990 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:51:09 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:51:09.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:09 smithi190 bash[23392]: cluster 2024-03-26T03:51:08.643311+0000 mgr.y (mgr.24431) 279 : cluster [DBG] pgmap v315: 452 pgs: 10 creating+peering, 118 unknown, 324 active+clean; 584 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:51:09.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:09 smithi190 bash[23392]: cluster 2024-03-26T03:51:08.659320+0000 mon.a (mon.0) 1775 : cluster [DBG] osdmap e212: 8 total, 8 up, 8 in 2024-03-26T03:51:09.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:09 smithi190 bash[23392]: audit 2024-03-26T03:51:08.662374+0000 mon.a (mon.0) 1776 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-25"}]: dispatch 2024-03-26T03:51:09.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:09 smithi190 bash[23392]: audit 2024-03-26T03:51:08.665482+0000 mon.c (mon.2) 369 : audit [INF] from='client.? 172.21.15.175:0/1802544349' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:09.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:09 smithi190 bash[23392]: audit 2024-03-26T03:51:08.675065+0000 mon.a (mon.0) 1777 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:09.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:09 smithi190 bash[23392]: audit 2024-03-26T03:51:09.646214+0000 mon.a (mon.0) 1778 : audit [INF] from='client.? 172.21.15.175:0/2339775793' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi175-64398-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi175-64398-35"}]': finished 2024-03-26T03:51:09.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:09 smithi190 bash[23392]: audit 2024-03-26T03:51:09.646517+0000 mon.a (mon.0) 1779 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-25"}]': finished 2024-03-26T03:51:09.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:09 smithi190 bash[23392]: audit 2024-03-26T03:51:09.646684+0000 mon.a (mon.0) 1780 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:09.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:09 smithi190 bash[23392]: cluster 2024-03-26T03:51:09.656283+0000 mon.a (mon.0) 1781 : cluster [DBG] osdmap e213: 8 total, 8 up, 8 in 2024-03-26T03:51:09.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:09 smithi190 bash[23392]: audit 2024-03-26T03:51:09.657766+0000 mon.a (mon.0) 1782 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-25", "mode": "writeback"}]: dispatch 2024-03-26T03:51:09.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:09 smithi175 bash[29284]: audit 2024-03-26T03:51:08.675065+0000 mon.a (mon.0) 1777 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:09.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:09 smithi175 bash[29284]: audit 2024-03-26T03:51:09.646214+0000 mon.a (mon.0) 1778 : audit [INF] from='client.? 172.21.15.175:0/2339775793' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi175-64398-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi175-64398-35"}]': finished 2024-03-26T03:51:09.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:09 smithi175 bash[29284]: audit 2024-03-26T03:51:09.646517+0000 mon.a (mon.0) 1779 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-25"}]': finished 2024-03-26T03:51:09.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:09 smithi175 bash[29284]: audit 2024-03-26T03:51:09.646684+0000 mon.a (mon.0) 1780 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:09.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:09 smithi175 bash[29284]: cluster 2024-03-26T03:51:09.656283+0000 mon.a (mon.0) 1781 : cluster [DBG] osdmap e213: 8 total, 8 up, 8 in 2024-03-26T03:51:09.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:09 smithi175 bash[29284]: audit 2024-03-26T03:51:09.657766+0000 mon.a (mon.0) 1782 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-25", "mode": "writeback"}]: dispatch 2024-03-26T03:51:09.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:09 smithi175 bash[21527]: cluster 2024-03-26T03:51:08.643311+0000 mgr.y (mgr.24431) 279 : cluster [DBG] pgmap v315: 452 pgs: 10 creating+peering, 118 unknown, 324 active+clean; 584 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:51:09.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:09 smithi175 bash[21527]: cluster 2024-03-26T03:51:08.659320+0000 mon.a (mon.0) 1775 : cluster [DBG] osdmap e212: 8 total, 8 up, 8 in 2024-03-26T03:51:09.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:09 smithi175 bash[21527]: audit 2024-03-26T03:51:08.662374+0000 mon.a (mon.0) 1776 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-25"}]: dispatch 2024-03-26T03:51:09.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:09 smithi175 bash[21527]: audit 2024-03-26T03:51:08.665482+0000 mon.c (mon.2) 369 : audit [INF] from='client.? 172.21.15.175:0/1802544349' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:09.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:09 smithi175 bash[21527]: audit 2024-03-26T03:51:08.675065+0000 mon.a (mon.0) 1777 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:09.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:09 smithi175 bash[21527]: audit 2024-03-26T03:51:09.646214+0000 mon.a (mon.0) 1778 : audit [INF] from='client.? 172.21.15.175:0/2339775793' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi175-64398-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi175-64398-35"}]': finished 2024-03-26T03:51:09.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:09 smithi175 bash[21527]: audit 2024-03-26T03:51:09.646517+0000 mon.a (mon.0) 1779 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-25"}]': finished 2024-03-26T03:51:09.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:09 smithi175 bash[21527]: audit 2024-03-26T03:51:09.646684+0000 mon.a (mon.0) 1780 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi175-64406-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:09.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:09 smithi175 bash[21527]: cluster 2024-03-26T03:51:09.656283+0000 mon.a (mon.0) 1781 : cluster [DBG] osdmap e213: 8 total, 8 up, 8 in 2024-03-26T03:51:09.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:09 smithi175 bash[21527]: audit 2024-03-26T03:51:09.657766+0000 mon.a (mon.0) 1782 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-25", "mode": "writeback"}]: dispatch 2024-03-26T03:51:10.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:10 smithi190 bash[23392]: audit 2024-03-26T03:51:09.825257+0000 mgr.y (mgr.24431) 280 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:51:10.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:10 smithi190 bash[23392]: cluster 2024-03-26T03:51:10.646722+0000 mon.a (mon.0) 1783 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:51:10.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:10 smithi190 bash[23392]: audit 2024-03-26T03:51:10.650977+0000 mon.a (mon.0) 1784 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-25", "mode": "writeback"}]': finished 2024-03-26T03:51:10.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:10 smithi190 bash[23392]: cluster 2024-03-26T03:51:10.659423+0000 mon.a (mon.0) 1785 : cluster [DBG] osdmap e214: 8 total, 8 up, 8 in 2024-03-26T03:51:10.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:10 smithi175 bash[21527]: audit 2024-03-26T03:51:09.825257+0000 mgr.y (mgr.24431) 280 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:51:10.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:10 smithi175 bash[21527]: cluster 2024-03-26T03:51:10.646722+0000 mon.a (mon.0) 1783 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:51:10.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:10 smithi175 bash[21527]: audit 2024-03-26T03:51:10.650977+0000 mon.a (mon.0) 1784 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-25", "mode": "writeback"}]': finished 2024-03-26T03:51:10.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:10 smithi175 bash[21527]: cluster 2024-03-26T03:51:10.659423+0000 mon.a (mon.0) 1785 : cluster [DBG] osdmap e214: 8 total, 8 up, 8 in 2024-03-26T03:51:10.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:10 smithi175 bash[29284]: audit 2024-03-26T03:51:09.825257+0000 mgr.y (mgr.24431) 280 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:51:10.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:10 smithi175 bash[29284]: cluster 2024-03-26T03:51:10.646722+0000 mon.a (mon.0) 1783 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:51:10.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:10 smithi175 bash[29284]: audit 2024-03-26T03:51:10.650977+0000 mon.a (mon.0) 1784 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-25", "mode": "writeback"}]': finished 2024-03-26T03:51:10.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:10 smithi175 bash[29284]: cluster 2024-03-26T03:51:10.659423+0000 mon.a (mon.0) 1785 : cluster [DBG] osdmap e214: 8 total, 8 up, 8 in 2024-03-26T03:51:11.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:11 smithi190 bash[23392]: cluster 2024-03-26T03:51:10.645559+0000 mgr.y (mgr.24431) 281 : cluster [DBG] pgmap v317: 460 pgs: 24 creating+peering, 5 unknown, 431 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 764 B/s wr, 2 op/s 2024-03-26T03:51:11.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:11 smithi190 bash[23392]: audit 2024-03-26T03:51:10.754186+0000 mon.a (mon.0) 1786 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:51:11.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:11 smithi190 bash[23392]: audit 2024-03-26T03:51:11.438108+0000 mon.a (mon.0) 1787 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:51:11.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:11 smithi190 bash[23392]: cluster 2024-03-26T03:51:11.450404+0000 mon.a (mon.0) 1788 : cluster [DBG] osdmap e215: 8 total, 8 up, 8 in 2024-03-26T03:51:11.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:11 smithi190 bash[23392]: audit 2024-03-26T03:51:11.454877+0000 mon.a (mon.0) 1789 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-25"}]: dispatch 2024-03-26T03:51:11.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:11 smithi190 bash[23392]: audit 2024-03-26T03:51:11.455866+0000 mon.a (mon.0) 1790 : audit [INF] from='client.? 172.21.15.175:0/2339775793' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi175-64398-35"}]: dispatch 2024-03-26T03:51:11.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:11 smithi175 bash[29284]: cluster 2024-03-26T03:51:10.645559+0000 mgr.y (mgr.24431) 281 : cluster [DBG] pgmap v317: 460 pgs: 24 creating+peering, 5 unknown, 431 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 764 B/s wr, 2 op/s 2024-03-26T03:51:11.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:11 smithi175 bash[29284]: audit 2024-03-26T03:51:10.754186+0000 mon.a (mon.0) 1786 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:51:11.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:11 smithi175 bash[29284]: audit 2024-03-26T03:51:11.438108+0000 mon.a (mon.0) 1787 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:51:11.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:11 smithi175 bash[29284]: cluster 2024-03-26T03:51:11.450404+0000 mon.a (mon.0) 1788 : cluster [DBG] osdmap e215: 8 total, 8 up, 8 in 2024-03-26T03:51:11.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:11 smithi175 bash[29284]: audit 2024-03-26T03:51:11.454877+0000 mon.a (mon.0) 1789 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-25"}]: dispatch 2024-03-26T03:51:11.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:11 smithi175 bash[29284]: audit 2024-03-26T03:51:11.455866+0000 mon.a (mon.0) 1790 : audit [INF] from='client.? 172.21.15.175:0/2339775793' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi175-64398-35"}]: dispatch 2024-03-26T03:51:11.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:11 smithi175 bash[21527]: cluster 2024-03-26T03:51:10.645559+0000 mgr.y (mgr.24431) 281 : cluster [DBG] pgmap v317: 460 pgs: 24 creating+peering, 5 unknown, 431 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 764 B/s wr, 2 op/s 2024-03-26T03:51:11.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:11 smithi175 bash[21527]: audit 2024-03-26T03:51:10.754186+0000 mon.a (mon.0) 1786 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:51:11.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:11 smithi175 bash[21527]: audit 2024-03-26T03:51:11.438108+0000 mon.a (mon.0) 1787 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:51:11.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:11 smithi175 bash[21527]: cluster 2024-03-26T03:51:11.450404+0000 mon.a (mon.0) 1788 : cluster [DBG] osdmap e215: 8 total, 8 up, 8 in 2024-03-26T03:51:11.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:11 smithi175 bash[21527]: audit 2024-03-26T03:51:11.454877+0000 mon.a (mon.0) 1789 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-25"}]: dispatch 2024-03-26T03:51:11.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:11 smithi175 bash[21527]: audit 2024-03-26T03:51:11.455866+0000 mon.a (mon.0) 1790 : audit [INF] from='client.? 172.21.15.175:0/2339775793' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi175-64398-35"}]: dispatch 2024-03-26T03:51:12.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:12 smithi190 bash[23392]: cluster 2024-03-26T03:51:12.438471+0000 mon.a (mon.0) 1791 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:51:12.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:12 smithi190 bash[23392]: audit 2024-03-26T03:51:12.441056+0000 mon.a (mon.0) 1792 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-25"}]': finished 2024-03-26T03:51:12.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:12 smithi190 bash[23392]: audit 2024-03-26T03:51:12.441222+0000 mon.a (mon.0) 1793 : audit [INF] from='client.? 172.21.15.175:0/2339775793' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi175-64398-35"}]': finished 2024-03-26T03:51:12.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:12 smithi190 bash[23392]: cluster 2024-03-26T03:51:12.451617+0000 mon.a (mon.0) 1794 : cluster [DBG] osdmap e216: 8 total, 8 up, 8 in 2024-03-26T03:51:12.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:12 smithi190 bash[23392]: audit 2024-03-26T03:51:12.452652+0000 mon.a (mon.0) 1795 : audit [INF] from='client.? 172.21.15.175:0/2339775793' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi175-64398-35"}]: dispatch 2024-03-26T03:51:12.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:12 smithi175 bash[29284]: cluster 2024-03-26T03:51:12.438471+0000 mon.a (mon.0) 1791 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:51:12.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:12 smithi175 bash[29284]: audit 2024-03-26T03:51:12.441056+0000 mon.a (mon.0) 1792 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-25"}]': finished 2024-03-26T03:51:12.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:12 smithi175 bash[29284]: audit 2024-03-26T03:51:12.441222+0000 mon.a (mon.0) 1793 : audit [INF] from='client.? 172.21.15.175:0/2339775793' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi175-64398-35"}]': finished 2024-03-26T03:51:12.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:12 smithi175 bash[29284]: cluster 2024-03-26T03:51:12.451617+0000 mon.a (mon.0) 1794 : cluster [DBG] osdmap e216: 8 total, 8 up, 8 in 2024-03-26T03:51:12.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:12 smithi175 bash[29284]: audit 2024-03-26T03:51:12.452652+0000 mon.a (mon.0) 1795 : audit [INF] from='client.? 172.21.15.175:0/2339775793' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi175-64398-35"}]: dispatch 2024-03-26T03:51:12.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:12 smithi175 bash[21527]: cluster 2024-03-26T03:51:12.438471+0000 mon.a (mon.0) 1791 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:51:12.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:12 smithi175 bash[21527]: audit 2024-03-26T03:51:12.441056+0000 mon.a (mon.0) 1792 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-25"}]': finished 2024-03-26T03:51:12.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:12 smithi175 bash[21527]: audit 2024-03-26T03:51:12.441222+0000 mon.a (mon.0) 1793 : audit [INF] from='client.? 172.21.15.175:0/2339775793' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi175-64398-35"}]': finished 2024-03-26T03:51:12.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:12 smithi175 bash[21527]: cluster 2024-03-26T03:51:12.451617+0000 mon.a (mon.0) 1794 : cluster [DBG] osdmap e216: 8 total, 8 up, 8 in 2024-03-26T03:51:12.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:12 smithi175 bash[21527]: audit 2024-03-26T03:51:12.452652+0000 mon.a (mon.0) 1795 : audit [INF] from='client.? 172.21.15.175:0/2339775793' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi175-64398-35"}]: dispatch 2024-03-26T03:51:13.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:13 smithi190 bash[23392]: cluster 2024-03-26T03:51:12.646472+0000 mgr.y (mgr.24431) 282 : cluster [DBG] pgmap v321: 356 pgs: 356 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-26T03:51:13.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:13 smithi190 bash[23392]: cluster 2024-03-26T03:51:13.441043+0000 mon.a (mon.0) 1796 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:51:13.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:13 smithi190 bash[23392]: audit 2024-03-26T03:51:13.452938+0000 mon.a (mon.0) 1797 : audit [INF] from='client.? 172.21.15.175:0/2339775793' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi175-64398-35"}]': finished 2024-03-26T03:51:13.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:13 smithi190 bash[23392]: cluster 2024-03-26T03:51:13.461056+0000 mon.a (mon.0) 1798 : cluster [DBG] osdmap e217: 8 total, 8 up, 8 in 2024-03-26T03:51:13.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:13 smithi190 bash[23392]: audit 2024-03-26T03:51:13.500717+0000 mon.c (mon.2) 370 : audit [INF] from='client.? 172.21.15.175:0/2979685069' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi175-64398-36"}]: dispatch 2024-03-26T03:51:13.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:13 smithi190 bash[23392]: audit 2024-03-26T03:51:13.501067+0000 mon.a (mon.0) 1799 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi175-64398-36"}]: dispatch 2024-03-26T03:51:13.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:13 smithi190 bash[23392]: audit 2024-03-26T03:51:13.501718+0000 mon.c (mon.2) 371 : audit [INF] from='client.? 172.21.15.175:0/2979685069' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi175-64398-36"}]: dispatch 2024-03-26T03:51:13.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:13 smithi190 bash[23392]: audit 2024-03-26T03:51:13.502029+0000 mon.a (mon.0) 1800 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi175-64398-36"}]: dispatch 2024-03-26T03:51:13.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:13 smithi190 bash[23392]: audit 2024-03-26T03:51:13.502529+0000 mon.c (mon.2) 372 : audit [INF] from='client.? 172.21.15.175:0/2979685069' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi175-64398-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:13.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:13 smithi190 bash[23392]: audit 2024-03-26T03:51:13.502805+0000 mon.a (mon.0) 1801 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi175-64398-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:13.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:13 smithi175 bash[21527]: cluster 2024-03-26T03:51:12.646472+0000 mgr.y (mgr.24431) 282 : cluster [DBG] pgmap v321: 356 pgs: 356 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-26T03:51:13.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:13 smithi175 bash[21527]: cluster 2024-03-26T03:51:13.441043+0000 mon.a (mon.0) 1796 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:51:13.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:13 smithi175 bash[21527]: audit 2024-03-26T03:51:13.452938+0000 mon.a (mon.0) 1797 : audit [INF] from='client.? 172.21.15.175:0/2339775793' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi175-64398-35"}]': finished 2024-03-26T03:51:13.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:13 smithi175 bash[21527]: cluster 2024-03-26T03:51:13.461056+0000 mon.a (mon.0) 1798 : cluster [DBG] osdmap e217: 8 total, 8 up, 8 in 2024-03-26T03:51:13.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:13 smithi175 bash[21527]: audit 2024-03-26T03:51:13.500717+0000 mon.c (mon.2) 370 : audit [INF] from='client.? 172.21.15.175:0/2979685069' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi175-64398-36"}]: dispatch 2024-03-26T03:51:13.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:13 smithi175 bash[21527]: audit 2024-03-26T03:51:13.501067+0000 mon.a (mon.0) 1799 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi175-64398-36"}]: dispatch 2024-03-26T03:51:13.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:13 smithi175 bash[21527]: audit 2024-03-26T03:51:13.501718+0000 mon.c (mon.2) 371 : audit [INF] from='client.? 172.21.15.175:0/2979685069' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi175-64398-36"}]: dispatch 2024-03-26T03:51:13.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:13 smithi175 bash[21527]: audit 2024-03-26T03:51:13.502029+0000 mon.a (mon.0) 1800 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi175-64398-36"}]: dispatch 2024-03-26T03:51:13.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:13 smithi175 bash[21527]: audit 2024-03-26T03:51:13.502529+0000 mon.c (mon.2) 372 : audit [INF] from='client.? 172.21.15.175:0/2979685069' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi175-64398-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:13.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:13 smithi175 bash[21527]: audit 2024-03-26T03:51:13.502805+0000 mon.a (mon.0) 1801 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi175-64398-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:13.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:13 smithi175 bash[29284]: cluster 2024-03-26T03:51:12.646472+0000 mgr.y (mgr.24431) 282 : cluster [DBG] pgmap v321: 356 pgs: 356 active+clean; 584 KiB data, 571 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-26T03:51:13.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:13 smithi175 bash[29284]: cluster 2024-03-26T03:51:13.441043+0000 mon.a (mon.0) 1796 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:51:13.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:13 smithi175 bash[29284]: audit 2024-03-26T03:51:13.452938+0000 mon.a (mon.0) 1797 : audit [INF] from='client.? 172.21.15.175:0/2339775793' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi175-64398-35"}]': finished 2024-03-26T03:51:13.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:13 smithi175 bash[29284]: cluster 2024-03-26T03:51:13.461056+0000 mon.a (mon.0) 1798 : cluster [DBG] osdmap e217: 8 total, 8 up, 8 in 2024-03-26T03:51:13.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:13 smithi175 bash[29284]: audit 2024-03-26T03:51:13.500717+0000 mon.c (mon.2) 370 : audit [INF] from='client.? 172.21.15.175:0/2979685069' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi175-64398-36"}]: dispatch 2024-03-26T03:51:13.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:13 smithi175 bash[29284]: audit 2024-03-26T03:51:13.501067+0000 mon.a (mon.0) 1799 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi175-64398-36"}]: dispatch 2024-03-26T03:51:13.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:13 smithi175 bash[29284]: audit 2024-03-26T03:51:13.501718+0000 mon.c (mon.2) 371 : audit [INF] from='client.? 172.21.15.175:0/2979685069' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi175-64398-36"}]: dispatch 2024-03-26T03:51:13.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:13 smithi175 bash[29284]: audit 2024-03-26T03:51:13.502029+0000 mon.a (mon.0) 1800 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi175-64398-36"}]: dispatch 2024-03-26T03:51:13.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:13 smithi175 bash[29284]: audit 2024-03-26T03:51:13.502529+0000 mon.c (mon.2) 372 : audit [INF] from='client.? 172.21.15.175:0/2979685069' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi175-64398-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:13.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:13 smithi175 bash[29284]: audit 2024-03-26T03:51:13.502805+0000 mon.a (mon.0) 1801 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi175-64398-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:15.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:15 smithi190 bash[23392]: audit 2024-03-26T03:51:14.456704+0000 mon.a (mon.0) 1802 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi175-64398-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:15.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:15 smithi190 bash[23392]: cluster 2024-03-26T03:51:14.466622+0000 mon.a (mon.0) 1803 : cluster [DBG] osdmap e218: 8 total, 8 up, 8 in 2024-03-26T03:51:15.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:15 smithi190 bash[23392]: audit 2024-03-26T03:51:14.471505+0000 mon.c (mon.2) 373 : audit [INF] from='client.? 172.21.15.175:0/2979685069' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi175-64398-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi175-64398-36"}]: dispatch 2024-03-26T03:51:15.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:15 smithi190 bash[23392]: audit 2024-03-26T03:51:14.472187+0000 mon.a (mon.0) 1804 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:15.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:15 smithi190 bash[23392]: audit 2024-03-26T03:51:14.473826+0000 mon.a (mon.0) 1805 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi175-64398-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi175-64398-36"}]: dispatch 2024-03-26T03:51:15.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:15 smithi175 bash[21527]: audit 2024-03-26T03:51:14.456704+0000 mon.a (mon.0) 1802 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi175-64398-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:15.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:15 smithi175 bash[21527]: cluster 2024-03-26T03:51:14.466622+0000 mon.a (mon.0) 1803 : cluster [DBG] osdmap e218: 8 total, 8 up, 8 in 2024-03-26T03:51:15.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:15 smithi175 bash[21527]: audit 2024-03-26T03:51:14.471505+0000 mon.c (mon.2) 373 : audit [INF] from='client.? 172.21.15.175:0/2979685069' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi175-64398-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi175-64398-36"}]: dispatch 2024-03-26T03:51:15.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:15 smithi175 bash[21527]: audit 2024-03-26T03:51:14.472187+0000 mon.a (mon.0) 1804 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:15.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:15 smithi175 bash[21527]: audit 2024-03-26T03:51:14.473826+0000 mon.a (mon.0) 1805 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi175-64398-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi175-64398-36"}]: dispatch 2024-03-26T03:51:15.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:15 smithi175 bash[29284]: audit 2024-03-26T03:51:14.456704+0000 mon.a (mon.0) 1802 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi175-64398-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:15.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:15 smithi175 bash[29284]: cluster 2024-03-26T03:51:14.466622+0000 mon.a (mon.0) 1803 : cluster [DBG] osdmap e218: 8 total, 8 up, 8 in 2024-03-26T03:51:15.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:15 smithi175 bash[29284]: audit 2024-03-26T03:51:14.471505+0000 mon.c (mon.2) 373 : audit [INF] from='client.? 172.21.15.175:0/2979685069' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi175-64398-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi175-64398-36"}]: dispatch 2024-03-26T03:51:15.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:15 smithi175 bash[29284]: audit 2024-03-26T03:51:14.472187+0000 mon.a (mon.0) 1804 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:15.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:15 smithi175 bash[29284]: audit 2024-03-26T03:51:14.473826+0000 mon.a (mon.0) 1805 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi175-64398-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi175-64398-36"}]: dispatch 2024-03-26T03:51:16.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:16 smithi190 bash[23392]: cluster 2024-03-26T03:51:14.648460+0000 mgr.y (mgr.24431) 283 : cluster [DBG] pgmap v324: 292 pgs: 25 creating+peering, 7 unknown, 260 active+clean; 584 KiB data, 575 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:51:16.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:16 smithi190 bash[23392]: audit 2024-03-26T03:51:15.470175+0000 mon.a (mon.0) 1806 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:16.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:16 smithi190 bash[23392]: cluster 2024-03-26T03:51:15.496582+0000 mon.a (mon.0) 1807 : cluster [DBG] osdmap e219: 8 total, 8 up, 8 in 2024-03-26T03:51:16.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:16 smithi190 bash[23392]: audit 2024-03-26T03:51:15.504998+0000 mon.a (mon.0) 1808 : audit [INF] from='client.? 172.21.15.175:0/2872959749' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi175-64406-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:16.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:16 smithi175 bash[21527]: cluster 2024-03-26T03:51:14.648460+0000 mgr.y (mgr.24431) 283 : cluster [DBG] pgmap v324: 292 pgs: 25 creating+peering, 7 unknown, 260 active+clean; 584 KiB data, 575 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:51:16.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:16 smithi175 bash[21527]: audit 2024-03-26T03:51:15.470175+0000 mon.a (mon.0) 1806 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:16.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:16 smithi175 bash[21527]: cluster 2024-03-26T03:51:15.496582+0000 mon.a (mon.0) 1807 : cluster [DBG] osdmap e219: 8 total, 8 up, 8 in 2024-03-26T03:51:16.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:16 smithi175 bash[21527]: audit 2024-03-26T03:51:15.504998+0000 mon.a (mon.0) 1808 : audit [INF] from='client.? 172.21.15.175:0/2872959749' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi175-64406-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:16.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:16 smithi175 bash[29284]: cluster 2024-03-26T03:51:14.648460+0000 mgr.y (mgr.24431) 283 : cluster [DBG] pgmap v324: 292 pgs: 25 creating+peering, 7 unknown, 260 active+clean; 584 KiB data, 575 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:51:16.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:16 smithi175 bash[29284]: audit 2024-03-26T03:51:15.470175+0000 mon.a (mon.0) 1806 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:16.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:16 smithi175 bash[29284]: cluster 2024-03-26T03:51:15.496582+0000 mon.a (mon.0) 1807 : cluster [DBG] osdmap e219: 8 total, 8 up, 8 in 2024-03-26T03:51:16.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:16 smithi175 bash[29284]: audit 2024-03-26T03:51:15.504998+0000 mon.a (mon.0) 1808 : audit [INF] from='client.? 172.21.15.175:0/2872959749' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi175-64406-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:17.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:17 smithi190 bash[23392]: audit 2024-03-26T03:51:16.484826+0000 mon.a (mon.0) 1809 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi175-64398-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi175-64398-36"}]': finished 2024-03-26T03:51:17.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:17 smithi190 bash[23392]: audit 2024-03-26T03:51:16.485168+0000 mon.a (mon.0) 1810 : audit [INF] from='client.? 172.21.15.175:0/2872959749' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi175-64406-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:17.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:17 smithi190 bash[23392]: cluster 2024-03-26T03:51:16.493818+0000 mon.a (mon.0) 1811 : cluster [DBG] osdmap e220: 8 total, 8 up, 8 in 2024-03-26T03:51:17.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:17 smithi190 bash[23392]: audit 2024-03-26T03:51:16.567956+0000 mon.a (mon.0) 1812 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:51:17.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:17 smithi175 bash[29284]: audit 2024-03-26T03:51:16.484826+0000 mon.a (mon.0) 1809 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi175-64398-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi175-64398-36"}]': finished 2024-03-26T03:51:17.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:17 smithi175 bash[29284]: audit 2024-03-26T03:51:16.485168+0000 mon.a (mon.0) 1810 : audit [INF] from='client.? 172.21.15.175:0/2872959749' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi175-64406-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:17.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:17 smithi175 bash[29284]: cluster 2024-03-26T03:51:16.493818+0000 mon.a (mon.0) 1811 : cluster [DBG] osdmap e220: 8 total, 8 up, 8 in 2024-03-26T03:51:17.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:17 smithi175 bash[29284]: audit 2024-03-26T03:51:16.567956+0000 mon.a (mon.0) 1812 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:51:17.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:17 smithi175 bash[21527]: audit 2024-03-26T03:51:16.484826+0000 mon.a (mon.0) 1809 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi175-64398-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi175-64398-36"}]': finished 2024-03-26T03:51:17.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:17 smithi175 bash[21527]: audit 2024-03-26T03:51:16.485168+0000 mon.a (mon.0) 1810 : audit [INF] from='client.? 172.21.15.175:0/2872959749' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi175-64406-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:17.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:17 smithi175 bash[21527]: cluster 2024-03-26T03:51:16.493818+0000 mon.a (mon.0) 1811 : cluster [DBG] osdmap e220: 8 total, 8 up, 8 in 2024-03-26T03:51:17.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:17 smithi175 bash[21527]: audit 2024-03-26T03:51:16.567956+0000 mon.a (mon.0) 1812 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:51:18.812 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:18 smithi175 bash[21527]: cluster 2024-03-26T03:51:16.649413+0000 mgr.y (mgr.24431) 284 : cluster [DBG] pgmap v327: 332 pgs: 40 unknown, 32 creating+peering, 260 active+clean; 584 KiB data, 559 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:51:18.812 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:18 smithi175 bash[21527]: audit 2024-03-26T03:51:17.488063+0000 mon.a (mon.0) 1813 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-27", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:51:18.812 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:18 smithi175 bash[21527]: cluster 2024-03-26T03:51:17.495744+0000 mon.a (mon.0) 1814 : cluster [DBG] osdmap e221: 8 total, 8 up, 8 in 2024-03-26T03:51:18.812 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:18 smithi175 bash[21527]: audit 2024-03-26T03:51:17.499775+0000 mon.a (mon.0) 1815 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-27"}]: dispatch 2024-03-26T03:51:18.813 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:18 smithi175 bash[21527]: audit 2024-03-26T03:51:18.492070+0000 mon.a (mon.0) 1816 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-27"}]': finished 2024-03-26T03:51:18.813 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:18 smithi175 bash[21527]: cluster 2024-03-26T03:51:18.502666+0000 mon.a (mon.0) 1817 : cluster [DBG] osdmap e222: 8 total, 8 up, 8 in 2024-03-26T03:51:18.813 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:18 smithi175 bash[29284]: cluster 2024-03-26T03:51:16.649413+0000 mgr.y (mgr.24431) 284 : cluster [DBG] pgmap v327: 332 pgs: 40 unknown, 32 creating+peering, 260 active+clean; 584 KiB data, 559 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:51:18.813 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:18 smithi175 bash[29284]: audit 2024-03-26T03:51:17.488063+0000 mon.a (mon.0) 1813 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-27", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:51:18.813 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:18 smithi175 bash[29284]: cluster 2024-03-26T03:51:17.495744+0000 mon.a (mon.0) 1814 : cluster [DBG] osdmap e221: 8 total, 8 up, 8 in 2024-03-26T03:51:18.813 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:18 smithi175 bash[29284]: audit 2024-03-26T03:51:17.499775+0000 mon.a (mon.0) 1815 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-27"}]: dispatch 2024-03-26T03:51:18.813 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:18 smithi175 bash[29284]: audit 2024-03-26T03:51:18.492070+0000 mon.a (mon.0) 1816 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-27"}]': finished 2024-03-26T03:51:18.813 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:18 smithi175 bash[29284]: cluster 2024-03-26T03:51:18.502666+0000 mon.a (mon.0) 1817 : cluster [DBG] osdmap e222: 8 total, 8 up, 8 in 2024-03-26T03:51:18.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:18 smithi190 bash[23392]: cluster 2024-03-26T03:51:16.649413+0000 mgr.y (mgr.24431) 284 : cluster [DBG] pgmap v327: 332 pgs: 40 unknown, 32 creating+peering, 260 active+clean; 584 KiB data, 559 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:51:18.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:18 smithi190 bash[23392]: audit 2024-03-26T03:51:17.488063+0000 mon.a (mon.0) 1813 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-27", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:51:18.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:18 smithi190 bash[23392]: cluster 2024-03-26T03:51:17.495744+0000 mon.a (mon.0) 1814 : cluster [DBG] osdmap e221: 8 total, 8 up, 8 in 2024-03-26T03:51:18.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:18 smithi190 bash[23392]: audit 2024-03-26T03:51:17.499775+0000 mon.a (mon.0) 1815 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-27"}]: dispatch 2024-03-26T03:51:18.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:18 smithi190 bash[23392]: audit 2024-03-26T03:51:18.492070+0000 mon.a (mon.0) 1816 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-27"}]': finished 2024-03-26T03:51:18.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:18 smithi190 bash[23392]: cluster 2024-03-26T03:51:18.502666+0000 mon.a (mon.0) 1817 : cluster [DBG] osdmap e222: 8 total, 8 up, 8 in 2024-03-26T03:51:19.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:51:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:51:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:51:19.835 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:19 smithi190 bash[23392]: audit 2024-03-26T03:51:18.508181+0000 mon.a (mon.0) 1818 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-27", "mode": "writeback"}]: dispatch 2024-03-26T03:51:19.835 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:19 smithi190 bash[23392]: audit 2024-03-26T03:51:18.509405+0000 mon.a (mon.0) 1819 : audit [INF] from='client.? 172.21.15.175:0/254601226' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi175-64406-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:19.835 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:19 smithi190 bash[23392]: audit 2024-03-26T03:51:18.512134+0000 mon.c (mon.2) 374 : audit [INF] from='client.? 172.21.15.175:0/2979685069' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi175-64398-36"}]: dispatch 2024-03-26T03:51:19.835 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:19 smithi190 bash[23392]: audit 2024-03-26T03:51:18.513254+0000 mon.a (mon.0) 1820 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi175-64398-36"}]: dispatch 2024-03-26T03:51:19.835 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:19 smithi190 bash[23392]: cluster 2024-03-26T03:51:18.650288+0000 mgr.y (mgr.24431) 285 : cluster [DBG] pgmap v330: 324 pgs: 32 unknown, 32 creating+peering, 260 active+clean; 584 KiB data, 559 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:51:19.835 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:19 smithi190 bash[23392]: audit 2024-03-26T03:51:19.373929+0000 mon.c (mon.2) 375 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:51:19.835 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:19 smithi190 bash[23392]: cluster 2024-03-26T03:51:19.492656+0000 mon.a (mon.0) 1821 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:51:19.835 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:19 smithi190 bash[23392]: audit 2024-03-26T03:51:19.495299+0000 mon.a (mon.0) 1822 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-27", "mode": "writeback"}]': finished 2024-03-26T03:51:19.835 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:19 smithi190 bash[23392]: audit 2024-03-26T03:51:19.495485+0000 mon.a (mon.0) 1823 : audit [INF] from='client.? 172.21.15.175:0/254601226' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi175-64406-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:19.835 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:19 smithi190 bash[23392]: audit 2024-03-26T03:51:19.495639+0000 mon.a (mon.0) 1824 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi175-64398-36"}]': finished 2024-03-26T03:51:19.835 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:19 smithi190 bash[23392]: cluster 2024-03-26T03:51:19.504643+0000 mon.a (mon.0) 1825 : cluster [DBG] osdmap e223: 8 total, 8 up, 8 in 2024-03-26T03:51:19.836 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:19 smithi190 bash[23392]: audit 2024-03-26T03:51:19.508392+0000 mon.c (mon.2) 376 : audit [INF] from='client.? 172.21.15.175:0/2979685069' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi175-64398-36"}]: dispatch 2024-03-26T03:51:19.836 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:19 smithi190 bash[23392]: audit 2024-03-26T03:51:19.509799+0000 mon.a (mon.0) 1826 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi175-64398-36"}]: dispatch 2024-03-26T03:51:19.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:19 smithi175 bash[29284]: audit 2024-03-26T03:51:18.508181+0000 mon.a (mon.0) 1818 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-27", "mode": "writeback"}]: dispatch 2024-03-26T03:51:19.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:19 smithi175 bash[29284]: audit 2024-03-26T03:51:18.509405+0000 mon.a (mon.0) 1819 : audit [INF] from='client.? 172.21.15.175:0/254601226' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi175-64406-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:19.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:19 smithi175 bash[29284]: audit 2024-03-26T03:51:18.512134+0000 mon.c (mon.2) 374 : audit [INF] from='client.? 172.21.15.175:0/2979685069' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi175-64398-36"}]: dispatch 2024-03-26T03:51:19.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:19 smithi175 bash[29284]: audit 2024-03-26T03:51:18.513254+0000 mon.a (mon.0) 1820 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi175-64398-36"}]: dispatch 2024-03-26T03:51:19.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:19 smithi175 bash[29284]: cluster 2024-03-26T03:51:18.650288+0000 mgr.y (mgr.24431) 285 : cluster [DBG] pgmap v330: 324 pgs: 32 unknown, 32 creating+peering, 260 active+clean; 584 KiB data, 559 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:51:19.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:19 smithi175 bash[29284]: audit 2024-03-26T03:51:19.373929+0000 mon.c (mon.2) 375 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:51:19.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:19 smithi175 bash[29284]: cluster 2024-03-26T03:51:19.492656+0000 mon.a (mon.0) 1821 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:51:19.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:19 smithi175 bash[29284]: audit 2024-03-26T03:51:19.495299+0000 mon.a (mon.0) 1822 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-27", "mode": "writeback"}]': finished 2024-03-26T03:51:19.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:19 smithi175 bash[29284]: audit 2024-03-26T03:51:19.495485+0000 mon.a (mon.0) 1823 : audit [INF] from='client.? 172.21.15.175:0/254601226' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi175-64406-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:19.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:19 smithi175 bash[29284]: audit 2024-03-26T03:51:19.495639+0000 mon.a (mon.0) 1824 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi175-64398-36"}]': finished 2024-03-26T03:51:19.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:19 smithi175 bash[29284]: cluster 2024-03-26T03:51:19.504643+0000 mon.a (mon.0) 1825 : cluster [DBG] osdmap e223: 8 total, 8 up, 8 in 2024-03-26T03:51:19.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:19 smithi175 bash[29284]: audit 2024-03-26T03:51:19.508392+0000 mon.c (mon.2) 376 : audit [INF] from='client.? 172.21.15.175:0/2979685069' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi175-64398-36"}]: dispatch 2024-03-26T03:51:19.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:19 smithi175 bash[29284]: audit 2024-03-26T03:51:19.509799+0000 mon.a (mon.0) 1826 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi175-64398-36"}]: dispatch 2024-03-26T03:51:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:19 smithi175 bash[21527]: audit 2024-03-26T03:51:18.508181+0000 mon.a (mon.0) 1818 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-27", "mode": "writeback"}]: dispatch 2024-03-26T03:51:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:19 smithi175 bash[21527]: audit 2024-03-26T03:51:18.509405+0000 mon.a (mon.0) 1819 : audit [INF] from='client.? 172.21.15.175:0/254601226' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi175-64406-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:19 smithi175 bash[21527]: audit 2024-03-26T03:51:18.512134+0000 mon.c (mon.2) 374 : audit [INF] from='client.? 172.21.15.175:0/2979685069' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi175-64398-36"}]: dispatch 2024-03-26T03:51:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:19 smithi175 bash[21527]: audit 2024-03-26T03:51:18.513254+0000 mon.a (mon.0) 1820 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi175-64398-36"}]: dispatch 2024-03-26T03:51:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:19 smithi175 bash[21527]: cluster 2024-03-26T03:51:18.650288+0000 mgr.y (mgr.24431) 285 : cluster [DBG] pgmap v330: 324 pgs: 32 unknown, 32 creating+peering, 260 active+clean; 584 KiB data, 559 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:51:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:19 smithi175 bash[21527]: audit 2024-03-26T03:51:19.373929+0000 mon.c (mon.2) 375 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:51:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:19 smithi175 bash[21527]: cluster 2024-03-26T03:51:19.492656+0000 mon.a (mon.0) 1821 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:51:19.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:19 smithi175 bash[21527]: audit 2024-03-26T03:51:19.495299+0000 mon.a (mon.0) 1822 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-27", "mode": "writeback"}]': finished 2024-03-26T03:51:19.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:19 smithi175 bash[21527]: audit 2024-03-26T03:51:19.495485+0000 mon.a (mon.0) 1823 : audit [INF] from='client.? 172.21.15.175:0/254601226' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi175-64406-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:19.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:19 smithi175 bash[21527]: audit 2024-03-26T03:51:19.495639+0000 mon.a (mon.0) 1824 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi175-64398-36"}]': finished 2024-03-26T03:51:19.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:19 smithi175 bash[21527]: cluster 2024-03-26T03:51:19.504643+0000 mon.a (mon.0) 1825 : cluster [DBG] osdmap e223: 8 total, 8 up, 8 in 2024-03-26T03:51:19.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:19 smithi175 bash[21527]: audit 2024-03-26T03:51:19.508392+0000 mon.c (mon.2) 376 : audit [INF] from='client.? 172.21.15.175:0/2979685069' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi175-64398-36"}]: dispatch 2024-03-26T03:51:19.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:19 smithi175 bash[21527]: audit 2024-03-26T03:51:19.509799+0000 mon.a (mon.0) 1826 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi175-64398-36"}]: dispatch 2024-03-26T03:51:20.240 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:51:19 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:51:21.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:21 smithi190 bash[23392]: audit 2024-03-26T03:51:19.836602+0000 mgr.y (mgr.24431) 286 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:51:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:21 smithi190 bash[23392]: audit 2024-03-26T03:51:20.499172+0000 mon.a (mon.0) 1827 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi175-64398-36"}]': finished 2024-03-26T03:51:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:21 smithi190 bash[23392]: cluster 2024-03-26T03:51:20.510381+0000 mon.a (mon.0) 1828 : cluster [DBG] osdmap e224: 8 total, 8 up, 8 in 2024-03-26T03:51:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:21 smithi190 bash[23392]: audit 2024-03-26T03:51:20.557500+0000 mon.c (mon.2) 377 : audit [INF] from='client.? 172.21.15.175:0/795941933' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi175-64406-46"}]: dispatch 2024-03-26T03:51:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:21 smithi190 bash[23392]: audit 2024-03-26T03:51:20.558081+0000 mon.a (mon.0) 1829 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi175-64406-46"}]: dispatch 2024-03-26T03:51:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:21 smithi190 bash[23392]: audit 2024-03-26T03:51:20.558928+0000 mon.c (mon.2) 378 : audit [INF] from='client.? 172.21.15.175:0/795941933' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi175-64406-46"}]: dispatch 2024-03-26T03:51:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:21 smithi190 bash[23392]: audit 2024-03-26T03:51:20.559378+0000 mon.a (mon.0) 1830 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi175-64406-46"}]: dispatch 2024-03-26T03:51:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:21 smithi190 bash[23392]: audit 2024-03-26T03:51:20.560034+0000 mon.c (mon.2) 379 : audit [INF] from='client.? 172.21.15.175:0/795941933' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi175-64406-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:21 smithi190 bash[23392]: audit 2024-03-26T03:51:20.560408+0000 mon.a (mon.0) 1831 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi175-64406-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:21 smithi190 bash[23392]: audit 2024-03-26T03:51:20.564169+0000 mon.c (mon.2) 380 : audit [INF] from='client.? 172.21.15.175:0/1747297285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi175-64398-37"}]: dispatch 2024-03-26T03:51:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:21 smithi190 bash[23392]: audit 2024-03-26T03:51:20.564520+0000 mon.a (mon.0) 1832 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi175-64398-37"}]: dispatch 2024-03-26T03:51:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:21 smithi190 bash[23392]: audit 2024-03-26T03:51:20.565585+0000 mon.c (mon.2) 381 : audit [INF] from='client.? 172.21.15.175:0/1747297285' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi175-64398-37"}]: dispatch 2024-03-26T03:51:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:21 smithi190 bash[23392]: audit 2024-03-26T03:51:20.565939+0000 mon.a (mon.0) 1833 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi175-64398-37"}]: dispatch 2024-03-26T03:51:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:21 smithi190 bash[23392]: audit 2024-03-26T03:51:20.566597+0000 mon.c (mon.2) 382 : audit [INF] from='client.? 172.21.15.175:0/1747297285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi175-64398-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:21.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:21 smithi190 bash[23392]: audit 2024-03-26T03:51:20.566900+0000 mon.a (mon.0) 1834 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi175-64398-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:21.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:21 smithi190 bash[23392]: audit 2024-03-26T03:51:20.585140+0000 mon.a (mon.0) 1835 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:51:21.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:21 smithi190 bash[23392]: cluster 2024-03-26T03:51:21.438410+0000 mon.a (mon.0) 1836 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:51:21.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[21527]: audit 2024-03-26T03:51:19.836602+0000 mgr.y (mgr.24431) 286 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:51:21.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[21527]: audit 2024-03-26T03:51:20.499172+0000 mon.a (mon.0) 1827 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi175-64398-36"}]': finished 2024-03-26T03:51:21.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[21527]: cluster 2024-03-26T03:51:20.510381+0000 mon.a (mon.0) 1828 : cluster [DBG] osdmap e224: 8 total, 8 up, 8 in 2024-03-26T03:51:21.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[21527]: audit 2024-03-26T03:51:20.557500+0000 mon.c (mon.2) 377 : audit [INF] from='client.? 172.21.15.175:0/795941933' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi175-64406-46"}]: dispatch 2024-03-26T03:51:21.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[21527]: audit 2024-03-26T03:51:20.558081+0000 mon.a (mon.0) 1829 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi175-64406-46"}]: dispatch 2024-03-26T03:51:21.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[21527]: audit 2024-03-26T03:51:20.558928+0000 mon.c (mon.2) 378 : audit [INF] from='client.? 172.21.15.175:0/795941933' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi175-64406-46"}]: dispatch 2024-03-26T03:51:21.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[21527]: audit 2024-03-26T03:51:20.559378+0000 mon.a (mon.0) 1830 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi175-64406-46"}]: dispatch 2024-03-26T03:51:21.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[21527]: audit 2024-03-26T03:51:20.560034+0000 mon.c (mon.2) 379 : audit [INF] from='client.? 172.21.15.175:0/795941933' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi175-64406-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:21.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[21527]: audit 2024-03-26T03:51:20.560408+0000 mon.a (mon.0) 1831 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi175-64406-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:21.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[21527]: audit 2024-03-26T03:51:20.564169+0000 mon.c (mon.2) 380 : audit [INF] from='client.? 172.21.15.175:0/1747297285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi175-64398-37"}]: dispatch 2024-03-26T03:51:21.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[21527]: audit 2024-03-26T03:51:20.564520+0000 mon.a (mon.0) 1832 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi175-64398-37"}]: dispatch 2024-03-26T03:51:21.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[21527]: audit 2024-03-26T03:51:20.565585+0000 mon.c (mon.2) 381 : audit [INF] from='client.? 172.21.15.175:0/1747297285' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi175-64398-37"}]: dispatch 2024-03-26T03:51:21.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[21527]: audit 2024-03-26T03:51:20.565939+0000 mon.a (mon.0) 1833 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi175-64398-37"}]: dispatch 2024-03-26T03:51:21.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[21527]: audit 2024-03-26T03:51:20.566597+0000 mon.c (mon.2) 382 : audit [INF] from='client.? 172.21.15.175:0/1747297285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi175-64398-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:21.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[21527]: audit 2024-03-26T03:51:20.566900+0000 mon.a (mon.0) 1834 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi175-64398-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:21.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[21527]: audit 2024-03-26T03:51:20.585140+0000 mon.a (mon.0) 1835 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:51:21.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[21527]: cluster 2024-03-26T03:51:21.438410+0000 mon.a (mon.0) 1836 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:51:21.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[29284]: audit 2024-03-26T03:51:19.836602+0000 mgr.y (mgr.24431) 286 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:51:21.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[29284]: audit 2024-03-26T03:51:20.499172+0000 mon.a (mon.0) 1827 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi175-64398-36"}]': finished 2024-03-26T03:51:21.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[29284]: cluster 2024-03-26T03:51:20.510381+0000 mon.a (mon.0) 1828 : cluster [DBG] osdmap e224: 8 total, 8 up, 8 in 2024-03-26T03:51:21.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[29284]: audit 2024-03-26T03:51:20.557500+0000 mon.c (mon.2) 377 : audit [INF] from='client.? 172.21.15.175:0/795941933' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi175-64406-46"}]: dispatch 2024-03-26T03:51:21.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[29284]: audit 2024-03-26T03:51:20.558081+0000 mon.a (mon.0) 1829 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi175-64406-46"}]: dispatch 2024-03-26T03:51:21.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[29284]: audit 2024-03-26T03:51:20.558928+0000 mon.c (mon.2) 378 : audit [INF] from='client.? 172.21.15.175:0/795941933' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi175-64406-46"}]: dispatch 2024-03-26T03:51:21.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[29284]: audit 2024-03-26T03:51:20.559378+0000 mon.a (mon.0) 1830 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi175-64406-46"}]: dispatch 2024-03-26T03:51:21.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[29284]: audit 2024-03-26T03:51:20.560034+0000 mon.c (mon.2) 379 : audit [INF] from='client.? 172.21.15.175:0/795941933' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi175-64406-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:21.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[29284]: audit 2024-03-26T03:51:20.560408+0000 mon.a (mon.0) 1831 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi175-64406-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:21.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[29284]: audit 2024-03-26T03:51:20.564169+0000 mon.c (mon.2) 380 : audit [INF] from='client.? 172.21.15.175:0/1747297285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi175-64398-37"}]: dispatch 2024-03-26T03:51:21.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[29284]: audit 2024-03-26T03:51:20.564520+0000 mon.a (mon.0) 1832 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi175-64398-37"}]: dispatch 2024-03-26T03:51:21.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[29284]: audit 2024-03-26T03:51:20.565585+0000 mon.c (mon.2) 381 : audit [INF] from='client.? 172.21.15.175:0/1747297285' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi175-64398-37"}]: dispatch 2024-03-26T03:51:21.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[29284]: audit 2024-03-26T03:51:20.565939+0000 mon.a (mon.0) 1833 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi175-64398-37"}]: dispatch 2024-03-26T03:51:21.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[29284]: audit 2024-03-26T03:51:20.566597+0000 mon.c (mon.2) 382 : audit [INF] from='client.? 172.21.15.175:0/1747297285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi175-64398-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:21.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[29284]: audit 2024-03-26T03:51:20.566900+0000 mon.a (mon.0) 1834 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi175-64398-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:21.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[29284]: audit 2024-03-26T03:51:20.585140+0000 mon.a (mon.0) 1835 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:51:21.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:21 smithi175 bash[29284]: cluster 2024-03-26T03:51:21.438410+0000 mon.a (mon.0) 1836 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:51:22.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:22 smithi190 bash[23392]: cluster 2024-03-26T03:51:20.651200+0000 mgr.y (mgr.24431) 287 : cluster [DBG] pgmap v333: 292 pgs: 292 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-03-26T03:51:22.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:22 smithi190 bash[23392]: audit 2024-03-26T03:51:21.510802+0000 mon.a (mon.0) 1837 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi175-64406-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:22.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:22 smithi190 bash[23392]: audit 2024-03-26T03:51:21.510998+0000 mon.a (mon.0) 1838 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi175-64398-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:22.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:22 smithi190 bash[23392]: audit 2024-03-26T03:51:21.511121+0000 mon.a (mon.0) 1839 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:51:22.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:22 smithi190 bash[23392]: cluster 2024-03-26T03:51:21.521564+0000 mon.a (mon.0) 1840 : cluster [DBG] osdmap e225: 8 total, 8 up, 8 in 2024-03-26T03:51:22.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:22 smithi190 bash[23392]: audit 2024-03-26T03:51:21.524151+0000 mon.c (mon.2) 383 : audit [INF] from='client.? 172.21.15.175:0/795941933' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi175-64406-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi175-64406-46"}]: dispatch 2024-03-26T03:51:22.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:22 smithi190 bash[23392]: audit 2024-03-26T03:51:21.525280+0000 mon.c (mon.2) 384 : audit [INF] from='client.? 172.21.15.175:0/1747297285' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi175-64398-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi175-64398-37"}]: dispatch 2024-03-26T03:51:22.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:22 smithi190 bash[23392]: audit 2024-03-26T03:51:21.525486+0000 mon.a (mon.0) 1841 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-27"}]: dispatch 2024-03-26T03:51:22.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:22 smithi190 bash[23392]: audit 2024-03-26T03:51:21.527765+0000 mon.a (mon.0) 1842 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi175-64406-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi175-64406-46"}]: dispatch 2024-03-26T03:51:22.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:22 smithi190 bash[23392]: audit 2024-03-26T03:51:21.528771+0000 mon.a (mon.0) 1843 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi175-64398-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi175-64398-37"}]: dispatch 2024-03-26T03:51:22.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:22 smithi175 bash[21527]: cluster 2024-03-26T03:51:20.651200+0000 mgr.y (mgr.24431) 287 : cluster [DBG] pgmap v333: 292 pgs: 292 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-03-26T03:51:22.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:22 smithi175 bash[21527]: audit 2024-03-26T03:51:21.510802+0000 mon.a (mon.0) 1837 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi175-64406-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:22.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:22 smithi175 bash[21527]: audit 2024-03-26T03:51:21.510998+0000 mon.a (mon.0) 1838 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi175-64398-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:22.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:22 smithi175 bash[21527]: audit 2024-03-26T03:51:21.511121+0000 mon.a (mon.0) 1839 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:51:22.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:22 smithi175 bash[21527]: cluster 2024-03-26T03:51:21.521564+0000 mon.a (mon.0) 1840 : cluster [DBG] osdmap e225: 8 total, 8 up, 8 in 2024-03-26T03:51:22.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:22 smithi175 bash[21527]: audit 2024-03-26T03:51:21.524151+0000 mon.c (mon.2) 383 : audit [INF] from='client.? 172.21.15.175:0/795941933' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi175-64406-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi175-64406-46"}]: dispatch 2024-03-26T03:51:22.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:22 smithi175 bash[21527]: audit 2024-03-26T03:51:21.525280+0000 mon.c (mon.2) 384 : audit [INF] from='client.? 172.21.15.175:0/1747297285' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi175-64398-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi175-64398-37"}]: dispatch 2024-03-26T03:51:22.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:22 smithi175 bash[21527]: audit 2024-03-26T03:51:21.525486+0000 mon.a (mon.0) 1841 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-27"}]: dispatch 2024-03-26T03:51:22.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:22 smithi175 bash[21527]: audit 2024-03-26T03:51:21.527765+0000 mon.a (mon.0) 1842 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi175-64406-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi175-64406-46"}]: dispatch 2024-03-26T03:51:22.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:22 smithi175 bash[21527]: audit 2024-03-26T03:51:21.528771+0000 mon.a (mon.0) 1843 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi175-64398-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi175-64398-37"}]: dispatch 2024-03-26T03:51:22.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:22 smithi175 bash[29284]: cluster 2024-03-26T03:51:20.651200+0000 mgr.y (mgr.24431) 287 : cluster [DBG] pgmap v333: 292 pgs: 292 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-03-26T03:51:22.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:22 smithi175 bash[29284]: audit 2024-03-26T03:51:21.510802+0000 mon.a (mon.0) 1837 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi175-64406-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:22.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:22 smithi175 bash[29284]: audit 2024-03-26T03:51:21.510998+0000 mon.a (mon.0) 1838 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi175-64398-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:22.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:22 smithi175 bash[29284]: audit 2024-03-26T03:51:21.511121+0000 mon.a (mon.0) 1839 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:51:22.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:22 smithi175 bash[29284]: cluster 2024-03-26T03:51:21.521564+0000 mon.a (mon.0) 1840 : cluster [DBG] osdmap e225: 8 total, 8 up, 8 in 2024-03-26T03:51:22.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:22 smithi175 bash[29284]: audit 2024-03-26T03:51:21.524151+0000 mon.c (mon.2) 383 : audit [INF] from='client.? 172.21.15.175:0/795941933' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi175-64406-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi175-64406-46"}]: dispatch 2024-03-26T03:51:22.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:22 smithi175 bash[29284]: audit 2024-03-26T03:51:21.525280+0000 mon.c (mon.2) 384 : audit [INF] from='client.? 172.21.15.175:0/1747297285' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi175-64398-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi175-64398-37"}]: dispatch 2024-03-26T03:51:22.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:22 smithi175 bash[29284]: audit 2024-03-26T03:51:21.525486+0000 mon.a (mon.0) 1841 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-27"}]: dispatch 2024-03-26T03:51:22.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:22 smithi175 bash[29284]: audit 2024-03-26T03:51:21.527765+0000 mon.a (mon.0) 1842 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi175-64406-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi175-64406-46"}]: dispatch 2024-03-26T03:51:22.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:22 smithi175 bash[29284]: audit 2024-03-26T03:51:21.528771+0000 mon.a (mon.0) 1843 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi175-64398-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi175-64398-37"}]: dispatch 2024-03-26T03:51:23.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:23 smithi190 bash[23392]: cluster 2024-03-26T03:51:22.511873+0000 mon.a (mon.0) 1844 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:51:23.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:23 smithi190 bash[23392]: audit 2024-03-26T03:51:22.514425+0000 mon.a (mon.0) 1845 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-27"}]': finished 2024-03-26T03:51:23.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:23 smithi190 bash[23392]: cluster 2024-03-26T03:51:22.546648+0000 mon.a (mon.0) 1846 : cluster [DBG] osdmap e226: 8 total, 8 up, 8 in 2024-03-26T03:51:23.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:23 smithi190 bash[23392]: cluster 2024-03-26T03:51:22.651853+0000 mgr.y (mgr.24431) 288 : cluster [DBG] pgmap v336: 292 pgs: 292 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-03-26T03:51:23.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:23 smithi190 bash[23392]: audit 2024-03-26T03:51:23.519868+0000 mon.a (mon.0) 1847 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi175-64406-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi175-64406-46"}]': finished 2024-03-26T03:51:23.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:23 smithi190 bash[23392]: audit 2024-03-26T03:51:23.520160+0000 mon.a (mon.0) 1848 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi175-64398-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi175-64398-37"}]': finished 2024-03-26T03:51:23.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:23 smithi190 bash[23392]: cluster 2024-03-26T03:51:23.527414+0000 mon.a (mon.0) 1849 : cluster [DBG] osdmap e227: 8 total, 8 up, 8 in 2024-03-26T03:51:23.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:23 smithi175 bash[29284]: cluster 2024-03-26T03:51:22.511873+0000 mon.a (mon.0) 1844 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:51:23.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:23 smithi175 bash[29284]: audit 2024-03-26T03:51:22.514425+0000 mon.a (mon.0) 1845 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-27"}]': finished 2024-03-26T03:51:23.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:23 smithi175 bash[29284]: cluster 2024-03-26T03:51:22.546648+0000 mon.a (mon.0) 1846 : cluster [DBG] osdmap e226: 8 total, 8 up, 8 in 2024-03-26T03:51:23.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:23 smithi175 bash[29284]: cluster 2024-03-26T03:51:22.651853+0000 mgr.y (mgr.24431) 288 : cluster [DBG] pgmap v336: 292 pgs: 292 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-03-26T03:51:23.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:23 smithi175 bash[29284]: audit 2024-03-26T03:51:23.519868+0000 mon.a (mon.0) 1847 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi175-64406-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi175-64406-46"}]': finished 2024-03-26T03:51:23.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:23 smithi175 bash[29284]: audit 2024-03-26T03:51:23.520160+0000 mon.a (mon.0) 1848 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi175-64398-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi175-64398-37"}]': finished 2024-03-26T03:51:23.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:23 smithi175 bash[29284]: cluster 2024-03-26T03:51:23.527414+0000 mon.a (mon.0) 1849 : cluster [DBG] osdmap e227: 8 total, 8 up, 8 in 2024-03-26T03:51:23.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:23 smithi175 bash[21527]: cluster 2024-03-26T03:51:22.511873+0000 mon.a (mon.0) 1844 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:51:23.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:23 smithi175 bash[21527]: audit 2024-03-26T03:51:22.514425+0000 mon.a (mon.0) 1845 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-27"}]': finished 2024-03-26T03:51:23.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:23 smithi175 bash[21527]: cluster 2024-03-26T03:51:22.546648+0000 mon.a (mon.0) 1846 : cluster [DBG] osdmap e226: 8 total, 8 up, 8 in 2024-03-26T03:51:23.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:23 smithi175 bash[21527]: cluster 2024-03-26T03:51:22.651853+0000 mgr.y (mgr.24431) 288 : cluster [DBG] pgmap v336: 292 pgs: 292 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-03-26T03:51:23.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:23 smithi175 bash[21527]: audit 2024-03-26T03:51:23.519868+0000 mon.a (mon.0) 1847 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi175-64406-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi175-64406-46"}]': finished 2024-03-26T03:51:23.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:23 smithi175 bash[21527]: audit 2024-03-26T03:51:23.520160+0000 mon.a (mon.0) 1848 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi175-64398-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi175-64398-37"}]': finished 2024-03-26T03:51:23.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:23 smithi175 bash[21527]: cluster 2024-03-26T03:51:23.527414+0000 mon.a (mon.0) 1849 : cluster [DBG] osdmap e227: 8 total, 8 up, 8 in 2024-03-26T03:51:25.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:25 smithi190 bash[23392]: cluster 2024-03-26T03:51:24.535470+0000 mon.a (mon.0) 1850 : cluster [DBG] osdmap e228: 8 total, 8 up, 8 in 2024-03-26T03:51:25.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:25 smithi190 bash[23392]: audit 2024-03-26T03:51:24.545776+0000 mon.a (mon.0) 1851 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:25.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:25 smithi190 bash[23392]: cluster 2024-03-26T03:51:24.653782+0000 mgr.y (mgr.24431) 289 : cluster [DBG] pgmap v339: 308 pgs: 18 creating+peering, 20 unknown, 270 active+clean; 584 KiB data, 565 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:51:25.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:25 smithi175 bash[29284]: cluster 2024-03-26T03:51:24.535470+0000 mon.a (mon.0) 1850 : cluster [DBG] osdmap e228: 8 total, 8 up, 8 in 2024-03-26T03:51:25.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:25 smithi175 bash[29284]: audit 2024-03-26T03:51:24.545776+0000 mon.a (mon.0) 1851 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:25.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:25 smithi175 bash[29284]: cluster 2024-03-26T03:51:24.653782+0000 mgr.y (mgr.24431) 289 : cluster [DBG] pgmap v339: 308 pgs: 18 creating+peering, 20 unknown, 270 active+clean; 584 KiB data, 565 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:51:25.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:25 smithi175 bash[21527]: cluster 2024-03-26T03:51:24.535470+0000 mon.a (mon.0) 1850 : cluster [DBG] osdmap e228: 8 total, 8 up, 8 in 2024-03-26T03:51:25.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:25 smithi175 bash[21527]: audit 2024-03-26T03:51:24.545776+0000 mon.a (mon.0) 1851 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:25.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:25 smithi175 bash[21527]: cluster 2024-03-26T03:51:24.653782+0000 mgr.y (mgr.24431) 289 : cluster [DBG] pgmap v339: 308 pgs: 18 creating+peering, 20 unknown, 270 active+clean; 584 KiB data, 565 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:51:26.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:26 smithi190 bash[23392]: audit 2024-03-26T03:51:25.532743+0000 mon.a (mon.0) 1852 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:26.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:26 smithi190 bash[23392]: cluster 2024-03-26T03:51:25.542459+0000 mon.a (mon.0) 1853 : cluster [DBG] osdmap e229: 8 total, 8 up, 8 in 2024-03-26T03:51:26.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:26 smithi190 bash[23392]: audit 2024-03-26T03:51:25.554224+0000 mon.c (mon.2) 385 : audit [INF] from='client.? 172.21.15.175:0/795941933' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi175-64406-46"}]: dispatch 2024-03-26T03:51:26.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:26 smithi190 bash[23392]: audit 2024-03-26T03:51:25.555217+0000 mon.a (mon.0) 1854 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi175-64406-46"}]: dispatch 2024-03-26T03:51:26.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:26 smithi190 bash[23392]: audit 2024-03-26T03:51:25.555985+0000 mon.c (mon.2) 386 : audit [INF] from='client.? 172.21.15.175:0/1747297285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi175-64398-37"}]: dispatch 2024-03-26T03:51:26.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:26 smithi190 bash[23392]: audit 2024-03-26T03:51:25.559450+0000 mon.a (mon.0) 1855 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi175-64398-37"}]: dispatch 2024-03-26T03:51:26.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:26 smithi190 bash[23392]: cluster 2024-03-26T03:51:26.439796+0000 mon.a (mon.0) 1856 : cluster [WRN] Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:51:26.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:26 smithi175 bash[29284]: audit 2024-03-26T03:51:25.532743+0000 mon.a (mon.0) 1852 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:26.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:26 smithi175 bash[29284]: cluster 2024-03-26T03:51:25.542459+0000 mon.a (mon.0) 1853 : cluster [DBG] osdmap e229: 8 total, 8 up, 8 in 2024-03-26T03:51:26.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:26 smithi175 bash[29284]: audit 2024-03-26T03:51:25.554224+0000 mon.c (mon.2) 385 : audit [INF] from='client.? 172.21.15.175:0/795941933' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi175-64406-46"}]: dispatch 2024-03-26T03:51:26.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:26 smithi175 bash[29284]: audit 2024-03-26T03:51:25.555217+0000 mon.a (mon.0) 1854 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi175-64406-46"}]: dispatch 2024-03-26T03:51:26.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:26 smithi175 bash[29284]: audit 2024-03-26T03:51:25.555985+0000 mon.c (mon.2) 386 : audit [INF] from='client.? 172.21.15.175:0/1747297285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi175-64398-37"}]: dispatch 2024-03-26T03:51:26.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:26 smithi175 bash[29284]: audit 2024-03-26T03:51:25.559450+0000 mon.a (mon.0) 1855 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi175-64398-37"}]: dispatch 2024-03-26T03:51:26.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:26 smithi175 bash[29284]: cluster 2024-03-26T03:51:26.439796+0000 mon.a (mon.0) 1856 : cluster [WRN] Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:51:26.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:26 smithi175 bash[21527]: audit 2024-03-26T03:51:25.532743+0000 mon.a (mon.0) 1852 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:26.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:26 smithi175 bash[21527]: cluster 2024-03-26T03:51:25.542459+0000 mon.a (mon.0) 1853 : cluster [DBG] osdmap e229: 8 total, 8 up, 8 in 2024-03-26T03:51:26.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:26 smithi175 bash[21527]: audit 2024-03-26T03:51:25.554224+0000 mon.c (mon.2) 385 : audit [INF] from='client.? 172.21.15.175:0/795941933' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi175-64406-46"}]: dispatch 2024-03-26T03:51:26.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:26 smithi175 bash[21527]: audit 2024-03-26T03:51:25.555217+0000 mon.a (mon.0) 1854 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi175-64406-46"}]: dispatch 2024-03-26T03:51:26.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:26 smithi175 bash[21527]: audit 2024-03-26T03:51:25.555985+0000 mon.c (mon.2) 386 : audit [INF] from='client.? 172.21.15.175:0/1747297285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi175-64398-37"}]: dispatch 2024-03-26T03:51:26.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:26 smithi175 bash[21527]: audit 2024-03-26T03:51:25.559450+0000 mon.a (mon.0) 1855 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi175-64398-37"}]: dispatch 2024-03-26T03:51:26.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:26 smithi175 bash[21527]: cluster 2024-03-26T03:51:26.439796+0000 mon.a (mon.0) 1856 : cluster [WRN] Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:51:27.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:27 smithi190 bash[23392]: audit 2024-03-26T03:51:26.547013+0000 mon.a (mon.0) 1857 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi175-64406-46"}]': finished 2024-03-26T03:51:27.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:27 smithi190 bash[23392]: audit 2024-03-26T03:51:26.547253+0000 mon.a (mon.0) 1858 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi175-64398-37"}]': finished 2024-03-26T03:51:27.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:27 smithi190 bash[23392]: cluster 2024-03-26T03:51:26.556692+0000 mon.a (mon.0) 1859 : cluster [DBG] osdmap e230: 8 total, 8 up, 8 in 2024-03-26T03:51:27.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:27 smithi190 bash[23392]: audit 2024-03-26T03:51:26.558818+0000 mon.c (mon.2) 387 : audit [INF] from='client.? 172.21.15.175:0/1747297285' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi175-64398-37"}]: dispatch 2024-03-26T03:51:27.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:27 smithi190 bash[23392]: audit 2024-03-26T03:51:26.559431+0000 mon.c (mon.2) 388 : audit [INF] from='client.? 172.21.15.175:0/795941933' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi175-64406-46"}]: dispatch 2024-03-26T03:51:27.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:27 smithi190 bash[23392]: audit 2024-03-26T03:51:26.559982+0000 mon.a (mon.0) 1860 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi175-64398-37"}]: dispatch 2024-03-26T03:51:27.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:27 smithi190 bash[23392]: audit 2024-03-26T03:51:26.562393+0000 mon.a (mon.0) 1861 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi175-64406-46"}]: dispatch 2024-03-26T03:51:27.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:27 smithi190 bash[23392]: cluster 2024-03-26T03:51:26.654664+0000 mgr.y (mgr.24431) 290 : cluster [DBG] pgmap v342: 292 pgs: 32 creating+peering, 260 active+clean; 584 KiB data, 565 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T03:51:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:27 smithi175 bash[21527]: audit 2024-03-26T03:51:26.547013+0000 mon.a (mon.0) 1857 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi175-64406-46"}]': finished 2024-03-26T03:51:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:27 smithi175 bash[21527]: audit 2024-03-26T03:51:26.547253+0000 mon.a (mon.0) 1858 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi175-64398-37"}]': finished 2024-03-26T03:51:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:27 smithi175 bash[21527]: cluster 2024-03-26T03:51:26.556692+0000 mon.a (mon.0) 1859 : cluster [DBG] osdmap e230: 8 total, 8 up, 8 in 2024-03-26T03:51:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:27 smithi175 bash[21527]: audit 2024-03-26T03:51:26.558818+0000 mon.c (mon.2) 387 : audit [INF] from='client.? 172.21.15.175:0/1747297285' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi175-64398-37"}]: dispatch 2024-03-26T03:51:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:27 smithi175 bash[21527]: audit 2024-03-26T03:51:26.559431+0000 mon.c (mon.2) 388 : audit [INF] from='client.? 172.21.15.175:0/795941933' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi175-64406-46"}]: dispatch 2024-03-26T03:51:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:27 smithi175 bash[21527]: audit 2024-03-26T03:51:26.559982+0000 mon.a (mon.0) 1860 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi175-64398-37"}]: dispatch 2024-03-26T03:51:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:27 smithi175 bash[21527]: audit 2024-03-26T03:51:26.562393+0000 mon.a (mon.0) 1861 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi175-64406-46"}]: dispatch 2024-03-26T03:51:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:27 smithi175 bash[21527]: cluster 2024-03-26T03:51:26.654664+0000 mgr.y (mgr.24431) 290 : cluster [DBG] pgmap v342: 292 pgs: 32 creating+peering, 260 active+clean; 584 KiB data, 565 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T03:51:27.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:27 smithi175 bash[29284]: audit 2024-03-26T03:51:26.547013+0000 mon.a (mon.0) 1857 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi175-64406-46"}]': finished 2024-03-26T03:51:27.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:27 smithi175 bash[29284]: audit 2024-03-26T03:51:26.547253+0000 mon.a (mon.0) 1858 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi175-64398-37"}]': finished 2024-03-26T03:51:27.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:27 smithi175 bash[29284]: cluster 2024-03-26T03:51:26.556692+0000 mon.a (mon.0) 1859 : cluster [DBG] osdmap e230: 8 total, 8 up, 8 in 2024-03-26T03:51:27.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:27 smithi175 bash[29284]: audit 2024-03-26T03:51:26.558818+0000 mon.c (mon.2) 387 : audit [INF] from='client.? 172.21.15.175:0/1747297285' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi175-64398-37"}]: dispatch 2024-03-26T03:51:27.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:27 smithi175 bash[29284]: audit 2024-03-26T03:51:26.559431+0000 mon.c (mon.2) 388 : audit [INF] from='client.? 172.21.15.175:0/795941933' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi175-64406-46"}]: dispatch 2024-03-26T03:51:27.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:27 smithi175 bash[29284]: audit 2024-03-26T03:51:26.559982+0000 mon.a (mon.0) 1860 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi175-64398-37"}]: dispatch 2024-03-26T03:51:27.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:27 smithi175 bash[29284]: audit 2024-03-26T03:51:26.562393+0000 mon.a (mon.0) 1861 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi175-64406-46"}]: dispatch 2024-03-26T03:51:27.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:27 smithi175 bash[29284]: cluster 2024-03-26T03:51:26.654664+0000 mgr.y (mgr.24431) 290 : cluster [DBG] pgmap v342: 292 pgs: 32 creating+peering, 260 active+clean; 584 KiB data, 565 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T03:51:28.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:28 smithi190 bash[23392]: audit 2024-03-26T03:51:27.557740+0000 mon.a (mon.0) 1862 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi175-64398-37"}]': finished 2024-03-26T03:51:28.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:28 smithi190 bash[23392]: audit 2024-03-26T03:51:27.557919+0000 mon.a (mon.0) 1863 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi175-64406-46"}]': finished 2024-03-26T03:51:28.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:28 smithi190 bash[23392]: cluster 2024-03-26T03:51:27.570378+0000 mon.a (mon.0) 1864 : cluster [DBG] osdmap e231: 8 total, 8 up, 8 in 2024-03-26T03:51:28.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:28 smithi190 bash[23392]: audit 2024-03-26T03:51:27.588315+0000 mon.a (mon.0) 1865 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:51:28.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:28 smithi190 bash[23392]: audit 2024-03-26T03:51:27.608558+0000 mon.c (mon.2) 389 : audit [INF] from='client.? 172.21.15.175:0/2527120587' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi175-64406-47"}]: dispatch 2024-03-26T03:51:28.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:28 smithi190 bash[23392]: audit 2024-03-26T03:51:27.608961+0000 mon.a (mon.0) 1866 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi175-64406-47"}]: dispatch 2024-03-26T03:51:28.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:28 smithi190 bash[23392]: audit 2024-03-26T03:51:27.609710+0000 mon.c (mon.2) 390 : audit [INF] from='client.? 172.21.15.175:0/2527120587' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi175-64406-47"}]: dispatch 2024-03-26T03:51:28.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:28 smithi190 bash[23392]: audit 2024-03-26T03:51:27.609893+0000 mon.c (mon.2) 391 : audit [INF] from='client.? 172.21.15.175:0/3718266538' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi175-64398-38"}]: dispatch 2024-03-26T03:51:28.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:28 smithi190 bash[23392]: audit 2024-03-26T03:51:27.610047+0000 mon.a (mon.0) 1867 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi175-64406-47"}]: dispatch 2024-03-26T03:51:28.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:28 smithi190 bash[23392]: audit 2024-03-26T03:51:27.610584+0000 mon.a (mon.0) 1868 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi175-64398-38"}]: dispatch 2024-03-26T03:51:28.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:28 smithi190 bash[23392]: audit 2024-03-26T03:51:27.610925+0000 mon.c (mon.2) 392 : audit [INF] from='client.? 172.21.15.175:0/2527120587' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi175-64406-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:28.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:28 smithi190 bash[23392]: audit 2024-03-26T03:51:27.611415+0000 mon.c (mon.2) 393 : audit [INF] from='client.? 172.21.15.175:0/3718266538' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi175-64398-38"}]: dispatch 2024-03-26T03:51:28.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:28 smithi190 bash[23392]: audit 2024-03-26T03:51:27.611577+0000 mon.a (mon.0) 1869 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi175-64406-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:28.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:28 smithi190 bash[23392]: audit 2024-03-26T03:51:27.612029+0000 mon.a (mon.0) 1870 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi175-64398-38"}]: dispatch 2024-03-26T03:51:28.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:28 smithi190 bash[23392]: audit 2024-03-26T03:51:27.612561+0000 mon.c (mon.2) 394 : audit [INF] from='client.? 172.21.15.175:0/3718266538' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi175-64398-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:28.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:28 smithi190 bash[23392]: audit 2024-03-26T03:51:27.612869+0000 mon.a (mon.0) 1871 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi175-64398-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:28.992 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:51:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:51:28.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[21527]: audit 2024-03-26T03:51:27.557740+0000 mon.a (mon.0) 1862 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi175-64398-37"}]': finished 2024-03-26T03:51:28.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[21527]: audit 2024-03-26T03:51:27.557919+0000 mon.a (mon.0) 1863 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi175-64406-46"}]': finished 2024-03-26T03:51:28.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[21527]: cluster 2024-03-26T03:51:27.570378+0000 mon.a (mon.0) 1864 : cluster [DBG] osdmap e231: 8 total, 8 up, 8 in 2024-03-26T03:51:28.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[21527]: audit 2024-03-26T03:51:27.588315+0000 mon.a (mon.0) 1865 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:51:28.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[21527]: audit 2024-03-26T03:51:27.608558+0000 mon.c (mon.2) 389 : audit [INF] from='client.? 172.21.15.175:0/2527120587' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi175-64406-47"}]: dispatch 2024-03-26T03:51:28.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[21527]: audit 2024-03-26T03:51:27.608961+0000 mon.a (mon.0) 1866 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi175-64406-47"}]: dispatch 2024-03-26T03:51:28.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[21527]: audit 2024-03-26T03:51:27.609710+0000 mon.c (mon.2) 390 : audit [INF] from='client.? 172.21.15.175:0/2527120587' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi175-64406-47"}]: dispatch 2024-03-26T03:51:28.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[21527]: audit 2024-03-26T03:51:27.609893+0000 mon.c (mon.2) 391 : audit [INF] from='client.? 172.21.15.175:0/3718266538' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi175-64398-38"}]: dispatch 2024-03-26T03:51:28.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[21527]: audit 2024-03-26T03:51:27.610047+0000 mon.a (mon.0) 1867 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi175-64406-47"}]: dispatch 2024-03-26T03:51:28.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[21527]: audit 2024-03-26T03:51:27.610584+0000 mon.a (mon.0) 1868 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi175-64398-38"}]: dispatch 2024-03-26T03:51:28.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[21527]: audit 2024-03-26T03:51:27.610925+0000 mon.c (mon.2) 392 : audit [INF] from='client.? 172.21.15.175:0/2527120587' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi175-64406-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:28.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[21527]: audit 2024-03-26T03:51:27.611415+0000 mon.c (mon.2) 393 : audit [INF] from='client.? 172.21.15.175:0/3718266538' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi175-64398-38"}]: dispatch 2024-03-26T03:51:28.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[21527]: audit 2024-03-26T03:51:27.611577+0000 mon.a (mon.0) 1869 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi175-64406-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:28.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[21527]: audit 2024-03-26T03:51:27.612029+0000 mon.a (mon.0) 1870 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi175-64398-38"}]: dispatch 2024-03-26T03:51:28.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[21527]: audit 2024-03-26T03:51:27.612561+0000 mon.c (mon.2) 394 : audit [INF] from='client.? 172.21.15.175:0/3718266538' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi175-64398-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:28.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[21527]: audit 2024-03-26T03:51:27.612869+0000 mon.a (mon.0) 1871 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi175-64398-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:28.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[29284]: audit 2024-03-26T03:51:27.557740+0000 mon.a (mon.0) 1862 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi175-64398-37"}]': finished 2024-03-26T03:51:28.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[29284]: audit 2024-03-26T03:51:27.557919+0000 mon.a (mon.0) 1863 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi175-64406-46"}]': finished 2024-03-26T03:51:28.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[29284]: cluster 2024-03-26T03:51:27.570378+0000 mon.a (mon.0) 1864 : cluster [DBG] osdmap e231: 8 total, 8 up, 8 in 2024-03-26T03:51:28.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[29284]: audit 2024-03-26T03:51:27.588315+0000 mon.a (mon.0) 1865 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:51:28.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[29284]: audit 2024-03-26T03:51:27.608558+0000 mon.c (mon.2) 389 : audit [INF] from='client.? 172.21.15.175:0/2527120587' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi175-64406-47"}]: dispatch 2024-03-26T03:51:28.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[29284]: audit 2024-03-26T03:51:27.608961+0000 mon.a (mon.0) 1866 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi175-64406-47"}]: dispatch 2024-03-26T03:51:28.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[29284]: audit 2024-03-26T03:51:27.609710+0000 mon.c (mon.2) 390 : audit [INF] from='client.? 172.21.15.175:0/2527120587' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi175-64406-47"}]: dispatch 2024-03-26T03:51:28.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[29284]: audit 2024-03-26T03:51:27.609893+0000 mon.c (mon.2) 391 : audit [INF] from='client.? 172.21.15.175:0/3718266538' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi175-64398-38"}]: dispatch 2024-03-26T03:51:28.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[29284]: audit 2024-03-26T03:51:27.610047+0000 mon.a (mon.0) 1867 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi175-64406-47"}]: dispatch 2024-03-26T03:51:28.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[29284]: audit 2024-03-26T03:51:27.610584+0000 mon.a (mon.0) 1868 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi175-64398-38"}]: dispatch 2024-03-26T03:51:28.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[29284]: audit 2024-03-26T03:51:27.610925+0000 mon.c (mon.2) 392 : audit [INF] from='client.? 172.21.15.175:0/2527120587' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi175-64406-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:28.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[29284]: audit 2024-03-26T03:51:27.611415+0000 mon.c (mon.2) 393 : audit [INF] from='client.? 172.21.15.175:0/3718266538' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi175-64398-38"}]: dispatch 2024-03-26T03:51:28.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[29284]: audit 2024-03-26T03:51:27.611577+0000 mon.a (mon.0) 1869 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi175-64406-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:28.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[29284]: audit 2024-03-26T03:51:27.612029+0000 mon.a (mon.0) 1870 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi175-64398-38"}]: dispatch 2024-03-26T03:51:28.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[29284]: audit 2024-03-26T03:51:27.612561+0000 mon.c (mon.2) 394 : audit [INF] from='client.? 172.21.15.175:0/3718266538' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi175-64398-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:28.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:28 smithi175 bash[29284]: audit 2024-03-26T03:51:27.612869+0000 mon.a (mon.0) 1871 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi175-64398-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:29.990 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:51:29 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:51:29.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:29 smithi190 bash[23392]: audit 2024-03-26T03:51:28.560521+0000 mon.a (mon.0) 1872 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-29", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:51:29.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:29 smithi190 bash[23392]: audit 2024-03-26T03:51:28.560680+0000 mon.a (mon.0) 1873 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi175-64406-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:29.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:29 smithi190 bash[23392]: audit 2024-03-26T03:51:28.560816+0000 mon.a (mon.0) 1874 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi175-64398-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:29.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:29 smithi190 bash[23392]: cluster 2024-03-26T03:51:28.568178+0000 mon.a (mon.0) 1875 : cluster [DBG] osdmap e232: 8 total, 8 up, 8 in 2024-03-26T03:51:29.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:29 smithi190 bash[23392]: audit 2024-03-26T03:51:28.570268+0000 mon.c (mon.2) 395 : audit [INF] from='client.? 172.21.15.175:0/2527120587' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi175-64406-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi175-64406-47"}]: dispatch 2024-03-26T03:51:29.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:29 smithi190 bash[23392]: audit 2024-03-26T03:51:28.570874+0000 mon.a (mon.0) 1876 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-29"}]: dispatch 2024-03-26T03:51:29.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:29 smithi190 bash[23392]: audit 2024-03-26T03:51:28.571308+0000 mon.a (mon.0) 1877 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi175-64406-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi175-64406-47"}]: dispatch 2024-03-26T03:51:29.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:29 smithi190 bash[23392]: audit 2024-03-26T03:51:28.573024+0000 mon.c (mon.2) 396 : audit [INF] from='client.? 172.21.15.175:0/3718266538' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi175-64398-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi175-64398-38"}]: dispatch 2024-03-26T03:51:29.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:29 smithi190 bash[23392]: audit 2024-03-26T03:51:28.585601+0000 mon.a (mon.0) 1878 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi175-64398-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi175-64398-38"}]: dispatch 2024-03-26T03:51:29.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:29 smithi190 bash[23392]: cluster 2024-03-26T03:51:28.655674+0000 mgr.y (mgr.24431) 291 : cluster [DBG] pgmap v345: 292 pgs: 32 creating+peering, 260 active+clean; 584 KiB data, 565 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:51:29.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:29 smithi190 bash[23392]: audit 2024-03-26T03:51:29.564168+0000 mon.a (mon.0) 1879 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-29"}]': finished 2024-03-26T03:51:29.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:29 smithi175 bash[21527]: audit 2024-03-26T03:51:28.560521+0000 mon.a (mon.0) 1872 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-29", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:51:29.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:29 smithi175 bash[21527]: audit 2024-03-26T03:51:28.560680+0000 mon.a (mon.0) 1873 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi175-64406-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:29.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:29 smithi175 bash[21527]: audit 2024-03-26T03:51:28.560816+0000 mon.a (mon.0) 1874 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi175-64398-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:29.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:29 smithi175 bash[21527]: cluster 2024-03-26T03:51:28.568178+0000 mon.a (mon.0) 1875 : cluster [DBG] osdmap e232: 8 total, 8 up, 8 in 2024-03-26T03:51:29.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:29 smithi175 bash[21527]: audit 2024-03-26T03:51:28.570268+0000 mon.c (mon.2) 395 : audit [INF] from='client.? 172.21.15.175:0/2527120587' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi175-64406-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi175-64406-47"}]: dispatch 2024-03-26T03:51:29.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:29 smithi175 bash[21527]: audit 2024-03-26T03:51:28.570874+0000 mon.a (mon.0) 1876 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-29"}]: dispatch 2024-03-26T03:51:29.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:29 smithi175 bash[21527]: audit 2024-03-26T03:51:28.571308+0000 mon.a (mon.0) 1877 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi175-64406-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi175-64406-47"}]: dispatch 2024-03-26T03:51:29.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:29 smithi175 bash[21527]: audit 2024-03-26T03:51:28.573024+0000 mon.c (mon.2) 396 : audit [INF] from='client.? 172.21.15.175:0/3718266538' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi175-64398-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi175-64398-38"}]: dispatch 2024-03-26T03:51:29.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:29 smithi175 bash[21527]: audit 2024-03-26T03:51:28.585601+0000 mon.a (mon.0) 1878 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi175-64398-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi175-64398-38"}]: dispatch 2024-03-26T03:51:29.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:29 smithi175 bash[21527]: cluster 2024-03-26T03:51:28.655674+0000 mgr.y (mgr.24431) 291 : cluster [DBG] pgmap v345: 292 pgs: 32 creating+peering, 260 active+clean; 584 KiB data, 565 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:51:29.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:29 smithi175 bash[21527]: audit 2024-03-26T03:51:29.564168+0000 mon.a (mon.0) 1879 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-29"}]': finished 2024-03-26T03:51:29.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:29 smithi175 bash[29284]: audit 2024-03-26T03:51:28.560521+0000 mon.a (mon.0) 1872 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-29", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:51:29.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:29 smithi175 bash[29284]: audit 2024-03-26T03:51:28.560680+0000 mon.a (mon.0) 1873 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi175-64406-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:29.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:29 smithi175 bash[29284]: audit 2024-03-26T03:51:28.560816+0000 mon.a (mon.0) 1874 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi175-64398-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:29.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:29 smithi175 bash[29284]: cluster 2024-03-26T03:51:28.568178+0000 mon.a (mon.0) 1875 : cluster [DBG] osdmap e232: 8 total, 8 up, 8 in 2024-03-26T03:51:29.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:29 smithi175 bash[29284]: audit 2024-03-26T03:51:28.570268+0000 mon.c (mon.2) 395 : audit [INF] from='client.? 172.21.15.175:0/2527120587' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi175-64406-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi175-64406-47"}]: dispatch 2024-03-26T03:51:29.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:29 smithi175 bash[29284]: audit 2024-03-26T03:51:28.570874+0000 mon.a (mon.0) 1876 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-29"}]: dispatch 2024-03-26T03:51:29.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:29 smithi175 bash[29284]: audit 2024-03-26T03:51:28.571308+0000 mon.a (mon.0) 1877 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi175-64406-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi175-64406-47"}]: dispatch 2024-03-26T03:51:29.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:29 smithi175 bash[29284]: audit 2024-03-26T03:51:28.573024+0000 mon.c (mon.2) 396 : audit [INF] from='client.? 172.21.15.175:0/3718266538' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi175-64398-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi175-64398-38"}]: dispatch 2024-03-26T03:51:29.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:29 smithi175 bash[29284]: audit 2024-03-26T03:51:28.585601+0000 mon.a (mon.0) 1878 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi175-64398-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi175-64398-38"}]: dispatch 2024-03-26T03:51:29.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:29 smithi175 bash[29284]: cluster 2024-03-26T03:51:28.655674+0000 mgr.y (mgr.24431) 291 : cluster [DBG] pgmap v345: 292 pgs: 32 creating+peering, 260 active+clean; 584 KiB data, 565 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:51:29.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:29 smithi175 bash[29284]: audit 2024-03-26T03:51:29.564168+0000 mon.a (mon.0) 1879 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-29"}]': finished 2024-03-26T03:51:30.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:30 smithi190 bash[23392]: cluster 2024-03-26T03:51:29.594284+0000 mon.a (mon.0) 1880 : cluster [DBG] osdmap e233: 8 total, 8 up, 8 in 2024-03-26T03:51:30.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:30 smithi190 bash[23392]: audit 2024-03-26T03:51:29.596230+0000 mon.a (mon.0) 1881 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-29", "mode": "writeback"}]: dispatch 2024-03-26T03:51:30.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:30 smithi190 bash[23392]: audit 2024-03-26T03:51:29.840355+0000 mgr.y (mgr.24431) 292 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:51:30.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:30 smithi190 bash[23392]: cluster 2024-03-26T03:51:30.564889+0000 mon.a (mon.0) 1882 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:51:30.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:30 smithi190 bash[23392]: audit 2024-03-26T03:51:30.567591+0000 mon.a (mon.0) 1883 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi175-64406-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi175-64406-47"}]': finished 2024-03-26T03:51:30.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:30 smithi190 bash[23392]: audit 2024-03-26T03:51:30.567889+0000 mon.a (mon.0) 1884 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi175-64398-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi175-64398-38"}]': finished 2024-03-26T03:51:30.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:30 smithi190 bash[23392]: audit 2024-03-26T03:51:30.568109+0000 mon.a (mon.0) 1885 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-29", "mode": "writeback"}]': finished 2024-03-26T03:51:30.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:30 smithi190 bash[23392]: cluster 2024-03-26T03:51:30.575191+0000 mon.a (mon.0) 1886 : cluster [DBG] osdmap e234: 8 total, 8 up, 8 in 2024-03-26T03:51:30.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:30 smithi175 bash[29284]: cluster 2024-03-26T03:51:29.594284+0000 mon.a (mon.0) 1880 : cluster [DBG] osdmap e233: 8 total, 8 up, 8 in 2024-03-26T03:51:30.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:30 smithi175 bash[29284]: audit 2024-03-26T03:51:29.596230+0000 mon.a (mon.0) 1881 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-29", "mode": "writeback"}]: dispatch 2024-03-26T03:51:30.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:30 smithi175 bash[29284]: audit 2024-03-26T03:51:29.840355+0000 mgr.y (mgr.24431) 292 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:51:30.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:30 smithi175 bash[29284]: cluster 2024-03-26T03:51:30.564889+0000 mon.a (mon.0) 1882 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:51:30.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:30 smithi175 bash[29284]: audit 2024-03-26T03:51:30.567591+0000 mon.a (mon.0) 1883 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi175-64406-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi175-64406-47"}]': finished 2024-03-26T03:51:30.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:30 smithi175 bash[29284]: audit 2024-03-26T03:51:30.567889+0000 mon.a (mon.0) 1884 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi175-64398-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi175-64398-38"}]': finished 2024-03-26T03:51:30.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:30 smithi175 bash[29284]: audit 2024-03-26T03:51:30.568109+0000 mon.a (mon.0) 1885 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-29", "mode": "writeback"}]': finished 2024-03-26T03:51:30.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:30 smithi175 bash[29284]: cluster 2024-03-26T03:51:30.575191+0000 mon.a (mon.0) 1886 : cluster [DBG] osdmap e234: 8 total, 8 up, 8 in 2024-03-26T03:51:30.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:30 smithi175 bash[21527]: cluster 2024-03-26T03:51:29.594284+0000 mon.a (mon.0) 1880 : cluster [DBG] osdmap e233: 8 total, 8 up, 8 in 2024-03-26T03:51:30.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:30 smithi175 bash[21527]: audit 2024-03-26T03:51:29.596230+0000 mon.a (mon.0) 1881 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-29", "mode": "writeback"}]: dispatch 2024-03-26T03:51:30.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:30 smithi175 bash[21527]: audit 2024-03-26T03:51:29.840355+0000 mgr.y (mgr.24431) 292 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:51:30.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:30 smithi175 bash[21527]: cluster 2024-03-26T03:51:30.564889+0000 mon.a (mon.0) 1882 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:51:30.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:30 smithi175 bash[21527]: audit 2024-03-26T03:51:30.567591+0000 mon.a (mon.0) 1883 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi175-64406-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi175-64406-47"}]': finished 2024-03-26T03:51:30.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:30 smithi175 bash[21527]: audit 2024-03-26T03:51:30.567889+0000 mon.a (mon.0) 1884 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi175-64398-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi175-64398-38"}]': finished 2024-03-26T03:51:30.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:30 smithi175 bash[21527]: audit 2024-03-26T03:51:30.568109+0000 mon.a (mon.0) 1885 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-29", "mode": "writeback"}]': finished 2024-03-26T03:51:30.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:30 smithi175 bash[21527]: cluster 2024-03-26T03:51:30.575191+0000 mon.a (mon.0) 1886 : cluster [DBG] osdmap e234: 8 total, 8 up, 8 in 2024-03-26T03:51:31.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:31 smithi190 bash[23392]: cluster 2024-03-26T03:51:30.656622+0000 mgr.y (mgr.24431) 293 : cluster [DBG] pgmap v348: 308 pgs: 16 unknown, 292 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-03-26T03:51:31.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:31 smithi190 bash[23392]: cluster 2024-03-26T03:51:31.441229+0000 mon.a (mon.0) 1887 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:51:31.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:31 smithi190 bash[23392]: cluster 2024-03-26T03:51:31.452852+0000 mon.a (mon.0) 1888 : cluster [DBG] osdmap e235: 8 total, 8 up, 8 in 2024-03-26T03:51:31.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:31 smithi175 bash[29284]: cluster 2024-03-26T03:51:30.656622+0000 mgr.y (mgr.24431) 293 : cluster [DBG] pgmap v348: 308 pgs: 16 unknown, 292 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-03-26T03:51:31.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:31 smithi175 bash[29284]: cluster 2024-03-26T03:51:31.441229+0000 mon.a (mon.0) 1887 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:51:31.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:31 smithi175 bash[29284]: cluster 2024-03-26T03:51:31.452852+0000 mon.a (mon.0) 1888 : cluster [DBG] osdmap e235: 8 total, 8 up, 8 in 2024-03-26T03:51:31.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:31 smithi175 bash[21527]: cluster 2024-03-26T03:51:30.656622+0000 mgr.y (mgr.24431) 293 : cluster [DBG] pgmap v348: 308 pgs: 16 unknown, 292 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-03-26T03:51:31.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:31 smithi175 bash[21527]: cluster 2024-03-26T03:51:31.441229+0000 mon.a (mon.0) 1887 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:51:31.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:31 smithi175 bash[21527]: cluster 2024-03-26T03:51:31.452852+0000 mon.a (mon.0) 1888 : cluster [DBG] osdmap e235: 8 total, 8 up, 8 in 2024-03-26T03:51:33.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:33 smithi190 bash[23392]: cluster 2024-03-26T03:51:32.455294+0000 mon.a (mon.0) 1889 : cluster [DBG] osdmap e236: 8 total, 8 up, 8 in 2024-03-26T03:51:33.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:33 smithi190 bash[23392]: audit 2024-03-26T03:51:32.459541+0000 mon.c (mon.2) 397 : audit [INF] from='client.? 172.21.15.175:0/2527120587' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi175-64406-47"}]: dispatch 2024-03-26T03:51:33.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:33 smithi190 bash[23392]: audit 2024-03-26T03:51:32.461387+0000 mon.c (mon.2) 398 : audit [INF] from='client.? 172.21.15.175:0/3718266538' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi175-64398-38"}]: dispatch 2024-03-26T03:51:33.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:33 smithi190 bash[23392]: audit 2024-03-26T03:51:32.461407+0000 mon.a (mon.0) 1890 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi175-64406-47"}]: dispatch 2024-03-26T03:51:33.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:33 smithi190 bash[23392]: audit 2024-03-26T03:51:32.462292+0000 mon.a (mon.0) 1891 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi175-64398-38"}]: dispatch 2024-03-26T03:51:33.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:33 smithi175 bash[29284]: cluster 2024-03-26T03:51:32.455294+0000 mon.a (mon.0) 1889 : cluster [DBG] osdmap e236: 8 total, 8 up, 8 in 2024-03-26T03:51:33.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:33 smithi175 bash[29284]: audit 2024-03-26T03:51:32.459541+0000 mon.c (mon.2) 397 : audit [INF] from='client.? 172.21.15.175:0/2527120587' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi175-64406-47"}]: dispatch 2024-03-26T03:51:33.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:33 smithi175 bash[29284]: audit 2024-03-26T03:51:32.461387+0000 mon.c (mon.2) 398 : audit [INF] from='client.? 172.21.15.175:0/3718266538' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi175-64398-38"}]: dispatch 2024-03-26T03:51:33.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:33 smithi175 bash[29284]: audit 2024-03-26T03:51:32.461407+0000 mon.a (mon.0) 1890 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi175-64406-47"}]: dispatch 2024-03-26T03:51:33.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:33 smithi175 bash[29284]: audit 2024-03-26T03:51:32.462292+0000 mon.a (mon.0) 1891 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi175-64398-38"}]: dispatch 2024-03-26T03:51:33.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:33 smithi175 bash[21527]: cluster 2024-03-26T03:51:32.455294+0000 mon.a (mon.0) 1889 : cluster [DBG] osdmap e236: 8 total, 8 up, 8 in 2024-03-26T03:51:33.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:33 smithi175 bash[21527]: audit 2024-03-26T03:51:32.459541+0000 mon.c (mon.2) 397 : audit [INF] from='client.? 172.21.15.175:0/2527120587' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi175-64406-47"}]: dispatch 2024-03-26T03:51:33.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:33 smithi175 bash[21527]: audit 2024-03-26T03:51:32.461387+0000 mon.c (mon.2) 398 : audit [INF] from='client.? 172.21.15.175:0/3718266538' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi175-64398-38"}]: dispatch 2024-03-26T03:51:33.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:33 smithi175 bash[21527]: audit 2024-03-26T03:51:32.461407+0000 mon.a (mon.0) 1890 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi175-64406-47"}]: dispatch 2024-03-26T03:51:33.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:33 smithi175 bash[21527]: audit 2024-03-26T03:51:32.462292+0000 mon.a (mon.0) 1891 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi175-64398-38"}]: dispatch 2024-03-26T03:51:34.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:34 smithi190 bash[23392]: cluster 2024-03-26T03:51:32.657501+0000 mgr.y (mgr.24431) 294 : cluster [DBG] pgmap v351: 292 pgs: 292 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-03-26T03:51:34.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:34 smithi190 bash[23392]: audit 2024-03-26T03:51:33.459530+0000 mon.a (mon.0) 1892 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi175-64406-47"}]': finished 2024-03-26T03:51:34.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:34 smithi190 bash[23392]: audit 2024-03-26T03:51:33.459749+0000 mon.a (mon.0) 1893 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi175-64398-38"}]': finished 2024-03-26T03:51:34.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:34 smithi190 bash[23392]: audit 2024-03-26T03:51:33.472098+0000 mon.c (mon.2) 399 : audit [INF] from='client.? 172.21.15.175:0/2527120587' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi175-64406-47"}]: dispatch 2024-03-26T03:51:34.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:34 smithi190 bash[23392]: audit 2024-03-26T03:51:33.472362+0000 mon.c (mon.2) 400 : audit [INF] from='client.? 172.21.15.175:0/3718266538' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi175-64398-38"}]: dispatch 2024-03-26T03:51:34.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:34 smithi190 bash[23392]: cluster 2024-03-26T03:51:33.472825+0000 mon.a (mon.0) 1894 : cluster [DBG] osdmap e237: 8 total, 8 up, 8 in 2024-03-26T03:51:34.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:34 smithi190 bash[23392]: audit 2024-03-26T03:51:33.476601+0000 mon.a (mon.0) 1895 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi175-64406-47"}]: dispatch 2024-03-26T03:51:34.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:34 smithi190 bash[23392]: audit 2024-03-26T03:51:33.477514+0000 mon.a (mon.0) 1896 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi175-64398-38"}]: dispatch 2024-03-26T03:51:34.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:34 smithi190 bash[23392]: audit 2024-03-26T03:51:33.595794+0000 mon.a (mon.0) 1897 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:51:34.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:34 smithi190 bash[23392]: audit 2024-03-26T03:51:34.416124+0000 mon.a (mon.0) 1898 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:51:34.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:34 smithi190 bash[23392]: audit 2024-03-26T03:51:34.419029+0000 mon.c (mon.2) 401 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:51:34.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:34 smithi175 bash[21527]: cluster 2024-03-26T03:51:32.657501+0000 mgr.y (mgr.24431) 294 : cluster [DBG] pgmap v351: 292 pgs: 292 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-03-26T03:51:34.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:34 smithi175 bash[21527]: audit 2024-03-26T03:51:33.459530+0000 mon.a (mon.0) 1892 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi175-64406-47"}]': finished 2024-03-26T03:51:34.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:34 smithi175 bash[21527]: audit 2024-03-26T03:51:33.459749+0000 mon.a (mon.0) 1893 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi175-64398-38"}]': finished 2024-03-26T03:51:34.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:34 smithi175 bash[21527]: audit 2024-03-26T03:51:33.472098+0000 mon.c (mon.2) 399 : audit [INF] from='client.? 172.21.15.175:0/2527120587' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi175-64406-47"}]: dispatch 2024-03-26T03:51:34.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:34 smithi175 bash[21527]: audit 2024-03-26T03:51:33.472362+0000 mon.c (mon.2) 400 : audit [INF] from='client.? 172.21.15.175:0/3718266538' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi175-64398-38"}]: dispatch 2024-03-26T03:51:34.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:34 smithi175 bash[21527]: cluster 2024-03-26T03:51:33.472825+0000 mon.a (mon.0) 1894 : cluster [DBG] osdmap e237: 8 total, 8 up, 8 in 2024-03-26T03:51:34.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:34 smithi175 bash[21527]: audit 2024-03-26T03:51:33.476601+0000 mon.a (mon.0) 1895 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi175-64406-47"}]: dispatch 2024-03-26T03:51:34.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:34 smithi175 bash[21527]: audit 2024-03-26T03:51:33.477514+0000 mon.a (mon.0) 1896 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi175-64398-38"}]: dispatch 2024-03-26T03:51:34.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:34 smithi175 bash[21527]: audit 2024-03-26T03:51:33.595794+0000 mon.a (mon.0) 1897 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:51:34.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:34 smithi175 bash[21527]: audit 2024-03-26T03:51:34.416124+0000 mon.a (mon.0) 1898 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:51:34.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:34 smithi175 bash[21527]: audit 2024-03-26T03:51:34.419029+0000 mon.c (mon.2) 401 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:51:34.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:34 smithi175 bash[29284]: cluster 2024-03-26T03:51:32.657501+0000 mgr.y (mgr.24431) 294 : cluster [DBG] pgmap v351: 292 pgs: 292 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-03-26T03:51:34.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:34 smithi175 bash[29284]: audit 2024-03-26T03:51:33.459530+0000 mon.a (mon.0) 1892 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi175-64406-47"}]': finished 2024-03-26T03:51:34.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:34 smithi175 bash[29284]: audit 2024-03-26T03:51:33.459749+0000 mon.a (mon.0) 1893 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi175-64398-38"}]': finished 2024-03-26T03:51:34.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:34 smithi175 bash[29284]: audit 2024-03-26T03:51:33.472098+0000 mon.c (mon.2) 399 : audit [INF] from='client.? 172.21.15.175:0/2527120587' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi175-64406-47"}]: dispatch 2024-03-26T03:51:34.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:34 smithi175 bash[29284]: audit 2024-03-26T03:51:33.472362+0000 mon.c (mon.2) 400 : audit [INF] from='client.? 172.21.15.175:0/3718266538' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi175-64398-38"}]: dispatch 2024-03-26T03:51:34.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:34 smithi175 bash[29284]: cluster 2024-03-26T03:51:33.472825+0000 mon.a (mon.0) 1894 : cluster [DBG] osdmap e237: 8 total, 8 up, 8 in 2024-03-26T03:51:34.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:34 smithi175 bash[29284]: audit 2024-03-26T03:51:33.476601+0000 mon.a (mon.0) 1895 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi175-64406-47"}]: dispatch 2024-03-26T03:51:34.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:34 smithi175 bash[29284]: audit 2024-03-26T03:51:33.477514+0000 mon.a (mon.0) 1896 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi175-64398-38"}]: dispatch 2024-03-26T03:51:34.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:34 smithi175 bash[29284]: audit 2024-03-26T03:51:33.595794+0000 mon.a (mon.0) 1897 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:51:34.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:34 smithi175 bash[29284]: audit 2024-03-26T03:51:34.416124+0000 mon.a (mon.0) 1898 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:51:34.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:34 smithi175 bash[29284]: audit 2024-03-26T03:51:34.419029+0000 mon.c (mon.2) 401 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:51:35.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:35 smithi190 bash[23392]: audit 2024-03-26T03:51:34.463079+0000 mon.a (mon.0) 1899 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi175-64406-47"}]': finished 2024-03-26T03:51:35.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:35 smithi190 bash[23392]: audit 2024-03-26T03:51:34.463255+0000 mon.a (mon.0) 1900 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi175-64398-38"}]': finished 2024-03-26T03:51:35.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:35 smithi190 bash[23392]: audit 2024-03-26T03:51:34.463441+0000 mon.a (mon.0) 1901 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:51:35.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:35 smithi190 bash[23392]: cluster 2024-03-26T03:51:34.471130+0000 mon.a (mon.0) 1902 : cluster [DBG] osdmap e238: 8 total, 8 up, 8 in 2024-03-26T03:51:35.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:35 smithi190 bash[23392]: audit 2024-03-26T03:51:34.484082+0000 mon.a (mon.0) 1903 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-29"}]: dispatch 2024-03-26T03:51:35.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:35 smithi190 bash[23392]: audit 2024-03-26T03:51:34.505200+0000 mon.c (mon.2) 402 : audit [INF] from='client.? 172.21.15.175:0/3064582945' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi175-64398-39"}]: dispatch 2024-03-26T03:51:35.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:35 smithi190 bash[23392]: audit 2024-03-26T03:51:34.505709+0000 mon.a (mon.0) 1904 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi175-64398-39"}]: dispatch 2024-03-26T03:51:35.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:35 smithi190 bash[23392]: audit 2024-03-26T03:51:34.506812+0000 mon.c (mon.2) 403 : audit [INF] from='client.? 172.21.15.175:0/3064582945' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi175-64398-39"}]: dispatch 2024-03-26T03:51:35.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:35 smithi190 bash[23392]: audit 2024-03-26T03:51:34.507125+0000 mon.c (mon.2) 404 : audit [INF] from='client.? 172.21.15.175:0/425412779' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi175-64406-48"}]: dispatch 2024-03-26T03:51:35.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:35 smithi190 bash[23392]: audit 2024-03-26T03:51:34.507169+0000 mon.a (mon.0) 1905 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi175-64398-39"}]: dispatch 2024-03-26T03:51:35.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:35 smithi190 bash[23392]: audit 2024-03-26T03:51:34.507635+0000 mon.a (mon.0) 1906 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi175-64406-48"}]: dispatch 2024-03-26T03:51:35.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:35 smithi190 bash[23392]: audit 2024-03-26T03:51:34.507878+0000 mon.c (mon.2) 405 : audit [INF] from='client.? 172.21.15.175:0/3064582945' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi175-64398-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:35.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:35 smithi190 bash[23392]: audit 2024-03-26T03:51:34.508360+0000 mon.a (mon.0) 1907 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi175-64398-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:35.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:35 smithi190 bash[23392]: audit 2024-03-26T03:51:34.508712+0000 mon.c (mon.2) 406 : audit [INF] from='client.? 172.21.15.175:0/425412779' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi175-64406-48"}]: dispatch 2024-03-26T03:51:35.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:35 smithi190 bash[23392]: audit 2024-03-26T03:51:34.508977+0000 mon.a (mon.0) 1908 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi175-64406-48"}]: dispatch 2024-03-26T03:51:35.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:35 smithi190 bash[23392]: audit 2024-03-26T03:51:34.509458+0000 mon.c (mon.2) 407 : audit [INF] from='client.? 172.21.15.175:0/425412779' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi175-64406-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:35.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:35 smithi190 bash[23392]: audit 2024-03-26T03:51:34.509727+0000 mon.a (mon.0) 1909 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi175-64406-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:35.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:35 smithi190 bash[23392]: cluster 2024-03-26T03:51:35.463486+0000 mon.a (mon.0) 1910 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:51:35.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:35 smithi190 bash[23392]: audit 2024-03-26T03:51:35.466043+0000 mon.a (mon.0) 1911 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-29"}]': finished 2024-03-26T03:51:35.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:35 smithi190 bash[23392]: audit 2024-03-26T03:51:35.466243+0000 mon.a (mon.0) 1912 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi175-64398-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:35.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:35 smithi190 bash[23392]: audit 2024-03-26T03:51:35.466452+0000 mon.a (mon.0) 1913 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi175-64406-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:35.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:35 smithi190 bash[23392]: cluster 2024-03-26T03:51:35.474634+0000 mon.a (mon.0) 1914 : cluster [DBG] osdmap e239: 8 total, 8 up, 8 in 2024-03-26T03:51:35.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:35 smithi190 bash[23392]: audit 2024-03-26T03:51:35.482217+0000 mon.c (mon.2) 408 : audit [INF] from='client.? 172.21.15.175:0/3064582945' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi175-64398-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi175-64398-39"}]: dispatch 2024-03-26T03:51:35.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:35 smithi190 bash[23392]: audit 2024-03-26T03:51:35.482863+0000 mon.c (mon.2) 409 : audit [INF] from='client.? 172.21.15.175:0/425412779' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi175-64406-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi175-64406-48"}]: dispatch 2024-03-26T03:51:35.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[21527]: audit 2024-03-26T03:51:34.463079+0000 mon.a (mon.0) 1899 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi175-64406-47"}]': finished 2024-03-26T03:51:35.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[21527]: audit 2024-03-26T03:51:34.463255+0000 mon.a (mon.0) 1900 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi175-64398-38"}]': finished 2024-03-26T03:51:35.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[21527]: audit 2024-03-26T03:51:34.463441+0000 mon.a (mon.0) 1901 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:51:35.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[21527]: cluster 2024-03-26T03:51:34.471130+0000 mon.a (mon.0) 1902 : cluster [DBG] osdmap e238: 8 total, 8 up, 8 in 2024-03-26T03:51:35.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[21527]: audit 2024-03-26T03:51:34.484082+0000 mon.a (mon.0) 1903 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-29"}]: dispatch 2024-03-26T03:51:35.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[21527]: audit 2024-03-26T03:51:34.505200+0000 mon.c (mon.2) 402 : audit [INF] from='client.? 172.21.15.175:0/3064582945' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi175-64398-39"}]: dispatch 2024-03-26T03:51:35.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[21527]: audit 2024-03-26T03:51:34.505709+0000 mon.a (mon.0) 1904 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi175-64398-39"}]: dispatch 2024-03-26T03:51:35.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[21527]: audit 2024-03-26T03:51:34.506812+0000 mon.c (mon.2) 403 : audit [INF] from='client.? 172.21.15.175:0/3064582945' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi175-64398-39"}]: dispatch 2024-03-26T03:51:35.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[21527]: audit 2024-03-26T03:51:34.507125+0000 mon.c (mon.2) 404 : audit [INF] from='client.? 172.21.15.175:0/425412779' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi175-64406-48"}]: dispatch 2024-03-26T03:51:35.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[21527]: audit 2024-03-26T03:51:34.507169+0000 mon.a (mon.0) 1905 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi175-64398-39"}]: dispatch 2024-03-26T03:51:35.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[21527]: audit 2024-03-26T03:51:34.507635+0000 mon.a (mon.0) 1906 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi175-64406-48"}]: dispatch 2024-03-26T03:51:35.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[21527]: audit 2024-03-26T03:51:34.507878+0000 mon.c (mon.2) 405 : audit [INF] from='client.? 172.21.15.175:0/3064582945' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi175-64398-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:35.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[21527]: audit 2024-03-26T03:51:34.508360+0000 mon.a (mon.0) 1907 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi175-64398-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:35.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[21527]: audit 2024-03-26T03:51:34.508712+0000 mon.c (mon.2) 406 : audit [INF] from='client.? 172.21.15.175:0/425412779' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi175-64406-48"}]: dispatch 2024-03-26T03:51:35.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[21527]: audit 2024-03-26T03:51:34.508977+0000 mon.a (mon.0) 1908 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi175-64406-48"}]: dispatch 2024-03-26T03:51:35.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[21527]: audit 2024-03-26T03:51:34.509458+0000 mon.c (mon.2) 407 : audit [INF] from='client.? 172.21.15.175:0/425412779' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi175-64406-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:35.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[21527]: audit 2024-03-26T03:51:34.509727+0000 mon.a (mon.0) 1909 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi175-64406-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:35.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[21527]: cluster 2024-03-26T03:51:35.463486+0000 mon.a (mon.0) 1910 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:51:35.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[21527]: audit 2024-03-26T03:51:35.466043+0000 mon.a (mon.0) 1911 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-29"}]': finished 2024-03-26T03:51:35.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[21527]: audit 2024-03-26T03:51:35.466243+0000 mon.a (mon.0) 1912 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi175-64398-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:35.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[21527]: audit 2024-03-26T03:51:35.466452+0000 mon.a (mon.0) 1913 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi175-64406-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:35.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[21527]: cluster 2024-03-26T03:51:35.474634+0000 mon.a (mon.0) 1914 : cluster [DBG] osdmap e239: 8 total, 8 up, 8 in 2024-03-26T03:51:35.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[21527]: audit 2024-03-26T03:51:35.482217+0000 mon.c (mon.2) 408 : audit [INF] from='client.? 172.21.15.175:0/3064582945' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi175-64398-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi175-64398-39"}]: dispatch 2024-03-26T03:51:35.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[21527]: audit 2024-03-26T03:51:35.482863+0000 mon.c (mon.2) 409 : audit [INF] from='client.? 172.21.15.175:0/425412779' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi175-64406-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi175-64406-48"}]: dispatch 2024-03-26T03:51:35.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[29284]: audit 2024-03-26T03:51:34.463079+0000 mon.a (mon.0) 1899 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi175-64406-47"}]': finished 2024-03-26T03:51:35.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[29284]: audit 2024-03-26T03:51:34.463255+0000 mon.a (mon.0) 1900 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi175-64398-38"}]': finished 2024-03-26T03:51:35.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[29284]: audit 2024-03-26T03:51:34.463441+0000 mon.a (mon.0) 1901 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:51:35.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[29284]: cluster 2024-03-26T03:51:34.471130+0000 mon.a (mon.0) 1902 : cluster [DBG] osdmap e238: 8 total, 8 up, 8 in 2024-03-26T03:51:35.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[29284]: audit 2024-03-26T03:51:34.484082+0000 mon.a (mon.0) 1903 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-29"}]: dispatch 2024-03-26T03:51:35.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[29284]: audit 2024-03-26T03:51:34.505200+0000 mon.c (mon.2) 402 : audit [INF] from='client.? 172.21.15.175:0/3064582945' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi175-64398-39"}]: dispatch 2024-03-26T03:51:35.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[29284]: audit 2024-03-26T03:51:34.505709+0000 mon.a (mon.0) 1904 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi175-64398-39"}]: dispatch 2024-03-26T03:51:35.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[29284]: audit 2024-03-26T03:51:34.506812+0000 mon.c (mon.2) 403 : audit [INF] from='client.? 172.21.15.175:0/3064582945' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi175-64398-39"}]: dispatch 2024-03-26T03:51:35.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[29284]: audit 2024-03-26T03:51:34.507125+0000 mon.c (mon.2) 404 : audit [INF] from='client.? 172.21.15.175:0/425412779' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi175-64406-48"}]: dispatch 2024-03-26T03:51:35.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[29284]: audit 2024-03-26T03:51:34.507169+0000 mon.a (mon.0) 1905 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi175-64398-39"}]: dispatch 2024-03-26T03:51:35.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[29284]: audit 2024-03-26T03:51:34.507635+0000 mon.a (mon.0) 1906 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi175-64406-48"}]: dispatch 2024-03-26T03:51:35.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[29284]: audit 2024-03-26T03:51:34.507878+0000 mon.c (mon.2) 405 : audit [INF] from='client.? 172.21.15.175:0/3064582945' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi175-64398-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:35.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[29284]: audit 2024-03-26T03:51:34.508360+0000 mon.a (mon.0) 1907 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi175-64398-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:35.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[29284]: audit 2024-03-26T03:51:34.508712+0000 mon.c (mon.2) 406 : audit [INF] from='client.? 172.21.15.175:0/425412779' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi175-64406-48"}]: dispatch 2024-03-26T03:51:35.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[29284]: audit 2024-03-26T03:51:34.508977+0000 mon.a (mon.0) 1908 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi175-64406-48"}]: dispatch 2024-03-26T03:51:35.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[29284]: audit 2024-03-26T03:51:34.509458+0000 mon.c (mon.2) 407 : audit [INF] from='client.? 172.21.15.175:0/425412779' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi175-64406-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:35.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[29284]: audit 2024-03-26T03:51:34.509727+0000 mon.a (mon.0) 1909 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi175-64406-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:35.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[29284]: cluster 2024-03-26T03:51:35.463486+0000 mon.a (mon.0) 1910 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:51:35.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[29284]: audit 2024-03-26T03:51:35.466043+0000 mon.a (mon.0) 1911 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-29"}]': finished 2024-03-26T03:51:35.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[29284]: audit 2024-03-26T03:51:35.466243+0000 mon.a (mon.0) 1912 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi175-64398-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:35.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[29284]: audit 2024-03-26T03:51:35.466452+0000 mon.a (mon.0) 1913 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi175-64406-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:35.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[29284]: cluster 2024-03-26T03:51:35.474634+0000 mon.a (mon.0) 1914 : cluster [DBG] osdmap e239: 8 total, 8 up, 8 in 2024-03-26T03:51:35.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[29284]: audit 2024-03-26T03:51:35.482217+0000 mon.c (mon.2) 408 : audit [INF] from='client.? 172.21.15.175:0/3064582945' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi175-64398-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi175-64398-39"}]: dispatch 2024-03-26T03:51:35.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:35 smithi175 bash[29284]: audit 2024-03-26T03:51:35.482863+0000 mon.c (mon.2) 409 : audit [INF] from='client.? 172.21.15.175:0/425412779' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi175-64406-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi175-64406-48"}]: dispatch 2024-03-26T03:51:36.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:36 smithi190 bash[23392]: cluster 2024-03-26T03:51:34.659535+0000 mgr.y (mgr.24431) 295 : cluster [DBG] pgmap v354: 292 pgs: 292 active+clean; 4.6 MiB data, 582 MiB used, 715 GiB / 715 GiB avail; 13 KiB/s rd, 1024 KiB/s wr, 25 op/s 2024-03-26T03:51:36.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:36 smithi190 bash[23392]: audit 2024-03-26T03:51:35.483888+0000 mon.a (mon.0) 1915 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi175-64398-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi175-64398-39"}]: dispatch 2024-03-26T03:51:36.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:36 smithi190 bash[23392]: audit 2024-03-26T03:51:35.486216+0000 mon.a (mon.0) 1916 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi175-64406-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi175-64406-48"}]: dispatch 2024-03-26T03:51:36.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:36 smithi190 bash[23392]: cluster 2024-03-26T03:51:36.504615+0000 mon.a (mon.0) 1917 : cluster [DBG] osdmap e240: 8 total, 8 up, 8 in 2024-03-26T03:51:36.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:36 smithi175 bash[21527]: cluster 2024-03-26T03:51:34.659535+0000 mgr.y (mgr.24431) 295 : cluster [DBG] pgmap v354: 292 pgs: 292 active+clean; 4.6 MiB data, 582 MiB used, 715 GiB / 715 GiB avail; 13 KiB/s rd, 1024 KiB/s wr, 25 op/s 2024-03-26T03:51:36.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:36 smithi175 bash[21527]: audit 2024-03-26T03:51:35.483888+0000 mon.a (mon.0) 1915 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi175-64398-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi175-64398-39"}]: dispatch 2024-03-26T03:51:36.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:36 smithi175 bash[21527]: audit 2024-03-26T03:51:35.486216+0000 mon.a (mon.0) 1916 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi175-64406-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi175-64406-48"}]: dispatch 2024-03-26T03:51:36.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:36 smithi175 bash[21527]: cluster 2024-03-26T03:51:36.504615+0000 mon.a (mon.0) 1917 : cluster [DBG] osdmap e240: 8 total, 8 up, 8 in 2024-03-26T03:51:36.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:36 smithi175 bash[29284]: cluster 2024-03-26T03:51:34.659535+0000 mgr.y (mgr.24431) 295 : cluster [DBG] pgmap v354: 292 pgs: 292 active+clean; 4.6 MiB data, 582 MiB used, 715 GiB / 715 GiB avail; 13 KiB/s rd, 1024 KiB/s wr, 25 op/s 2024-03-26T03:51:36.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:36 smithi175 bash[29284]: audit 2024-03-26T03:51:35.483888+0000 mon.a (mon.0) 1915 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi175-64398-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi175-64398-39"}]: dispatch 2024-03-26T03:51:36.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:36 smithi175 bash[29284]: audit 2024-03-26T03:51:35.486216+0000 mon.a (mon.0) 1916 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi175-64406-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi175-64406-48"}]: dispatch 2024-03-26T03:51:36.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:36 smithi175 bash[29284]: cluster 2024-03-26T03:51:36.504615+0000 mon.a (mon.0) 1917 : cluster [DBG] osdmap e240: 8 total, 8 up, 8 in 2024-03-26T03:51:38.767 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:38 smithi175 bash[21527]: cluster 2024-03-26T03:51:36.660361+0000 mgr.y (mgr.24431) 296 : cluster [DBG] pgmap v357: 260 pgs: 260 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 KiB/s wr, 1 op/s 2024-03-26T03:51:38.768 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:38 smithi175 bash[21527]: audit 2024-03-26T03:51:37.473802+0000 mon.a (mon.0) 1918 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi175-64398-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi175-64398-39"}]': finished 2024-03-26T03:51:38.768 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:38 smithi175 bash[21527]: audit 2024-03-26T03:51:37.474111+0000 mon.a (mon.0) 1919 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi175-64406-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi175-64406-48"}]': finished 2024-03-26T03:51:38.768 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:38 smithi175 bash[21527]: cluster 2024-03-26T03:51:37.483482+0000 mon.a (mon.0) 1920 : cluster [DBG] osdmap e241: 8 total, 8 up, 8 in 2024-03-26T03:51:38.768 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:38 smithi175 bash[21527]: audit 2024-03-26T03:51:37.487652+0000 mon.a (mon.0) 1921 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:38.768 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:38 smithi175 bash[29284]: cluster 2024-03-26T03:51:36.660361+0000 mgr.y (mgr.24431) 296 : cluster [DBG] pgmap v357: 260 pgs: 260 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 KiB/s wr, 1 op/s 2024-03-26T03:51:38.768 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:38 smithi175 bash[29284]: audit 2024-03-26T03:51:37.473802+0000 mon.a (mon.0) 1918 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi175-64398-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi175-64398-39"}]': finished 2024-03-26T03:51:38.768 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:38 smithi175 bash[29284]: audit 2024-03-26T03:51:37.474111+0000 mon.a (mon.0) 1919 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi175-64406-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi175-64406-48"}]': finished 2024-03-26T03:51:38.768 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:38 smithi175 bash[29284]: cluster 2024-03-26T03:51:37.483482+0000 mon.a (mon.0) 1920 : cluster [DBG] osdmap e241: 8 total, 8 up, 8 in 2024-03-26T03:51:38.768 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:38 smithi175 bash[29284]: audit 2024-03-26T03:51:37.487652+0000 mon.a (mon.0) 1921 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:38.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:38 smithi190 bash[23392]: cluster 2024-03-26T03:51:36.660361+0000 mgr.y (mgr.24431) 296 : cluster [DBG] pgmap v357: 260 pgs: 260 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 KiB/s wr, 1 op/s 2024-03-26T03:51:38.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:38 smithi190 bash[23392]: audit 2024-03-26T03:51:37.473802+0000 mon.a (mon.0) 1918 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi175-64398-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi175-64398-39"}]': finished 2024-03-26T03:51:38.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:38 smithi190 bash[23392]: audit 2024-03-26T03:51:37.474111+0000 mon.a (mon.0) 1919 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi175-64406-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi175-64406-48"}]': finished 2024-03-26T03:51:38.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:38 smithi190 bash[23392]: cluster 2024-03-26T03:51:37.483482+0000 mon.a (mon.0) 1920 : cluster [DBG] osdmap e241: 8 total, 8 up, 8 in 2024-03-26T03:51:38.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:38 smithi190 bash[23392]: audit 2024-03-26T03:51:37.487652+0000 mon.a (mon.0) 1921 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:39.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:51:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:51:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:51:39.847 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:39 smithi190 bash[23392]: audit 2024-03-26T03:51:38.477252+0000 mon.a (mon.0) 1922 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:39.847 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:39 smithi190 bash[23392]: cluster 2024-03-26T03:51:38.487909+0000 mon.a (mon.0) 1923 : cluster [DBG] osdmap e242: 8 total, 8 up, 8 in 2024-03-26T03:51:39.847 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:39 smithi190 bash[23392]: audit 2024-03-26T03:51:38.500123+0000 mon.a (mon.0) 1924 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:51:39.847 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:39 smithi190 bash[23392]: cluster 2024-03-26T03:51:38.661282+0000 mgr.y (mgr.24431) 297 : cluster [DBG] pgmap v360: 308 pgs: 48 unknown, 260 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:51:39.847 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:39 smithi190 bash[23392]: audit 2024-03-26T03:51:39.480230+0000 mon.a (mon.0) 1925 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-31", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:51:39.847 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:39 smithi190 bash[23392]: cluster 2024-03-26T03:51:39.488533+0000 mon.a (mon.0) 1926 : cluster [DBG] osdmap e243: 8 total, 8 up, 8 in 2024-03-26T03:51:39.847 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:39 smithi190 bash[23392]: audit 2024-03-26T03:51:39.490468+0000 mon.a (mon.0) 1927 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-31"}]: dispatch 2024-03-26T03:51:39.847 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:39 smithi190 bash[23392]: audit 2024-03-26T03:51:39.492845+0000 mon.c (mon.2) 410 : audit [INF] from='client.? 172.21.15.175:0/425412779' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi175-64406-48"}]: dispatch 2024-03-26T03:51:39.847 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:39 smithi190 bash[23392]: audit 2024-03-26T03:51:39.493249+0000 mon.c (mon.2) 411 : audit [INF] from='client.? 172.21.15.175:0/3064582945' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi175-64398-39"}]: dispatch 2024-03-26T03:51:39.847 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:39 smithi190 bash[23392]: audit 2024-03-26T03:51:39.494236+0000 mon.a (mon.0) 1928 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi175-64406-48"}]: dispatch 2024-03-26T03:51:39.847 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:39 smithi190 bash[23392]: audit 2024-03-26T03:51:39.494689+0000 mon.a (mon.0) 1929 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi175-64398-39"}]: dispatch 2024-03-26T03:51:39.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:39 smithi175 bash[21527]: audit 2024-03-26T03:51:38.477252+0000 mon.a (mon.0) 1922 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:39.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:39 smithi175 bash[21527]: cluster 2024-03-26T03:51:38.487909+0000 mon.a (mon.0) 1923 : cluster [DBG] osdmap e242: 8 total, 8 up, 8 in 2024-03-26T03:51:39.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:39 smithi175 bash[21527]: audit 2024-03-26T03:51:38.500123+0000 mon.a (mon.0) 1924 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:51:39.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:39 smithi175 bash[21527]: cluster 2024-03-26T03:51:38.661282+0000 mgr.y (mgr.24431) 297 : cluster [DBG] pgmap v360: 308 pgs: 48 unknown, 260 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:51:39.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:39 smithi175 bash[21527]: audit 2024-03-26T03:51:39.480230+0000 mon.a (mon.0) 1925 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-31", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:51:39.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:39 smithi175 bash[21527]: cluster 2024-03-26T03:51:39.488533+0000 mon.a (mon.0) 1926 : cluster [DBG] osdmap e243: 8 total, 8 up, 8 in 2024-03-26T03:51:39.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:39 smithi175 bash[21527]: audit 2024-03-26T03:51:39.490468+0000 mon.a (mon.0) 1927 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-31"}]: dispatch 2024-03-26T03:51:39.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:39 smithi175 bash[21527]: audit 2024-03-26T03:51:39.492845+0000 mon.c (mon.2) 410 : audit [INF] from='client.? 172.21.15.175:0/425412779' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi175-64406-48"}]: dispatch 2024-03-26T03:51:39.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:39 smithi175 bash[21527]: audit 2024-03-26T03:51:39.493249+0000 mon.c (mon.2) 411 : audit [INF] from='client.? 172.21.15.175:0/3064582945' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi175-64398-39"}]: dispatch 2024-03-26T03:51:39.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:39 smithi175 bash[21527]: audit 2024-03-26T03:51:39.494236+0000 mon.a (mon.0) 1928 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi175-64406-48"}]: dispatch 2024-03-26T03:51:39.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:39 smithi175 bash[21527]: audit 2024-03-26T03:51:39.494689+0000 mon.a (mon.0) 1929 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi175-64398-39"}]: dispatch 2024-03-26T03:51:39.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:39 smithi175 bash[29284]: audit 2024-03-26T03:51:38.477252+0000 mon.a (mon.0) 1922 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:39.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:39 smithi175 bash[29284]: cluster 2024-03-26T03:51:38.487909+0000 mon.a (mon.0) 1923 : cluster [DBG] osdmap e242: 8 total, 8 up, 8 in 2024-03-26T03:51:39.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:39 smithi175 bash[29284]: audit 2024-03-26T03:51:38.500123+0000 mon.a (mon.0) 1924 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:51:39.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:39 smithi175 bash[29284]: cluster 2024-03-26T03:51:38.661282+0000 mgr.y (mgr.24431) 297 : cluster [DBG] pgmap v360: 308 pgs: 48 unknown, 260 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:51:39.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:39 smithi175 bash[29284]: audit 2024-03-26T03:51:39.480230+0000 mon.a (mon.0) 1925 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-31", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:51:39.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:39 smithi175 bash[29284]: cluster 2024-03-26T03:51:39.488533+0000 mon.a (mon.0) 1926 : cluster [DBG] osdmap e243: 8 total, 8 up, 8 in 2024-03-26T03:51:39.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:39 smithi175 bash[29284]: audit 2024-03-26T03:51:39.490468+0000 mon.a (mon.0) 1927 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-31"}]: dispatch 2024-03-26T03:51:39.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:39 smithi175 bash[29284]: audit 2024-03-26T03:51:39.492845+0000 mon.c (mon.2) 410 : audit [INF] from='client.? 172.21.15.175:0/425412779' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi175-64406-48"}]: dispatch 2024-03-26T03:51:39.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:39 smithi175 bash[29284]: audit 2024-03-26T03:51:39.493249+0000 mon.c (mon.2) 411 : audit [INF] from='client.? 172.21.15.175:0/3064582945' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi175-64398-39"}]: dispatch 2024-03-26T03:51:39.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:39 smithi175 bash[29284]: audit 2024-03-26T03:51:39.494236+0000 mon.a (mon.0) 1928 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi175-64406-48"}]: dispatch 2024-03-26T03:51:39.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:39 smithi175 bash[29284]: audit 2024-03-26T03:51:39.494689+0000 mon.a (mon.0) 1929 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi175-64398-39"}]: dispatch 2024-03-26T03:51:40.239 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:51:39 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:51:40.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:40 smithi190 bash[23392]: cluster 2024-03-26T03:51:39.511847+0000 mon.a (mon.0) 1930 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:51:40.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:40 smithi190 bash[23392]: audit 2024-03-26T03:51:39.848146+0000 mgr.y (mgr.24431) 298 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:51:40.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:40 smithi190 bash[23392]: audit 2024-03-26T03:51:40.482946+0000 mon.a (mon.0) 1931 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-31"}]': finished 2024-03-26T03:51:40.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:40 smithi190 bash[23392]: audit 2024-03-26T03:51:40.483114+0000 mon.a (mon.0) 1932 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi175-64406-48"}]': finished 2024-03-26T03:51:40.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:40 smithi190 bash[23392]: audit 2024-03-26T03:51:40.483240+0000 mon.a (mon.0) 1933 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi175-64398-39"}]': finished 2024-03-26T03:51:40.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:40 smithi190 bash[23392]: cluster 2024-03-26T03:51:40.491644+0000 mon.a (mon.0) 1934 : cluster [DBG] osdmap e244: 8 total, 8 up, 8 in 2024-03-26T03:51:40.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:40 smithi190 bash[23392]: audit 2024-03-26T03:51:40.493376+0000 mon.a (mon.0) 1935 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-31", "mode": "writeback"}]: dispatch 2024-03-26T03:51:40.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:40 smithi190 bash[23392]: audit 2024-03-26T03:51:40.494115+0000 mon.c (mon.2) 412 : audit [INF] from='client.? 172.21.15.175:0/425412779' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi175-64406-48"}]: dispatch 2024-03-26T03:51:40.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:40 smithi190 bash[23392]: audit 2024-03-26T03:51:40.495854+0000 mon.a (mon.0) 1936 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi175-64406-48"}]: dispatch 2024-03-26T03:51:40.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:40 smithi190 bash[23392]: audit 2024-03-26T03:51:40.498437+0000 mon.c (mon.2) 413 : audit [INF] from='client.? 172.21.15.175:0/3064582945' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi175-64398-39"}]: dispatch 2024-03-26T03:51:40.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:40 smithi190 bash[23392]: audit 2024-03-26T03:51:40.499789+0000 mon.a (mon.0) 1937 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi175-64398-39"}]: dispatch 2024-03-26T03:51:40.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:40 smithi175 bash[29284]: cluster 2024-03-26T03:51:39.511847+0000 mon.a (mon.0) 1930 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:51:40.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:40 smithi175 bash[29284]: audit 2024-03-26T03:51:39.848146+0000 mgr.y (mgr.24431) 298 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:51:40.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:40 smithi175 bash[29284]: audit 2024-03-26T03:51:40.482946+0000 mon.a (mon.0) 1931 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-31"}]': finished 2024-03-26T03:51:40.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:40 smithi175 bash[29284]: audit 2024-03-26T03:51:40.483114+0000 mon.a (mon.0) 1932 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi175-64406-48"}]': finished 2024-03-26T03:51:40.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:40 smithi175 bash[29284]: audit 2024-03-26T03:51:40.483240+0000 mon.a (mon.0) 1933 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi175-64398-39"}]': finished 2024-03-26T03:51:40.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:40 smithi175 bash[29284]: cluster 2024-03-26T03:51:40.491644+0000 mon.a (mon.0) 1934 : cluster [DBG] osdmap e244: 8 total, 8 up, 8 in 2024-03-26T03:51:40.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:40 smithi175 bash[29284]: audit 2024-03-26T03:51:40.493376+0000 mon.a (mon.0) 1935 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-31", "mode": "writeback"}]: dispatch 2024-03-26T03:51:40.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:40 smithi175 bash[29284]: audit 2024-03-26T03:51:40.494115+0000 mon.c (mon.2) 412 : audit [INF] from='client.? 172.21.15.175:0/425412779' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi175-64406-48"}]: dispatch 2024-03-26T03:51:40.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:40 smithi175 bash[29284]: audit 2024-03-26T03:51:40.495854+0000 mon.a (mon.0) 1936 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi175-64406-48"}]: dispatch 2024-03-26T03:51:40.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:40 smithi175 bash[29284]: audit 2024-03-26T03:51:40.498437+0000 mon.c (mon.2) 413 : audit [INF] from='client.? 172.21.15.175:0/3064582945' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi175-64398-39"}]: dispatch 2024-03-26T03:51:40.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:40 smithi175 bash[29284]: audit 2024-03-26T03:51:40.499789+0000 mon.a (mon.0) 1937 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi175-64398-39"}]: dispatch 2024-03-26T03:51:40.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:40 smithi175 bash[21527]: cluster 2024-03-26T03:51:39.511847+0000 mon.a (mon.0) 1930 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:51:40.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:40 smithi175 bash[21527]: audit 2024-03-26T03:51:39.848146+0000 mgr.y (mgr.24431) 298 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:51:40.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:40 smithi175 bash[21527]: audit 2024-03-26T03:51:40.482946+0000 mon.a (mon.0) 1931 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-31"}]': finished 2024-03-26T03:51:40.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:40 smithi175 bash[21527]: audit 2024-03-26T03:51:40.483114+0000 mon.a (mon.0) 1932 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi175-64406-48"}]': finished 2024-03-26T03:51:40.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:40 smithi175 bash[21527]: audit 2024-03-26T03:51:40.483240+0000 mon.a (mon.0) 1933 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi175-64398-39"}]': finished 2024-03-26T03:51:40.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:40 smithi175 bash[21527]: cluster 2024-03-26T03:51:40.491644+0000 mon.a (mon.0) 1934 : cluster [DBG] osdmap e244: 8 total, 8 up, 8 in 2024-03-26T03:51:40.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:40 smithi175 bash[21527]: audit 2024-03-26T03:51:40.493376+0000 mon.a (mon.0) 1935 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-31", "mode": "writeback"}]: dispatch 2024-03-26T03:51:40.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:40 smithi175 bash[21527]: audit 2024-03-26T03:51:40.494115+0000 mon.c (mon.2) 412 : audit [INF] from='client.? 172.21.15.175:0/425412779' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi175-64406-48"}]: dispatch 2024-03-26T03:51:40.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:40 smithi175 bash[21527]: audit 2024-03-26T03:51:40.495854+0000 mon.a (mon.0) 1936 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi175-64406-48"}]: dispatch 2024-03-26T03:51:40.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:40 smithi175 bash[21527]: audit 2024-03-26T03:51:40.498437+0000 mon.c (mon.2) 413 : audit [INF] from='client.? 172.21.15.175:0/3064582945' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi175-64398-39"}]: dispatch 2024-03-26T03:51:40.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:40 smithi175 bash[21527]: audit 2024-03-26T03:51:40.499789+0000 mon.a (mon.0) 1937 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi175-64398-39"}]: dispatch 2024-03-26T03:51:41.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:41 smithi190 bash[23392]: cluster 2024-03-26T03:51:40.662132+0000 mgr.y (mgr.24431) 299 : cluster [DBG] pgmap v363: 292 pgs: 292 active+clean; 4.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:51:41.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:41 smithi190 bash[23392]: cluster 2024-03-26T03:51:41.447695+0000 mon.a (mon.0) 1938 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:51:41.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:41 smithi190 bash[23392]: audit 2024-03-26T03:51:41.450231+0000 mon.a (mon.0) 1939 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-31", "mode": "writeback"}]': finished 2024-03-26T03:51:41.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:41 smithi190 bash[23392]: audit 2024-03-26T03:51:41.450427+0000 mon.a (mon.0) 1940 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi175-64406-48"}]': finished 2024-03-26T03:51:41.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:41 smithi190 bash[23392]: audit 2024-03-26T03:51:41.450586+0000 mon.a (mon.0) 1941 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi175-64398-39"}]': finished 2024-03-26T03:51:41.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:41 smithi190 bash[23392]: cluster 2024-03-26T03:51:41.457454+0000 mon.a (mon.0) 1942 : cluster [DBG] osdmap e245: 8 total, 8 up, 8 in 2024-03-26T03:51:41.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:41 smithi190 bash[23392]: audit 2024-03-26T03:51:41.509358+0000 mon.a (mon.0) 1943 : audit [INF] from='client.? 172.21.15.175:0/437476285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi175-64398-40"}]: dispatch 2024-03-26T03:51:41.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:41 smithi190 bash[23392]: audit 2024-03-26T03:51:41.509578+0000 mon.a (mon.0) 1944 : audit [INF] from='client.? 172.21.15.175:0/2762811080' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi175-64406-49"}]: dispatch 2024-03-26T03:51:41.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:41 smithi190 bash[23392]: audit 2024-03-26T03:51:41.510798+0000 mon.a (mon.0) 1945 : audit [INF] from='client.? 172.21.15.175:0/437476285' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi175-64398-40"}]: dispatch 2024-03-26T03:51:41.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:41 smithi190 bash[23392]: audit 2024-03-26T03:51:41.511521+0000 mon.a (mon.0) 1946 : audit [INF] from='client.? 172.21.15.175:0/437476285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi175-64398-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:41.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:41 smithi190 bash[23392]: audit 2024-03-26T03:51:41.511908+0000 mon.a (mon.0) 1947 : audit [INF] from='client.? 172.21.15.175:0/2762811080' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi175-64406-49"}]: dispatch 2024-03-26T03:51:41.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:41 smithi190 bash[23392]: audit 2024-03-26T03:51:41.512405+0000 mon.a (mon.0) 1948 : audit [INF] from='client.? 172.21.15.175:0/2762811080' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi175-64406-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:41.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:41 smithi175 bash[21527]: cluster 2024-03-26T03:51:40.662132+0000 mgr.y (mgr.24431) 299 : cluster [DBG] pgmap v363: 292 pgs: 292 active+clean; 4.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:51:41.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:41 smithi175 bash[21527]: cluster 2024-03-26T03:51:41.447695+0000 mon.a (mon.0) 1938 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:51:41.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:41 smithi175 bash[21527]: audit 2024-03-26T03:51:41.450231+0000 mon.a (mon.0) 1939 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-31", "mode": "writeback"}]': finished 2024-03-26T03:51:41.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:41 smithi175 bash[21527]: audit 2024-03-26T03:51:41.450427+0000 mon.a (mon.0) 1940 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi175-64406-48"}]': finished 2024-03-26T03:51:41.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:41 smithi175 bash[21527]: audit 2024-03-26T03:51:41.450586+0000 mon.a (mon.0) 1941 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi175-64398-39"}]': finished 2024-03-26T03:51:41.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:41 smithi175 bash[21527]: cluster 2024-03-26T03:51:41.457454+0000 mon.a (mon.0) 1942 : cluster [DBG] osdmap e245: 8 total, 8 up, 8 in 2024-03-26T03:51:41.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:41 smithi175 bash[21527]: audit 2024-03-26T03:51:41.509358+0000 mon.a (mon.0) 1943 : audit [INF] from='client.? 172.21.15.175:0/437476285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi175-64398-40"}]: dispatch 2024-03-26T03:51:41.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:41 smithi175 bash[21527]: audit 2024-03-26T03:51:41.509578+0000 mon.a (mon.0) 1944 : audit [INF] from='client.? 172.21.15.175:0/2762811080' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi175-64406-49"}]: dispatch 2024-03-26T03:51:41.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:41 smithi175 bash[21527]: audit 2024-03-26T03:51:41.510798+0000 mon.a (mon.0) 1945 : audit [INF] from='client.? 172.21.15.175:0/437476285' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi175-64398-40"}]: dispatch 2024-03-26T03:51:41.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:41 smithi175 bash[21527]: audit 2024-03-26T03:51:41.511521+0000 mon.a (mon.0) 1946 : audit [INF] from='client.? 172.21.15.175:0/437476285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi175-64398-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:41.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:41 smithi175 bash[21527]: audit 2024-03-26T03:51:41.511908+0000 mon.a (mon.0) 1947 : audit [INF] from='client.? 172.21.15.175:0/2762811080' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi175-64406-49"}]: dispatch 2024-03-26T03:51:41.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:41 smithi175 bash[21527]: audit 2024-03-26T03:51:41.512405+0000 mon.a (mon.0) 1948 : audit [INF] from='client.? 172.21.15.175:0/2762811080' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi175-64406-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:41 smithi175 bash[29284]: cluster 2024-03-26T03:51:40.662132+0000 mgr.y (mgr.24431) 299 : cluster [DBG] pgmap v363: 292 pgs: 292 active+clean; 4.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:51:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:41 smithi175 bash[29284]: cluster 2024-03-26T03:51:41.447695+0000 mon.a (mon.0) 1938 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:51:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:41 smithi175 bash[29284]: audit 2024-03-26T03:51:41.450231+0000 mon.a (mon.0) 1939 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-31", "mode": "writeback"}]': finished 2024-03-26T03:51:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:41 smithi175 bash[29284]: audit 2024-03-26T03:51:41.450427+0000 mon.a (mon.0) 1940 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi175-64406-48"}]': finished 2024-03-26T03:51:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:41 smithi175 bash[29284]: audit 2024-03-26T03:51:41.450586+0000 mon.a (mon.0) 1941 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi175-64398-39"}]': finished 2024-03-26T03:51:41.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:41 smithi175 bash[29284]: cluster 2024-03-26T03:51:41.457454+0000 mon.a (mon.0) 1942 : cluster [DBG] osdmap e245: 8 total, 8 up, 8 in 2024-03-26T03:51:41.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:41 smithi175 bash[29284]: audit 2024-03-26T03:51:41.509358+0000 mon.a (mon.0) 1943 : audit [INF] from='client.? 172.21.15.175:0/437476285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi175-64398-40"}]: dispatch 2024-03-26T03:51:41.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:41 smithi175 bash[29284]: audit 2024-03-26T03:51:41.509578+0000 mon.a (mon.0) 1944 : audit [INF] from='client.? 172.21.15.175:0/2762811080' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi175-64406-49"}]: dispatch 2024-03-26T03:51:41.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:41 smithi175 bash[29284]: audit 2024-03-26T03:51:41.510798+0000 mon.a (mon.0) 1945 : audit [INF] from='client.? 172.21.15.175:0/437476285' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi175-64398-40"}]: dispatch 2024-03-26T03:51:41.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:41 smithi175 bash[29284]: audit 2024-03-26T03:51:41.511521+0000 mon.a (mon.0) 1946 : audit [INF] from='client.? 172.21.15.175:0/437476285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi175-64398-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:41.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:41 smithi175 bash[29284]: audit 2024-03-26T03:51:41.511908+0000 mon.a (mon.0) 1947 : audit [INF] from='client.? 172.21.15.175:0/2762811080' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi175-64406-49"}]: dispatch 2024-03-26T03:51:41.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:41 smithi175 bash[29284]: audit 2024-03-26T03:51:41.512405+0000 mon.a (mon.0) 1948 : audit [INF] from='client.? 172.21.15.175:0/2762811080' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi175-64406-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:42.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:42 smithi190 bash[23392]: audit 2024-03-26T03:51:41.613979+0000 mon.a (mon.0) 1949 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:51:42.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:42 smithi190 bash[23392]: audit 2024-03-26T03:51:42.453186+0000 mon.a (mon.0) 1950 : audit [INF] from='client.? 172.21.15.175:0/437476285' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi175-64398-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:42.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:42 smithi190 bash[23392]: audit 2024-03-26T03:51:42.453411+0000 mon.a (mon.0) 1951 : audit [INF] from='client.? 172.21.15.175:0/2762811080' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi175-64406-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:42.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:42 smithi190 bash[23392]: audit 2024-03-26T03:51:42.453595+0000 mon.a (mon.0) 1952 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:51:42.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:42 smithi190 bash[23392]: cluster 2024-03-26T03:51:42.461518+0000 mon.a (mon.0) 1953 : cluster [DBG] osdmap e246: 8 total, 8 up, 8 in 2024-03-26T03:51:42.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:42 smithi190 bash[23392]: audit 2024-03-26T03:51:42.464343+0000 mon.a (mon.0) 1954 : audit [INF] from='client.? 172.21.15.175:0/437476285' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi175-64398-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi175-64398-40"}]: dispatch 2024-03-26T03:51:42.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:42 smithi190 bash[23392]: audit 2024-03-26T03:51:42.465354+0000 mon.a (mon.0) 1955 : audit [INF] from='client.? 172.21.15.175:0/2762811080' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi175-64406-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi175-64406-49"}]: dispatch 2024-03-26T03:51:42.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:42 smithi190 bash[23392]: audit 2024-03-26T03:51:42.466275+0000 mon.a (mon.0) 1956 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-31"}]: dispatch 2024-03-26T03:51:42.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:42 smithi175 bash[21527]: audit 2024-03-26T03:51:41.613979+0000 mon.a (mon.0) 1949 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:51:42.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:42 smithi175 bash[21527]: audit 2024-03-26T03:51:42.453186+0000 mon.a (mon.0) 1950 : audit [INF] from='client.? 172.21.15.175:0/437476285' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi175-64398-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:42.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:42 smithi175 bash[21527]: audit 2024-03-26T03:51:42.453411+0000 mon.a (mon.0) 1951 : audit [INF] from='client.? 172.21.15.175:0/2762811080' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi175-64406-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:42.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:42 smithi175 bash[21527]: audit 2024-03-26T03:51:42.453595+0000 mon.a (mon.0) 1952 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:51:42.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:42 smithi175 bash[21527]: cluster 2024-03-26T03:51:42.461518+0000 mon.a (mon.0) 1953 : cluster [DBG] osdmap e246: 8 total, 8 up, 8 in 2024-03-26T03:51:42.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:42 smithi175 bash[21527]: audit 2024-03-26T03:51:42.464343+0000 mon.a (mon.0) 1954 : audit [INF] from='client.? 172.21.15.175:0/437476285' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi175-64398-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi175-64398-40"}]: dispatch 2024-03-26T03:51:42.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:42 smithi175 bash[21527]: audit 2024-03-26T03:51:42.465354+0000 mon.a (mon.0) 1955 : audit [INF] from='client.? 172.21.15.175:0/2762811080' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi175-64406-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi175-64406-49"}]: dispatch 2024-03-26T03:51:42.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:42 smithi175 bash[21527]: audit 2024-03-26T03:51:42.466275+0000 mon.a (mon.0) 1956 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-31"}]: dispatch 2024-03-26T03:51:42.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:42 smithi175 bash[29284]: audit 2024-03-26T03:51:41.613979+0000 mon.a (mon.0) 1949 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:51:42.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:42 smithi175 bash[29284]: audit 2024-03-26T03:51:42.453186+0000 mon.a (mon.0) 1950 : audit [INF] from='client.? 172.21.15.175:0/437476285' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi175-64398-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:42.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:42 smithi175 bash[29284]: audit 2024-03-26T03:51:42.453411+0000 mon.a (mon.0) 1951 : audit [INF] from='client.? 172.21.15.175:0/2762811080' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi175-64406-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:42.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:42 smithi175 bash[29284]: audit 2024-03-26T03:51:42.453595+0000 mon.a (mon.0) 1952 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:51:42.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:42 smithi175 bash[29284]: cluster 2024-03-26T03:51:42.461518+0000 mon.a (mon.0) 1953 : cluster [DBG] osdmap e246: 8 total, 8 up, 8 in 2024-03-26T03:51:42.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:42 smithi175 bash[29284]: audit 2024-03-26T03:51:42.464343+0000 mon.a (mon.0) 1954 : audit [INF] from='client.? 172.21.15.175:0/437476285' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi175-64398-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi175-64398-40"}]: dispatch 2024-03-26T03:51:42.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:42 smithi175 bash[29284]: audit 2024-03-26T03:51:42.465354+0000 mon.a (mon.0) 1955 : audit [INF] from='client.? 172.21.15.175:0/2762811080' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi175-64406-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi175-64406-49"}]: dispatch 2024-03-26T03:51:42.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:42 smithi175 bash[29284]: audit 2024-03-26T03:51:42.466275+0000 mon.a (mon.0) 1956 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-31"}]: dispatch 2024-03-26T03:51:43.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:43 smithi190 bash[23392]: cluster 2024-03-26T03:51:42.663028+0000 mgr.y (mgr.24431) 300 : cluster [DBG] pgmap v366: 292 pgs: 292 active+clean; 4.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:51:43.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:43 smithi190 bash[23392]: cluster 2024-03-26T03:51:43.454435+0000 mon.a (mon.0) 1957 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:51:43.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:43 smithi190 bash[23392]: audit 2024-03-26T03:51:43.457061+0000 mon.a (mon.0) 1958 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-31"}]': finished 2024-03-26T03:51:43.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:43 smithi190 bash[23392]: cluster 2024-03-26T03:51:43.488033+0000 mon.a (mon.0) 1959 : cluster [DBG] osdmap e247: 8 total, 8 up, 8 in 2024-03-26T03:51:43.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:43 smithi175 bash[21527]: cluster 2024-03-26T03:51:42.663028+0000 mgr.y (mgr.24431) 300 : cluster [DBG] pgmap v366: 292 pgs: 292 active+clean; 4.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:51:43.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:43 smithi175 bash[21527]: cluster 2024-03-26T03:51:43.454435+0000 mon.a (mon.0) 1957 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:51:43.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:43 smithi175 bash[21527]: audit 2024-03-26T03:51:43.457061+0000 mon.a (mon.0) 1958 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-31"}]': finished 2024-03-26T03:51:43.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:43 smithi175 bash[21527]: cluster 2024-03-26T03:51:43.488033+0000 mon.a (mon.0) 1959 : cluster [DBG] osdmap e247: 8 total, 8 up, 8 in 2024-03-26T03:51:43.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:43 smithi175 bash[29284]: cluster 2024-03-26T03:51:42.663028+0000 mgr.y (mgr.24431) 300 : cluster [DBG] pgmap v366: 292 pgs: 292 active+clean; 4.6 MiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:51:43.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:43 smithi175 bash[29284]: cluster 2024-03-26T03:51:43.454435+0000 mon.a (mon.0) 1957 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:51:43.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:43 smithi175 bash[29284]: audit 2024-03-26T03:51:43.457061+0000 mon.a (mon.0) 1958 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-31"}]': finished 2024-03-26T03:51:43.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:43 smithi175 bash[29284]: cluster 2024-03-26T03:51:43.488033+0000 mon.a (mon.0) 1959 : cluster [DBG] osdmap e247: 8 total, 8 up, 8 in 2024-03-26T03:51:45.739 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:45 smithi190 bash[23392]: audit 2024-03-26T03:51:44.460600+0000 mon.a (mon.0) 1960 : audit [INF] from='client.? 172.21.15.175:0/437476285' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi175-64398-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi175-64398-40"}]': finished 2024-03-26T03:51:45.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:45 smithi190 bash[23392]: audit 2024-03-26T03:51:44.460885+0000 mon.a (mon.0) 1961 : audit [INF] from='client.? 172.21.15.175:0/2762811080' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi175-64406-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi175-64406-49"}]': finished 2024-03-26T03:51:45.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:45 smithi190 bash[23392]: cluster 2024-03-26T03:51:44.468835+0000 mon.a (mon.0) 1962 : cluster [DBG] osdmap e248: 8 total, 8 up, 8 in 2024-03-26T03:51:45.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:45 smithi175 bash[21527]: audit 2024-03-26T03:51:44.460600+0000 mon.a (mon.0) 1960 : audit [INF] from='client.? 172.21.15.175:0/437476285' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi175-64398-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi175-64398-40"}]': finished 2024-03-26T03:51:45.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:45 smithi175 bash[21527]: audit 2024-03-26T03:51:44.460885+0000 mon.a (mon.0) 1961 : audit [INF] from='client.? 172.21.15.175:0/2762811080' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi175-64406-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi175-64406-49"}]': finished 2024-03-26T03:51:45.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:45 smithi175 bash[21527]: cluster 2024-03-26T03:51:44.468835+0000 mon.a (mon.0) 1962 : cluster [DBG] osdmap e248: 8 total, 8 up, 8 in 2024-03-26T03:51:45.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:45 smithi175 bash[29284]: audit 2024-03-26T03:51:44.460600+0000 mon.a (mon.0) 1960 : audit [INF] from='client.? 172.21.15.175:0/437476285' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi175-64398-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi175-64398-40"}]': finished 2024-03-26T03:51:45.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:45 smithi175 bash[29284]: audit 2024-03-26T03:51:44.460885+0000 mon.a (mon.0) 1961 : audit [INF] from='client.? 172.21.15.175:0/2762811080' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi175-64406-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi175-64406-49"}]': finished 2024-03-26T03:51:45.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:45 smithi175 bash[29284]: cluster 2024-03-26T03:51:44.468835+0000 mon.a (mon.0) 1962 : cluster [DBG] osdmap e248: 8 total, 8 up, 8 in 2024-03-26T03:51:46.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:46 smithi190 bash[23392]: cluster 2024-03-26T03:51:44.664957+0000 mgr.y (mgr.24431) 301 : cluster [DBG] pgmap v369: 276 pgs: 12 creating+peering, 4 unknown, 260 active+clean; 4.6 MiB data, 596 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:51:46.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:46 smithi190 bash[23392]: cluster 2024-03-26T03:51:45.460654+0000 mon.a (mon.0) 1963 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:51:46.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:46 smithi190 bash[23392]: cluster 2024-03-26T03:51:45.487498+0000 mon.a (mon.0) 1964 : cluster [DBG] osdmap e249: 8 total, 8 up, 8 in 2024-03-26T03:51:46.740 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:46 smithi190 bash[23392]: audit 2024-03-26T03:51:45.491388+0000 mon.a (mon.0) 1965 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:46.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:46 smithi175 bash[29284]: cluster 2024-03-26T03:51:44.664957+0000 mgr.y (mgr.24431) 301 : cluster [DBG] pgmap v369: 276 pgs: 12 creating+peering, 4 unknown, 260 active+clean; 4.6 MiB data, 596 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:51:46.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:46 smithi175 bash[29284]: cluster 2024-03-26T03:51:45.460654+0000 mon.a (mon.0) 1963 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:51:46.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:46 smithi175 bash[29284]: cluster 2024-03-26T03:51:45.487498+0000 mon.a (mon.0) 1964 : cluster [DBG] osdmap e249: 8 total, 8 up, 8 in 2024-03-26T03:51:46.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:46 smithi175 bash[29284]: audit 2024-03-26T03:51:45.491388+0000 mon.a (mon.0) 1965 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:46.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:46 smithi175 bash[21527]: cluster 2024-03-26T03:51:44.664957+0000 mgr.y (mgr.24431) 301 : cluster [DBG] pgmap v369: 276 pgs: 12 creating+peering, 4 unknown, 260 active+clean; 4.6 MiB data, 596 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:51:46.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:46 smithi175 bash[21527]: cluster 2024-03-26T03:51:45.460654+0000 mon.a (mon.0) 1963 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:51:46.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:46 smithi175 bash[21527]: cluster 2024-03-26T03:51:45.487498+0000 mon.a (mon.0) 1964 : cluster [DBG] osdmap e249: 8 total, 8 up, 8 in 2024-03-26T03:51:46.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:46 smithi175 bash[21527]: audit 2024-03-26T03:51:45.491388+0000 mon.a (mon.0) 1965 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:47.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:47 smithi190 bash[23392]: audit 2024-03-26T03:51:46.488230+0000 mon.a (mon.0) 1966 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:47.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:47 smithi190 bash[23392]: cluster 2024-03-26T03:51:46.500931+0000 mon.a (mon.0) 1967 : cluster [DBG] osdmap e250: 8 total, 8 up, 8 in 2024-03-26T03:51:47.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:47 smithi190 bash[23392]: audit 2024-03-26T03:51:46.503600+0000 mon.a (mon.0) 1968 : audit [INF] from='client.? 172.21.15.175:0/2762811080' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi175-64406-49"}]: dispatch 2024-03-26T03:51:47.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:47 smithi190 bash[23392]: audit 2024-03-26T03:51:46.504225+0000 mon.a (mon.0) 1969 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:51:47.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:47 smithi190 bash[23392]: audit 2024-03-26T03:51:46.506604+0000 mon.a (mon.0) 1970 : audit [INF] from='client.? 172.21.15.175:0/437476285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi175-64398-40"}]: dispatch 2024-03-26T03:51:47.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:47 smithi175 bash[29284]: audit 2024-03-26T03:51:46.488230+0000 mon.a (mon.0) 1966 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:47.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:47 smithi175 bash[29284]: cluster 2024-03-26T03:51:46.500931+0000 mon.a (mon.0) 1967 : cluster [DBG] osdmap e250: 8 total, 8 up, 8 in 2024-03-26T03:51:47.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:47 smithi175 bash[29284]: audit 2024-03-26T03:51:46.503600+0000 mon.a (mon.0) 1968 : audit [INF] from='client.? 172.21.15.175:0/2762811080' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi175-64406-49"}]: dispatch 2024-03-26T03:51:47.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:47 smithi175 bash[29284]: audit 2024-03-26T03:51:46.504225+0000 mon.a (mon.0) 1969 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:51:47.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:47 smithi175 bash[29284]: audit 2024-03-26T03:51:46.506604+0000 mon.a (mon.0) 1970 : audit [INF] from='client.? 172.21.15.175:0/437476285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi175-64398-40"}]: dispatch 2024-03-26T03:51:47.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:47 smithi175 bash[21527]: audit 2024-03-26T03:51:46.488230+0000 mon.a (mon.0) 1966 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:47.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:47 smithi175 bash[21527]: cluster 2024-03-26T03:51:46.500931+0000 mon.a (mon.0) 1967 : cluster [DBG] osdmap e250: 8 total, 8 up, 8 in 2024-03-26T03:51:47.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:47 smithi175 bash[21527]: audit 2024-03-26T03:51:46.503600+0000 mon.a (mon.0) 1968 : audit [INF] from='client.? 172.21.15.175:0/2762811080' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi175-64406-49"}]: dispatch 2024-03-26T03:51:47.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:47 smithi175 bash[21527]: audit 2024-03-26T03:51:46.504225+0000 mon.a (mon.0) 1969 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:51:47.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:47 smithi175 bash[21527]: audit 2024-03-26T03:51:46.506604+0000 mon.a (mon.0) 1970 : audit [INF] from='client.? 172.21.15.175:0/437476285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi175-64398-40"}]: dispatch 2024-03-26T03:51:48.767 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:48 smithi175 bash[29284]: cluster 2024-03-26T03:51:46.665870+0000 mgr.y (mgr.24431) 302 : cluster [DBG] pgmap v372: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:51:48.767 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:48 smithi175 bash[29284]: audit 2024-03-26T03:51:47.499989+0000 mon.a (mon.0) 1971 : audit [INF] from='client.? 172.21.15.175:0/2762811080' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi175-64406-49"}]': finished 2024-03-26T03:51:48.767 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:48 smithi175 bash[29284]: audit 2024-03-26T03:51:47.500155+0000 mon.a (mon.0) 1972 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-33", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:51:48.767 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:48 smithi175 bash[29284]: audit 2024-03-26T03:51:47.500340+0000 mon.a (mon.0) 1973 : audit [INF] from='client.? 172.21.15.175:0/437476285' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi175-64398-40"}]': finished 2024-03-26T03:51:48.767 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:48 smithi175 bash[29284]: cluster 2024-03-26T03:51:47.510943+0000 mon.a (mon.0) 1974 : cluster [DBG] osdmap e251: 8 total, 8 up, 8 in 2024-03-26T03:51:48.767 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:48 smithi175 bash[29284]: audit 2024-03-26T03:51:47.512672+0000 mon.a (mon.0) 1975 : audit [INF] from='client.? 172.21.15.175:0/2762811080' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi175-64406-49"}]: dispatch 2024-03-26T03:51:48.767 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:48 smithi175 bash[29284]: audit 2024-03-26T03:51:47.513429+0000 mon.a (mon.0) 1976 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-33"}]: dispatch 2024-03-26T03:51:48.768 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:48 smithi175 bash[29284]: audit 2024-03-26T03:51:47.513814+0000 mon.a (mon.0) 1977 : audit [INF] from='client.? 172.21.15.175:0/437476285' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi175-64398-40"}]: dispatch 2024-03-26T03:51:48.768 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:48 smithi175 bash[21527]: cluster 2024-03-26T03:51:46.665870+0000 mgr.y (mgr.24431) 302 : cluster [DBG] pgmap v372: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:51:48.768 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:48 smithi175 bash[21527]: audit 2024-03-26T03:51:47.499989+0000 mon.a (mon.0) 1971 : audit [INF] from='client.? 172.21.15.175:0/2762811080' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi175-64406-49"}]': finished 2024-03-26T03:51:48.768 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:48 smithi175 bash[21527]: audit 2024-03-26T03:51:47.500155+0000 mon.a (mon.0) 1972 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-33", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:51:48.768 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:48 smithi175 bash[21527]: audit 2024-03-26T03:51:47.500340+0000 mon.a (mon.0) 1973 : audit [INF] from='client.? 172.21.15.175:0/437476285' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi175-64398-40"}]': finished 2024-03-26T03:51:48.768 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:48 smithi175 bash[21527]: cluster 2024-03-26T03:51:47.510943+0000 mon.a (mon.0) 1974 : cluster [DBG] osdmap e251: 8 total, 8 up, 8 in 2024-03-26T03:51:48.768 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:48 smithi175 bash[21527]: audit 2024-03-26T03:51:47.512672+0000 mon.a (mon.0) 1975 : audit [INF] from='client.? 172.21.15.175:0/2762811080' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi175-64406-49"}]: dispatch 2024-03-26T03:51:48.768 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:48 smithi175 bash[21527]: audit 2024-03-26T03:51:47.513429+0000 mon.a (mon.0) 1976 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-33"}]: dispatch 2024-03-26T03:51:48.768 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:48 smithi175 bash[21527]: audit 2024-03-26T03:51:47.513814+0000 mon.a (mon.0) 1977 : audit [INF] from='client.? 172.21.15.175:0/437476285' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi175-64398-40"}]: dispatch 2024-03-26T03:51:48.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:48 smithi190 bash[23392]: cluster 2024-03-26T03:51:46.665870+0000 mgr.y (mgr.24431) 302 : cluster [DBG] pgmap v372: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:51:48.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:48 smithi190 bash[23392]: audit 2024-03-26T03:51:47.499989+0000 mon.a (mon.0) 1971 : audit [INF] from='client.? 172.21.15.175:0/2762811080' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi175-64406-49"}]': finished 2024-03-26T03:51:48.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:48 smithi190 bash[23392]: audit 2024-03-26T03:51:47.500155+0000 mon.a (mon.0) 1972 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-33", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:51:48.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:48 smithi190 bash[23392]: audit 2024-03-26T03:51:47.500340+0000 mon.a (mon.0) 1973 : audit [INF] from='client.? 172.21.15.175:0/437476285' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi175-64398-40"}]': finished 2024-03-26T03:51:48.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:48 smithi190 bash[23392]: cluster 2024-03-26T03:51:47.510943+0000 mon.a (mon.0) 1974 : cluster [DBG] osdmap e251: 8 total, 8 up, 8 in 2024-03-26T03:51:48.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:48 smithi190 bash[23392]: audit 2024-03-26T03:51:47.512672+0000 mon.a (mon.0) 1975 : audit [INF] from='client.? 172.21.15.175:0/2762811080' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi175-64406-49"}]: dispatch 2024-03-26T03:51:48.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:48 smithi190 bash[23392]: audit 2024-03-26T03:51:47.513429+0000 mon.a (mon.0) 1976 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-33"}]: dispatch 2024-03-26T03:51:48.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:48 smithi190 bash[23392]: audit 2024-03-26T03:51:47.513814+0000 mon.a (mon.0) 1977 : audit [INF] from='client.? 172.21.15.175:0/437476285' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi175-64398-40"}]: dispatch 2024-03-26T03:51:49.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:51:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:51:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:51:49.858 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:49 smithi190 bash[23392]: audit 2024-03-26T03:51:48.514271+0000 mon.a (mon.0) 1978 : audit [INF] from='client.? 172.21.15.175:0/2762811080' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi175-64406-49"}]': finished 2024-03-26T03:51:49.858 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:49 smithi190 bash[23392]: audit 2024-03-26T03:51:48.514707+0000 mon.a (mon.0) 1979 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-33"}]': finished 2024-03-26T03:51:49.858 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:49 smithi190 bash[23392]: audit 2024-03-26T03:51:48.514962+0000 mon.a (mon.0) 1980 : audit [INF] from='client.? 172.21.15.175:0/437476285' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi175-64398-40"}]': finished 2024-03-26T03:51:49.858 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:49 smithi190 bash[23392]: cluster 2024-03-26T03:51:48.523222+0000 mon.a (mon.0) 1981 : cluster [DBG] osdmap e252: 8 total, 8 up, 8 in 2024-03-26T03:51:49.858 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:49 smithi190 bash[23392]: audit 2024-03-26T03:51:48.524518+0000 mon.a (mon.0) 1982 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-33", "mode": "writeback"}]: dispatch 2024-03-26T03:51:49.859 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:49 smithi190 bash[23392]: audit 2024-03-26T03:51:48.571892+0000 mon.c (mon.2) 414 : audit [INF] from='client.? 172.21.15.175:0/188292743' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi175-64398-41"}]: dispatch 2024-03-26T03:51:49.859 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:49 smithi190 bash[23392]: audit 2024-03-26T03:51:48.572239+0000 mon.a (mon.0) 1983 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi175-64398-41"}]: dispatch 2024-03-26T03:51:49.859 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:49 smithi190 bash[23392]: audit 2024-03-26T03:51:48.573342+0000 mon.c (mon.2) 415 : audit [INF] from='client.? 172.21.15.175:0/188292743' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi175-64398-41"}]: dispatch 2024-03-26T03:51:49.859 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:49 smithi190 bash[23392]: audit 2024-03-26T03:51:48.573717+0000 mon.a (mon.0) 1984 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi175-64398-41"}]: dispatch 2024-03-26T03:51:49.859 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:49 smithi190 bash[23392]: audit 2024-03-26T03:51:48.574465+0000 mon.c (mon.2) 416 : audit [INF] from='client.? 172.21.15.175:0/188292743' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi175-64398-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:49.859 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:49 smithi190 bash[23392]: audit 2024-03-26T03:51:48.574867+0000 mon.a (mon.0) 1985 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi175-64398-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:49.859 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:49 smithi190 bash[23392]: audit 2024-03-26T03:51:48.576649+0000 mon.a (mon.0) 1986 : audit [INF] from='client.? 172.21.15.175:0/3739364783' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi175-64406-50"}]: dispatch 2024-03-26T03:51:49.859 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:49 smithi190 bash[23392]: audit 2024-03-26T03:51:48.577568+0000 mon.a (mon.0) 1987 : audit [INF] from='client.? 172.21.15.175:0/3739364783' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi175-64406-50"}]: dispatch 2024-03-26T03:51:49.859 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:49 smithi190 bash[23392]: audit 2024-03-26T03:51:48.577980+0000 mon.a (mon.0) 1988 : audit [INF] from='client.? 172.21.15.175:0/3739364783' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi175-64406-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:49.859 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:49 smithi190 bash[23392]: cluster 2024-03-26T03:51:48.666768+0000 mgr.y (mgr.24431) 303 : cluster [DBG] pgmap v375: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:51:49.859 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:49 smithi190 bash[23392]: audit 2024-03-26T03:51:49.460362+0000 mon.a (mon.0) 1989 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:51:49.859 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:49 smithi190 bash[23392]: audit 2024-03-26T03:51:49.463161+0000 mon.c (mon.2) 417 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:51:49.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[21527]: audit 2024-03-26T03:51:48.514271+0000 mon.a (mon.0) 1978 : audit [INF] from='client.? 172.21.15.175:0/2762811080' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi175-64406-49"}]': finished 2024-03-26T03:51:49.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[21527]: audit 2024-03-26T03:51:48.514707+0000 mon.a (mon.0) 1979 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-33"}]': finished 2024-03-26T03:51:49.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[21527]: audit 2024-03-26T03:51:48.514962+0000 mon.a (mon.0) 1980 : audit [INF] from='client.? 172.21.15.175:0/437476285' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi175-64398-40"}]': finished 2024-03-26T03:51:49.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[21527]: cluster 2024-03-26T03:51:48.523222+0000 mon.a (mon.0) 1981 : cluster [DBG] osdmap e252: 8 total, 8 up, 8 in 2024-03-26T03:51:49.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[21527]: audit 2024-03-26T03:51:48.524518+0000 mon.a (mon.0) 1982 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-33", "mode": "writeback"}]: dispatch 2024-03-26T03:51:49.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[21527]: audit 2024-03-26T03:51:48.571892+0000 mon.c (mon.2) 414 : audit [INF] from='client.? 172.21.15.175:0/188292743' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi175-64398-41"}]: dispatch 2024-03-26T03:51:49.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[21527]: audit 2024-03-26T03:51:48.572239+0000 mon.a (mon.0) 1983 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi175-64398-41"}]: dispatch 2024-03-26T03:51:49.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[21527]: audit 2024-03-26T03:51:48.573342+0000 mon.c (mon.2) 415 : audit [INF] from='client.? 172.21.15.175:0/188292743' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi175-64398-41"}]: dispatch 2024-03-26T03:51:49.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[21527]: audit 2024-03-26T03:51:48.573717+0000 mon.a (mon.0) 1984 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi175-64398-41"}]: dispatch 2024-03-26T03:51:49.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[21527]: audit 2024-03-26T03:51:48.574465+0000 mon.c (mon.2) 416 : audit [INF] from='client.? 172.21.15.175:0/188292743' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi175-64398-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:49.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[21527]: audit 2024-03-26T03:51:48.574867+0000 mon.a (mon.0) 1985 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi175-64398-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:49.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[21527]: audit 2024-03-26T03:51:48.576649+0000 mon.a (mon.0) 1986 : audit [INF] from='client.? 172.21.15.175:0/3739364783' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi175-64406-50"}]: dispatch 2024-03-26T03:51:49.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[21527]: audit 2024-03-26T03:51:48.577568+0000 mon.a (mon.0) 1987 : audit [INF] from='client.? 172.21.15.175:0/3739364783' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi175-64406-50"}]: dispatch 2024-03-26T03:51:49.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[21527]: audit 2024-03-26T03:51:48.577980+0000 mon.a (mon.0) 1988 : audit [INF] from='client.? 172.21.15.175:0/3739364783' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi175-64406-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:49.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[29284]: audit 2024-03-26T03:51:48.514271+0000 mon.a (mon.0) 1978 : audit [INF] from='client.? 172.21.15.175:0/2762811080' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi175-64406-49"}]': finished 2024-03-26T03:51:49.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[29284]: audit 2024-03-26T03:51:48.514707+0000 mon.a (mon.0) 1979 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-33"}]': finished 2024-03-26T03:51:49.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[29284]: audit 2024-03-26T03:51:48.514962+0000 mon.a (mon.0) 1980 : audit [INF] from='client.? 172.21.15.175:0/437476285' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi175-64398-40"}]': finished 2024-03-26T03:51:49.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[29284]: cluster 2024-03-26T03:51:48.523222+0000 mon.a (mon.0) 1981 : cluster [DBG] osdmap e252: 8 total, 8 up, 8 in 2024-03-26T03:51:49.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[29284]: audit 2024-03-26T03:51:48.524518+0000 mon.a (mon.0) 1982 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-33", "mode": "writeback"}]: dispatch 2024-03-26T03:51:49.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[29284]: audit 2024-03-26T03:51:48.571892+0000 mon.c (mon.2) 414 : audit [INF] from='client.? 172.21.15.175:0/188292743' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi175-64398-41"}]: dispatch 2024-03-26T03:51:49.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[29284]: audit 2024-03-26T03:51:48.572239+0000 mon.a (mon.0) 1983 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi175-64398-41"}]: dispatch 2024-03-26T03:51:49.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[29284]: audit 2024-03-26T03:51:48.573342+0000 mon.c (mon.2) 415 : audit [INF] from='client.? 172.21.15.175:0/188292743' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi175-64398-41"}]: dispatch 2024-03-26T03:51:49.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[29284]: audit 2024-03-26T03:51:48.573717+0000 mon.a (mon.0) 1984 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi175-64398-41"}]: dispatch 2024-03-26T03:51:49.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[29284]: audit 2024-03-26T03:51:48.574465+0000 mon.c (mon.2) 416 : audit [INF] from='client.? 172.21.15.175:0/188292743' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi175-64398-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:49.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[29284]: audit 2024-03-26T03:51:48.574867+0000 mon.a (mon.0) 1985 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi175-64398-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:49.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[29284]: audit 2024-03-26T03:51:48.576649+0000 mon.a (mon.0) 1986 : audit [INF] from='client.? 172.21.15.175:0/3739364783' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi175-64406-50"}]: dispatch 2024-03-26T03:51:49.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[29284]: audit 2024-03-26T03:51:48.577568+0000 mon.a (mon.0) 1987 : audit [INF] from='client.? 172.21.15.175:0/3739364783' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi175-64406-50"}]: dispatch 2024-03-26T03:51:49.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[29284]: audit 2024-03-26T03:51:48.577980+0000 mon.a (mon.0) 1988 : audit [INF] from='client.? 172.21.15.175:0/3739364783' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi175-64406-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:49.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[29284]: cluster 2024-03-26T03:51:48.666768+0000 mgr.y (mgr.24431) 303 : cluster [DBG] pgmap v375: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:51:49.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[29284]: audit 2024-03-26T03:51:49.460362+0000 mon.a (mon.0) 1989 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:51:49.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[29284]: audit 2024-03-26T03:51:49.463161+0000 mon.c (mon.2) 417 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:51:49.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[21527]: cluster 2024-03-26T03:51:48.666768+0000 mgr.y (mgr.24431) 303 : cluster [DBG] pgmap v375: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:51:49.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[21527]: audit 2024-03-26T03:51:49.460362+0000 mon.a (mon.0) 1989 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:51:49.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:49 smithi175 bash[21527]: audit 2024-03-26T03:51:49.463161+0000 mon.c (mon.2) 417 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:51:50.240 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:51:49 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:51:50.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:50 smithi190 bash[23392]: cluster 2024-03-26T03:51:49.515456+0000 mon.a (mon.0) 1990 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:51:50.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:50 smithi190 bash[23392]: audit 2024-03-26T03:51:49.525705+0000 mon.a (mon.0) 1991 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-33", "mode": "writeback"}]': finished 2024-03-26T03:51:50.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:50 smithi190 bash[23392]: audit 2024-03-26T03:51:49.525844+0000 mon.a (mon.0) 1992 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi175-64398-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:50.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:50 smithi190 bash[23392]: audit 2024-03-26T03:51:49.526003+0000 mon.a (mon.0) 1993 : audit [INF] from='client.? 172.21.15.175:0/3739364783' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi175-64406-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:50.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:50 smithi190 bash[23392]: cluster 2024-03-26T03:51:49.537580+0000 mon.a (mon.0) 1994 : cluster [DBG] osdmap e253: 8 total, 8 up, 8 in 2024-03-26T03:51:50.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:50 smithi190 bash[23392]: audit 2024-03-26T03:51:49.542209+0000 mon.a (mon.0) 1995 : audit [INF] from='client.? 172.21.15.175:0/3739364783' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi175-64406-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi175-64406-50"}]: dispatch 2024-03-26T03:51:50.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:50 smithi190 bash[23392]: audit 2024-03-26T03:51:49.548677+0000 mon.c (mon.2) 418 : audit [INF] from='client.? 172.21.15.175:0/188292743' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi175-64398-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi175-64398-41"}]: dispatch 2024-03-26T03:51:50.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:50 smithi190 bash[23392]: audit 2024-03-26T03:51:49.549493+0000 mon.a (mon.0) 1996 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi175-64398-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi175-64398-41"}]: dispatch 2024-03-26T03:51:50.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:50 smithi190 bash[23392]: audit 2024-03-26T03:51:49.804956+0000 mon.a (mon.0) 1997 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:51:50.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:50 smithi190 bash[23392]: audit 2024-03-26T03:51:49.859005+0000 mgr.y (mgr.24431) 304 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:51:50.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:50 smithi175 bash[21527]: cluster 2024-03-26T03:51:49.515456+0000 mon.a (mon.0) 1990 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:51:50.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:50 smithi175 bash[21527]: audit 2024-03-26T03:51:49.525705+0000 mon.a (mon.0) 1991 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-33", "mode": "writeback"}]': finished 2024-03-26T03:51:50.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:50 smithi175 bash[21527]: audit 2024-03-26T03:51:49.525844+0000 mon.a (mon.0) 1992 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi175-64398-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:50 smithi175 bash[21527]: audit 2024-03-26T03:51:49.526003+0000 mon.a (mon.0) 1993 : audit [INF] from='client.? 172.21.15.175:0/3739364783' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi175-64406-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:50 smithi175 bash[21527]: cluster 2024-03-26T03:51:49.537580+0000 mon.a (mon.0) 1994 : cluster [DBG] osdmap e253: 8 total, 8 up, 8 in 2024-03-26T03:51:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:50 smithi175 bash[21527]: audit 2024-03-26T03:51:49.542209+0000 mon.a (mon.0) 1995 : audit [INF] from='client.? 172.21.15.175:0/3739364783' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi175-64406-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi175-64406-50"}]: dispatch 2024-03-26T03:51:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:50 smithi175 bash[21527]: audit 2024-03-26T03:51:49.548677+0000 mon.c (mon.2) 418 : audit [INF] from='client.? 172.21.15.175:0/188292743' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi175-64398-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi175-64398-41"}]: dispatch 2024-03-26T03:51:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:50 smithi175 bash[21527]: audit 2024-03-26T03:51:49.549493+0000 mon.a (mon.0) 1996 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi175-64398-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi175-64398-41"}]: dispatch 2024-03-26T03:51:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:50 smithi175 bash[21527]: audit 2024-03-26T03:51:49.804956+0000 mon.a (mon.0) 1997 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:51:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:50 smithi175 bash[21527]: audit 2024-03-26T03:51:49.859005+0000 mgr.y (mgr.24431) 304 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:51:50.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:50 smithi175 bash[29284]: cluster 2024-03-26T03:51:49.515456+0000 mon.a (mon.0) 1990 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:51:50.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:50 smithi175 bash[29284]: audit 2024-03-26T03:51:49.525705+0000 mon.a (mon.0) 1991 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-33", "mode": "writeback"}]': finished 2024-03-26T03:51:50.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:50 smithi175 bash[29284]: audit 2024-03-26T03:51:49.525844+0000 mon.a (mon.0) 1992 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi175-64398-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:50.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:50 smithi175 bash[29284]: audit 2024-03-26T03:51:49.526003+0000 mon.a (mon.0) 1993 : audit [INF] from='client.? 172.21.15.175:0/3739364783' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi175-64406-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:50.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:50 smithi175 bash[29284]: cluster 2024-03-26T03:51:49.537580+0000 mon.a (mon.0) 1994 : cluster [DBG] osdmap e253: 8 total, 8 up, 8 in 2024-03-26T03:51:50.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:50 smithi175 bash[29284]: audit 2024-03-26T03:51:49.542209+0000 mon.a (mon.0) 1995 : audit [INF] from='client.? 172.21.15.175:0/3739364783' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi175-64406-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi175-64406-50"}]: dispatch 2024-03-26T03:51:50.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:50 smithi175 bash[29284]: audit 2024-03-26T03:51:49.548677+0000 mon.c (mon.2) 418 : audit [INF] from='client.? 172.21.15.175:0/188292743' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi175-64398-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi175-64398-41"}]: dispatch 2024-03-26T03:51:50.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:50 smithi175 bash[29284]: audit 2024-03-26T03:51:49.549493+0000 mon.a (mon.0) 1996 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi175-64398-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi175-64398-41"}]: dispatch 2024-03-26T03:51:50.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:50 smithi175 bash[29284]: audit 2024-03-26T03:51:49.804956+0000 mon.a (mon.0) 1997 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:51:50.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:50 smithi175 bash[29284]: audit 2024-03-26T03:51:49.859005+0000 mgr.y (mgr.24431) 304 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:51:51.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:51 smithi190 bash[23392]: audit 2024-03-26T03:51:50.538872+0000 mon.a (mon.0) 1998 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:51:52.029 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:51 smithi190 bash[23392]: cluster 2024-03-26T03:51:50.571166+0000 mon.a (mon.0) 1999 : cluster [DBG] osdmap e254: 8 total, 8 up, 8 in 2024-03-26T03:51:52.029 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:51 smithi190 bash[23392]: audit 2024-03-26T03:51:50.571833+0000 mon.a (mon.0) 2000 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-33"}]: dispatch 2024-03-26T03:51:52.029 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:51 smithi190 bash[23392]: cluster 2024-03-26T03:51:50.667625+0000 mgr.y (mgr.24431) 305 : cluster [DBG] pgmap v378: 292 pgs: 292 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 3 op/s 2024-03-26T03:51:52.030 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:51 smithi190 bash[23392]: cluster 2024-03-26T03:51:51.450778+0000 mon.a (mon.0) 2001 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:51:52.030 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:51 smithi175 bash[21527]: audit 2024-03-26T03:51:50.538872+0000 mon.a (mon.0) 1998 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:51:52.030 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:51 smithi175 bash[21527]: cluster 2024-03-26T03:51:50.571166+0000 mon.a (mon.0) 1999 : cluster [DBG] osdmap e254: 8 total, 8 up, 8 in 2024-03-26T03:51:52.030 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:51 smithi175 bash[21527]: audit 2024-03-26T03:51:50.571833+0000 mon.a (mon.0) 2000 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-33"}]: dispatch 2024-03-26T03:51:52.030 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:51 smithi175 bash[21527]: cluster 2024-03-26T03:51:50.667625+0000 mgr.y (mgr.24431) 305 : cluster [DBG] pgmap v378: 292 pgs: 292 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 3 op/s 2024-03-26T03:51:52.030 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:51 smithi175 bash[21527]: cluster 2024-03-26T03:51:51.450778+0000 mon.a (mon.0) 2001 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:51:52.031 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:51 smithi175 bash[29284]: audit 2024-03-26T03:51:50.538872+0000 mon.a (mon.0) 1998 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:51:52.031 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:51 smithi175 bash[29284]: cluster 2024-03-26T03:51:50.571166+0000 mon.a (mon.0) 1999 : cluster [DBG] osdmap e254: 8 total, 8 up, 8 in 2024-03-26T03:51:52.031 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:51 smithi175 bash[29284]: audit 2024-03-26T03:51:50.571833+0000 mon.a (mon.0) 2000 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-33"}]: dispatch 2024-03-26T03:51:52.031 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:51 smithi175 bash[29284]: cluster 2024-03-26T03:51:50.667625+0000 mgr.y (mgr.24431) 305 : cluster [DBG] pgmap v378: 292 pgs: 292 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 3 op/s 2024-03-26T03:51:52.031 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:51 smithi175 bash[29284]: cluster 2024-03-26T03:51:51.450778+0000 mon.a (mon.0) 2001 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:51:52.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:52 smithi190 bash[23392]: cluster 2024-03-26T03:51:51.540672+0000 mon.a (mon.0) 2002 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:51:52.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:52 smithi190 bash[23392]: audit 2024-03-26T03:51:51.552564+0000 mon.a (mon.0) 2003 : audit [INF] from='client.? 172.21.15.175:0/3739364783' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi175-64406-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi175-64406-50"}]': finished 2024-03-26T03:51:52.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:52 smithi190 bash[23392]: audit 2024-03-26T03:51:51.552798+0000 mon.a (mon.0) 2004 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi175-64398-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi175-64398-41"}]': finished 2024-03-26T03:51:52.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:52 smithi190 bash[23392]: audit 2024-03-26T03:51:51.553016+0000 mon.a (mon.0) 2005 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-33"}]': finished 2024-03-26T03:51:52.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:52 smithi190 bash[23392]: cluster 2024-03-26T03:51:51.563907+0000 mon.a (mon.0) 2006 : cluster [DBG] osdmap e255: 8 total, 8 up, 8 in 2024-03-26T03:51:52.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:52 smithi190 bash[23392]: audit 2024-03-26T03:51:51.983812+0000 mon.c (mon.2) 419 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:51:52.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:52 smithi175 bash[21527]: cluster 2024-03-26T03:51:51.540672+0000 mon.a (mon.0) 2002 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:51:52.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:52 smithi175 bash[21527]: audit 2024-03-26T03:51:51.552564+0000 mon.a (mon.0) 2003 : audit [INF] from='client.? 172.21.15.175:0/3739364783' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi175-64406-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi175-64406-50"}]': finished 2024-03-26T03:51:52.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:52 smithi175 bash[21527]: audit 2024-03-26T03:51:51.552798+0000 mon.a (mon.0) 2004 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi175-64398-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi175-64398-41"}]': finished 2024-03-26T03:51:52.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:52 smithi175 bash[21527]: audit 2024-03-26T03:51:51.553016+0000 mon.a (mon.0) 2005 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-33"}]': finished 2024-03-26T03:51:52.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:52 smithi175 bash[21527]: cluster 2024-03-26T03:51:51.563907+0000 mon.a (mon.0) 2006 : cluster [DBG] osdmap e255: 8 total, 8 up, 8 in 2024-03-26T03:51:52.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:52 smithi175 bash[21527]: audit 2024-03-26T03:51:51.983812+0000 mon.c (mon.2) 419 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:51:52.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:52 smithi175 bash[29284]: cluster 2024-03-26T03:51:51.540672+0000 mon.a (mon.0) 2002 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:51:52.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:52 smithi175 bash[29284]: audit 2024-03-26T03:51:51.552564+0000 mon.a (mon.0) 2003 : audit [INF] from='client.? 172.21.15.175:0/3739364783' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi175-64406-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi175-64406-50"}]': finished 2024-03-26T03:51:52.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:52 smithi175 bash[29284]: audit 2024-03-26T03:51:51.552798+0000 mon.a (mon.0) 2004 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi175-64398-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi175-64398-41"}]': finished 2024-03-26T03:51:52.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:52 smithi175 bash[29284]: audit 2024-03-26T03:51:51.553016+0000 mon.a (mon.0) 2005 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-33"}]': finished 2024-03-26T03:51:52.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:52 smithi175 bash[29284]: cluster 2024-03-26T03:51:51.563907+0000 mon.a (mon.0) 2006 : cluster [DBG] osdmap e255: 8 total, 8 up, 8 in 2024-03-26T03:51:52.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:52 smithi175 bash[29284]: audit 2024-03-26T03:51:51.983812+0000 mon.c (mon.2) 419 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:51:53.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:53 smithi190 bash[23392]: cluster 2024-03-26T03:51:52.571175+0000 mon.a (mon.0) 2007 : cluster [DBG] osdmap e256: 8 total, 8 up, 8 in 2024-03-26T03:51:53.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:53 smithi190 bash[23392]: cluster 2024-03-26T03:51:52.668171+0000 mgr.y (mgr.24431) 306 : cluster [DBG] pgmap v381: 276 pgs: 16 unknown, 260 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:51:53.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:53 smithi190 bash[23392]: audit 2024-03-26T03:51:52.719178+0000 mon.c (mon.2) 420 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:51:53.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:53 smithi190 bash[23392]: audit 2024-03-26T03:51:52.720634+0000 mon.c (mon.2) 421 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:51:53.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:53 smithi190 bash[23392]: audit 2024-03-26T03:51:52.730762+0000 mon.a (mon.0) 2008 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:51:53.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:53 smithi175 bash[21527]: cluster 2024-03-26T03:51:52.571175+0000 mon.a (mon.0) 2007 : cluster [DBG] osdmap e256: 8 total, 8 up, 8 in 2024-03-26T03:51:53.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:53 smithi175 bash[21527]: cluster 2024-03-26T03:51:52.668171+0000 mgr.y (mgr.24431) 306 : cluster [DBG] pgmap v381: 276 pgs: 16 unknown, 260 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:51:53.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:53 smithi175 bash[21527]: audit 2024-03-26T03:51:52.719178+0000 mon.c (mon.2) 420 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:51:53.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:53 smithi175 bash[21527]: audit 2024-03-26T03:51:52.720634+0000 mon.c (mon.2) 421 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:51:53.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:53 smithi175 bash[21527]: audit 2024-03-26T03:51:52.730762+0000 mon.a (mon.0) 2008 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:51:53.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:53 smithi175 bash[29284]: cluster 2024-03-26T03:51:52.571175+0000 mon.a (mon.0) 2007 : cluster [DBG] osdmap e256: 8 total, 8 up, 8 in 2024-03-26T03:51:53.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:53 smithi175 bash[29284]: cluster 2024-03-26T03:51:52.668171+0000 mgr.y (mgr.24431) 306 : cluster [DBG] pgmap v381: 276 pgs: 16 unknown, 260 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:51:53.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:53 smithi175 bash[29284]: audit 2024-03-26T03:51:52.719178+0000 mon.c (mon.2) 420 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:51:53.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:53 smithi175 bash[29284]: audit 2024-03-26T03:51:52.720634+0000 mon.c (mon.2) 421 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:51:53.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:53 smithi175 bash[29284]: audit 2024-03-26T03:51:52.730762+0000 mon.a (mon.0) 2008 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:51:54.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:54 smithi190 bash[23392]: cluster 2024-03-26T03:51:53.586582+0000 mon.a (mon.0) 2009 : cluster [DBG] osdmap e257: 8 total, 8 up, 8 in 2024-03-26T03:51:54.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:54 smithi190 bash[23392]: audit 2024-03-26T03:51:53.589432+0000 mon.c (mon.2) 422 : audit [INF] from='client.? 172.21.15.175:0/188292743' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi175-64398-41"}]: dispatch 2024-03-26T03:51:54.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:54 smithi190 bash[23392]: audit 2024-03-26T03:51:53.591935+0000 mon.a (mon.0) 2010 : audit [INF] from='client.? 172.21.15.175:0/3739364783' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi175-64406-50"}]: dispatch 2024-03-26T03:51:54.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:54 smithi190 bash[23392]: audit 2024-03-26T03:51:53.592537+0000 mon.a (mon.0) 2011 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi175-64398-41"}]: dispatch 2024-03-26T03:51:54.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:54 smithi190 bash[23392]: audit 2024-03-26T03:51:53.593035+0000 mon.a (mon.0) 2012 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:54.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:54 smithi175 bash[21527]: cluster 2024-03-26T03:51:53.586582+0000 mon.a (mon.0) 2009 : cluster [DBG] osdmap e257: 8 total, 8 up, 8 in 2024-03-26T03:51:54.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:54 smithi175 bash[21527]: audit 2024-03-26T03:51:53.589432+0000 mon.c (mon.2) 422 : audit [INF] from='client.? 172.21.15.175:0/188292743' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi175-64398-41"}]: dispatch 2024-03-26T03:51:54.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:54 smithi175 bash[21527]: audit 2024-03-26T03:51:53.591935+0000 mon.a (mon.0) 2010 : audit [INF] from='client.? 172.21.15.175:0/3739364783' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi175-64406-50"}]: dispatch 2024-03-26T03:51:54.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:54 smithi175 bash[21527]: audit 2024-03-26T03:51:53.592537+0000 mon.a (mon.0) 2011 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi175-64398-41"}]: dispatch 2024-03-26T03:51:54.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:54 smithi175 bash[21527]: audit 2024-03-26T03:51:53.593035+0000 mon.a (mon.0) 2012 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:54.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:54 smithi175 bash[29284]: cluster 2024-03-26T03:51:53.586582+0000 mon.a (mon.0) 2009 : cluster [DBG] osdmap e257: 8 total, 8 up, 8 in 2024-03-26T03:51:54.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:54 smithi175 bash[29284]: audit 2024-03-26T03:51:53.589432+0000 mon.c (mon.2) 422 : audit [INF] from='client.? 172.21.15.175:0/188292743' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi175-64398-41"}]: dispatch 2024-03-26T03:51:54.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:54 smithi175 bash[29284]: audit 2024-03-26T03:51:53.591935+0000 mon.a (mon.0) 2010 : audit [INF] from='client.? 172.21.15.175:0/3739364783' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi175-64406-50"}]: dispatch 2024-03-26T03:51:54.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:54 smithi175 bash[29284]: audit 2024-03-26T03:51:53.592537+0000 mon.a (mon.0) 2011 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi175-64398-41"}]: dispatch 2024-03-26T03:51:54.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:54 smithi175 bash[29284]: audit 2024-03-26T03:51:53.593035+0000 mon.a (mon.0) 2012 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:55.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:55 smithi190 bash[23392]: audit 2024-03-26T03:51:54.589849+0000 mon.a (mon.0) 2013 : audit [INF] from='client.? 172.21.15.175:0/3739364783' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi175-64406-50"}]': finished 2024-03-26T03:51:55.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:55 smithi190 bash[23392]: audit 2024-03-26T03:51:54.590027+0000 mon.a (mon.0) 2014 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi175-64398-41"}]': finished 2024-03-26T03:51:55.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:55 smithi190 bash[23392]: audit 2024-03-26T03:51:54.590239+0000 mon.a (mon.0) 2015 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:55.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:55 smithi190 bash[23392]: cluster 2024-03-26T03:51:54.597199+0000 mon.a (mon.0) 2016 : cluster [DBG] osdmap e258: 8 total, 8 up, 8 in 2024-03-26T03:51:55.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:55 smithi190 bash[23392]: audit 2024-03-26T03:51:54.599228+0000 mon.a (mon.0) 2017 : audit [INF] from='client.? 172.21.15.175:0/3739364783' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi175-64406-50"}]: dispatch 2024-03-26T03:51:55.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:55 smithi190 bash[23392]: audit 2024-03-26T03:51:54.603684+0000 mon.a (mon.0) 2018 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:51:55.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:55 smithi190 bash[23392]: audit 2024-03-26T03:51:54.604439+0000 mon.c (mon.2) 423 : audit [INF] from='client.? 172.21.15.175:0/188292743' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi175-64398-41"}]: dispatch 2024-03-26T03:51:55.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:55 smithi190 bash[23392]: audit 2024-03-26T03:51:54.605655+0000 mon.a (mon.0) 2019 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi175-64398-41"}]: dispatch 2024-03-26T03:51:55.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:55 smithi190 bash[23392]: cluster 2024-03-26T03:51:54.669688+0000 mgr.y (mgr.24431) 307 : cluster [DBG] pgmap v384: 292 pgs: 2 active, 8 creating+activating, 13 unknown, 269 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:51:55.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:55 smithi175 bash[21527]: audit 2024-03-26T03:51:54.589849+0000 mon.a (mon.0) 2013 : audit [INF] from='client.? 172.21.15.175:0/3739364783' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi175-64406-50"}]': finished 2024-03-26T03:51:55.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:55 smithi175 bash[21527]: audit 2024-03-26T03:51:54.590027+0000 mon.a (mon.0) 2014 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi175-64398-41"}]': finished 2024-03-26T03:51:55.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:55 smithi175 bash[21527]: audit 2024-03-26T03:51:54.590239+0000 mon.a (mon.0) 2015 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:55.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:55 smithi175 bash[21527]: cluster 2024-03-26T03:51:54.597199+0000 mon.a (mon.0) 2016 : cluster [DBG] osdmap e258: 8 total, 8 up, 8 in 2024-03-26T03:51:55.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:55 smithi175 bash[21527]: audit 2024-03-26T03:51:54.599228+0000 mon.a (mon.0) 2017 : audit [INF] from='client.? 172.21.15.175:0/3739364783' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi175-64406-50"}]: dispatch 2024-03-26T03:51:55.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:55 smithi175 bash[21527]: audit 2024-03-26T03:51:54.603684+0000 mon.a (mon.0) 2018 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:51:55.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:55 smithi175 bash[21527]: audit 2024-03-26T03:51:54.604439+0000 mon.c (mon.2) 423 : audit [INF] from='client.? 172.21.15.175:0/188292743' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi175-64398-41"}]: dispatch 2024-03-26T03:51:55.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:55 smithi175 bash[21527]: audit 2024-03-26T03:51:54.605655+0000 mon.a (mon.0) 2019 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi175-64398-41"}]: dispatch 2024-03-26T03:51:55.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:55 smithi175 bash[21527]: cluster 2024-03-26T03:51:54.669688+0000 mgr.y (mgr.24431) 307 : cluster [DBG] pgmap v384: 292 pgs: 2 active, 8 creating+activating, 13 unknown, 269 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:51:55.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:55 smithi175 bash[29284]: audit 2024-03-26T03:51:54.589849+0000 mon.a (mon.0) 2013 : audit [INF] from='client.? 172.21.15.175:0/3739364783' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi175-64406-50"}]': finished 2024-03-26T03:51:55.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:55 smithi175 bash[29284]: audit 2024-03-26T03:51:54.590027+0000 mon.a (mon.0) 2014 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi175-64398-41"}]': finished 2024-03-26T03:51:55.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:55 smithi175 bash[29284]: audit 2024-03-26T03:51:54.590239+0000 mon.a (mon.0) 2015 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:55.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:55 smithi175 bash[29284]: cluster 2024-03-26T03:51:54.597199+0000 mon.a (mon.0) 2016 : cluster [DBG] osdmap e258: 8 total, 8 up, 8 in 2024-03-26T03:51:55.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:55 smithi175 bash[29284]: audit 2024-03-26T03:51:54.599228+0000 mon.a (mon.0) 2017 : audit [INF] from='client.? 172.21.15.175:0/3739364783' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi175-64406-50"}]: dispatch 2024-03-26T03:51:55.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:55 smithi175 bash[29284]: audit 2024-03-26T03:51:54.603684+0000 mon.a (mon.0) 2018 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:51:55.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:55 smithi175 bash[29284]: audit 2024-03-26T03:51:54.604439+0000 mon.c (mon.2) 423 : audit [INF] from='client.? 172.21.15.175:0/188292743' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi175-64398-41"}]: dispatch 2024-03-26T03:51:55.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:55 smithi175 bash[29284]: audit 2024-03-26T03:51:54.605655+0000 mon.a (mon.0) 2019 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi175-64398-41"}]: dispatch 2024-03-26T03:51:55.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:55 smithi175 bash[29284]: cluster 2024-03-26T03:51:54.669688+0000 mgr.y (mgr.24431) 307 : cluster [DBG] pgmap v384: 292 pgs: 2 active, 8 creating+activating, 13 unknown, 269 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:51:56.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:56 smithi190 bash[23392]: audit 2024-03-26T03:51:55.601890+0000 mon.a (mon.0) 2020 : audit [INF] from='client.? 172.21.15.175:0/3739364783' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi175-64406-50"}]': finished 2024-03-26T03:51:56.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:56 smithi190 bash[23392]: audit 2024-03-26T03:51:55.602055+0000 mon.a (mon.0) 2021 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-35", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:51:56.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:56 smithi190 bash[23392]: audit 2024-03-26T03:51:55.602253+0000 mon.a (mon.0) 2022 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi175-64398-41"}]': finished 2024-03-26T03:51:56.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:56 smithi190 bash[23392]: cluster 2024-03-26T03:51:55.612261+0000 mon.a (mon.0) 2023 : cluster [DBG] osdmap e259: 8 total, 8 up, 8 in 2024-03-26T03:51:56.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:56 smithi190 bash[23392]: audit 2024-03-26T03:51:55.613798+0000 mon.a (mon.0) 2024 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-35"}]: dispatch 2024-03-26T03:51:56.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:56 smithi190 bash[23392]: audit 2024-03-26T03:51:55.648108+0000 mon.c (mon.2) 424 : audit [INF] from='client.? 172.21.15.175:0/2898360639' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi175-64398-42"}]: dispatch 2024-03-26T03:51:56.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:56 smithi190 bash[23392]: audit 2024-03-26T03:51:55.648488+0000 mon.a (mon.0) 2025 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi175-64398-42"}]: dispatch 2024-03-26T03:51:56.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:56 smithi190 bash[23392]: audit 2024-03-26T03:51:55.649140+0000 mon.c (mon.2) 425 : audit [INF] from='client.? 172.21.15.175:0/2898360639' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi175-64398-42"}]: dispatch 2024-03-26T03:51:56.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:56 smithi190 bash[23392]: audit 2024-03-26T03:51:55.649520+0000 mon.a (mon.0) 2026 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi175-64398-42"}]: dispatch 2024-03-26T03:51:56.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:56 smithi190 bash[23392]: audit 2024-03-26T03:51:55.650126+0000 mon.c (mon.2) 426 : audit [INF] from='client.? 172.21.15.175:0/2898360639' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi175-64398-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:56.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:56 smithi190 bash[23392]: audit 2024-03-26T03:51:55.650416+0000 mon.a (mon.0) 2027 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi175-64398-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:56.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:56 smithi175 bash[29284]: audit 2024-03-26T03:51:55.601890+0000 mon.a (mon.0) 2020 : audit [INF] from='client.? 172.21.15.175:0/3739364783' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi175-64406-50"}]': finished 2024-03-26T03:51:56.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:56 smithi175 bash[29284]: audit 2024-03-26T03:51:55.602055+0000 mon.a (mon.0) 2021 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-35", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:51:56.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:56 smithi175 bash[29284]: audit 2024-03-26T03:51:55.602253+0000 mon.a (mon.0) 2022 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi175-64398-41"}]': finished 2024-03-26T03:51:56.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:56 smithi175 bash[29284]: cluster 2024-03-26T03:51:55.612261+0000 mon.a (mon.0) 2023 : cluster [DBG] osdmap e259: 8 total, 8 up, 8 in 2024-03-26T03:51:56.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:56 smithi175 bash[29284]: audit 2024-03-26T03:51:55.613798+0000 mon.a (mon.0) 2024 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-35"}]: dispatch 2024-03-26T03:51:56.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:56 smithi175 bash[29284]: audit 2024-03-26T03:51:55.648108+0000 mon.c (mon.2) 424 : audit [INF] from='client.? 172.21.15.175:0/2898360639' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi175-64398-42"}]: dispatch 2024-03-26T03:51:56.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:56 smithi175 bash[29284]: audit 2024-03-26T03:51:55.648488+0000 mon.a (mon.0) 2025 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi175-64398-42"}]: dispatch 2024-03-26T03:51:56.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:56 smithi175 bash[29284]: audit 2024-03-26T03:51:55.649140+0000 mon.c (mon.2) 425 : audit [INF] from='client.? 172.21.15.175:0/2898360639' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi175-64398-42"}]: dispatch 2024-03-26T03:51:56.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:56 smithi175 bash[29284]: audit 2024-03-26T03:51:55.649520+0000 mon.a (mon.0) 2026 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi175-64398-42"}]: dispatch 2024-03-26T03:51:56.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:56 smithi175 bash[29284]: audit 2024-03-26T03:51:55.650126+0000 mon.c (mon.2) 426 : audit [INF] from='client.? 172.21.15.175:0/2898360639' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi175-64398-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:56.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:56 smithi175 bash[29284]: audit 2024-03-26T03:51:55.650416+0000 mon.a (mon.0) 2027 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi175-64398-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:56.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:56 smithi175 bash[21527]: audit 2024-03-26T03:51:55.601890+0000 mon.a (mon.0) 2020 : audit [INF] from='client.? 172.21.15.175:0/3739364783' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi175-64406-50"}]': finished 2024-03-26T03:51:56.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:56 smithi175 bash[21527]: audit 2024-03-26T03:51:55.602055+0000 mon.a (mon.0) 2021 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-35", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:51:56.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:56 smithi175 bash[21527]: audit 2024-03-26T03:51:55.602253+0000 mon.a (mon.0) 2022 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi175-64398-41"}]': finished 2024-03-26T03:51:56.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:56 smithi175 bash[21527]: cluster 2024-03-26T03:51:55.612261+0000 mon.a (mon.0) 2023 : cluster [DBG] osdmap e259: 8 total, 8 up, 8 in 2024-03-26T03:51:56.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:56 smithi175 bash[21527]: audit 2024-03-26T03:51:55.613798+0000 mon.a (mon.0) 2024 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-35"}]: dispatch 2024-03-26T03:51:56.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:56 smithi175 bash[21527]: audit 2024-03-26T03:51:55.648108+0000 mon.c (mon.2) 424 : audit [INF] from='client.? 172.21.15.175:0/2898360639' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi175-64398-42"}]: dispatch 2024-03-26T03:51:56.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:56 smithi175 bash[21527]: audit 2024-03-26T03:51:55.648488+0000 mon.a (mon.0) 2025 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi175-64398-42"}]: dispatch 2024-03-26T03:51:56.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:56 smithi175 bash[21527]: audit 2024-03-26T03:51:55.649140+0000 mon.c (mon.2) 425 : audit [INF] from='client.? 172.21.15.175:0/2898360639' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi175-64398-42"}]: dispatch 2024-03-26T03:51:56.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:56 smithi175 bash[21527]: audit 2024-03-26T03:51:55.649520+0000 mon.a (mon.0) 2026 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi175-64398-42"}]: dispatch 2024-03-26T03:51:56.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:56 smithi175 bash[21527]: audit 2024-03-26T03:51:55.650126+0000 mon.c (mon.2) 426 : audit [INF] from='client.? 172.21.15.175:0/2898360639' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi175-64398-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:56.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:56 smithi175 bash[21527]: audit 2024-03-26T03:51:55.650416+0000 mon.a (mon.0) 2027 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi175-64398-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:57.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:57 smithi190 bash[23392]: audit 2024-03-26T03:51:56.605808+0000 mon.a (mon.0) 2028 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-35"}]': finished 2024-03-26T03:51:57.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:57 smithi190 bash[23392]: audit 2024-03-26T03:51:56.606040+0000 mon.a (mon.0) 2029 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi175-64398-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:57.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:57 smithi190 bash[23392]: cluster 2024-03-26T03:51:56.616310+0000 mon.a (mon.0) 2030 : cluster [DBG] osdmap e260: 8 total, 8 up, 8 in 2024-03-26T03:51:57.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:57 smithi190 bash[23392]: audit 2024-03-26T03:51:56.621290+0000 mon.c (mon.2) 427 : audit [INF] from='client.? 172.21.15.175:0/2898360639' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi175-64398-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi175-64398-42"}]: dispatch 2024-03-26T03:51:57.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:57 smithi190 bash[23392]: audit 2024-03-26T03:51:56.622616+0000 mon.a (mon.0) 2031 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-35", "mode": "writeback"}]: dispatch 2024-03-26T03:51:57.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:57 smithi190 bash[23392]: audit 2024-03-26T03:51:56.623904+0000 mon.a (mon.0) 2032 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi175-64398-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi175-64398-42"}]: dispatch 2024-03-26T03:51:57.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:57 smithi190 bash[23392]: audit 2024-03-26T03:51:56.626895+0000 mon.c (mon.2) 428 : audit [INF] from='client.? 172.21.15.175:0/2246972559' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi175-64406-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:57.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:57 smithi190 bash[23392]: audit 2024-03-26T03:51:56.641205+0000 mon.a (mon.0) 2033 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi175-64406-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:57.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:57 smithi190 bash[23392]: cluster 2024-03-26T03:51:56.670500+0000 mgr.y (mgr.24431) 308 : cluster [DBG] pgmap v387: 324 pgs: 32 unknown, 2 active, 8 creating+activating, 282 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:51:57.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:57 smithi190 bash[23392]: cluster 2024-03-26T03:51:57.607231+0000 mon.a (mon.0) 2034 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:51:57.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:57 smithi190 bash[23392]: audit 2024-03-26T03:51:57.610152+0000 mon.a (mon.0) 2035 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-35", "mode": "writeback"}]': finished 2024-03-26T03:51:57.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:57 smithi175 bash[21527]: audit 2024-03-26T03:51:56.605808+0000 mon.a (mon.0) 2028 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-35"}]': finished 2024-03-26T03:51:57.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:57 smithi175 bash[21527]: audit 2024-03-26T03:51:56.606040+0000 mon.a (mon.0) 2029 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi175-64398-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:57.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:57 smithi175 bash[21527]: cluster 2024-03-26T03:51:56.616310+0000 mon.a (mon.0) 2030 : cluster [DBG] osdmap e260: 8 total, 8 up, 8 in 2024-03-26T03:51:57.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:57 smithi175 bash[21527]: audit 2024-03-26T03:51:56.621290+0000 mon.c (mon.2) 427 : audit [INF] from='client.? 172.21.15.175:0/2898360639' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi175-64398-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi175-64398-42"}]: dispatch 2024-03-26T03:51:57.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:57 smithi175 bash[21527]: audit 2024-03-26T03:51:56.622616+0000 mon.a (mon.0) 2031 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-35", "mode": "writeback"}]: dispatch 2024-03-26T03:51:57.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:57 smithi175 bash[21527]: audit 2024-03-26T03:51:56.623904+0000 mon.a (mon.0) 2032 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi175-64398-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi175-64398-42"}]: dispatch 2024-03-26T03:51:57.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:57 smithi175 bash[21527]: audit 2024-03-26T03:51:56.626895+0000 mon.c (mon.2) 428 : audit [INF] from='client.? 172.21.15.175:0/2246972559' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi175-64406-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:57.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:57 smithi175 bash[21527]: audit 2024-03-26T03:51:56.641205+0000 mon.a (mon.0) 2033 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi175-64406-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:57.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:57 smithi175 bash[21527]: cluster 2024-03-26T03:51:56.670500+0000 mgr.y (mgr.24431) 308 : cluster [DBG] pgmap v387: 324 pgs: 32 unknown, 2 active, 8 creating+activating, 282 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:51:57.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:57 smithi175 bash[21527]: cluster 2024-03-26T03:51:57.607231+0000 mon.a (mon.0) 2034 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:51:57.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:57 smithi175 bash[21527]: audit 2024-03-26T03:51:57.610152+0000 mon.a (mon.0) 2035 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-35", "mode": "writeback"}]': finished 2024-03-26T03:51:57.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:57 smithi175 bash[29284]: audit 2024-03-26T03:51:56.605808+0000 mon.a (mon.0) 2028 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-35"}]': finished 2024-03-26T03:51:57.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:57 smithi175 bash[29284]: audit 2024-03-26T03:51:56.606040+0000 mon.a (mon.0) 2029 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi175-64398-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:57.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:57 smithi175 bash[29284]: cluster 2024-03-26T03:51:56.616310+0000 mon.a (mon.0) 2030 : cluster [DBG] osdmap e260: 8 total, 8 up, 8 in 2024-03-26T03:51:57.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:57 smithi175 bash[29284]: audit 2024-03-26T03:51:56.621290+0000 mon.c (mon.2) 427 : audit [INF] from='client.? 172.21.15.175:0/2898360639' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi175-64398-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi175-64398-42"}]: dispatch 2024-03-26T03:51:57.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:57 smithi175 bash[29284]: audit 2024-03-26T03:51:56.622616+0000 mon.a (mon.0) 2031 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-35", "mode": "writeback"}]: dispatch 2024-03-26T03:51:57.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:57 smithi175 bash[29284]: audit 2024-03-26T03:51:56.623904+0000 mon.a (mon.0) 2032 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi175-64398-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi175-64398-42"}]: dispatch 2024-03-26T03:51:57.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:57 smithi175 bash[29284]: audit 2024-03-26T03:51:56.626895+0000 mon.c (mon.2) 428 : audit [INF] from='client.? 172.21.15.175:0/2246972559' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi175-64406-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:57.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:57 smithi175 bash[29284]: audit 2024-03-26T03:51:56.641205+0000 mon.a (mon.0) 2033 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi175-64406-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:51:57.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:57 smithi175 bash[29284]: cluster 2024-03-26T03:51:56.670500+0000 mgr.y (mgr.24431) 308 : cluster [DBG] pgmap v387: 324 pgs: 32 unknown, 2 active, 8 creating+activating, 282 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:51:57.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:57 smithi175 bash[29284]: cluster 2024-03-26T03:51:57.607231+0000 mon.a (mon.0) 2034 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:51:57.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:57 smithi175 bash[29284]: audit 2024-03-26T03:51:57.610152+0000 mon.a (mon.0) 2035 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-35", "mode": "writeback"}]': finished 2024-03-26T03:51:58.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:58 smithi190 bash[23392]: audit 2024-03-26T03:51:57.610462+0000 mon.a (mon.0) 2036 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi175-64406-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:58.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:58 smithi190 bash[23392]: cluster 2024-03-26T03:51:57.632117+0000 mon.a (mon.0) 2037 : cluster [DBG] osdmap e261: 8 total, 8 up, 8 in 2024-03-26T03:51:58.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:58 smithi190 bash[23392]: cluster 2024-03-26T03:51:57.632525+0000 mon.a (mon.0) 2038 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:51:58.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:58 smithi190 bash[23392]: audit 2024-03-26T03:51:58.613261+0000 mon.a (mon.0) 2039 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi175-64398-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi175-64398-42"}]': finished 2024-03-26T03:51:58.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:58 smithi190 bash[23392]: cluster 2024-03-26T03:51:58.621527+0000 mon.a (mon.0) 2040 : cluster [DBG] osdmap e262: 8 total, 8 up, 8 in 2024-03-26T03:51:58.992 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:51:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:51:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:51:58.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:58 smithi175 bash[21527]: audit 2024-03-26T03:51:57.610462+0000 mon.a (mon.0) 2036 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi175-64406-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:58.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:58 smithi175 bash[21527]: cluster 2024-03-26T03:51:57.632117+0000 mon.a (mon.0) 2037 : cluster [DBG] osdmap e261: 8 total, 8 up, 8 in 2024-03-26T03:51:58.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:58 smithi175 bash[21527]: cluster 2024-03-26T03:51:57.632525+0000 mon.a (mon.0) 2038 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:51:58.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:58 smithi175 bash[21527]: audit 2024-03-26T03:51:58.613261+0000 mon.a (mon.0) 2039 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi175-64398-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi175-64398-42"}]': finished 2024-03-26T03:51:58.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:58 smithi175 bash[21527]: cluster 2024-03-26T03:51:58.621527+0000 mon.a (mon.0) 2040 : cluster [DBG] osdmap e262: 8 total, 8 up, 8 in 2024-03-26T03:51:58.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:58 smithi175 bash[29284]: audit 2024-03-26T03:51:57.610462+0000 mon.a (mon.0) 2036 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi175-64406-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:51:58.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:58 smithi175 bash[29284]: cluster 2024-03-26T03:51:57.632117+0000 mon.a (mon.0) 2037 : cluster [DBG] osdmap e261: 8 total, 8 up, 8 in 2024-03-26T03:51:58.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:58 smithi175 bash[29284]: cluster 2024-03-26T03:51:57.632525+0000 mon.a (mon.0) 2038 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:51:58.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:58 smithi175 bash[29284]: audit 2024-03-26T03:51:58.613261+0000 mon.a (mon.0) 2039 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi175-64398-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi175-64398-42"}]': finished 2024-03-26T03:51:58.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:58 smithi175 bash[29284]: cluster 2024-03-26T03:51:58.621527+0000 mon.a (mon.0) 2040 : cluster [DBG] osdmap e262: 8 total, 8 up, 8 in 2024-03-26T03:51:59.990 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:51:59 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:51:59.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:59 smithi190 bash[23392]: cluster 2024-03-26T03:51:58.671147+0000 mgr.y (mgr.24431) 309 : cluster [DBG] pgmap v390: 300 pgs: 8 unknown, 2 active, 8 creating+activating, 282 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:51:59.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:59 smithi190 bash[23392]: audit 2024-03-26T03:51:58.674540+0000 mon.c (mon.2) 429 : audit [INF] from='client.? 172.21.15.175:0/200872624' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi175-64406-52"}]: dispatch 2024-03-26T03:51:59.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:59 smithi190 bash[23392]: audit 2024-03-26T03:51:58.674904+0000 mon.a (mon.0) 2041 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi175-64406-52"}]: dispatch 2024-03-26T03:51:59.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:59 smithi190 bash[23392]: audit 2024-03-26T03:51:58.675535+0000 mon.c (mon.2) 430 : audit [INF] from='client.? 172.21.15.175:0/200872624' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi175-64406-52"}]: dispatch 2024-03-26T03:51:59.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:59 smithi190 bash[23392]: audit 2024-03-26T03:51:58.675822+0000 mon.a (mon.0) 2042 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi175-64406-52"}]: dispatch 2024-03-26T03:51:59.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:59 smithi190 bash[23392]: audit 2024-03-26T03:51:58.676308+0000 mon.c (mon.2) 431 : audit [INF] from='client.? 172.21.15.175:0/200872624' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi175-64406-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:59.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:59 smithi190 bash[23392]: audit 2024-03-26T03:51:58.676604+0000 mon.a (mon.0) 2043 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi175-64406-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:59.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:59 smithi190 bash[23392]: audit 2024-03-26T03:51:59.615927+0000 mon.a (mon.0) 2044 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi175-64406-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:59.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:59 smithi190 bash[23392]: cluster 2024-03-26T03:51:59.623684+0000 mon.a (mon.0) 2045 : cluster [DBG] osdmap e263: 8 total, 8 up, 8 in 2024-03-26T03:51:59.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:59 smithi190 bash[23392]: audit 2024-03-26T03:51:59.629349+0000 mon.c (mon.2) 432 : audit [INF] from='client.? 172.21.15.175:0/200872624' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi175-64406-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi175-64406-52"}]: dispatch 2024-03-26T03:51:59.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:51:59 smithi190 bash[23392]: audit 2024-03-26T03:51:59.637957+0000 mon.a (mon.0) 2046 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi175-64406-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi175-64406-52"}]: dispatch 2024-03-26T03:51:59.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:59 smithi175 bash[29284]: cluster 2024-03-26T03:51:58.671147+0000 mgr.y (mgr.24431) 309 : cluster [DBG] pgmap v390: 300 pgs: 8 unknown, 2 active, 8 creating+activating, 282 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:51:59.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:59 smithi175 bash[29284]: audit 2024-03-26T03:51:58.674540+0000 mon.c (mon.2) 429 : audit [INF] from='client.? 172.21.15.175:0/200872624' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi175-64406-52"}]: dispatch 2024-03-26T03:51:59.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:59 smithi175 bash[29284]: audit 2024-03-26T03:51:58.674904+0000 mon.a (mon.0) 2041 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi175-64406-52"}]: dispatch 2024-03-26T03:51:59.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:59 smithi175 bash[29284]: audit 2024-03-26T03:51:58.675535+0000 mon.c (mon.2) 430 : audit [INF] from='client.? 172.21.15.175:0/200872624' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi175-64406-52"}]: dispatch 2024-03-26T03:51:59.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:59 smithi175 bash[29284]: audit 2024-03-26T03:51:58.675822+0000 mon.a (mon.0) 2042 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi175-64406-52"}]: dispatch 2024-03-26T03:51:59.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:59 smithi175 bash[29284]: audit 2024-03-26T03:51:58.676308+0000 mon.c (mon.2) 431 : audit [INF] from='client.? 172.21.15.175:0/200872624' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi175-64406-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:59.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:59 smithi175 bash[29284]: audit 2024-03-26T03:51:58.676604+0000 mon.a (mon.0) 2043 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi175-64406-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:59.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:59 smithi175 bash[29284]: audit 2024-03-26T03:51:59.615927+0000 mon.a (mon.0) 2044 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi175-64406-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:59.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:59 smithi175 bash[29284]: cluster 2024-03-26T03:51:59.623684+0000 mon.a (mon.0) 2045 : cluster [DBG] osdmap e263: 8 total, 8 up, 8 in 2024-03-26T03:51:59.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:59 smithi175 bash[29284]: audit 2024-03-26T03:51:59.629349+0000 mon.c (mon.2) 432 : audit [INF] from='client.? 172.21.15.175:0/200872624' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi175-64406-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi175-64406-52"}]: dispatch 2024-03-26T03:51:59.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:51:59 smithi175 bash[29284]: audit 2024-03-26T03:51:59.637957+0000 mon.a (mon.0) 2046 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi175-64406-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi175-64406-52"}]: dispatch 2024-03-26T03:51:59.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:59 smithi175 bash[21527]: cluster 2024-03-26T03:51:58.671147+0000 mgr.y (mgr.24431) 309 : cluster [DBG] pgmap v390: 300 pgs: 8 unknown, 2 active, 8 creating+activating, 282 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:51:59.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:59 smithi175 bash[21527]: audit 2024-03-26T03:51:58.674540+0000 mon.c (mon.2) 429 : audit [INF] from='client.? 172.21.15.175:0/200872624' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi175-64406-52"}]: dispatch 2024-03-26T03:51:59.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:59 smithi175 bash[21527]: audit 2024-03-26T03:51:58.674904+0000 mon.a (mon.0) 2041 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi175-64406-52"}]: dispatch 2024-03-26T03:51:59.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:59 smithi175 bash[21527]: audit 2024-03-26T03:51:58.675535+0000 mon.c (mon.2) 430 : audit [INF] from='client.? 172.21.15.175:0/200872624' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi175-64406-52"}]: dispatch 2024-03-26T03:51:59.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:59 smithi175 bash[21527]: audit 2024-03-26T03:51:58.675822+0000 mon.a (mon.0) 2042 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi175-64406-52"}]: dispatch 2024-03-26T03:51:59.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:59 smithi175 bash[21527]: audit 2024-03-26T03:51:58.676308+0000 mon.c (mon.2) 431 : audit [INF] from='client.? 172.21.15.175:0/200872624' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi175-64406-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:59.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:59 smithi175 bash[21527]: audit 2024-03-26T03:51:58.676604+0000 mon.a (mon.0) 2043 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi175-64406-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:51:59.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:59 smithi175 bash[21527]: audit 2024-03-26T03:51:59.615927+0000 mon.a (mon.0) 2044 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi175-64406-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:51:59.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:59 smithi175 bash[21527]: cluster 2024-03-26T03:51:59.623684+0000 mon.a (mon.0) 2045 : cluster [DBG] osdmap e263: 8 total, 8 up, 8 in 2024-03-26T03:51:59.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:59 smithi175 bash[21527]: audit 2024-03-26T03:51:59.629349+0000 mon.c (mon.2) 432 : audit [INF] from='client.? 172.21.15.175:0/200872624' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi175-64406-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi175-64406-52"}]: dispatch 2024-03-26T03:51:59.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:51:59 smithi175 bash[21527]: audit 2024-03-26T03:51:59.637957+0000 mon.a (mon.0) 2046 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi175-64406-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi175-64406-52"}]: dispatch 2024-03-26T03:52:00.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:00 smithi190 bash[23392]: audit 2024-03-26T03:51:59.742408+0000 mon.a (mon.0) 2047 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:52:00.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:00 smithi190 bash[23392]: audit 2024-03-26T03:51:59.869962+0000 mgr.y (mgr.24431) 310 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:52:00.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:00 smithi190 bash[23392]: audit 2024-03-26T03:52:00.619554+0000 mon.a (mon.0) 2048 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:52:00.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:00 smithi190 bash[23392]: cluster 2024-03-26T03:52:00.639116+0000 mon.a (mon.0) 2049 : cluster [DBG] osdmap e264: 8 total, 8 up, 8 in 2024-03-26T03:52:00.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:00 smithi190 bash[23392]: audit 2024-03-26T03:52:00.642135+0000 mon.c (mon.2) 433 : audit [INF] from='client.? 172.21.15.175:0/2898360639' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi175-64398-42"}]: dispatch 2024-03-26T03:52:00.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:00 smithi190 bash[23392]: audit 2024-03-26T03:52:00.642872+0000 mon.a (mon.0) 2050 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi175-64398-42"}]: dispatch 2024-03-26T03:52:00.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:00 smithi190 bash[23392]: audit 2024-03-26T03:52:00.656653+0000 mon.a (mon.0) 2051 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-35"}]: dispatch 2024-03-26T03:52:00.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:00 smithi175 bash[21527]: audit 2024-03-26T03:51:59.742408+0000 mon.a (mon.0) 2047 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:52:00.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:00 smithi175 bash[21527]: audit 2024-03-26T03:51:59.869962+0000 mgr.y (mgr.24431) 310 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:52:00.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:00 smithi175 bash[21527]: audit 2024-03-26T03:52:00.619554+0000 mon.a (mon.0) 2048 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:52:00.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:00 smithi175 bash[21527]: cluster 2024-03-26T03:52:00.639116+0000 mon.a (mon.0) 2049 : cluster [DBG] osdmap e264: 8 total, 8 up, 8 in 2024-03-26T03:52:00.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:00 smithi175 bash[21527]: audit 2024-03-26T03:52:00.642135+0000 mon.c (mon.2) 433 : audit [INF] from='client.? 172.21.15.175:0/2898360639' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi175-64398-42"}]: dispatch 2024-03-26T03:52:00.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:00 smithi175 bash[21527]: audit 2024-03-26T03:52:00.642872+0000 mon.a (mon.0) 2050 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi175-64398-42"}]: dispatch 2024-03-26T03:52:00.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:00 smithi175 bash[21527]: audit 2024-03-26T03:52:00.656653+0000 mon.a (mon.0) 2051 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-35"}]: dispatch 2024-03-26T03:52:00.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:00 smithi175 bash[29284]: audit 2024-03-26T03:51:59.742408+0000 mon.a (mon.0) 2047 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:52:00.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:00 smithi175 bash[29284]: audit 2024-03-26T03:51:59.869962+0000 mgr.y (mgr.24431) 310 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:52:00.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:00 smithi175 bash[29284]: audit 2024-03-26T03:52:00.619554+0000 mon.a (mon.0) 2048 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:52:00.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:00 smithi175 bash[29284]: cluster 2024-03-26T03:52:00.639116+0000 mon.a (mon.0) 2049 : cluster [DBG] osdmap e264: 8 total, 8 up, 8 in 2024-03-26T03:52:00.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:00 smithi175 bash[29284]: audit 2024-03-26T03:52:00.642135+0000 mon.c (mon.2) 433 : audit [INF] from='client.? 172.21.15.175:0/2898360639' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi175-64398-42"}]: dispatch 2024-03-26T03:52:00.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:00 smithi175 bash[29284]: audit 2024-03-26T03:52:00.642872+0000 mon.a (mon.0) 2050 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi175-64398-42"}]: dispatch 2024-03-26T03:52:00.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:00 smithi175 bash[29284]: audit 2024-03-26T03:52:00.656653+0000 mon.a (mon.0) 2051 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-35"}]: dispatch 2024-03-26T03:52:01.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:01 smithi190 bash[23392]: cluster 2024-03-26T03:52:00.671823+0000 mgr.y (mgr.24431) 311 : cluster [DBG] pgmap v393: 292 pgs: 292 active+clean; 4.6 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-03-26T03:52:01.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:01 smithi190 bash[23392]: audit 2024-03-26T03:52:01.622848+0000 mon.a (mon.0) 2052 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi175-64406-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi175-64406-52"}]': finished 2024-03-26T03:52:01.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:01 smithi190 bash[23392]: audit 2024-03-26T03:52:01.623108+0000 mon.a (mon.0) 2053 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi175-64398-42"}]': finished 2024-03-26T03:52:01.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:01 smithi190 bash[23392]: audit 2024-03-26T03:52:01.623242+0000 mon.a (mon.0) 2054 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-35"}]': finished 2024-03-26T03:52:01.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:01 smithi190 bash[23392]: cluster 2024-03-26T03:52:01.631223+0000 mon.a (mon.0) 2055 : cluster [DBG] osdmap e265: 8 total, 8 up, 8 in 2024-03-26T03:52:01.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:01 smithi190 bash[23392]: audit 2024-03-26T03:52:01.633670+0000 mon.c (mon.2) 434 : audit [INF] from='client.? 172.21.15.175:0/2898360639' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi175-64398-42"}]: dispatch 2024-03-26T03:52:01.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:01 smithi190 bash[23392]: audit 2024-03-26T03:52:01.634723+0000 mon.a (mon.0) 2056 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi175-64398-42"}]: dispatch 2024-03-26T03:52:01.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:01 smithi175 bash[29284]: cluster 2024-03-26T03:52:00.671823+0000 mgr.y (mgr.24431) 311 : cluster [DBG] pgmap v393: 292 pgs: 292 active+clean; 4.6 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-03-26T03:52:01.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:01 smithi175 bash[29284]: audit 2024-03-26T03:52:01.622848+0000 mon.a (mon.0) 2052 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi175-64406-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi175-64406-52"}]': finished 2024-03-26T03:52:01.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:01 smithi175 bash[29284]: audit 2024-03-26T03:52:01.623108+0000 mon.a (mon.0) 2053 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi175-64398-42"}]': finished 2024-03-26T03:52:01.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:01 smithi175 bash[29284]: audit 2024-03-26T03:52:01.623242+0000 mon.a (mon.0) 2054 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-35"}]': finished 2024-03-26T03:52:01.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:01 smithi175 bash[29284]: cluster 2024-03-26T03:52:01.631223+0000 mon.a (mon.0) 2055 : cluster [DBG] osdmap e265: 8 total, 8 up, 8 in 2024-03-26T03:52:01.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:01 smithi175 bash[29284]: audit 2024-03-26T03:52:01.633670+0000 mon.c (mon.2) 434 : audit [INF] from='client.? 172.21.15.175:0/2898360639' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi175-64398-42"}]: dispatch 2024-03-26T03:52:01.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:01 smithi175 bash[29284]: audit 2024-03-26T03:52:01.634723+0000 mon.a (mon.0) 2056 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi175-64398-42"}]: dispatch 2024-03-26T03:52:01.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:01 smithi175 bash[21527]: cluster 2024-03-26T03:52:00.671823+0000 mgr.y (mgr.24431) 311 : cluster [DBG] pgmap v393: 292 pgs: 292 active+clean; 4.6 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-03-26T03:52:01.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:01 smithi175 bash[21527]: audit 2024-03-26T03:52:01.622848+0000 mon.a (mon.0) 2052 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi175-64406-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi175-64406-52"}]': finished 2024-03-26T03:52:01.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:01 smithi175 bash[21527]: audit 2024-03-26T03:52:01.623108+0000 mon.a (mon.0) 2053 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi175-64398-42"}]': finished 2024-03-26T03:52:01.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:01 smithi175 bash[21527]: audit 2024-03-26T03:52:01.623242+0000 mon.a (mon.0) 2054 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-35"}]': finished 2024-03-26T03:52:01.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:01 smithi175 bash[21527]: cluster 2024-03-26T03:52:01.631223+0000 mon.a (mon.0) 2055 : cluster [DBG] osdmap e265: 8 total, 8 up, 8 in 2024-03-26T03:52:01.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:01 smithi175 bash[21527]: audit 2024-03-26T03:52:01.633670+0000 mon.c (mon.2) 434 : audit [INF] from='client.? 172.21.15.175:0/2898360639' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi175-64398-42"}]: dispatch 2024-03-26T03:52:01.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:01 smithi175 bash[21527]: audit 2024-03-26T03:52:01.634723+0000 mon.a (mon.0) 2056 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi175-64398-42"}]: dispatch 2024-03-26T03:52:02.650 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAio api_aio: [ OK ] LibRadosAioEC.SimpleWrite (7033 ms) 2024-03-26T03:52:02.650 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForComplete 2024-03-26T03:52:02.650 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (6955 ms) 2024-03-26T03:52:02.650 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2024-03-26T03:52:02.650 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (7029 ms) 2024-03-26T03:52:02.650 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2024-03-26T03:52:02.650 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (6994 ms) 2024-03-26T03:52:02.650 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2024-03-26T03:52:02.650 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (7038 ms) 2024-03-26T03:52:02.650 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2024-03-26T03:52:02.651 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (7021 ms) 2024-03-26T03:52:02.651 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2024-03-26T03:52:02.651 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (7070 ms) 2024-03-26T03:52:02.651 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2024-03-26T03:52:02.651 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (7040 ms) 2024-03-26T03:52:02.651 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2024-03-26T03:52:02.651 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (6826 ms) 2024-03-26T03:52:02.651 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2024-03-26T03:52:02.651 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (7051 ms) 2024-03-26T03:52:02.651 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-03-26T03:52:02.651 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (7061 ms) 2024-03-26T03:52:02.651 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2024-03-26T03:52:02.651 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (6899 ms) 2024-03-26T03:52:02.651 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2024-03-26T03:52:02.651 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (6988 ms) 2024-03-26T03:52:02.651 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2024-03-26T03:52:02.651 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (7065 ms) 2024-03-26T03:52:02.652 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2024-03-26T03:52:02.652 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (7091 ms) 2024-03-26T03:52:02.652 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2024-03-26T03:52:02.652 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (7024 ms) 2024-03-26T03:52:02.652 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (112187 ms total) 2024-03-26T03:52:02.652 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: 2024-03-26T03:52:02.652 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [----------] Global test environment tear-down 2024-03-26T03:52:02.652 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [==========] 42 tests from 2 test suites ran. (190445 ms total) 2024-03-26T03:52:02.652 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio: [ PASSED ] 42 tests. 2024-03-26T03:52:03.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:03 smithi190 bash[23392]: audit 2024-03-26T03:52:02.626785+0000 mon.a (mon.0) 2057 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi175-64398-42"}]': finished 2024-03-26T03:52:03.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:03 smithi190 bash[23392]: cluster 2024-03-26T03:52:02.635180+0000 mon.a (mon.0) 2058 : cluster [DBG] osdmap e266: 8 total, 8 up, 8 in 2024-03-26T03:52:03.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:03 smithi190 bash[23392]: cluster 2024-03-26T03:52:02.672581+0000 mgr.y (mgr.24431) 312 : cluster [DBG] pgmap v396: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-03-26T03:52:03.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:03 smithi190 bash[23392]: audit 2024-03-26T03:52:02.701521+0000 mon.a (mon.0) 2059 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:52:03.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:03 smithi175 bash[29284]: audit 2024-03-26T03:52:02.626785+0000 mon.a (mon.0) 2057 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi175-64398-42"}]': finished 2024-03-26T03:52:03.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:03 smithi175 bash[29284]: cluster 2024-03-26T03:52:02.635180+0000 mon.a (mon.0) 2058 : cluster [DBG] osdmap e266: 8 total, 8 up, 8 in 2024-03-26T03:52:03.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:03 smithi175 bash[29284]: cluster 2024-03-26T03:52:02.672581+0000 mgr.y (mgr.24431) 312 : cluster [DBG] pgmap v396: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-03-26T03:52:03.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:03 smithi175 bash[29284]: audit 2024-03-26T03:52:02.701521+0000 mon.a (mon.0) 2059 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:52:03.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:03 smithi175 bash[21527]: audit 2024-03-26T03:52:02.626785+0000 mon.a (mon.0) 2057 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi175-64398-42"}]': finished 2024-03-26T03:52:03.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:03 smithi175 bash[21527]: cluster 2024-03-26T03:52:02.635180+0000 mon.a (mon.0) 2058 : cluster [DBG] osdmap e266: 8 total, 8 up, 8 in 2024-03-26T03:52:03.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:03 smithi175 bash[21527]: cluster 2024-03-26T03:52:02.672581+0000 mgr.y (mgr.24431) 312 : cluster [DBG] pgmap v396: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-03-26T03:52:03.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:03 smithi175 bash[21527]: audit 2024-03-26T03:52:02.701521+0000 mon.a (mon.0) 2059 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:52:04.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:04 smithi190 bash[23392]: cluster 2024-03-26T03:52:03.626728+0000 mon.a (mon.0) 2060 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:52:04.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:04 smithi190 bash[23392]: audit 2024-03-26T03:52:03.638394+0000 mon.a (mon.0) 2061 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:52:04.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:04 smithi190 bash[23392]: cluster 2024-03-26T03:52:03.649822+0000 mon.a (mon.0) 2062 : cluster [DBG] osdmap e267: 8 total, 8 up, 8 in 2024-03-26T03:52:04.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:04 smithi190 bash[23392]: audit 2024-03-26T03:52:03.650926+0000 mon.a (mon.0) 2063 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-35"}]: dispatch 2024-03-26T03:52:04.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:04 smithi190 bash[23392]: audit 2024-03-26T03:52:03.652256+0000 mon.c (mon.2) 435 : audit [INF] from='client.? 172.21.15.175:0/200872624' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi175-64406-52"}]: dispatch 2024-03-26T03:52:04.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:04 smithi190 bash[23392]: audit 2024-03-26T03:52:03.652979+0000 mon.a (mon.0) 2064 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi175-64406-52"}]: dispatch 2024-03-26T03:52:04.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:04 smithi190 bash[23392]: audit 2024-03-26T03:52:04.506897+0000 mon.a (mon.0) 2065 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:52:04.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:04 smithi190 bash[23392]: audit 2024-03-26T03:52:04.510076+0000 mon.c (mon.2) 436 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:52:04.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:04 smithi175 bash[21527]: cluster 2024-03-26T03:52:03.626728+0000 mon.a (mon.0) 2060 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:52:04.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:04 smithi175 bash[21527]: audit 2024-03-26T03:52:03.638394+0000 mon.a (mon.0) 2061 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:52:04.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:04 smithi175 bash[21527]: cluster 2024-03-26T03:52:03.649822+0000 mon.a (mon.0) 2062 : cluster [DBG] osdmap e267: 8 total, 8 up, 8 in 2024-03-26T03:52:04.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:04 smithi175 bash[21527]: audit 2024-03-26T03:52:03.650926+0000 mon.a (mon.0) 2063 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-35"}]: dispatch 2024-03-26T03:52:04.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:04 smithi175 bash[21527]: audit 2024-03-26T03:52:03.652256+0000 mon.c (mon.2) 435 : audit [INF] from='client.? 172.21.15.175:0/200872624' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi175-64406-52"}]: dispatch 2024-03-26T03:52:04.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:04 smithi175 bash[21527]: audit 2024-03-26T03:52:03.652979+0000 mon.a (mon.0) 2064 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi175-64406-52"}]: dispatch 2024-03-26T03:52:04.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:04 smithi175 bash[21527]: audit 2024-03-26T03:52:04.506897+0000 mon.a (mon.0) 2065 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:52:04.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:04 smithi175 bash[21527]: audit 2024-03-26T03:52:04.510076+0000 mon.c (mon.2) 436 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:52:04.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:04 smithi175 bash[29284]: cluster 2024-03-26T03:52:03.626728+0000 mon.a (mon.0) 2060 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:52:04.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:04 smithi175 bash[29284]: audit 2024-03-26T03:52:03.638394+0000 mon.a (mon.0) 2061 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:52:04.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:04 smithi175 bash[29284]: cluster 2024-03-26T03:52:03.649822+0000 mon.a (mon.0) 2062 : cluster [DBG] osdmap e267: 8 total, 8 up, 8 in 2024-03-26T03:52:04.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:04 smithi175 bash[29284]: audit 2024-03-26T03:52:03.650926+0000 mon.a (mon.0) 2063 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-35"}]: dispatch 2024-03-26T03:52:04.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:04 smithi175 bash[29284]: audit 2024-03-26T03:52:03.652256+0000 mon.c (mon.2) 435 : audit [INF] from='client.? 172.21.15.175:0/200872624' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi175-64406-52"}]: dispatch 2024-03-26T03:52:04.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:04 smithi175 bash[29284]: audit 2024-03-26T03:52:03.652979+0000 mon.a (mon.0) 2064 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi175-64406-52"}]: dispatch 2024-03-26T03:52:04.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:04 smithi175 bash[29284]: audit 2024-03-26T03:52:04.506897+0000 mon.a (mon.0) 2065 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:52:04.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:04 smithi175 bash[29284]: audit 2024-03-26T03:52:04.510076+0000 mon.c (mon.2) 436 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:52:05.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:05 smithi190 bash[23392]: cluster 2024-03-26T03:52:04.638926+0000 mon.a (mon.0) 2066 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:52:05.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:05 smithi190 bash[23392]: audit 2024-03-26T03:52:04.641220+0000 mon.a (mon.0) 2067 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-35"}]': finished 2024-03-26T03:52:05.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:05 smithi190 bash[23392]: audit 2024-03-26T03:52:04.641418+0000 mon.a (mon.0) 2068 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi175-64406-52"}]': finished 2024-03-26T03:52:05.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:05 smithi190 bash[23392]: cluster 2024-03-26T03:52:04.648430+0000 mon.a (mon.0) 2069 : cluster [DBG] osdmap e268: 8 total, 8 up, 8 in 2024-03-26T03:52:05.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:05 smithi190 bash[23392]: audit 2024-03-26T03:52:04.652392+0000 mon.c (mon.2) 437 : audit [INF] from='client.? 172.21.15.175:0/200872624' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi175-64406-52"}]: dispatch 2024-03-26T03:52:05.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:05 smithi190 bash[23392]: audit 2024-03-26T03:52:04.668952+0000 mon.a (mon.0) 2070 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi175-64406-52"}]: dispatch 2024-03-26T03:52:05.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:05 smithi190 bash[23392]: cluster 2024-03-26T03:52:04.673813+0000 mgr.y (mgr.24431) 313 : cluster [DBG] pgmap v399: 292 pgs: 292 active+clean; 4.6 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 767 B/s wr, 1 op/s 2024-03-26T03:52:05.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:05 smithi175 bash[21527]: cluster 2024-03-26T03:52:04.638926+0000 mon.a (mon.0) 2066 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:52:05.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:05 smithi175 bash[21527]: audit 2024-03-26T03:52:04.641220+0000 mon.a (mon.0) 2067 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-35"}]': finished 2024-03-26T03:52:05.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:05 smithi175 bash[21527]: audit 2024-03-26T03:52:04.641418+0000 mon.a (mon.0) 2068 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi175-64406-52"}]': finished 2024-03-26T03:52:05.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:05 smithi175 bash[21527]: cluster 2024-03-26T03:52:04.648430+0000 mon.a (mon.0) 2069 : cluster [DBG] osdmap e268: 8 total, 8 up, 8 in 2024-03-26T03:52:05.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:05 smithi175 bash[21527]: audit 2024-03-26T03:52:04.652392+0000 mon.c (mon.2) 437 : audit [INF] from='client.? 172.21.15.175:0/200872624' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi175-64406-52"}]: dispatch 2024-03-26T03:52:05.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:05 smithi175 bash[21527]: audit 2024-03-26T03:52:04.668952+0000 mon.a (mon.0) 2070 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi175-64406-52"}]: dispatch 2024-03-26T03:52:05.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:05 smithi175 bash[21527]: cluster 2024-03-26T03:52:04.673813+0000 mgr.y (mgr.24431) 313 : cluster [DBG] pgmap v399: 292 pgs: 292 active+clean; 4.6 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 767 B/s wr, 1 op/s 2024-03-26T03:52:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:05 smithi175 bash[29284]: cluster 2024-03-26T03:52:04.638926+0000 mon.a (mon.0) 2066 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:52:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:05 smithi175 bash[29284]: audit 2024-03-26T03:52:04.641220+0000 mon.a (mon.0) 2067 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-35"}]': finished 2024-03-26T03:52:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:05 smithi175 bash[29284]: audit 2024-03-26T03:52:04.641418+0000 mon.a (mon.0) 2068 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi175-64406-52"}]': finished 2024-03-26T03:52:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:05 smithi175 bash[29284]: cluster 2024-03-26T03:52:04.648430+0000 mon.a (mon.0) 2069 : cluster [DBG] osdmap e268: 8 total, 8 up, 8 in 2024-03-26T03:52:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:05 smithi175 bash[29284]: audit 2024-03-26T03:52:04.652392+0000 mon.c (mon.2) 437 : audit [INF] from='client.? 172.21.15.175:0/200872624' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi175-64406-52"}]: dispatch 2024-03-26T03:52:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:05 smithi175 bash[29284]: audit 2024-03-26T03:52:04.668952+0000 mon.a (mon.0) 2070 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi175-64406-52"}]: dispatch 2024-03-26T03:52:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:05 smithi175 bash[29284]: cluster 2024-03-26T03:52:04.673813+0000 mgr.y (mgr.24431) 313 : cluster [DBG] pgmap v399: 292 pgs: 292 active+clean; 4.6 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 767 B/s wr, 1 op/s 2024-03-26T03:52:06.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:06 smithi190 bash[23392]: audit 2024-03-26T03:52:05.658893+0000 mon.a (mon.0) 2071 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi175-64406-52"}]': finished 2024-03-26T03:52:06.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:06 smithi190 bash[23392]: cluster 2024-03-26T03:52:05.667187+0000 mon.a (mon.0) 2072 : cluster [DBG] osdmap e269: 8 total, 8 up, 8 in 2024-03-26T03:52:06.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:06 smithi190 bash[23392]: audit 2024-03-26T03:52:05.705154+0000 mon.c (mon.2) 438 : audit [INF] from='client.? 172.21.15.175:0/3341683102' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi175-64406-53"}]: dispatch 2024-03-26T03:52:06.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:06 smithi190 bash[23392]: audit 2024-03-26T03:52:05.705500+0000 mon.a (mon.0) 2073 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi175-64406-53"}]: dispatch 2024-03-26T03:52:06.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:06 smithi190 bash[23392]: audit 2024-03-26T03:52:05.706167+0000 mon.c (mon.2) 439 : audit [INF] from='client.? 172.21.15.175:0/3341683102' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi175-64406-53"}]: dispatch 2024-03-26T03:52:06.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:06 smithi190 bash[23392]: audit 2024-03-26T03:52:05.706479+0000 mon.a (mon.0) 2074 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi175-64406-53"}]: dispatch 2024-03-26T03:52:06.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:06 smithi190 bash[23392]: audit 2024-03-26T03:52:05.706986+0000 mon.c (mon.2) 440 : audit [INF] from='client.? 172.21.15.175:0/3341683102' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi175-64406-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:06.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:06 smithi190 bash[23392]: audit 2024-03-26T03:52:05.707282+0000 mon.a (mon.0) 2075 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi175-64406-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:06.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:06 smithi190 bash[23392]: audit 2024-03-26T03:52:06.662660+0000 mon.a (mon.0) 2076 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi175-64406-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:52:06.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:06 smithi190 bash[23392]: cluster 2024-03-26T03:52:06.670788+0000 mon.a (mon.0) 2077 : cluster [DBG] osdmap e270: 8 total, 8 up, 8 in 2024-03-26T03:52:06.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:06 smithi190 bash[23392]: audit 2024-03-26T03:52:06.673271+0000 mon.c (mon.2) 441 : audit [INF] from='client.? 172.21.15.175:0/3341683102' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi175-64406-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi175-64406-53"}]: dispatch 2024-03-26T03:52:06.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:06 smithi190 bash[23392]: audit 2024-03-26T03:52:06.675457+0000 mon.a (mon.0) 2078 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:52:06.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:06 smithi190 bash[23392]: audit 2024-03-26T03:52:06.676359+0000 mon.a (mon.0) 2079 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi175-64406-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi175-64406-53"}]: dispatch 2024-03-26T03:52:06.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:06 smithi175 bash[21527]: audit 2024-03-26T03:52:05.658893+0000 mon.a (mon.0) 2071 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi175-64406-52"}]': finished 2024-03-26T03:52:06.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:06 smithi175 bash[21527]: cluster 2024-03-26T03:52:05.667187+0000 mon.a (mon.0) 2072 : cluster [DBG] osdmap e269: 8 total, 8 up, 8 in 2024-03-26T03:52:06.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:06 smithi175 bash[21527]: audit 2024-03-26T03:52:05.705154+0000 mon.c (mon.2) 438 : audit [INF] from='client.? 172.21.15.175:0/3341683102' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi175-64406-53"}]: dispatch 2024-03-26T03:52:06.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:06 smithi175 bash[21527]: audit 2024-03-26T03:52:05.705500+0000 mon.a (mon.0) 2073 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi175-64406-53"}]: dispatch 2024-03-26T03:52:06.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:06 smithi175 bash[21527]: audit 2024-03-26T03:52:05.706167+0000 mon.c (mon.2) 439 : audit [INF] from='client.? 172.21.15.175:0/3341683102' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi175-64406-53"}]: dispatch 2024-03-26T03:52:06.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:06 smithi175 bash[21527]: audit 2024-03-26T03:52:05.706479+0000 mon.a (mon.0) 2074 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi175-64406-53"}]: dispatch 2024-03-26T03:52:06.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:06 smithi175 bash[21527]: audit 2024-03-26T03:52:05.706986+0000 mon.c (mon.2) 440 : audit [INF] from='client.? 172.21.15.175:0/3341683102' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi175-64406-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:06.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:06 smithi175 bash[21527]: audit 2024-03-26T03:52:05.707282+0000 mon.a (mon.0) 2075 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi175-64406-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:06.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:06 smithi175 bash[21527]: audit 2024-03-26T03:52:06.662660+0000 mon.a (mon.0) 2076 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi175-64406-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:52:06.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:06 smithi175 bash[21527]: cluster 2024-03-26T03:52:06.670788+0000 mon.a (mon.0) 2077 : cluster [DBG] osdmap e270: 8 total, 8 up, 8 in 2024-03-26T03:52:06.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:06 smithi175 bash[21527]: audit 2024-03-26T03:52:06.673271+0000 mon.c (mon.2) 441 : audit [INF] from='client.? 172.21.15.175:0/3341683102' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi175-64406-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi175-64406-53"}]: dispatch 2024-03-26T03:52:06.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:06 smithi175 bash[21527]: audit 2024-03-26T03:52:06.675457+0000 mon.a (mon.0) 2078 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:52:06.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:06 smithi175 bash[21527]: audit 2024-03-26T03:52:06.676359+0000 mon.a (mon.0) 2079 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi175-64406-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi175-64406-53"}]: dispatch 2024-03-26T03:52:06.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:06 smithi175 bash[29284]: audit 2024-03-26T03:52:05.658893+0000 mon.a (mon.0) 2071 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi175-64406-52"}]': finished 2024-03-26T03:52:06.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:06 smithi175 bash[29284]: cluster 2024-03-26T03:52:05.667187+0000 mon.a (mon.0) 2072 : cluster [DBG] osdmap e269: 8 total, 8 up, 8 in 2024-03-26T03:52:06.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:06 smithi175 bash[29284]: audit 2024-03-26T03:52:05.705154+0000 mon.c (mon.2) 438 : audit [INF] from='client.? 172.21.15.175:0/3341683102' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi175-64406-53"}]: dispatch 2024-03-26T03:52:06.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:06 smithi175 bash[29284]: audit 2024-03-26T03:52:05.705500+0000 mon.a (mon.0) 2073 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi175-64406-53"}]: dispatch 2024-03-26T03:52:06.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:06 smithi175 bash[29284]: audit 2024-03-26T03:52:05.706167+0000 mon.c (mon.2) 439 : audit [INF] from='client.? 172.21.15.175:0/3341683102' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi175-64406-53"}]: dispatch 2024-03-26T03:52:06.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:06 smithi175 bash[29284]: audit 2024-03-26T03:52:05.706479+0000 mon.a (mon.0) 2074 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi175-64406-53"}]: dispatch 2024-03-26T03:52:06.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:06 smithi175 bash[29284]: audit 2024-03-26T03:52:05.706986+0000 mon.c (mon.2) 440 : audit [INF] from='client.? 172.21.15.175:0/3341683102' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi175-64406-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:06.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:06 smithi175 bash[29284]: audit 2024-03-26T03:52:05.707282+0000 mon.a (mon.0) 2075 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi175-64406-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:06.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:06 smithi175 bash[29284]: audit 2024-03-26T03:52:06.662660+0000 mon.a (mon.0) 2076 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi175-64406-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:52:06.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:06 smithi175 bash[29284]: cluster 2024-03-26T03:52:06.670788+0000 mon.a (mon.0) 2077 : cluster [DBG] osdmap e270: 8 total, 8 up, 8 in 2024-03-26T03:52:06.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:06 smithi175 bash[29284]: audit 2024-03-26T03:52:06.673271+0000 mon.c (mon.2) 441 : audit [INF] from='client.? 172.21.15.175:0/3341683102' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi175-64406-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi175-64406-53"}]: dispatch 2024-03-26T03:52:06.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:06 smithi175 bash[29284]: audit 2024-03-26T03:52:06.675457+0000 mon.a (mon.0) 2078 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:52:06.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:06 smithi175 bash[29284]: audit 2024-03-26T03:52:06.676359+0000 mon.a (mon.0) 2079 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi175-64406-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi175-64406-53"}]: dispatch 2024-03-26T03:52:07.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:07 smithi190 bash[23392]: cluster 2024-03-26T03:52:06.674790+0000 mgr.y (mgr.24431) 314 : cluster [DBG] pgmap v402: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 607 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-03-26T03:52:07.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:07 smithi190 bash[23392]: audit 2024-03-26T03:52:07.666677+0000 mon.a (mon.0) 2080 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:52:07.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:07 smithi190 bash[23392]: cluster 2024-03-26T03:52:07.689521+0000 mon.a (mon.0) 2081 : cluster [DBG] osdmap e271: 8 total, 8 up, 8 in 2024-03-26T03:52:07.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:07 smithi175 bash[29284]: cluster 2024-03-26T03:52:06.674790+0000 mgr.y (mgr.24431) 314 : cluster [DBG] pgmap v402: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 607 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-03-26T03:52:07.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:07 smithi175 bash[29284]: audit 2024-03-26T03:52:07.666677+0000 mon.a (mon.0) 2080 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:52:07.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:07 smithi175 bash[29284]: cluster 2024-03-26T03:52:07.689521+0000 mon.a (mon.0) 2081 : cluster [DBG] osdmap e271: 8 total, 8 up, 8 in 2024-03-26T03:52:07.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:07 smithi175 bash[21527]: cluster 2024-03-26T03:52:06.674790+0000 mgr.y (mgr.24431) 314 : cluster [DBG] pgmap v402: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 607 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-03-26T03:52:07.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:07 smithi175 bash[21527]: audit 2024-03-26T03:52:07.666677+0000 mon.a (mon.0) 2080 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:52:07.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:07 smithi175 bash[21527]: cluster 2024-03-26T03:52:07.689521+0000 mon.a (mon.0) 2081 : cluster [DBG] osdmap e271: 8 total, 8 up, 8 in 2024-03-26T03:52:08.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:08 smithi190 bash[23392]: audit 2024-03-26T03:52:07.715450+0000 mon.a (mon.0) 2082 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:52:08.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:08 smithi190 bash[23392]: audit 2024-03-26T03:52:08.670054+0000 mon.a (mon.0) 2083 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi175-64406-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi175-64406-53"}]': finished 2024-03-26T03:52:08.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:08 smithi190 bash[23392]: audit 2024-03-26T03:52:08.670294+0000 mon.a (mon.0) 2084 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-37", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:52:08.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:08 smithi190 bash[23392]: cluster 2024-03-26T03:52:08.677698+0000 mon.a (mon.0) 2085 : cluster [DBG] osdmap e272: 8 total, 8 up, 8 in 2024-03-26T03:52:08.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:08 smithi190 bash[23392]: audit 2024-03-26T03:52:08.681658+0000 mon.a (mon.0) 2086 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-37"}]: dispatch 2024-03-26T03:52:08.992 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:52:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:52:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:52:08.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:08 smithi175 bash[21527]: audit 2024-03-26T03:52:07.715450+0000 mon.a (mon.0) 2082 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:52:08.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:08 smithi175 bash[21527]: audit 2024-03-26T03:52:08.670054+0000 mon.a (mon.0) 2083 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi175-64406-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi175-64406-53"}]': finished 2024-03-26T03:52:08.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:08 smithi175 bash[21527]: audit 2024-03-26T03:52:08.670294+0000 mon.a (mon.0) 2084 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-37", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:52:08.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:08 smithi175 bash[21527]: cluster 2024-03-26T03:52:08.677698+0000 mon.a (mon.0) 2085 : cluster [DBG] osdmap e272: 8 total, 8 up, 8 in 2024-03-26T03:52:08.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:08 smithi175 bash[21527]: audit 2024-03-26T03:52:08.681658+0000 mon.a (mon.0) 2086 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-37"}]: dispatch 2024-03-26T03:52:08.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:08 smithi175 bash[29284]: audit 2024-03-26T03:52:07.715450+0000 mon.a (mon.0) 2082 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:52:08.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:08 smithi175 bash[29284]: audit 2024-03-26T03:52:08.670054+0000 mon.a (mon.0) 2083 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi175-64406-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi175-64406-53"}]': finished 2024-03-26T03:52:08.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:08 smithi175 bash[29284]: audit 2024-03-26T03:52:08.670294+0000 mon.a (mon.0) 2084 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-37", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:52:08.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:08 smithi175 bash[29284]: cluster 2024-03-26T03:52:08.677698+0000 mon.a (mon.0) 2085 : cluster [DBG] osdmap e272: 8 total, 8 up, 8 in 2024-03-26T03:52:08.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:08 smithi175 bash[29284]: audit 2024-03-26T03:52:08.681658+0000 mon.a (mon.0) 2086 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-37"}]: dispatch 2024-03-26T03:52:09.989 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:52:09 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:52:09.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:09 smithi190 bash[23392]: cluster 2024-03-26T03:52:08.681387+0000 mgr.y (mgr.24431) 315 : cluster [DBG] pgmap v405: 300 pgs: 40 unknown, 260 active+clean; 4.6 MiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:09.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:09 smithi190 bash[23392]: audit 2024-03-26T03:52:09.681780+0000 mon.a (mon.0) 2087 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-37"}]': finished 2024-03-26T03:52:09.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:09 smithi190 bash[23392]: cluster 2024-03-26T03:52:09.690668+0000 mon.a (mon.0) 2088 : cluster [DBG] osdmap e273: 8 total, 8 up, 8 in 2024-03-26T03:52:09.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:09 smithi190 bash[23392]: audit 2024-03-26T03:52:09.692750+0000 mon.a (mon.0) 2089 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-37", "mode": "writeback"}]: dispatch 2024-03-26T03:52:09.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:09 smithi175 bash[29284]: cluster 2024-03-26T03:52:08.681387+0000 mgr.y (mgr.24431) 315 : cluster [DBG] pgmap v405: 300 pgs: 40 unknown, 260 active+clean; 4.6 MiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:09.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:09 smithi175 bash[29284]: audit 2024-03-26T03:52:09.681780+0000 mon.a (mon.0) 2087 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-37"}]': finished 2024-03-26T03:52:09.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:09 smithi175 bash[29284]: cluster 2024-03-26T03:52:09.690668+0000 mon.a (mon.0) 2088 : cluster [DBG] osdmap e273: 8 total, 8 up, 8 in 2024-03-26T03:52:09.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:09 smithi175 bash[29284]: audit 2024-03-26T03:52:09.692750+0000 mon.a (mon.0) 2089 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-37", "mode": "writeback"}]: dispatch 2024-03-26T03:52:09.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:09 smithi175 bash[21527]: cluster 2024-03-26T03:52:08.681387+0000 mgr.y (mgr.24431) 315 : cluster [DBG] pgmap v405: 300 pgs: 40 unknown, 260 active+clean; 4.6 MiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:09.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:09 smithi175 bash[21527]: audit 2024-03-26T03:52:09.681780+0000 mon.a (mon.0) 2087 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-37"}]': finished 2024-03-26T03:52:09.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:09 smithi175 bash[21527]: cluster 2024-03-26T03:52:09.690668+0000 mon.a (mon.0) 2088 : cluster [DBG] osdmap e273: 8 total, 8 up, 8 in 2024-03-26T03:52:09.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:09 smithi175 bash[21527]: audit 2024-03-26T03:52:09.692750+0000 mon.a (mon.0) 2089 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-37", "mode": "writeback"}]: dispatch 2024-03-26T03:52:10.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:10 smithi190 bash[23392]: audit 2024-03-26T03:52:09.881208+0000 mgr.y (mgr.24431) 316 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:52:10.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:10 smithi190 bash[23392]: cluster 2024-03-26T03:52:10.682318+0000 mon.a (mon.0) 2090 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:52:10.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:10 smithi190 bash[23392]: audit 2024-03-26T03:52:10.685469+0000 mon.a (mon.0) 2091 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-37", "mode": "writeback"}]': finished 2024-03-26T03:52:10.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:10 smithi190 bash[23392]: cluster 2024-03-26T03:52:10.696825+0000 mon.a (mon.0) 2092 : cluster [DBG] osdmap e274: 8 total, 8 up, 8 in 2024-03-26T03:52:10.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:10 smithi190 bash[23392]: audit 2024-03-26T03:52:10.705660+0000 mon.c (mon.2) 442 : audit [INF] from='client.? 172.21.15.175:0/3341683102' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi175-64406-53"}]: dispatch 2024-03-26T03:52:10.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:10 smithi190 bash[23392]: audit 2024-03-26T03:52:10.713535+0000 mon.a (mon.0) 2093 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi175-64406-53"}]: dispatch 2024-03-26T03:52:10.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:10 smithi175 bash[21527]: audit 2024-03-26T03:52:09.881208+0000 mgr.y (mgr.24431) 316 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:52:10.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:10 smithi175 bash[21527]: cluster 2024-03-26T03:52:10.682318+0000 mon.a (mon.0) 2090 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:52:10.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:10 smithi175 bash[21527]: audit 2024-03-26T03:52:10.685469+0000 mon.a (mon.0) 2091 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-37", "mode": "writeback"}]': finished 2024-03-26T03:52:10.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:10 smithi175 bash[21527]: cluster 2024-03-26T03:52:10.696825+0000 mon.a (mon.0) 2092 : cluster [DBG] osdmap e274: 8 total, 8 up, 8 in 2024-03-26T03:52:10.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:10 smithi175 bash[21527]: audit 2024-03-26T03:52:10.705660+0000 mon.c (mon.2) 442 : audit [INF] from='client.? 172.21.15.175:0/3341683102' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi175-64406-53"}]: dispatch 2024-03-26T03:52:10.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:10 smithi175 bash[21527]: audit 2024-03-26T03:52:10.713535+0000 mon.a (mon.0) 2093 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi175-64406-53"}]: dispatch 2024-03-26T03:52:10.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:10 smithi175 bash[29284]: audit 2024-03-26T03:52:09.881208+0000 mgr.y (mgr.24431) 316 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:52:10.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:10 smithi175 bash[29284]: cluster 2024-03-26T03:52:10.682318+0000 mon.a (mon.0) 2090 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:52:10.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:10 smithi175 bash[29284]: audit 2024-03-26T03:52:10.685469+0000 mon.a (mon.0) 2091 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-37", "mode": "writeback"}]': finished 2024-03-26T03:52:10.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:10 smithi175 bash[29284]: cluster 2024-03-26T03:52:10.696825+0000 mon.a (mon.0) 2092 : cluster [DBG] osdmap e274: 8 total, 8 up, 8 in 2024-03-26T03:52:10.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:10 smithi175 bash[29284]: audit 2024-03-26T03:52:10.705660+0000 mon.c (mon.2) 442 : audit [INF] from='client.? 172.21.15.175:0/3341683102' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi175-64406-53"}]: dispatch 2024-03-26T03:52:10.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:10 smithi175 bash[29284]: audit 2024-03-26T03:52:10.713535+0000 mon.a (mon.0) 2093 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi175-64406-53"}]: dispatch 2024-03-26T03:52:11.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:11 smithi190 bash[23392]: cluster 2024-03-26T03:52:10.683431+0000 mgr.y (mgr.24431) 317 : cluster [DBG] pgmap v407: 300 pgs: 6 creating+peering, 294 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:11.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:11 smithi190 bash[23392]: audit 2024-03-26T03:52:10.807384+0000 mon.a (mon.0) 2094 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:52:11.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:11 smithi190 bash[23392]: audit 2024-03-26T03:52:11.463535+0000 mon.a (mon.0) 2095 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi175-64406-53"}]': finished 2024-03-26T03:52:11.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:11 smithi190 bash[23392]: audit 2024-03-26T03:52:11.463753+0000 mon.a (mon.0) 2096 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:52:11.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:11 smithi190 bash[23392]: cluster 2024-03-26T03:52:11.474213+0000 mon.a (mon.0) 2097 : cluster [DBG] osdmap e275: 8 total, 8 up, 8 in 2024-03-26T03:52:11.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:11 smithi190 bash[23392]: audit 2024-03-26T03:52:11.475261+0000 mon.a (mon.0) 2098 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-37"}]: dispatch 2024-03-26T03:52:11.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:11 smithi190 bash[23392]: audit 2024-03-26T03:52:11.476313+0000 mon.c (mon.2) 443 : audit [INF] from='client.? 172.21.15.175:0/3341683102' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi175-64406-53"}]: dispatch 2024-03-26T03:52:11.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:11 smithi190 bash[23392]: audit 2024-03-26T03:52:11.477212+0000 mon.a (mon.0) 2099 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi175-64406-53"}]: dispatch 2024-03-26T03:52:11.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:11 smithi175 bash[21527]: cluster 2024-03-26T03:52:10.683431+0000 mgr.y (mgr.24431) 317 : cluster [DBG] pgmap v407: 300 pgs: 6 creating+peering, 294 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:11.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:11 smithi175 bash[21527]: audit 2024-03-26T03:52:10.807384+0000 mon.a (mon.0) 2094 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:52:11.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:11 smithi175 bash[21527]: audit 2024-03-26T03:52:11.463535+0000 mon.a (mon.0) 2095 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi175-64406-53"}]': finished 2024-03-26T03:52:11.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:11 smithi175 bash[21527]: audit 2024-03-26T03:52:11.463753+0000 mon.a (mon.0) 2096 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:52:11.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:11 smithi175 bash[21527]: cluster 2024-03-26T03:52:11.474213+0000 mon.a (mon.0) 2097 : cluster [DBG] osdmap e275: 8 total, 8 up, 8 in 2024-03-26T03:52:11.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:11 smithi175 bash[21527]: audit 2024-03-26T03:52:11.475261+0000 mon.a (mon.0) 2098 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-37"}]: dispatch 2024-03-26T03:52:11.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:11 smithi175 bash[21527]: audit 2024-03-26T03:52:11.476313+0000 mon.c (mon.2) 443 : audit [INF] from='client.? 172.21.15.175:0/3341683102' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi175-64406-53"}]: dispatch 2024-03-26T03:52:11.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:11 smithi175 bash[21527]: audit 2024-03-26T03:52:11.477212+0000 mon.a (mon.0) 2099 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi175-64406-53"}]: dispatch 2024-03-26T03:52:11.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:11 smithi175 bash[29284]: cluster 2024-03-26T03:52:10.683431+0000 mgr.y (mgr.24431) 317 : cluster [DBG] pgmap v407: 300 pgs: 6 creating+peering, 294 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:11.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:11 smithi175 bash[29284]: audit 2024-03-26T03:52:10.807384+0000 mon.a (mon.0) 2094 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:52:11.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:11 smithi175 bash[29284]: audit 2024-03-26T03:52:11.463535+0000 mon.a (mon.0) 2095 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi175-64406-53"}]': finished 2024-03-26T03:52:11.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:11 smithi175 bash[29284]: audit 2024-03-26T03:52:11.463753+0000 mon.a (mon.0) 2096 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:52:11.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:11 smithi175 bash[29284]: cluster 2024-03-26T03:52:11.474213+0000 mon.a (mon.0) 2097 : cluster [DBG] osdmap e275: 8 total, 8 up, 8 in 2024-03-26T03:52:11.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:11 smithi175 bash[29284]: audit 2024-03-26T03:52:11.475261+0000 mon.a (mon.0) 2098 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-37"}]: dispatch 2024-03-26T03:52:11.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:11 smithi175 bash[29284]: audit 2024-03-26T03:52:11.476313+0000 mon.c (mon.2) 443 : audit [INF] from='client.? 172.21.15.175:0/3341683102' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi175-64406-53"}]: dispatch 2024-03-26T03:52:11.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:11 smithi175 bash[29284]: audit 2024-03-26T03:52:11.477212+0000 mon.a (mon.0) 2099 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi175-64406-53"}]: dispatch 2024-03-26T03:52:12.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:12 smithi190 bash[23392]: cluster 2024-03-26T03:52:12.464895+0000 mon.a (mon.0) 2100 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:52:12.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:12 smithi190 bash[23392]: audit 2024-03-26T03:52:12.467531+0000 mon.a (mon.0) 2101 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-37"}]': finished 2024-03-26T03:52:12.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:12 smithi190 bash[23392]: audit 2024-03-26T03:52:12.467739+0000 mon.a (mon.0) 2102 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi175-64406-53"}]': finished 2024-03-26T03:52:12.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:12 smithi190 bash[23392]: cluster 2024-03-26T03:52:12.475350+0000 mon.a (mon.0) 2103 : cluster [DBG] osdmap e276: 8 total, 8 up, 8 in 2024-03-26T03:52:12.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:12 smithi190 bash[23392]: audit 2024-03-26T03:52:12.530393+0000 mon.a (mon.0) 2104 : audit [INF] from='client.? 172.21.15.175:0/757500619' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi175-64406-54"}]: dispatch 2024-03-26T03:52:12.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:12 smithi190 bash[23392]: audit 2024-03-26T03:52:12.531541+0000 mon.a (mon.0) 2105 : audit [INF] from='client.? 172.21.15.175:0/757500619' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi175-64406-54"}]: dispatch 2024-03-26T03:52:12.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:12 smithi190 bash[23392]: audit 2024-03-26T03:52:12.531996+0000 mon.a (mon.0) 2106 : audit [INF] from='client.? 172.21.15.175:0/757500619' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi175-64406-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:12.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:12 smithi175 bash[21527]: cluster 2024-03-26T03:52:12.464895+0000 mon.a (mon.0) 2100 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:52:12.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:12 smithi175 bash[21527]: audit 2024-03-26T03:52:12.467531+0000 mon.a (mon.0) 2101 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-37"}]': finished 2024-03-26T03:52:12.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:12 smithi175 bash[21527]: audit 2024-03-26T03:52:12.467739+0000 mon.a (mon.0) 2102 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi175-64406-53"}]': finished 2024-03-26T03:52:12.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:12 smithi175 bash[21527]: cluster 2024-03-26T03:52:12.475350+0000 mon.a (mon.0) 2103 : cluster [DBG] osdmap e276: 8 total, 8 up, 8 in 2024-03-26T03:52:12.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:12 smithi175 bash[21527]: audit 2024-03-26T03:52:12.530393+0000 mon.a (mon.0) 2104 : audit [INF] from='client.? 172.21.15.175:0/757500619' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi175-64406-54"}]: dispatch 2024-03-26T03:52:12.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:12 smithi175 bash[21527]: audit 2024-03-26T03:52:12.531541+0000 mon.a (mon.0) 2105 : audit [INF] from='client.? 172.21.15.175:0/757500619' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi175-64406-54"}]: dispatch 2024-03-26T03:52:12.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:12 smithi175 bash[21527]: audit 2024-03-26T03:52:12.531996+0000 mon.a (mon.0) 2106 : audit [INF] from='client.? 172.21.15.175:0/757500619' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi175-64406-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:12.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:12 smithi175 bash[29284]: cluster 2024-03-26T03:52:12.464895+0000 mon.a (mon.0) 2100 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:52:12.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:12 smithi175 bash[29284]: audit 2024-03-26T03:52:12.467531+0000 mon.a (mon.0) 2101 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-37"}]': finished 2024-03-26T03:52:12.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:12 smithi175 bash[29284]: audit 2024-03-26T03:52:12.467739+0000 mon.a (mon.0) 2102 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi175-64406-53"}]': finished 2024-03-26T03:52:12.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:12 smithi175 bash[29284]: cluster 2024-03-26T03:52:12.475350+0000 mon.a (mon.0) 2103 : cluster [DBG] osdmap e276: 8 total, 8 up, 8 in 2024-03-26T03:52:12.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:12 smithi175 bash[29284]: audit 2024-03-26T03:52:12.530393+0000 mon.a (mon.0) 2104 : audit [INF] from='client.? 172.21.15.175:0/757500619' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi175-64406-54"}]: dispatch 2024-03-26T03:52:12.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:12 smithi175 bash[29284]: audit 2024-03-26T03:52:12.531541+0000 mon.a (mon.0) 2105 : audit [INF] from='client.? 172.21.15.175:0/757500619' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi175-64406-54"}]: dispatch 2024-03-26T03:52:12.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:12 smithi175 bash[29284]: audit 2024-03-26T03:52:12.531996+0000 mon.a (mon.0) 2106 : audit [INF] from='client.? 172.21.15.175:0/757500619' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi175-64406-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:13.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:13 smithi190 bash[23392]: cluster 2024-03-26T03:52:12.684370+0000 mgr.y (mgr.24431) 318 : cluster [DBG] pgmap v411: 292 pgs: 292 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:13.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:13 smithi190 bash[23392]: cluster 2024-03-26T03:52:13.467539+0000 mon.a (mon.0) 2107 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:52:13.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:13 smithi190 bash[23392]: audit 2024-03-26T03:52:13.477422+0000 mon.a (mon.0) 2108 : audit [INF] from='client.? 172.21.15.175:0/757500619' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi175-64406-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:52:13.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:13 smithi190 bash[23392]: cluster 2024-03-26T03:52:13.483475+0000 mon.a (mon.0) 2109 : cluster [DBG] osdmap e277: 8 total, 8 up, 8 in 2024-03-26T03:52:13.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:13 smithi190 bash[23392]: audit 2024-03-26T03:52:13.484467+0000 mon.a (mon.0) 2110 : audit [INF] from='client.? 172.21.15.175:0/757500619' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi175-64406-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi175-64406-54"}]: dispatch 2024-03-26T03:52:13.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:13 smithi175 bash[21527]: cluster 2024-03-26T03:52:12.684370+0000 mgr.y (mgr.24431) 318 : cluster [DBG] pgmap v411: 292 pgs: 292 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:13.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:13 smithi175 bash[21527]: cluster 2024-03-26T03:52:13.467539+0000 mon.a (mon.0) 2107 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:52:13.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:13 smithi175 bash[21527]: audit 2024-03-26T03:52:13.477422+0000 mon.a (mon.0) 2108 : audit [INF] from='client.? 172.21.15.175:0/757500619' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi175-64406-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:52:13.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:13 smithi175 bash[21527]: cluster 2024-03-26T03:52:13.483475+0000 mon.a (mon.0) 2109 : cluster [DBG] osdmap e277: 8 total, 8 up, 8 in 2024-03-26T03:52:13.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:13 smithi175 bash[21527]: audit 2024-03-26T03:52:13.484467+0000 mon.a (mon.0) 2110 : audit [INF] from='client.? 172.21.15.175:0/757500619' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi175-64406-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi175-64406-54"}]: dispatch 2024-03-26T03:52:13.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:13 smithi175 bash[29284]: cluster 2024-03-26T03:52:12.684370+0000 mgr.y (mgr.24431) 318 : cluster [DBG] pgmap v411: 292 pgs: 292 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:13.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:13 smithi175 bash[29284]: cluster 2024-03-26T03:52:13.467539+0000 mon.a (mon.0) 2107 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:52:13.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:13 smithi175 bash[29284]: audit 2024-03-26T03:52:13.477422+0000 mon.a (mon.0) 2108 : audit [INF] from='client.? 172.21.15.175:0/757500619' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi175-64406-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:52:13.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:13 smithi175 bash[29284]: cluster 2024-03-26T03:52:13.483475+0000 mon.a (mon.0) 2109 : cluster [DBG] osdmap e277: 8 total, 8 up, 8 in 2024-03-26T03:52:13.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:13 smithi175 bash[29284]: audit 2024-03-26T03:52:13.484467+0000 mon.a (mon.0) 2110 : audit [INF] from='client.? 172.21.15.175:0/757500619' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi175-64406-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi175-64406-54"}]: dispatch 2024-03-26T03:52:15.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:15 smithi190 bash[23392]: cluster 2024-03-26T03:52:14.508811+0000 mon.a (mon.0) 2111 : cluster [DBG] osdmap e278: 8 total, 8 up, 8 in 2024-03-26T03:52:15.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:15 smithi190 bash[23392]: audit 2024-03-26T03:52:14.513663+0000 mon.a (mon.0) 2112 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:52:15.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:15 smithi175 bash[29284]: cluster 2024-03-26T03:52:14.508811+0000 mon.a (mon.0) 2111 : cluster [DBG] osdmap e278: 8 total, 8 up, 8 in 2024-03-26T03:52:15.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:15 smithi175 bash[29284]: audit 2024-03-26T03:52:14.513663+0000 mon.a (mon.0) 2112 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:52:15.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:15 smithi175 bash[21527]: cluster 2024-03-26T03:52:14.508811+0000 mon.a (mon.0) 2111 : cluster [DBG] osdmap e278: 8 total, 8 up, 8 in 2024-03-26T03:52:15.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:15 smithi175 bash[21527]: audit 2024-03-26T03:52:14.513663+0000 mon.a (mon.0) 2112 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:52:16.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:16 smithi190 bash[23392]: cluster 2024-03-26T03:52:14.686285+0000 mgr.y (mgr.24431) 319 : cluster [DBG] pgmap v414: 292 pgs: 21 creating+peering, 11 unknown, 260 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s wr, 1 op/s 2024-03-26T03:52:16.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:16 smithi190 bash[23392]: audit 2024-03-26T03:52:15.496365+0000 mon.a (mon.0) 2113 : audit [INF] from='client.? 172.21.15.175:0/757500619' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi175-64406-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi175-64406-54"}]': finished 2024-03-26T03:52:16.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:16 smithi190 bash[23392]: audit 2024-03-26T03:52:15.496524+0000 mon.a (mon.0) 2114 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:52:16.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:16 smithi190 bash[23392]: cluster 2024-03-26T03:52:15.507998+0000 mon.a (mon.0) 2115 : cluster [DBG] osdmap e279: 8 total, 8 up, 8 in 2024-03-26T03:52:16.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:16 smithi190 bash[23392]: audit 2024-03-26T03:52:15.511016+0000 mon.a (mon.0) 2116 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:52:16.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:16 smithi175 bash[29284]: cluster 2024-03-26T03:52:14.686285+0000 mgr.y (mgr.24431) 319 : cluster [DBG] pgmap v414: 292 pgs: 21 creating+peering, 11 unknown, 260 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s wr, 1 op/s 2024-03-26T03:52:16.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:16 smithi175 bash[29284]: audit 2024-03-26T03:52:15.496365+0000 mon.a (mon.0) 2113 : audit [INF] from='client.? 172.21.15.175:0/757500619' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi175-64406-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi175-64406-54"}]': finished 2024-03-26T03:52:16.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:16 smithi175 bash[29284]: audit 2024-03-26T03:52:15.496524+0000 mon.a (mon.0) 2114 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:52:16.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:16 smithi175 bash[29284]: cluster 2024-03-26T03:52:15.507998+0000 mon.a (mon.0) 2115 : cluster [DBG] osdmap e279: 8 total, 8 up, 8 in 2024-03-26T03:52:16.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:16 smithi175 bash[29284]: audit 2024-03-26T03:52:15.511016+0000 mon.a (mon.0) 2116 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:52:16.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:16 smithi175 bash[21527]: cluster 2024-03-26T03:52:14.686285+0000 mgr.y (mgr.24431) 319 : cluster [DBG] pgmap v414: 292 pgs: 21 creating+peering, 11 unknown, 260 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s wr, 1 op/s 2024-03-26T03:52:16.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:16 smithi175 bash[21527]: audit 2024-03-26T03:52:15.496365+0000 mon.a (mon.0) 2113 : audit [INF] from='client.? 172.21.15.175:0/757500619' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi175-64406-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi175-64406-54"}]': finished 2024-03-26T03:52:16.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:16 smithi175 bash[21527]: audit 2024-03-26T03:52:15.496524+0000 mon.a (mon.0) 2114 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:52:16.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:16 smithi175 bash[21527]: cluster 2024-03-26T03:52:15.507998+0000 mon.a (mon.0) 2115 : cluster [DBG] osdmap e279: 8 total, 8 up, 8 in 2024-03-26T03:52:16.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:16 smithi175 bash[21527]: audit 2024-03-26T03:52:15.511016+0000 mon.a (mon.0) 2116 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:52:17.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:17 smithi190 bash[23392]: audit 2024-03-26T03:52:16.511998+0000 mon.a (mon.0) 2117 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-39", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:52:17.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:17 smithi190 bash[23392]: cluster 2024-03-26T03:52:16.520793+0000 mon.a (mon.0) 2118 : cluster [DBG] osdmap e280: 8 total, 8 up, 8 in 2024-03-26T03:52:17.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:17 smithi190 bash[23392]: audit 2024-03-26T03:52:16.523799+0000 mon.a (mon.0) 2119 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-39"}]: dispatch 2024-03-26T03:52:17.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:17 smithi175 bash[29284]: audit 2024-03-26T03:52:16.511998+0000 mon.a (mon.0) 2117 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-39", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:52:17.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:17 smithi175 bash[29284]: cluster 2024-03-26T03:52:16.520793+0000 mon.a (mon.0) 2118 : cluster [DBG] osdmap e280: 8 total, 8 up, 8 in 2024-03-26T03:52:17.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:17 smithi175 bash[29284]: audit 2024-03-26T03:52:16.523799+0000 mon.a (mon.0) 2119 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-39"}]: dispatch 2024-03-26T03:52:17.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:17 smithi175 bash[21527]: audit 2024-03-26T03:52:16.511998+0000 mon.a (mon.0) 2117 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-39", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:52:17.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:17 smithi175 bash[21527]: cluster 2024-03-26T03:52:16.520793+0000 mon.a (mon.0) 2118 : cluster [DBG] osdmap e280: 8 total, 8 up, 8 in 2024-03-26T03:52:17.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:17 smithi175 bash[21527]: audit 2024-03-26T03:52:16.523799+0000 mon.a (mon.0) 2119 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-39"}]: dispatch 2024-03-26T03:52:18.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:18 smithi190 bash[23392]: cluster 2024-03-26T03:52:16.687232+0000 mgr.y (mgr.24431) 320 : cluster [DBG] pgmap v417: 300 pgs: 8 unknown, 32 creating+peering, 260 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-26T03:52:18.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:18 smithi190 bash[23392]: audit 2024-03-26T03:52:17.521097+0000 mon.a (mon.0) 2120 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-39"}]': finished 2024-03-26T03:52:18.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:18 smithi190 bash[23392]: cluster 2024-03-26T03:52:17.532287+0000 mon.a (mon.0) 2121 : cluster [DBG] osdmap e281: 8 total, 8 up, 8 in 2024-03-26T03:52:18.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:18 smithi190 bash[23392]: audit 2024-03-26T03:52:17.534399+0000 mon.a (mon.0) 2122 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-39", "mode": "writeback"}]: dispatch 2024-03-26T03:52:18.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:18 smithi190 bash[23392]: audit 2024-03-26T03:52:17.536511+0000 mon.a (mon.0) 2123 : audit [INF] from='client.? 172.21.15.175:0/757500619' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi175-64406-54"}]: dispatch 2024-03-26T03:52:18.992 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:52:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:52:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:52:18.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:18 smithi175 bash[21527]: cluster 2024-03-26T03:52:16.687232+0000 mgr.y (mgr.24431) 320 : cluster [DBG] pgmap v417: 300 pgs: 8 unknown, 32 creating+peering, 260 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-26T03:52:18.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:18 smithi175 bash[21527]: audit 2024-03-26T03:52:17.521097+0000 mon.a (mon.0) 2120 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-39"}]': finished 2024-03-26T03:52:18.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:18 smithi175 bash[21527]: cluster 2024-03-26T03:52:17.532287+0000 mon.a (mon.0) 2121 : cluster [DBG] osdmap e281: 8 total, 8 up, 8 in 2024-03-26T03:52:18.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:18 smithi175 bash[21527]: audit 2024-03-26T03:52:17.534399+0000 mon.a (mon.0) 2122 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-39", "mode": "writeback"}]: dispatch 2024-03-26T03:52:18.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:18 smithi175 bash[21527]: audit 2024-03-26T03:52:17.536511+0000 mon.a (mon.0) 2123 : audit [INF] from='client.? 172.21.15.175:0/757500619' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi175-64406-54"}]: dispatch 2024-03-26T03:52:18.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:18 smithi175 bash[29284]: cluster 2024-03-26T03:52:16.687232+0000 mgr.y (mgr.24431) 320 : cluster [DBG] pgmap v417: 300 pgs: 8 unknown, 32 creating+peering, 260 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-26T03:52:18.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:18 smithi175 bash[29284]: audit 2024-03-26T03:52:17.521097+0000 mon.a (mon.0) 2120 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-39"}]': finished 2024-03-26T03:52:18.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:18 smithi175 bash[29284]: cluster 2024-03-26T03:52:17.532287+0000 mon.a (mon.0) 2121 : cluster [DBG] osdmap e281: 8 total, 8 up, 8 in 2024-03-26T03:52:18.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:18 smithi175 bash[29284]: audit 2024-03-26T03:52:17.534399+0000 mon.a (mon.0) 2122 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-39", "mode": "writeback"}]: dispatch 2024-03-26T03:52:18.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:18 smithi175 bash[29284]: audit 2024-03-26T03:52:17.536511+0000 mon.a (mon.0) 2123 : audit [INF] from='client.? 172.21.15.175:0/757500619' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi175-64406-54"}]: dispatch 2024-03-26T03:52:19.886 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:19 smithi190 bash[23392]: cluster 2024-03-26T03:52:18.521584+0000 mon.a (mon.0) 2124 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:52:19.886 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:19 smithi190 bash[23392]: audit 2024-03-26T03:52:18.523854+0000 mon.a (mon.0) 2125 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-39", "mode": "writeback"}]': finished 2024-03-26T03:52:19.886 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:19 smithi190 bash[23392]: audit 2024-03-26T03:52:18.524027+0000 mon.a (mon.0) 2126 : audit [INF] from='client.? 172.21.15.175:0/757500619' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi175-64406-54"}]': finished 2024-03-26T03:52:19.886 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:19 smithi190 bash[23392]: cluster 2024-03-26T03:52:18.531013+0000 mon.a (mon.0) 2127 : cluster [DBG] osdmap e282: 8 total, 8 up, 8 in 2024-03-26T03:52:19.886 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:19 smithi190 bash[23392]: audit 2024-03-26T03:52:18.533298+0000 mon.a (mon.0) 2128 : audit [INF] from='client.? 172.21.15.175:0/757500619' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi175-64406-54"}]: dispatch 2024-03-26T03:52:19.886 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:19 smithi190 bash[23392]: cluster 2024-03-26T03:52:18.688002+0000 mgr.y (mgr.24431) 321 : cluster [DBG] pgmap v420: 292 pgs: 32 creating+peering, 260 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:19.886 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:19 smithi190 bash[23392]: audit 2024-03-26T03:52:18.778205+0000 mon.a (mon.0) 2129 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:52:19.887 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:19 smithi190 bash[23392]: audit 2024-03-26T03:52:19.527488+0000 mon.a (mon.0) 2130 : audit [INF] from='client.? 172.21.15.175:0/757500619' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi175-64406-54"}]': finished 2024-03-26T03:52:19.887 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:19 smithi190 bash[23392]: audit 2024-03-26T03:52:19.527692+0000 mon.a (mon.0) 2131 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:52:19.887 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:19 smithi190 bash[23392]: cluster 2024-03-26T03:52:19.535000+0000 mon.a (mon.0) 2132 : cluster [DBG] osdmap e283: 8 total, 8 up, 8 in 2024-03-26T03:52:19.887 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:19 smithi190 bash[23392]: audit 2024-03-26T03:52:19.536748+0000 mon.a (mon.0) 2133 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-39"}]: dispatch 2024-03-26T03:52:19.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:19 smithi175 bash[21527]: cluster 2024-03-26T03:52:18.521584+0000 mon.a (mon.0) 2124 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:52:19.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:19 smithi175 bash[21527]: audit 2024-03-26T03:52:18.523854+0000 mon.a (mon.0) 2125 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-39", "mode": "writeback"}]': finished 2024-03-26T03:52:19.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:19 smithi175 bash[21527]: audit 2024-03-26T03:52:18.524027+0000 mon.a (mon.0) 2126 : audit [INF] from='client.? 172.21.15.175:0/757500619' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi175-64406-54"}]': finished 2024-03-26T03:52:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:19 smithi175 bash[21527]: cluster 2024-03-26T03:52:18.531013+0000 mon.a (mon.0) 2127 : cluster [DBG] osdmap e282: 8 total, 8 up, 8 in 2024-03-26T03:52:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:19 smithi175 bash[21527]: audit 2024-03-26T03:52:18.533298+0000 mon.a (mon.0) 2128 : audit [INF] from='client.? 172.21.15.175:0/757500619' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi175-64406-54"}]: dispatch 2024-03-26T03:52:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:19 smithi175 bash[21527]: cluster 2024-03-26T03:52:18.688002+0000 mgr.y (mgr.24431) 321 : cluster [DBG] pgmap v420: 292 pgs: 32 creating+peering, 260 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:19 smithi175 bash[21527]: audit 2024-03-26T03:52:18.778205+0000 mon.a (mon.0) 2129 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:52:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:19 smithi175 bash[21527]: audit 2024-03-26T03:52:19.527488+0000 mon.a (mon.0) 2130 : audit [INF] from='client.? 172.21.15.175:0/757500619' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi175-64406-54"}]': finished 2024-03-26T03:52:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:19 smithi175 bash[21527]: audit 2024-03-26T03:52:19.527692+0000 mon.a (mon.0) 2131 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:52:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:19 smithi175 bash[21527]: cluster 2024-03-26T03:52:19.535000+0000 mon.a (mon.0) 2132 : cluster [DBG] osdmap e283: 8 total, 8 up, 8 in 2024-03-26T03:52:19.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:19 smithi175 bash[21527]: audit 2024-03-26T03:52:19.536748+0000 mon.a (mon.0) 2133 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-39"}]: dispatch 2024-03-26T03:52:19.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:19 smithi175 bash[29284]: cluster 2024-03-26T03:52:18.521584+0000 mon.a (mon.0) 2124 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:52:19.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:19 smithi175 bash[29284]: audit 2024-03-26T03:52:18.523854+0000 mon.a (mon.0) 2125 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-39", "mode": "writeback"}]': finished 2024-03-26T03:52:19.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:19 smithi175 bash[29284]: audit 2024-03-26T03:52:18.524027+0000 mon.a (mon.0) 2126 : audit [INF] from='client.? 172.21.15.175:0/757500619' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi175-64406-54"}]': finished 2024-03-26T03:52:19.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:19 smithi175 bash[29284]: cluster 2024-03-26T03:52:18.531013+0000 mon.a (mon.0) 2127 : cluster [DBG] osdmap e282: 8 total, 8 up, 8 in 2024-03-26T03:52:19.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:19 smithi175 bash[29284]: audit 2024-03-26T03:52:18.533298+0000 mon.a (mon.0) 2128 : audit [INF] from='client.? 172.21.15.175:0/757500619' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi175-64406-54"}]: dispatch 2024-03-26T03:52:19.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:19 smithi175 bash[29284]: cluster 2024-03-26T03:52:18.688002+0000 mgr.y (mgr.24431) 321 : cluster [DBG] pgmap v420: 292 pgs: 32 creating+peering, 260 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:19.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:19 smithi175 bash[29284]: audit 2024-03-26T03:52:18.778205+0000 mon.a (mon.0) 2129 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:52:19.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:19 smithi175 bash[29284]: audit 2024-03-26T03:52:19.527488+0000 mon.a (mon.0) 2130 : audit [INF] from='client.? 172.21.15.175:0/757500619' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi175-64406-54"}]': finished 2024-03-26T03:52:19.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:19 smithi175 bash[29284]: audit 2024-03-26T03:52:19.527692+0000 mon.a (mon.0) 2131 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:52:19.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:19 smithi175 bash[29284]: cluster 2024-03-26T03:52:19.535000+0000 mon.a (mon.0) 2132 : cluster [DBG] osdmap e283: 8 total, 8 up, 8 in 2024-03-26T03:52:19.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:19 smithi175 bash[29284]: audit 2024-03-26T03:52:19.536748+0000 mon.a (mon.0) 2133 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-39"}]: dispatch 2024-03-26T03:52:20.239 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:52:19 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:52:20.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:20 smithi190 bash[23392]: cluster 2024-03-26T03:52:19.543850+0000 mon.a (mon.0) 2134 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:52:20.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:20 smithi190 bash[23392]: audit 2024-03-26T03:52:19.550026+0000 mon.c (mon.2) 444 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:52:20.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:20 smithi190 bash[23392]: audit 2024-03-26T03:52:19.583253+0000 mon.c (mon.2) 445 : audit [INF] from='client.? 172.21.15.175:0/2595517674' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi175-64406-55"}]: dispatch 2024-03-26T03:52:20.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:20 smithi190 bash[23392]: audit 2024-03-26T03:52:19.583686+0000 mon.a (mon.0) 2135 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi175-64406-55"}]: dispatch 2024-03-26T03:52:20.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:20 smithi190 bash[23392]: audit 2024-03-26T03:52:19.584323+0000 mon.c (mon.2) 446 : audit [INF] from='client.? 172.21.15.175:0/2595517674' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi175-64406-55"}]: dispatch 2024-03-26T03:52:20.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:20 smithi190 bash[23392]: audit 2024-03-26T03:52:19.584649+0000 mon.a (mon.0) 2136 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi175-64406-55"}]: dispatch 2024-03-26T03:52:20.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:20 smithi190 bash[23392]: audit 2024-03-26T03:52:19.585147+0000 mon.c (mon.2) 447 : audit [INF] from='client.? 172.21.15.175:0/2595517674' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi175-64406-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:20.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:20 smithi190 bash[23392]: audit 2024-03-26T03:52:19.585445+0000 mon.a (mon.0) 2137 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi175-64406-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:20.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:20 smithi190 bash[23392]: audit 2024-03-26T03:52:19.887647+0000 mgr.y (mgr.24431) 322 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:52:20.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:20 smithi190 bash[23392]: cluster 2024-03-26T03:52:20.527773+0000 mon.a (mon.0) 2138 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:52:20.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:20 smithi190 bash[23392]: audit 2024-03-26T03:52:20.530061+0000 mon.a (mon.0) 2139 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-39"}]': finished 2024-03-26T03:52:20.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:20 smithi190 bash[23392]: audit 2024-03-26T03:52:20.530280+0000 mon.a (mon.0) 2140 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi175-64406-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:52:20.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:20 smithi190 bash[23392]: cluster 2024-03-26T03:52:20.537170+0000 mon.a (mon.0) 2141 : cluster [DBG] osdmap e284: 8 total, 8 up, 8 in 2024-03-26T03:52:20.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:20 smithi190 bash[23392]: audit 2024-03-26T03:52:20.542274+0000 mon.c (mon.2) 448 : audit [INF] from='client.? 172.21.15.175:0/2595517674' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi175-64406-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi175-64406-55"}]: dispatch 2024-03-26T03:52:20.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:20 smithi190 bash[23392]: audit 2024-03-26T03:52:20.543656+0000 mon.a (mon.0) 2142 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi175-64406-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi175-64406-55"}]: dispatch 2024-03-26T03:52:20.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:20 smithi175 bash[21527]: cluster 2024-03-26T03:52:19.543850+0000 mon.a (mon.0) 2134 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:52:20.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:20 smithi175 bash[21527]: audit 2024-03-26T03:52:19.550026+0000 mon.c (mon.2) 444 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:52:20.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:20 smithi175 bash[21527]: audit 2024-03-26T03:52:19.583253+0000 mon.c (mon.2) 445 : audit [INF] from='client.? 172.21.15.175:0/2595517674' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi175-64406-55"}]: dispatch 2024-03-26T03:52:20.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:20 smithi175 bash[21527]: audit 2024-03-26T03:52:19.583686+0000 mon.a (mon.0) 2135 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi175-64406-55"}]: dispatch 2024-03-26T03:52:20.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:20 smithi175 bash[21527]: audit 2024-03-26T03:52:19.584323+0000 mon.c (mon.2) 446 : audit [INF] from='client.? 172.21.15.175:0/2595517674' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi175-64406-55"}]: dispatch 2024-03-26T03:52:20.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:20 smithi175 bash[21527]: audit 2024-03-26T03:52:19.584649+0000 mon.a (mon.0) 2136 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi175-64406-55"}]: dispatch 2024-03-26T03:52:20.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:20 smithi175 bash[21527]: audit 2024-03-26T03:52:19.585147+0000 mon.c (mon.2) 447 : audit [INF] from='client.? 172.21.15.175:0/2595517674' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi175-64406-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:20.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:20 smithi175 bash[21527]: audit 2024-03-26T03:52:19.585445+0000 mon.a (mon.0) 2137 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi175-64406-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:20.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:20 smithi175 bash[21527]: audit 2024-03-26T03:52:19.887647+0000 mgr.y (mgr.24431) 322 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:52:20.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:20 smithi175 bash[21527]: cluster 2024-03-26T03:52:20.527773+0000 mon.a (mon.0) 2138 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:52:20.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:20 smithi175 bash[21527]: audit 2024-03-26T03:52:20.530061+0000 mon.a (mon.0) 2139 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-39"}]': finished 2024-03-26T03:52:20.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:20 smithi175 bash[21527]: audit 2024-03-26T03:52:20.530280+0000 mon.a (mon.0) 2140 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi175-64406-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:52:20.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:20 smithi175 bash[21527]: cluster 2024-03-26T03:52:20.537170+0000 mon.a (mon.0) 2141 : cluster [DBG] osdmap e284: 8 total, 8 up, 8 in 2024-03-26T03:52:20.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:20 smithi175 bash[21527]: audit 2024-03-26T03:52:20.542274+0000 mon.c (mon.2) 448 : audit [INF] from='client.? 172.21.15.175:0/2595517674' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi175-64406-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi175-64406-55"}]: dispatch 2024-03-26T03:52:20.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:20 smithi175 bash[21527]: audit 2024-03-26T03:52:20.543656+0000 mon.a (mon.0) 2142 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi175-64406-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi175-64406-55"}]: dispatch 2024-03-26T03:52:20.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:20 smithi175 bash[29284]: cluster 2024-03-26T03:52:19.543850+0000 mon.a (mon.0) 2134 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:52:20.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:20 smithi175 bash[29284]: audit 2024-03-26T03:52:19.550026+0000 mon.c (mon.2) 444 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:52:20.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:20 smithi175 bash[29284]: audit 2024-03-26T03:52:19.583253+0000 mon.c (mon.2) 445 : audit [INF] from='client.? 172.21.15.175:0/2595517674' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi175-64406-55"}]: dispatch 2024-03-26T03:52:20.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:20 smithi175 bash[29284]: audit 2024-03-26T03:52:19.583686+0000 mon.a (mon.0) 2135 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi175-64406-55"}]: dispatch 2024-03-26T03:52:20.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:20 smithi175 bash[29284]: audit 2024-03-26T03:52:19.584323+0000 mon.c (mon.2) 446 : audit [INF] from='client.? 172.21.15.175:0/2595517674' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi175-64406-55"}]: dispatch 2024-03-26T03:52:20.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:20 smithi175 bash[29284]: audit 2024-03-26T03:52:19.584649+0000 mon.a (mon.0) 2136 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi175-64406-55"}]: dispatch 2024-03-26T03:52:20.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:20 smithi175 bash[29284]: audit 2024-03-26T03:52:19.585147+0000 mon.c (mon.2) 447 : audit [INF] from='client.? 172.21.15.175:0/2595517674' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi175-64406-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:20.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:20 smithi175 bash[29284]: audit 2024-03-26T03:52:19.585445+0000 mon.a (mon.0) 2137 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi175-64406-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:20.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:20 smithi175 bash[29284]: audit 2024-03-26T03:52:19.887647+0000 mgr.y (mgr.24431) 322 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:52:20.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:20 smithi175 bash[29284]: cluster 2024-03-26T03:52:20.527773+0000 mon.a (mon.0) 2138 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:52:20.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:20 smithi175 bash[29284]: audit 2024-03-26T03:52:20.530061+0000 mon.a (mon.0) 2139 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-39"}]': finished 2024-03-26T03:52:20.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:20 smithi175 bash[29284]: audit 2024-03-26T03:52:20.530280+0000 mon.a (mon.0) 2140 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi175-64406-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:52:20.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:20 smithi175 bash[29284]: cluster 2024-03-26T03:52:20.537170+0000 mon.a (mon.0) 2141 : cluster [DBG] osdmap e284: 8 total, 8 up, 8 in 2024-03-26T03:52:20.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:20 smithi175 bash[29284]: audit 2024-03-26T03:52:20.542274+0000 mon.c (mon.2) 448 : audit [INF] from='client.? 172.21.15.175:0/2595517674' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi175-64406-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi175-64406-55"}]: dispatch 2024-03-26T03:52:20.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:20 smithi175 bash[29284]: audit 2024-03-26T03:52:20.543656+0000 mon.a (mon.0) 2142 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi175-64406-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi175-64406-55"}]: dispatch 2024-03-26T03:52:21.557 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [==========] Running 77 tests from 4 test suites. 2024-03-26T03:52:21.557 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [----------] Global test environment set-up. 2024-03-26T03:52:21.557 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2024-03-26T03:52:21.558 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: seed 64594 2024-03-26T03:52:21.558 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2024-03-26T03:52:21.558 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (461 ms) 2024-03-26T03:52:21.558 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2024-03-26T03:52:21.558 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (10624 ms) 2024-03-26T03:52:21.558 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2024-03-26T03:52:21.558 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (76 ms) 2024-03-26T03:52:21.558 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (11161 ms total) 2024-03-26T03:52:21.558 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: 2024-03-26T03:52:21.558 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP 2024-03-26T03:52:21.558 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2024-03-26T03:52:21.558 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (7063 ms) 2024-03-26T03:52:21.558 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2024-03-26T03:52:21.558 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (7945 ms) 2024-03-26T03:52:21.558 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-03-26T03:52:21.558 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (10024 ms) 2024-03-26T03:52:21.559 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-03-26T03:52:21.559 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: my_snaps [3] 2024-03-26T03:52:21.559 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: my_snaps [4,3] 2024-03-26T03:52:21.559 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: my_snaps [5,4,3] 2024-03-26T03:52:21.559 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: my_snaps [6,5,4,3] 2024-03-26T03:52:21.559 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: promoting some heads 2024-03-26T03:52:21.559 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: promoting from clones for snap 6 2024-03-26T03:52:21.559 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: promoting from clones for snap 5 2024-03-26T03:52:21.559 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: promoting from clones for snap 4 2024-03-26T03:52:21.559 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: promoting from clones for snap 3 2024-03-26T03:52:21.559 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: waiting for scrubs... 2024-03-26T03:52:21.559 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: done waiting 2024-03-26T03:52:21.559 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (47085 ms) 2024-03-26T03:52:21.559 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-03-26T03:52:21.559 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (9942 ms) 2024-03-26T03:52:21.559 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-03-26T03:52:21.560 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (7999 ms) 2024-03-26T03:52:21.560 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-03-26T03:52:21.560 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (8036 ms) 2024-03-26T03:52:21.560 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2024-03-26T03:52:21.560 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (8049 ms) 2024-03-26T03:52:21.560 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-03-26T03:52:21.560 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (10101 ms) 2024-03-26T03:52:21.560 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-03-26T03:52:21.560 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (8811 ms) 2024-03-26T03:52:21.560 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-03-26T03:52:21.560 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (10068 ms) 2024-03-26T03:52:21.560 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-03-26T03:52:21.560 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (12976 ms) 2024-03-26T03:52:21.560 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-03-26T03:52:21.560 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (7968 ms) 2024-03-26T03:52:21.560 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2024-03-26T03:52:21.560 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (8105 ms) 2024-03-26T03:52:21.561 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-03-26T03:52:21.561 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (13101 ms) 2024-03-26T03:52:21.561 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-03-26T03:52:21.561 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (7806 ms) 2024-03-26T03:52:21.561 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-03-26T03:52:21.561 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (8074 ms) 2024-03-26T03:52:21.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:21 smithi190 bash[23392]: cluster 2024-03-26T03:52:20.688928+0000 mgr.y (mgr.24431) 323 : cluster [DBG] pgmap v423: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1.9 MiB/s wr, 93 op/s 2024-03-26T03:52:21.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:21 smithi175 bash[29284]: cluster 2024-03-26T03:52:20.688928+0000 mgr.y (mgr.24431) 323 : cluster [DBG] pgmap v423: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1.9 MiB/s wr, 93 op/s 2024-03-26T03:52:21.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:21 smithi175 bash[21527]: cluster 2024-03-26T03:52:20.688928+0000 mgr.y (mgr.24431) 323 : cluster [DBG] pgmap v423: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1.9 MiB/s wr, 93 op/s 2024-03-26T03:52:22.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:22 smithi190 bash[23392]: cluster 2024-03-26T03:52:21.556823+0000 mon.a (mon.0) 2143 : cluster [DBG] osdmap e285: 8 total, 8 up, 8 in 2024-03-26T03:52:22.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:22 smithi190 bash[23392]: audit 2024-03-26T03:52:22.537454+0000 mon.a (mon.0) 2144 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi175-64406-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi175-64406-55"}]': finished 2024-03-26T03:52:22.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:22 smithi190 bash[23392]: cluster 2024-03-26T03:52:22.545956+0000 mon.a (mon.0) 2145 : cluster [DBG] osdmap e286: 8 total, 8 up, 8 in 2024-03-26T03:52:22.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:22 smithi190 bash[23392]: audit 2024-03-26T03:52:22.555550+0000 mon.a (mon.0) 2146 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:52:22.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:22 smithi175 bash[21527]: cluster 2024-03-26T03:52:21.556823+0000 mon.a (mon.0) 2143 : cluster [DBG] osdmap e285: 8 total, 8 up, 8 in 2024-03-26T03:52:22.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:22 smithi175 bash[21527]: audit 2024-03-26T03:52:22.537454+0000 mon.a (mon.0) 2144 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi175-64406-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi175-64406-55"}]': finished 2024-03-26T03:52:22.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:22 smithi175 bash[21527]: cluster 2024-03-26T03:52:22.545956+0000 mon.a (mon.0) 2145 : cluster [DBG] osdmap e286: 8 total, 8 up, 8 in 2024-03-26T03:52:22.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:22 smithi175 bash[21527]: audit 2024-03-26T03:52:22.555550+0000 mon.a (mon.0) 2146 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:52:22.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:22 smithi175 bash[29284]: cluster 2024-03-26T03:52:21.556823+0000 mon.a (mon.0) 2143 : cluster [DBG] osdmap e285: 8 total, 8 up, 8 in 2024-03-26T03:52:22.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:22 smithi175 bash[29284]: audit 2024-03-26T03:52:22.537454+0000 mon.a (mon.0) 2144 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi175-64406-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi175-64406-55"}]': finished 2024-03-26T03:52:22.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:22 smithi175 bash[29284]: cluster 2024-03-26T03:52:22.545956+0000 mon.a (mon.0) 2145 : cluster [DBG] osdmap e286: 8 total, 8 up, 8 in 2024-03-26T03:52:22.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:22 smithi175 bash[29284]: audit 2024-03-26T03:52:22.555550+0000 mon.a (mon.0) 2146 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:52:23.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:23 smithi190 bash[23392]: cluster 2024-03-26T03:52:22.689938+0000 mgr.y (mgr.24431) 324 : cluster [DBG] pgmap v426: 300 pgs: 40 unknown, 260 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 976 KiB/s wr, 1 op/s 2024-03-26T03:52:23.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:23 smithi190 bash[23392]: audit 2024-03-26T03:52:23.540118+0000 mon.a (mon.0) 2147 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:52:23.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:23 smithi190 bash[23392]: cluster 2024-03-26T03:52:23.548426+0000 mon.a (mon.0) 2148 : cluster [DBG] osdmap e287: 8 total, 8 up, 8 in 2024-03-26T03:52:23.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:23 smithi190 bash[23392]: audit 2024-03-26T03:52:23.558713+0000 mon.a (mon.0) 2149 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:52:23.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:23 smithi175 bash[21527]: cluster 2024-03-26T03:52:22.689938+0000 mgr.y (mgr.24431) 324 : cluster [DBG] pgmap v426: 300 pgs: 40 unknown, 260 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 976 KiB/s wr, 1 op/s 2024-03-26T03:52:23.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:23 smithi175 bash[21527]: audit 2024-03-26T03:52:23.540118+0000 mon.a (mon.0) 2147 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:52:23.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:23 smithi175 bash[21527]: cluster 2024-03-26T03:52:23.548426+0000 mon.a (mon.0) 2148 : cluster [DBG] osdmap e287: 8 total, 8 up, 8 in 2024-03-26T03:52:23.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:23 smithi175 bash[21527]: audit 2024-03-26T03:52:23.558713+0000 mon.a (mon.0) 2149 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:52:23.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:23 smithi175 bash[29284]: cluster 2024-03-26T03:52:22.689938+0000 mgr.y (mgr.24431) 324 : cluster [DBG] pgmap v426: 300 pgs: 40 unknown, 260 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 976 KiB/s wr, 1 op/s 2024-03-26T03:52:23.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:23 smithi175 bash[29284]: audit 2024-03-26T03:52:23.540118+0000 mon.a (mon.0) 2147 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:52:23.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:23 smithi175 bash[29284]: cluster 2024-03-26T03:52:23.548426+0000 mon.a (mon.0) 2148 : cluster [DBG] osdmap e287: 8 total, 8 up, 8 in 2024-03-26T03:52:23.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:23 smithi175 bash[29284]: audit 2024-03-26T03:52:23.558713+0000 mon.a (mon.0) 2149 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:52:25.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:25 smithi190 bash[23392]: audit 2024-03-26T03:52:24.542901+0000 mon.a (mon.0) 2150 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-41", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:52:25.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:25 smithi190 bash[23392]: cluster 2024-03-26T03:52:24.553523+0000 mon.a (mon.0) 2151 : cluster [DBG] osdmap e288: 8 total, 8 up, 8 in 2024-03-26T03:52:25.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:25 smithi190 bash[23392]: audit 2024-03-26T03:52:24.557262+0000 mon.a (mon.0) 2152 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-41","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T03:52:25.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:25 smithi190 bash[23392]: audit 2024-03-26T03:52:24.558027+0000 mon.c (mon.2) 449 : audit [INF] from='client.? 172.21.15.175:0/2595517674' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi175-64406-55"}]: dispatch 2024-03-26T03:52:25.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:25 smithi190 bash[23392]: audit 2024-03-26T03:52:24.559420+0000 mon.a (mon.0) 2153 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi175-64406-55"}]: dispatch 2024-03-26T03:52:25.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:25 smithi190 bash[23392]: cluster 2024-03-26T03:52:24.691920+0000 mgr.y (mgr.24431) 325 : cluster [DBG] pgmap v429: 292 pgs: 14 unknown, 278 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:52:25.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:25 smithi175 bash[21527]: audit 2024-03-26T03:52:24.542901+0000 mon.a (mon.0) 2150 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-41", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:52:25.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:25 smithi175 bash[21527]: cluster 2024-03-26T03:52:24.553523+0000 mon.a (mon.0) 2151 : cluster [DBG] osdmap e288: 8 total, 8 up, 8 in 2024-03-26T03:52:25.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:25 smithi175 bash[21527]: audit 2024-03-26T03:52:24.557262+0000 mon.a (mon.0) 2152 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-41","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T03:52:25.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:25 smithi175 bash[21527]: audit 2024-03-26T03:52:24.558027+0000 mon.c (mon.2) 449 : audit [INF] from='client.? 172.21.15.175:0/2595517674' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi175-64406-55"}]: dispatch 2024-03-26T03:52:25.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:25 smithi175 bash[21527]: audit 2024-03-26T03:52:24.559420+0000 mon.a (mon.0) 2153 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi175-64406-55"}]: dispatch 2024-03-26T03:52:25.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:25 smithi175 bash[21527]: cluster 2024-03-26T03:52:24.691920+0000 mgr.y (mgr.24431) 325 : cluster [DBG] pgmap v429: 292 pgs: 14 unknown, 278 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:52:25.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:25 smithi175 bash[29284]: audit 2024-03-26T03:52:24.542901+0000 mon.a (mon.0) 2150 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-41", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:52:25.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:25 smithi175 bash[29284]: cluster 2024-03-26T03:52:24.553523+0000 mon.a (mon.0) 2151 : cluster [DBG] osdmap e288: 8 total, 8 up, 8 in 2024-03-26T03:52:25.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:25 smithi175 bash[29284]: audit 2024-03-26T03:52:24.557262+0000 mon.a (mon.0) 2152 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-41","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T03:52:25.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:25 smithi175 bash[29284]: audit 2024-03-26T03:52:24.558027+0000 mon.c (mon.2) 449 : audit [INF] from='client.? 172.21.15.175:0/2595517674' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi175-64406-55"}]: dispatch 2024-03-26T03:52:25.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:25 smithi175 bash[29284]: audit 2024-03-26T03:52:24.559420+0000 mon.a (mon.0) 2153 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi175-64406-55"}]: dispatch 2024-03-26T03:52:25.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:25 smithi175 bash[29284]: cluster 2024-03-26T03:52:24.691920+0000 mgr.y (mgr.24431) 325 : cluster [DBG] pgmap v429: 292 pgs: 14 unknown, 278 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:52:26.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:26 smithi190 bash[23392]: cluster 2024-03-26T03:52:25.544005+0000 mon.a (mon.0) 2154 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:52:26.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:26 smithi190 bash[23392]: audit 2024-03-26T03:52:25.545665+0000 mon.a (mon.0) 2155 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-41","var": "hit_set_count","val": "2"}]': finished 2024-03-26T03:52:26.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:26 smithi190 bash[23392]: audit 2024-03-26T03:52:25.545950+0000 mon.a (mon.0) 2156 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi175-64406-55"}]': finished 2024-03-26T03:52:26.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:26 smithi190 bash[23392]: cluster 2024-03-26T03:52:25.553495+0000 mon.a (mon.0) 2157 : cluster [DBG] osdmap e289: 8 total, 8 up, 8 in 2024-03-26T03:52:26.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:26 smithi190 bash[23392]: audit 2024-03-26T03:52:25.554239+0000 mon.c (mon.2) 450 : audit [INF] from='client.? 172.21.15.175:0/2595517674' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi175-64406-55"}]: dispatch 2024-03-26T03:52:26.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:26 smithi190 bash[23392]: audit 2024-03-26T03:52:25.556366+0000 mon.a (mon.0) 2158 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-41","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T03:52:26.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:26 smithi190 bash[23392]: audit 2024-03-26T03:52:25.558062+0000 mon.a (mon.0) 2159 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi175-64406-55"}]: dispatch 2024-03-26T03:52:26.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:26 smithi190 bash[23392]: audit 2024-03-26T03:52:26.549409+0000 mon.a (mon.0) 2160 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-41","var": "hit_set_period","val": "600"}]': finished 2024-03-26T03:52:26.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:26 smithi190 bash[23392]: audit 2024-03-26T03:52:26.549599+0000 mon.a (mon.0) 2161 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi175-64406-55"}]': finished 2024-03-26T03:52:26.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:26 smithi190 bash[23392]: cluster 2024-03-26T03:52:26.557127+0000 mon.a (mon.0) 2162 : cluster [DBG] osdmap e290: 8 total, 8 up, 8 in 2024-03-26T03:52:26.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:26 smithi190 bash[23392]: audit 2024-03-26T03:52:26.558080+0000 mon.a (mon.0) 2163 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-26T03:52:26.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:26 smithi175 bash[21527]: cluster 2024-03-26T03:52:25.544005+0000 mon.a (mon.0) 2154 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:52:26.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:26 smithi175 bash[21527]: audit 2024-03-26T03:52:25.545665+0000 mon.a (mon.0) 2155 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-41","var": "hit_set_count","val": "2"}]': finished 2024-03-26T03:52:26.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:26 smithi175 bash[21527]: audit 2024-03-26T03:52:25.545950+0000 mon.a (mon.0) 2156 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi175-64406-55"}]': finished 2024-03-26T03:52:26.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:26 smithi175 bash[21527]: cluster 2024-03-26T03:52:25.553495+0000 mon.a (mon.0) 2157 : cluster [DBG] osdmap e289: 8 total, 8 up, 8 in 2024-03-26T03:52:26.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:26 smithi175 bash[21527]: audit 2024-03-26T03:52:25.554239+0000 mon.c (mon.2) 450 : audit [INF] from='client.? 172.21.15.175:0/2595517674' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi175-64406-55"}]: dispatch 2024-03-26T03:52:26.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:26 smithi175 bash[21527]: audit 2024-03-26T03:52:25.556366+0000 mon.a (mon.0) 2158 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-41","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T03:52:26.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:26 smithi175 bash[21527]: audit 2024-03-26T03:52:25.558062+0000 mon.a (mon.0) 2159 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi175-64406-55"}]: dispatch 2024-03-26T03:52:26.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:26 smithi175 bash[21527]: audit 2024-03-26T03:52:26.549409+0000 mon.a (mon.0) 2160 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-41","var": "hit_set_period","val": "600"}]': finished 2024-03-26T03:52:26.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:26 smithi175 bash[21527]: audit 2024-03-26T03:52:26.549599+0000 mon.a (mon.0) 2161 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi175-64406-55"}]': finished 2024-03-26T03:52:26.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:26 smithi175 bash[21527]: cluster 2024-03-26T03:52:26.557127+0000 mon.a (mon.0) 2162 : cluster [DBG] osdmap e290: 8 total, 8 up, 8 in 2024-03-26T03:52:26.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:26 smithi175 bash[21527]: audit 2024-03-26T03:52:26.558080+0000 mon.a (mon.0) 2163 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-26T03:52:26.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:26 smithi175 bash[29284]: cluster 2024-03-26T03:52:25.544005+0000 mon.a (mon.0) 2154 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:52:26.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:26 smithi175 bash[29284]: audit 2024-03-26T03:52:25.545665+0000 mon.a (mon.0) 2155 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-41","var": "hit_set_count","val": "2"}]': finished 2024-03-26T03:52:26.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:26 smithi175 bash[29284]: audit 2024-03-26T03:52:25.545950+0000 mon.a (mon.0) 2156 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi175-64406-55"}]': finished 2024-03-26T03:52:26.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:26 smithi175 bash[29284]: cluster 2024-03-26T03:52:25.553495+0000 mon.a (mon.0) 2157 : cluster [DBG] osdmap e289: 8 total, 8 up, 8 in 2024-03-26T03:52:26.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:26 smithi175 bash[29284]: audit 2024-03-26T03:52:25.554239+0000 mon.c (mon.2) 450 : audit [INF] from='client.? 172.21.15.175:0/2595517674' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi175-64406-55"}]: dispatch 2024-03-26T03:52:26.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:26 smithi175 bash[29284]: audit 2024-03-26T03:52:25.556366+0000 mon.a (mon.0) 2158 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-41","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T03:52:26.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:26 smithi175 bash[29284]: audit 2024-03-26T03:52:25.558062+0000 mon.a (mon.0) 2159 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi175-64406-55"}]: dispatch 2024-03-26T03:52:26.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:26 smithi175 bash[29284]: audit 2024-03-26T03:52:26.549409+0000 mon.a (mon.0) 2160 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-41","var": "hit_set_period","val": "600"}]': finished 2024-03-26T03:52:26.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:26 smithi175 bash[29284]: audit 2024-03-26T03:52:26.549599+0000 mon.a (mon.0) 2161 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi175-64406-55"}]': finished 2024-03-26T03:52:26.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:26 smithi175 bash[29284]: cluster 2024-03-26T03:52:26.557127+0000 mon.a (mon.0) 2162 : cluster [DBG] osdmap e290: 8 total, 8 up, 8 in 2024-03-26T03:52:26.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:26 smithi175 bash[29284]: audit 2024-03-26T03:52:26.558080+0000 mon.a (mon.0) 2163 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-26T03:52:27.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:27 smithi190 bash[23392]: audit 2024-03-26T03:52:26.584231+0000 mon.c (mon.2) 451 : audit [INF] from='client.? 172.21.15.175:0/3894546360' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi175-64406-56"}]: dispatch 2024-03-26T03:52:27.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:27 smithi190 bash[23392]: audit 2024-03-26T03:52:26.584554+0000 mon.a (mon.0) 2164 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi175-64406-56"}]: dispatch 2024-03-26T03:52:27.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:27 smithi190 bash[23392]: audit 2024-03-26T03:52:26.585256+0000 mon.c (mon.2) 452 : audit [INF] from='client.? 172.21.15.175:0/3894546360' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi175-64406-56"}]: dispatch 2024-03-26T03:52:27.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:27 smithi190 bash[23392]: audit 2024-03-26T03:52:26.585537+0000 mon.a (mon.0) 2165 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi175-64406-56"}]: dispatch 2024-03-26T03:52:27.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:27 smithi190 bash[23392]: audit 2024-03-26T03:52:26.586053+0000 mon.c (mon.2) 453 : audit [INF] from='client.? 172.21.15.175:0/3894546360' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi175-64406-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:27.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:27 smithi190 bash[23392]: audit 2024-03-26T03:52:26.586342+0000 mon.a (mon.0) 2166 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi175-64406-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:27.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:27 smithi190 bash[23392]: cluster 2024-03-26T03:52:26.692757+0000 mgr.y (mgr.24431) 326 : cluster [DBG] pgmap v432: 292 pgs: 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:27.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:27 smithi190 bash[23392]: audit 2024-03-26T03:52:27.552091+0000 mon.a (mon.0) 2167 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-26T03:52:27.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:27 smithi190 bash[23392]: audit 2024-03-26T03:52:27.552280+0000 mon.a (mon.0) 2168 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi175-64406-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:52:27.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:27 smithi190 bash[23392]: cluster 2024-03-26T03:52:27.559829+0000 mon.a (mon.0) 2169 : cluster [DBG] osdmap e291: 8 total, 8 up, 8 in 2024-03-26T03:52:27.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:27 smithi190 bash[23392]: audit 2024-03-26T03:52:27.562121+0000 mon.c (mon.2) 454 : audit [INF] from='client.? 172.21.15.175:0/3894546360' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi175-64406-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi175-64406-56"}]: dispatch 2024-03-26T03:52:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:27 smithi175 bash[21527]: audit 2024-03-26T03:52:26.584231+0000 mon.c (mon.2) 451 : audit [INF] from='client.? 172.21.15.175:0/3894546360' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi175-64406-56"}]: dispatch 2024-03-26T03:52:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:27 smithi175 bash[21527]: audit 2024-03-26T03:52:26.584554+0000 mon.a (mon.0) 2164 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi175-64406-56"}]: dispatch 2024-03-26T03:52:27.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:27 smithi175 bash[21527]: audit 2024-03-26T03:52:26.585256+0000 mon.c (mon.2) 452 : audit [INF] from='client.? 172.21.15.175:0/3894546360' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi175-64406-56"}]: dispatch 2024-03-26T03:52:27.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:27 smithi175 bash[21527]: audit 2024-03-26T03:52:26.585537+0000 mon.a (mon.0) 2165 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi175-64406-56"}]: dispatch 2024-03-26T03:52:27.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:27 smithi175 bash[21527]: audit 2024-03-26T03:52:26.586053+0000 mon.c (mon.2) 453 : audit [INF] from='client.? 172.21.15.175:0/3894546360' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi175-64406-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:27.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:27 smithi175 bash[21527]: audit 2024-03-26T03:52:26.586342+0000 mon.a (mon.0) 2166 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi175-64406-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:27.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:27 smithi175 bash[21527]: cluster 2024-03-26T03:52:26.692757+0000 mgr.y (mgr.24431) 326 : cluster [DBG] pgmap v432: 292 pgs: 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:27.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:27 smithi175 bash[21527]: audit 2024-03-26T03:52:27.552091+0000 mon.a (mon.0) 2167 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-26T03:52:27.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:27 smithi175 bash[21527]: audit 2024-03-26T03:52:27.552280+0000 mon.a (mon.0) 2168 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi175-64406-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:52:27.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:27 smithi175 bash[21527]: cluster 2024-03-26T03:52:27.559829+0000 mon.a (mon.0) 2169 : cluster [DBG] osdmap e291: 8 total, 8 up, 8 in 2024-03-26T03:52:27.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:27 smithi175 bash[21527]: audit 2024-03-26T03:52:27.562121+0000 mon.c (mon.2) 454 : audit [INF] from='client.? 172.21.15.175:0/3894546360' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi175-64406-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi175-64406-56"}]: dispatch 2024-03-26T03:52:27.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:27 smithi175 bash[29284]: audit 2024-03-26T03:52:26.584231+0000 mon.c (mon.2) 451 : audit [INF] from='client.? 172.21.15.175:0/3894546360' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi175-64406-56"}]: dispatch 2024-03-26T03:52:27.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:27 smithi175 bash[29284]: audit 2024-03-26T03:52:26.584554+0000 mon.a (mon.0) 2164 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi175-64406-56"}]: dispatch 2024-03-26T03:52:27.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:27 smithi175 bash[29284]: audit 2024-03-26T03:52:26.585256+0000 mon.c (mon.2) 452 : audit [INF] from='client.? 172.21.15.175:0/3894546360' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi175-64406-56"}]: dispatch 2024-03-26T03:52:27.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:27 smithi175 bash[29284]: audit 2024-03-26T03:52:26.585537+0000 mon.a (mon.0) 2165 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi175-64406-56"}]: dispatch 2024-03-26T03:52:27.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:27 smithi175 bash[29284]: audit 2024-03-26T03:52:26.586053+0000 mon.c (mon.2) 453 : audit [INF] from='client.? 172.21.15.175:0/3894546360' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi175-64406-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:27.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:27 smithi175 bash[29284]: audit 2024-03-26T03:52:26.586342+0000 mon.a (mon.0) 2166 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi175-64406-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:27.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:27 smithi175 bash[29284]: cluster 2024-03-26T03:52:26.692757+0000 mgr.y (mgr.24431) 326 : cluster [DBG] pgmap v432: 292 pgs: 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:27.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:27 smithi175 bash[29284]: audit 2024-03-26T03:52:27.552091+0000 mon.a (mon.0) 2167 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-26T03:52:27.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:27 smithi175 bash[29284]: audit 2024-03-26T03:52:27.552280+0000 mon.a (mon.0) 2168 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi175-64406-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:52:27.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:27 smithi175 bash[29284]: cluster 2024-03-26T03:52:27.559829+0000 mon.a (mon.0) 2169 : cluster [DBG] osdmap e291: 8 total, 8 up, 8 in 2024-03-26T03:52:27.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:27 smithi175 bash[29284]: audit 2024-03-26T03:52:27.562121+0000 mon.c (mon.2) 454 : audit [INF] from='client.? 172.21.15.175:0/3894546360' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi175-64406-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi175-64406-56"}]: dispatch 2024-03-26T03:52:28.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:28 smithi190 bash[23392]: audit 2024-03-26T03:52:27.563454+0000 mon.a (mon.0) 2170 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi175-64406-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi175-64406-56"}]: dispatch 2024-03-26T03:52:28.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:28 smithi175 bash[21527]: audit 2024-03-26T03:52:27.563454+0000 mon.a (mon.0) 2170 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi175-64406-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi175-64406-56"}]: dispatch 2024-03-26T03:52:28.992 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:52:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:52:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:52:28.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:28 smithi175 bash[29284]: audit 2024-03-26T03:52:27.563454+0000 mon.a (mon.0) 2170 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi175-64406-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi175-64406-56"}]: dispatch 2024-03-26T03:52:29.894 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:29 smithi190 bash[23392]: cluster 2024-03-26T03:52:28.574750+0000 mon.a (mon.0) 2171 : cluster [DBG] osdmap e292: 8 total, 8 up, 8 in 2024-03-26T03:52:29.894 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:29 smithi190 bash[23392]: audit 2024-03-26T03:52:28.638773+0000 mon.a (mon.0) 2172 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:52:29.894 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:29 smithi190 bash[23392]: audit 2024-03-26T03:52:28.639824+0000 mon.a (mon.0) 2173 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-41"}]: dispatch 2024-03-26T03:52:29.894 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:29 smithi190 bash[23392]: cluster 2024-03-26T03:52:28.693626+0000 mgr.y (mgr.24431) 327 : cluster [DBG] pgmap v435: 292 pgs: 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:29.894 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:29 smithi190 bash[23392]: audit 2024-03-26T03:52:29.558710+0000 mon.a (mon.0) 2174 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi175-64406-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi175-64406-56"}]': finished 2024-03-26T03:52:29.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:29 smithi175 bash[21527]: cluster 2024-03-26T03:52:28.574750+0000 mon.a (mon.0) 2171 : cluster [DBG] osdmap e292: 8 total, 8 up, 8 in 2024-03-26T03:52:29.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:29 smithi175 bash[21527]: audit 2024-03-26T03:52:28.638773+0000 mon.a (mon.0) 2172 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:52:29.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:29 smithi175 bash[21527]: audit 2024-03-26T03:52:28.639824+0000 mon.a (mon.0) 2173 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-41"}]: dispatch 2024-03-26T03:52:29.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:29 smithi175 bash[21527]: cluster 2024-03-26T03:52:28.693626+0000 mgr.y (mgr.24431) 327 : cluster [DBG] pgmap v435: 292 pgs: 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:29.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:29 smithi175 bash[21527]: audit 2024-03-26T03:52:29.558710+0000 mon.a (mon.0) 2174 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi175-64406-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi175-64406-56"}]': finished 2024-03-26T03:52:29.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:29 smithi175 bash[29284]: cluster 2024-03-26T03:52:28.574750+0000 mon.a (mon.0) 2171 : cluster [DBG] osdmap e292: 8 total, 8 up, 8 in 2024-03-26T03:52:29.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:29 smithi175 bash[29284]: audit 2024-03-26T03:52:28.638773+0000 mon.a (mon.0) 2172 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:52:29.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:29 smithi175 bash[29284]: audit 2024-03-26T03:52:28.639824+0000 mon.a (mon.0) 2173 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-41"}]: dispatch 2024-03-26T03:52:29.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:29 smithi175 bash[29284]: cluster 2024-03-26T03:52:28.693626+0000 mgr.y (mgr.24431) 327 : cluster [DBG] pgmap v435: 292 pgs: 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:29.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:29 smithi175 bash[29284]: audit 2024-03-26T03:52:29.558710+0000 mon.a (mon.0) 2174 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi175-64406-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi175-64406-56"}]': finished 2024-03-26T03:52:30.239 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:52:29 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:52:30.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:30 smithi190 bash[23392]: audit 2024-03-26T03:52:29.558940+0000 mon.a (mon.0) 2175 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-41"}]': finished 2024-03-26T03:52:30.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:30 smithi190 bash[23392]: cluster 2024-03-26T03:52:29.566532+0000 mon.a (mon.0) 2176 : cluster [DBG] osdmap e293: 8 total, 8 up, 8 in 2024-03-26T03:52:30.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:30 smithi190 bash[23392]: audit 2024-03-26T03:52:29.895518+0000 mgr.y (mgr.24431) 328 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:52:30.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:30 smithi190 bash[23392]: cluster 2024-03-26T03:52:30.572393+0000 mon.a (mon.0) 2177 : cluster [DBG] osdmap e294: 8 total, 8 up, 8 in 2024-03-26T03:52:30.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:30 smithi175 bash[21527]: audit 2024-03-26T03:52:29.558940+0000 mon.a (mon.0) 2175 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-41"}]': finished 2024-03-26T03:52:30.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:30 smithi175 bash[21527]: cluster 2024-03-26T03:52:29.566532+0000 mon.a (mon.0) 2176 : cluster [DBG] osdmap e293: 8 total, 8 up, 8 in 2024-03-26T03:52:30.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:30 smithi175 bash[21527]: audit 2024-03-26T03:52:29.895518+0000 mgr.y (mgr.24431) 328 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:52:30.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:30 smithi175 bash[21527]: cluster 2024-03-26T03:52:30.572393+0000 mon.a (mon.0) 2177 : cluster [DBG] osdmap e294: 8 total, 8 up, 8 in 2024-03-26T03:52:30.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:30 smithi175 bash[29284]: audit 2024-03-26T03:52:29.558940+0000 mon.a (mon.0) 2175 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-41"}]': finished 2024-03-26T03:52:30.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:30 smithi175 bash[29284]: cluster 2024-03-26T03:52:29.566532+0000 mon.a (mon.0) 2176 : cluster [DBG] osdmap e293: 8 total, 8 up, 8 in 2024-03-26T03:52:30.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:30 smithi175 bash[29284]: audit 2024-03-26T03:52:29.895518+0000 mgr.y (mgr.24431) 328 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:52:30.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:30 smithi175 bash[29284]: cluster 2024-03-26T03:52:30.572393+0000 mon.a (mon.0) 2177 : cluster [DBG] osdmap e294: 8 total, 8 up, 8 in 2024-03-26T03:52:31.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:31 smithi190 bash[23392]: cluster 2024-03-26T03:52:30.694500+0000 mgr.y (mgr.24431) 329 : cluster [DBG] pgmap v438: 268 pgs: 8 creating+peering, 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:31.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:31 smithi190 bash[23392]: cluster 2024-03-26T03:52:31.574116+0000 mon.a (mon.0) 2178 : cluster [DBG] osdmap e295: 8 total, 8 up, 8 in 2024-03-26T03:52:31.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:31 smithi190 bash[23392]: audit 2024-03-26T03:52:31.579952+0000 mon.a (mon.0) 2179 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:52:31.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:31 smithi190 bash[23392]: audit 2024-03-26T03:52:31.580727+0000 mon.c (mon.2) 455 : audit [INF] from='client.? 172.21.15.175:0/3894546360' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi175-64406-56"}]: dispatch 2024-03-26T03:52:31.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:31 smithi190 bash[23392]: audit 2024-03-26T03:52:31.582627+0000 mon.a (mon.0) 2180 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi175-64406-56"}]: dispatch 2024-03-26T03:52:31.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:31 smithi175 bash[21527]: cluster 2024-03-26T03:52:30.694500+0000 mgr.y (mgr.24431) 329 : cluster [DBG] pgmap v438: 268 pgs: 8 creating+peering, 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:31.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:31 smithi175 bash[21527]: cluster 2024-03-26T03:52:31.574116+0000 mon.a (mon.0) 2178 : cluster [DBG] osdmap e295: 8 total, 8 up, 8 in 2024-03-26T03:52:31.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:31 smithi175 bash[21527]: audit 2024-03-26T03:52:31.579952+0000 mon.a (mon.0) 2179 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:52:31.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:31 smithi175 bash[21527]: audit 2024-03-26T03:52:31.580727+0000 mon.c (mon.2) 455 : audit [INF] from='client.? 172.21.15.175:0/3894546360' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi175-64406-56"}]: dispatch 2024-03-26T03:52:31.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:31 smithi175 bash[21527]: audit 2024-03-26T03:52:31.582627+0000 mon.a (mon.0) 2180 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi175-64406-56"}]: dispatch 2024-03-26T03:52:31.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:31 smithi175 bash[29284]: cluster 2024-03-26T03:52:30.694500+0000 mgr.y (mgr.24431) 329 : cluster [DBG] pgmap v438: 268 pgs: 8 creating+peering, 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:31.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:31 smithi175 bash[29284]: cluster 2024-03-26T03:52:31.574116+0000 mon.a (mon.0) 2178 : cluster [DBG] osdmap e295: 8 total, 8 up, 8 in 2024-03-26T03:52:31.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:31 smithi175 bash[29284]: audit 2024-03-26T03:52:31.579952+0000 mon.a (mon.0) 2179 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:52:31.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:31 smithi175 bash[29284]: audit 2024-03-26T03:52:31.580727+0000 mon.c (mon.2) 455 : audit [INF] from='client.? 172.21.15.175:0/3894546360' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi175-64406-56"}]: dispatch 2024-03-26T03:52:31.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:31 smithi175 bash[29284]: audit 2024-03-26T03:52:31.582627+0000 mon.a (mon.0) 2180 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi175-64406-56"}]: dispatch 2024-03-26T03:52:32.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:32 smithi190 bash[23392]: cluster 2024-03-26T03:52:31.591806+0000 mon.a (mon.0) 2181 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:52:32.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:32 smithi190 bash[23392]: audit 2024-03-26T03:52:32.568355+0000 mon.a (mon.0) 2182 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:52:32.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:32 smithi190 bash[23392]: audit 2024-03-26T03:52:32.568561+0000 mon.a (mon.0) 2183 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi175-64406-56"}]': finished 2024-03-26T03:52:32.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:32 smithi190 bash[23392]: cluster 2024-03-26T03:52:32.576734+0000 mon.a (mon.0) 2184 : cluster [DBG] osdmap e296: 8 total, 8 up, 8 in 2024-03-26T03:52:32.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:32 smithi190 bash[23392]: audit 2024-03-26T03:52:32.579153+0000 mon.c (mon.2) 456 : audit [INF] from='client.? 172.21.15.175:0/3894546360' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi175-64406-56"}]: dispatch 2024-03-26T03:52:32.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:32 smithi190 bash[23392]: audit 2024-03-26T03:52:32.579807+0000 mon.a (mon.0) 2185 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi175-64594-6","var": "pg_num","format": "json"}]: dispatch 2024-03-26T03:52:32.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:32 smithi190 bash[23392]: audit 2024-03-26T03:52:32.581130+0000 mon.a (mon.0) 2186 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi175-64406-56"}]: dispatch 2024-03-26T03:52:32.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:32 smithi190 bash[23392]: audit 2024-03-26T03:52:32.581962+0000 mon.a (mon.0) 2187 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:52:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:32 smithi175 bash[21527]: cluster 2024-03-26T03:52:31.591806+0000 mon.a (mon.0) 2181 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:52:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:32 smithi175 bash[21527]: audit 2024-03-26T03:52:32.568355+0000 mon.a (mon.0) 2182 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:52:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:32 smithi175 bash[21527]: audit 2024-03-26T03:52:32.568561+0000 mon.a (mon.0) 2183 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi175-64406-56"}]': finished 2024-03-26T03:52:32.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:32 smithi175 bash[21527]: cluster 2024-03-26T03:52:32.576734+0000 mon.a (mon.0) 2184 : cluster [DBG] osdmap e296: 8 total, 8 up, 8 in 2024-03-26T03:52:32.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:32 smithi175 bash[21527]: audit 2024-03-26T03:52:32.579153+0000 mon.c (mon.2) 456 : audit [INF] from='client.? 172.21.15.175:0/3894546360' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi175-64406-56"}]: dispatch 2024-03-26T03:52:32.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:32 smithi175 bash[21527]: audit 2024-03-26T03:52:32.579807+0000 mon.a (mon.0) 2185 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi175-64594-6","var": "pg_num","format": "json"}]: dispatch 2024-03-26T03:52:32.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:32 smithi175 bash[21527]: audit 2024-03-26T03:52:32.581130+0000 mon.a (mon.0) 2186 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi175-64406-56"}]: dispatch 2024-03-26T03:52:32.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:32 smithi175 bash[21527]: audit 2024-03-26T03:52:32.581962+0000 mon.a (mon.0) 2187 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:52:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:32 smithi175 bash[29284]: cluster 2024-03-26T03:52:31.591806+0000 mon.a (mon.0) 2181 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:52:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:32 smithi175 bash[29284]: audit 2024-03-26T03:52:32.568355+0000 mon.a (mon.0) 2182 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:52:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:32 smithi175 bash[29284]: audit 2024-03-26T03:52:32.568561+0000 mon.a (mon.0) 2183 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi175-64406-56"}]': finished 2024-03-26T03:52:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:32 smithi175 bash[29284]: cluster 2024-03-26T03:52:32.576734+0000 mon.a (mon.0) 2184 : cluster [DBG] osdmap e296: 8 total, 8 up, 8 in 2024-03-26T03:52:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:32 smithi175 bash[29284]: audit 2024-03-26T03:52:32.579153+0000 mon.c (mon.2) 456 : audit [INF] from='client.? 172.21.15.175:0/3894546360' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi175-64406-56"}]: dispatch 2024-03-26T03:52:32.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:32 smithi175 bash[29284]: audit 2024-03-26T03:52:32.579807+0000 mon.a (mon.0) 2185 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi175-64594-6","var": "pg_num","format": "json"}]: dispatch 2024-03-26T03:52:32.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:32 smithi175 bash[29284]: audit 2024-03-26T03:52:32.581130+0000 mon.a (mon.0) 2186 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi175-64406-56"}]: dispatch 2024-03-26T03:52:32.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:32 smithi175 bash[29284]: audit 2024-03-26T03:52:32.581962+0000 mon.a (mon.0) 2187 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:52:33.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:33 smithi190 bash[23392]: cluster 2024-03-26T03:52:32.695338+0000 mgr.y (mgr.24431) 330 : cluster [DBG] pgmap v441: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:33.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:33 smithi190 bash[23392]: audit 2024-03-26T03:52:33.571837+0000 mon.a (mon.0) 2188 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi175-64406-56"}]': finished 2024-03-26T03:52:33.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:33 smithi190 bash[23392]: audit 2024-03-26T03:52:33.572118+0000 mon.a (mon.0) 2189 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-43", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:52:33.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:33 smithi190 bash[23392]: cluster 2024-03-26T03:52:33.579377+0000 mon.a (mon.0) 2190 : cluster [DBG] osdmap e297: 8 total, 8 up, 8 in 2024-03-26T03:52:33.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:33 smithi190 bash[23392]: audit 2024-03-26T03:52:33.580440+0000 mon.a (mon.0) 2191 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-43","var": "hit_set_count","val": "8"}]: dispatch 2024-03-26T03:52:33.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:33 smithi190 bash[23392]: audit 2024-03-26T03:52:33.617494+0000 mon.a (mon.0) 2192 : audit [INF] from='client.? 172.21.15.175:0/1490514685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi175-64406-57"}]: dispatch 2024-03-26T03:52:33.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:33 smithi175 bash[21527]: cluster 2024-03-26T03:52:32.695338+0000 mgr.y (mgr.24431) 330 : cluster [DBG] pgmap v441: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:33.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:33 smithi175 bash[21527]: audit 2024-03-26T03:52:33.571837+0000 mon.a (mon.0) 2188 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi175-64406-56"}]': finished 2024-03-26T03:52:33.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:33 smithi175 bash[21527]: audit 2024-03-26T03:52:33.572118+0000 mon.a (mon.0) 2189 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-43", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:52:33.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:33 smithi175 bash[21527]: cluster 2024-03-26T03:52:33.579377+0000 mon.a (mon.0) 2190 : cluster [DBG] osdmap e297: 8 total, 8 up, 8 in 2024-03-26T03:52:33.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:33 smithi175 bash[21527]: audit 2024-03-26T03:52:33.580440+0000 mon.a (mon.0) 2191 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-43","var": "hit_set_count","val": "8"}]: dispatch 2024-03-26T03:52:33.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:33 smithi175 bash[21527]: audit 2024-03-26T03:52:33.617494+0000 mon.a (mon.0) 2192 : audit [INF] from='client.? 172.21.15.175:0/1490514685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi175-64406-57"}]: dispatch 2024-03-26T03:52:33.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:33 smithi175 bash[29284]: cluster 2024-03-26T03:52:32.695338+0000 mgr.y (mgr.24431) 330 : cluster [DBG] pgmap v441: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:33.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:33 smithi175 bash[29284]: audit 2024-03-26T03:52:33.571837+0000 mon.a (mon.0) 2188 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi175-64406-56"}]': finished 2024-03-26T03:52:33.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:33 smithi175 bash[29284]: audit 2024-03-26T03:52:33.572118+0000 mon.a (mon.0) 2189 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-43", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:52:33.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:33 smithi175 bash[29284]: cluster 2024-03-26T03:52:33.579377+0000 mon.a (mon.0) 2190 : cluster [DBG] osdmap e297: 8 total, 8 up, 8 in 2024-03-26T03:52:33.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:33 smithi175 bash[29284]: audit 2024-03-26T03:52:33.580440+0000 mon.a (mon.0) 2191 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-43","var": "hit_set_count","val": "8"}]: dispatch 2024-03-26T03:52:33.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:33 smithi175 bash[29284]: audit 2024-03-26T03:52:33.617494+0000 mon.a (mon.0) 2192 : audit [INF] from='client.? 172.21.15.175:0/1490514685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi175-64406-57"}]: dispatch 2024-03-26T03:52:34.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:34 smithi190 bash[23392]: audit 2024-03-26T03:52:33.619167+0000 mon.a (mon.0) 2193 : audit [INF] from='client.? 172.21.15.175:0/1490514685' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi175-64406-57"}]: dispatch 2024-03-26T03:52:34.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:34 smithi190 bash[23392]: audit 2024-03-26T03:52:33.619634+0000 mon.a (mon.0) 2194 : audit [INF] from='client.? 172.21.15.175:0/1490514685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi175-64406-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:34.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:34 smithi190 bash[23392]: audit 2024-03-26T03:52:34.574363+0000 mon.a (mon.0) 2195 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-43","var": "hit_set_count","val": "8"}]': finished 2024-03-26T03:52:34.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:34 smithi190 bash[23392]: audit 2024-03-26T03:52:34.574515+0000 mon.a (mon.0) 2196 : audit [INF] from='client.? 172.21.15.175:0/1490514685' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi175-64406-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:52:34.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:34 smithi190 bash[23392]: cluster 2024-03-26T03:52:34.581762+0000 mon.a (mon.0) 2197 : cluster [DBG] osdmap e298: 8 total, 8 up, 8 in 2024-03-26T03:52:34.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:34 smithi190 bash[23392]: audit 2024-03-26T03:52:34.582613+0000 mon.a (mon.0) 2198 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-43","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T03:52:34.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:34 smithi190 bash[23392]: audit 2024-03-26T03:52:34.583284+0000 mon.a (mon.0) 2199 : audit [INF] from='client.? 172.21.15.175:0/1490514685' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi175-64406-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi175-64406-57"}]: dispatch 2024-03-26T03:52:34.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:34 smithi190 bash[23392]: audit 2024-03-26T03:52:34.592016+0000 mon.a (mon.0) 2200 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:52:34.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:34 smithi190 bash[23392]: audit 2024-03-26T03:52:34.599019+0000 mon.c (mon.2) 457 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:52:34.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:34 smithi175 bash[21527]: audit 2024-03-26T03:52:33.619167+0000 mon.a (mon.0) 2193 : audit [INF] from='client.? 172.21.15.175:0/1490514685' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi175-64406-57"}]: dispatch 2024-03-26T03:52:34.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:34 smithi175 bash[21527]: audit 2024-03-26T03:52:33.619634+0000 mon.a (mon.0) 2194 : audit [INF] from='client.? 172.21.15.175:0/1490514685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi175-64406-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:34.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:34 smithi175 bash[21527]: audit 2024-03-26T03:52:34.574363+0000 mon.a (mon.0) 2195 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-43","var": "hit_set_count","val": "8"}]': finished 2024-03-26T03:52:34.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:34 smithi175 bash[21527]: audit 2024-03-26T03:52:34.574515+0000 mon.a (mon.0) 2196 : audit [INF] from='client.? 172.21.15.175:0/1490514685' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi175-64406-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:52:34.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:34 smithi175 bash[21527]: cluster 2024-03-26T03:52:34.581762+0000 mon.a (mon.0) 2197 : cluster [DBG] osdmap e298: 8 total, 8 up, 8 in 2024-03-26T03:52:34.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:34 smithi175 bash[21527]: audit 2024-03-26T03:52:34.582613+0000 mon.a (mon.0) 2198 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-43","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T03:52:34.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:34 smithi175 bash[21527]: audit 2024-03-26T03:52:34.583284+0000 mon.a (mon.0) 2199 : audit [INF] from='client.? 172.21.15.175:0/1490514685' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi175-64406-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi175-64406-57"}]: dispatch 2024-03-26T03:52:34.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:34 smithi175 bash[21527]: audit 2024-03-26T03:52:34.592016+0000 mon.a (mon.0) 2200 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:52:34.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:34 smithi175 bash[21527]: audit 2024-03-26T03:52:34.599019+0000 mon.c (mon.2) 457 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:52:34.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:34 smithi175 bash[29284]: audit 2024-03-26T03:52:33.619167+0000 mon.a (mon.0) 2193 : audit [INF] from='client.? 172.21.15.175:0/1490514685' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi175-64406-57"}]: dispatch 2024-03-26T03:52:34.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:34 smithi175 bash[29284]: audit 2024-03-26T03:52:33.619634+0000 mon.a (mon.0) 2194 : audit [INF] from='client.? 172.21.15.175:0/1490514685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi175-64406-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:34.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:34 smithi175 bash[29284]: audit 2024-03-26T03:52:34.574363+0000 mon.a (mon.0) 2195 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-43","var": "hit_set_count","val": "8"}]': finished 2024-03-26T03:52:34.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:34 smithi175 bash[29284]: audit 2024-03-26T03:52:34.574515+0000 mon.a (mon.0) 2196 : audit [INF] from='client.? 172.21.15.175:0/1490514685' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi175-64406-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:52:34.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:34 smithi175 bash[29284]: cluster 2024-03-26T03:52:34.581762+0000 mon.a (mon.0) 2197 : cluster [DBG] osdmap e298: 8 total, 8 up, 8 in 2024-03-26T03:52:34.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:34 smithi175 bash[29284]: audit 2024-03-26T03:52:34.582613+0000 mon.a (mon.0) 2198 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-43","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T03:52:34.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:34 smithi175 bash[29284]: audit 2024-03-26T03:52:34.583284+0000 mon.a (mon.0) 2199 : audit [INF] from='client.? 172.21.15.175:0/1490514685' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi175-64406-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi175-64406-57"}]: dispatch 2024-03-26T03:52:34.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:34 smithi175 bash[29284]: audit 2024-03-26T03:52:34.592016+0000 mon.a (mon.0) 2200 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:52:34.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:34 smithi175 bash[29284]: audit 2024-03-26T03:52:34.599019+0000 mon.c (mon.2) 457 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:52:35.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:35 smithi190 bash[23392]: cluster 2024-03-26T03:52:34.697221+0000 mgr.y (mgr.24431) 331 : cluster [DBG] pgmap v444: 292 pgs: 9 unknown, 283 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:52:35.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:35 smithi190 bash[23392]: audit 2024-03-26T03:52:35.578198+0000 mon.a (mon.0) 2201 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-43","var": "hit_set_period","val": "600"}]': finished 2024-03-26T03:52:35.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:35 smithi190 bash[23392]: cluster 2024-03-26T03:52:35.596378+0000 mon.a (mon.0) 2202 : cluster [DBG] osdmap e299: 8 total, 8 up, 8 in 2024-03-26T03:52:35.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:35 smithi190 bash[23392]: audit 2024-03-26T03:52:35.597104+0000 mon.a (mon.0) 2203 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-03-26T03:52:35.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:35 smithi175 bash[21527]: cluster 2024-03-26T03:52:34.697221+0000 mgr.y (mgr.24431) 331 : cluster [DBG] pgmap v444: 292 pgs: 9 unknown, 283 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:52:35.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:35 smithi175 bash[21527]: audit 2024-03-26T03:52:35.578198+0000 mon.a (mon.0) 2201 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-43","var": "hit_set_period","val": "600"}]': finished 2024-03-26T03:52:35.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:35 smithi175 bash[21527]: cluster 2024-03-26T03:52:35.596378+0000 mon.a (mon.0) 2202 : cluster [DBG] osdmap e299: 8 total, 8 up, 8 in 2024-03-26T03:52:35.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:35 smithi175 bash[21527]: audit 2024-03-26T03:52:35.597104+0000 mon.a (mon.0) 2203 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-03-26T03:52:35.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:35 smithi175 bash[29284]: cluster 2024-03-26T03:52:34.697221+0000 mgr.y (mgr.24431) 331 : cluster [DBG] pgmap v444: 292 pgs: 9 unknown, 283 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:52:35.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:35 smithi175 bash[29284]: audit 2024-03-26T03:52:35.578198+0000 mon.a (mon.0) 2201 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-43","var": "hit_set_period","val": "600"}]': finished 2024-03-26T03:52:35.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:35 smithi175 bash[29284]: cluster 2024-03-26T03:52:35.596378+0000 mon.a (mon.0) 2202 : cluster [DBG] osdmap e299: 8 total, 8 up, 8 in 2024-03-26T03:52:35.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:35 smithi175 bash[29284]: audit 2024-03-26T03:52:35.597104+0000 mon.a (mon.0) 2203 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-03-26T03:52:37.189 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetRead 2024-03-26T03:52:37.189 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: hmm, no HitSet yet 2024-03-26T03:52:37.189 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: ok, hit_set contains 268:602f83fe:::foo:head 2024-03-26T03:52:37.189 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (9014 ms) 2024-03-26T03:52:37.189 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-03-26T03:52:37.190 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg_num = 32 2024-03-26T03:52:37.190 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg 0 ls 1711425157,0 2024-03-26T03:52:37.190 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg 1 ls 1711425157,0 2024-03-26T03:52:37.190 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg 2 ls 1711425157,0 2024-03-26T03:52:37.190 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg 3 ls 1711425157,0 2024-03-26T03:52:37.190 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg 4 ls 1711425157,0 2024-03-26T03:52:37.190 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg 5 ls 1711425157,0 2024-03-26T03:52:37.190 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg 6 ls 1711425157,0 2024-03-26T03:52:37.190 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg 7 ls 1711425157,0 2024-03-26T03:52:37.190 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg 8 ls 1711425157,0 2024-03-26T03:52:37.190 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg 9 ls 1711425157,0 2024-03-26T03:52:37.190 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg 10 ls 1711425157,0 2024-03-26T03:52:37.190 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg 11 ls 1711425157,0 2024-03-26T03:52:37.190 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg 12 ls 1711425157,0 2024-03-26T03:52:37.190 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg 13 ls 1711425157,0 2024-03-26T03:52:37.191 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg 14 ls 1711425157,0 2024-03-26T03:52:37.191 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg 15 ls 1711425157,0 2024-03-26T03:52:37.191 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg 16 ls 1711425157,0 2024-03-26T03:52:37.191 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg 17 ls 1711425157,0 2024-03-26T03:52:37.191 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg 18 ls 1711425157,0 2024-03-26T03:52:37.191 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg 19 ls 1711425157,0 2024-03-26T03:52:37.191 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg 20 ls 1711425157,0 2024-03-26T03:52:37.191 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg 21 ls 1711425157,0 2024-03-26T03:52:37.191 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg 22 ls 1711425157,0 2024-03-26T03:52:37.191 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg 23 ls 1711425157,0 2024-03-26T03:52:37.191 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg 24 ls 1711425157,0 2024-03-26T03:52:37.191 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg 25 ls 1711425157,0 2024-03-26T03:52:37.191 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg 26 ls 1711425157,0 2024-03-26T03:52:37.191 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg 27 ls 1711425157,0 2024-03-26T03:52:37.191 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg 28 ls 1711425157,0 2024-03-26T03:52:37.192 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg 29 ls 1711425157,0 2024-03-26T03:52:37.192 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg 30 ls 1711425157,0 2024-03-26T03:52:37.192 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg 31 ls 1711425157,0 2024-03-26T03:52:37.192 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: pg_num = 32 2024-03-26T03:52:37.192 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:6cac518f:::0:head 2024-03-26T03:52:37.192 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:02547ec2:::1:head 2024-03-26T03:52:37.192 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:f905c69b:::2:head 2024-03-26T03:52:37.192 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:cfc208b3:::3:head 2024-03-26T03:52:37.192 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:d83876eb:::4:head 2024-03-26T03:52:37.192 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:b29083e3:::5:head 2024-03-26T03:52:37.192 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:c4fdafeb:::6:head 2024-03-26T03:52:37.192 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:5c6b0b28:::7:head 2024-03-26T03:52:37.192 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:bd63b0f1:::8:head 2024-03-26T03:52:37.192 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:e960b815:::9:head 2024-03-26T03:52:37.192 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:52ea6a34:::10:head 2024-03-26T03:52:37.192 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:89d3ae78:::11:head 2024-03-26T03:52:37.193 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:de5d7c5f:::12:head 2024-03-26T03:52:37.193 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:566253c9:::13:head 2024-03-26T03:52:37.193 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:62a1935d:::14:head 2024-03-26T03:52:37.193 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:863748b0:::15:head 2024-03-26T03:52:37.193 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:3958e169:::16:head 2024-03-26T03:52:37.193 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:4d4dabf9:::17:head 2024-03-26T03:52:37.193 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:8391935d:::18:head 2024-03-26T03:52:37.193 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:28883081:::19:head 2024-03-26T03:52:37.193 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:69259c59:::20:head 2024-03-26T03:52:37.193 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:4bdb80b7:::21:head 2024-03-26T03:52:37.193 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:a11c5d71:::22:head 2024-03-26T03:52:37.193 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:271af37b:::23:head 2024-03-26T03:52:37.193 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:95b121be:::24:head 2024-03-26T03:52:37.193 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:58d1031b:::25:head 2024-03-26T03:52:37.193 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:0a050783:::26:head 2024-03-26T03:52:37.193 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:c709704c:::27:head 2024-03-26T03:52:37.194 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:cbe56eaf:::28:head 2024-03-26T03:52:37.194 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:86b4b162:::29:head 2024-03-26T03:52:37.194 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:70d89383:::30:head 2024-03-26T03:52:37.194 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:dd450c7c:::31:head 2024-03-26T03:52:37.194 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:6d5729b1:::32:head 2024-03-26T03:52:37.194 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:c388f3fb:::33:head 2024-03-26T03:52:37.194 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:56cfea31:::34:head 2024-03-26T03:52:37.194 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:9dbc1bf7:::35:head 2024-03-26T03:52:37.194 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:40b74ccd:::36:head 2024-03-26T03:52:37.194 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:4d5aaf42:::37:head 2024-03-26T03:52:37.194 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:920f362c:::38:head 2024-03-26T03:52:37.194 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:6cc53222:::39:head 2024-03-26T03:52:37.194 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:9cad833f:::40:head 2024-03-26T03:52:37.194 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:1ea84d41:::41:head 2024-03-26T03:52:37.194 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:c4480ef6:::42:head 2024-03-26T03:52:37.194 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:a694361e:::43:head 2024-03-26T03:52:37.195 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:d1bd33e9:::44:head 2024-03-26T03:52:37.195 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:ddc2cd5d:::45:head 2024-03-26T03:52:37.195 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:2b782207:::46:head 2024-03-26T03:52:37.195 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:7b187fca:::47:head 2024-03-26T03:52:37.195 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:90ecdf6f:::48:head 2024-03-26T03:52:37.195 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:a5ed95fe:::49:head 2024-03-26T03:52:37.195 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:ea0eaa55:::50:head 2024-03-26T03:52:37.195 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:f33ef17b:::51:head 2024-03-26T03:52:37.195 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:a0d1b2f6:::52:head 2024-03-26T03:52:37.195 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:60c5229e:::53:head 2024-03-26T03:52:37.195 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:edcbc575:::54:head 2024-03-26T03:52:37.195 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:102cf253:::55:head 2024-03-26T03:52:37.195 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:efb7fb0b:::56:head 2024-03-26T03:52:37.195 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:50d0a326:::57:head 2024-03-26T03:52:37.195 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:d4dc5daf:::58:head 2024-03-26T03:52:37.195 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:3a130462:::59:head 2024-03-26T03:52:37.196 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:ec87ed71:::60:head 2024-03-26T03:52:37.196 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:d5bc9454:::61:head 2024-03-26T03:52:37.196 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:3ddfe313:::62:head 2024-03-26T03:52:37.196 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:7c2816b9:::63:head 2024-03-26T03:52:37.196 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:47e00e4d:::64:head 2024-03-26T03:52:37.196 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:c6410c18:::65:head 2024-03-26T03:52:37.196 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:b48ed237:::66:head 2024-03-26T03:52:37.196 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:cd63ad31:::67:head 2024-03-26T03:52:37.196 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:b179e92b:::68:head 2024-03-26T03:52:37.196 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:0d9f741a:::69:head 2024-03-26T03:52:37.196 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:6d3352ae:::70:head 2024-03-26T03:52:37.196 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:c6d5c19e:::71:head 2024-03-26T03:52:37.196 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:bc4729c3:::72:head 2024-03-26T03:52:37.196 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:77e930b9:::73:head 2024-03-26T03:52:37.196 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:0abeecfd:::74:head 2024-03-26T03:52:37.196 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:b7c37e15:::75:head 2024-03-26T03:52:37.196 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:b6378398:::76:head 2024-03-26T03:52:37.197 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:02bd68de:::77:head 2024-03-26T03:52:37.197 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:cc795d2d:::78:head 2024-03-26T03:52:37.197 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:630d4fea:::79:head 2024-03-26T03:52:37.197 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:e0d29ef5:::80:head 2024-03-26T03:52:37.197 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:fd6f13d2:::81:head 2024-03-26T03:52:37.197 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:606461d5:::82:head 2024-03-26T03:52:37.197 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:eadbdc43:::83:head 2024-03-26T03:52:37.197 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:8761d0bb:::84:head 2024-03-26T03:52:37.197 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:9ef0186f:::85:head 2024-03-26T03:52:37.197 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:e0d41294:::86:head 2024-03-26T03:52:37.197 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:961de695:::87:head 2024-03-26T03:52:37.197 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:1423148f:::88:head 2024-03-26T03:52:37.197 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:633a8fa2:::89:head 2024-03-26T03:52:37.197 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:a8653809:::90:head 2024-03-26T03:52:37.197 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:3dac8b33:::91:head 2024-03-26T03:52:37.197 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:35aad435:::92:head 2024-03-26T03:52:37.198 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:f6dcc343:::93:head 2024-03-26T03:52:37.198 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:dbbdad87:::94:head 2024-03-26T03:52:37.198 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:1cb48ce0:::95:head 2024-03-26T03:52:37.198 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:03cd461c:::96:head 2024-03-26T03:52:37.198 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:17a4ea99:::97:head 2024-03-26T03:52:37.198 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:9993c9a7:::98:head 2024-03-26T03:52:37.198 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:6394211c:::99:head 2024-03-26T03:52:37.198 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:94c7ae57:::100:head 2024-03-26T03:52:37.198 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:6fdee5bb:::101:head 2024-03-26T03:52:37.198 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:9a477fd1:::102:head 2024-03-26T03:52:37.198 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:eb850916:::103:head 2024-03-26T03:52:37.198 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:affc56b9:::104:head 2024-03-26T03:52:37.198 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:b42dc814:::105:head 2024-03-26T03:52:37.198 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:f319f8f0:::106:head 2024-03-26T03:52:37.198 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:9a40b9de:::107:head 2024-03-26T03:52:37.199 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:8b524f28:::108:head 2024-03-26T03:52:37.199 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:e3de589f:::109:head 2024-03-26T03:52:37.199 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:90f90a5b:::110:head 2024-03-26T03:52:37.199 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:a7b4f1d7:::111:head 2024-03-26T03:52:37.199 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:af51766e:::112:head 2024-03-26T03:52:37.199 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:b6f90bd1:::113:head 2024-03-26T03:52:37.199 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:e0261208:::114:head 2024-03-26T03:52:37.199 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:c9569ef7:::115:head 2024-03-26T03:52:37.199 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:61bebe50:::116:head 2024-03-26T03:52:37.199 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:fe93412b:::117:head 2024-03-26T03:52:37.199 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:d3d38bee:::118:head 2024-03-26T03:52:37.199 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:3100ba0c:::119:head 2024-03-26T03:52:37.199 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:d0560ada:::120:head 2024-03-26T03:52:37.199 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:f0ea8b35:::121:head 2024-03-26T03:52:37.199 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:766f231a:::122:head 2024-03-26T03:52:37.200 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:a07a2582:::123:head 2024-03-26T03:52:37.200 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:bd7c6b3a:::124:head 2024-03-26T03:52:37.200 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:fb2ddaff:::125:head 2024-03-26T03:52:37.200 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:4408e1fe:::126:head 2024-03-26T03:52:37.200 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:ee1df7a7:::127:head 2024-03-26T03:52:37.200 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:c3002909:::128:head 2024-03-26T03:52:37.200 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:4f48ffa9:::129:head 2024-03-26T03:52:37.200 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:edf38733:::130:head 2024-03-26T03:52:37.200 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:c08425c0:::131:head 2024-03-26T03:52:37.200 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:5f902d98:::132:head 2024-03-26T03:52:37.200 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:41ea2c93:::133:head 2024-03-26T03:52:37.200 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:813cee13:::134:head 2024-03-26T03:52:37.200 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:0131818d:::135:head 2024-03-26T03:52:37.200 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:26ba5a85:::136:head 2024-03-26T03:52:37.200 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:381b8a5a:::137:head 2024-03-26T03:52:37.200 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:28797e47:::138:head 2024-03-26T03:52:37.200 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:bfca7f22:::139:head 2024-03-26T03:52:37.201 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:36807075:::140:head 2024-03-26T03:52:37.201 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:80b03975:::141:head 2024-03-26T03:52:37.201 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:5c15709b:::142:head 2024-03-26T03:52:37.201 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:f39ea15e:::143:head 2024-03-26T03:52:37.201 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:ea992956:::144:head 2024-03-26T03:52:37.201 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:48887b1c:::145:head 2024-03-26T03:52:37.201 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:9f24a9dd:::146:head 2024-03-26T03:52:37.201 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:987f100b:::147:head 2024-03-26T03:52:37.201 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:d2dd3581:::148:head 2024-03-26T03:52:37.201 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:7fed1808:::149:head 2024-03-26T03:52:37.201 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:c80b70e9:::150:head 2024-03-26T03:52:37.201 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:85ed90f9:::151:head 2024-03-26T03:52:37.201 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:36428b24:::152:head 2024-03-26T03:52:37.201 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:d044c34a:::153:head 2024-03-26T03:52:37.201 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:7c18bf58:::154:head 2024-03-26T03:52:37.201 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:d1c21232:::155:head 2024-03-26T03:52:37.201 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:a7a3c575:::156:head 2024-03-26T03:52:37.202 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:87da0633:::157:head 2024-03-26T03:52:37.202 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:d5ac3822:::158:head 2024-03-26T03:52:37.202 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:3f20522d:::159:head 2024-03-26T03:52:37.202 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:6ca26563:::160:head 2024-03-26T03:52:37.202 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:532ce135:::161:head 2024-03-26T03:52:37.202 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:c78863e6:::162:head 2024-03-26T03:52:37.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:37 smithi190 bash[23392]: audit 2024-03-26T03:52:36.581077+0000 mon.a (mon.0) 2204 : audit [INF] from='client.? 172.21.15.175:0/1490514685' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi175-64406-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi175-64406-57"}]': finished 2024-03-26T03:52:37.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:37 smithi190 bash[23392]: audit 2024-03-26T03:52:36.581266+0000 mon.a (mon.0) 2205 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-03-26T03:52:37.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:37 smithi190 bash[23392]: cluster 2024-03-26T03:52:36.589302+0000 mon.a (mon.0) 2206 : cluster [DBG] osdmap e300: 8 total, 8 up, 8 in 2024-03-26T03:52:37.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:37 smithi190 bash[23392]: cluster 2024-03-26T03:52:36.698177+0000 mgr.y (mgr.24431) 332 : cluster [DBG] pgmap v447: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:37.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:37 smithi190 bash[23392]: audit 2024-03-26T03:52:37.190808+0000 mon.a (mon.0) 2207 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi175-64594-43","var": "pg_num","format": "json"}]: dispatch 2024-03-26T03:52:37.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:37 smithi190 bash[23392]: audit 2024-03-26T03:52:37.531261+0000 mon.a (mon.0) 2208 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:52:37.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:37 smithi190 bash[23392]: audit 2024-03-26T03:52:37.532349+0000 mon.a (mon.0) 2209 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-43"}]: dispatch 2024-03-26T03:52:37.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:37 smithi175 bash[21527]: audit 2024-03-26T03:52:36.581077+0000 mon.a (mon.0) 2204 : audit [INF] from='client.? 172.21.15.175:0/1490514685' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi175-64406-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi175-64406-57"}]': finished 2024-03-26T03:52:37.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:37 smithi175 bash[21527]: audit 2024-03-26T03:52:36.581266+0000 mon.a (mon.0) 2205 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-03-26T03:52:37.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:37 smithi175 bash[21527]: cluster 2024-03-26T03:52:36.589302+0000 mon.a (mon.0) 2206 : cluster [DBG] osdmap e300: 8 total, 8 up, 8 in 2024-03-26T03:52:37.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:37 smithi175 bash[21527]: cluster 2024-03-26T03:52:36.698177+0000 mgr.y (mgr.24431) 332 : cluster [DBG] pgmap v447: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:37.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:37 smithi175 bash[21527]: audit 2024-03-26T03:52:37.190808+0000 mon.a (mon.0) 2207 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi175-64594-43","var": "pg_num","format": "json"}]: dispatch 2024-03-26T03:52:37.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:37 smithi175 bash[21527]: audit 2024-03-26T03:52:37.531261+0000 mon.a (mon.0) 2208 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:52:37.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:37 smithi175 bash[21527]: audit 2024-03-26T03:52:37.532349+0000 mon.a (mon.0) 2209 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-43"}]: dispatch 2024-03-26T03:52:37.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:37 smithi175 bash[29284]: audit 2024-03-26T03:52:36.581077+0000 mon.a (mon.0) 2204 : audit [INF] from='client.? 172.21.15.175:0/1490514685' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi175-64406-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi175-64406-57"}]': finished 2024-03-26T03:52:37.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:37 smithi175 bash[29284]: audit 2024-03-26T03:52:36.581266+0000 mon.a (mon.0) 2205 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-03-26T03:52:37.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:37 smithi175 bash[29284]: cluster 2024-03-26T03:52:36.589302+0000 mon.a (mon.0) 2206 : cluster [DBG] osdmap e300: 8 total, 8 up, 8 in 2024-03-26T03:52:37.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:37 smithi175 bash[29284]: cluster 2024-03-26T03:52:36.698177+0000 mgr.y (mgr.24431) 332 : cluster [DBG] pgmap v447: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:37.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:37 smithi175 bash[29284]: audit 2024-03-26T03:52:37.190808+0000 mon.a (mon.0) 2207 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi175-64594-43","var": "pg_num","format": "json"}]: dispatch 2024-03-26T03:52:37.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:37 smithi175 bash[29284]: audit 2024-03-26T03:52:37.531261+0000 mon.a (mon.0) 2208 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:52:37.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:37 smithi175 bash[29284]: audit 2024-03-26T03:52:37.532349+0000 mon.a (mon.0) 2209 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-43"}]: dispatch 2024-03-26T03:52:38.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:38 smithi190 bash[23392]: cluster 2024-03-26T03:52:37.581124+0000 mon.a (mon.0) 2210 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:52:38.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:38 smithi190 bash[23392]: audit 2024-03-26T03:52:37.590823+0000 mon.a (mon.0) 2211 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-43"}]': finished 2024-03-26T03:52:38.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:38 smithi190 bash[23392]: cluster 2024-03-26T03:52:37.600076+0000 mon.a (mon.0) 2212 : cluster [DBG] osdmap e301: 8 total, 8 up, 8 in 2024-03-26T03:52:38.992 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:52:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:52:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:52:38.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:38 smithi175 bash[21527]: cluster 2024-03-26T03:52:37.581124+0000 mon.a (mon.0) 2210 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:52:38.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:38 smithi175 bash[21527]: audit 2024-03-26T03:52:37.590823+0000 mon.a (mon.0) 2211 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-43"}]': finished 2024-03-26T03:52:38.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:38 smithi175 bash[21527]: cluster 2024-03-26T03:52:37.600076+0000 mon.a (mon.0) 2212 : cluster [DBG] osdmap e301: 8 total, 8 up, 8 in 2024-03-26T03:52:38.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:38 smithi175 bash[29284]: cluster 2024-03-26T03:52:37.581124+0000 mon.a (mon.0) 2210 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:52:38.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:38 smithi175 bash[29284]: audit 2024-03-26T03:52:37.590823+0000 mon.a (mon.0) 2211 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-43"}]': finished 2024-03-26T03:52:38.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:38 smithi175 bash[29284]: cluster 2024-03-26T03:52:37.600076+0000 mon.a (mon.0) 2212 : cluster [DBG] osdmap e301: 8 total, 8 up, 8 in 2024-03-26T03:52:39.902 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:39 smithi190 bash[23392]: cluster 2024-03-26T03:52:38.602199+0000 mon.a (mon.0) 2213 : cluster [DBG] osdmap e302: 8 total, 8 up, 8 in 2024-03-26T03:52:39.902 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:39 smithi190 bash[23392]: audit 2024-03-26T03:52:38.606340+0000 mon.a (mon.0) 2214 : audit [INF] from='client.? 172.21.15.175:0/1490514685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi175-64406-57"}]: dispatch 2024-03-26T03:52:39.902 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:39 smithi190 bash[23392]: cluster 2024-03-26T03:52:38.699114+0000 mgr.y (mgr.24431) 333 : cluster [DBG] pgmap v450: 260 pgs: 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:39.902 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:39 smithi190 bash[23392]: audit 2024-03-26T03:52:39.597299+0000 mon.a (mon.0) 2215 : audit [INF] from='client.? 172.21.15.175:0/1490514685' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi175-64406-57"}]': finished 2024-03-26T03:52:39.902 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:39 smithi190 bash[23392]: cluster 2024-03-26T03:52:39.605554+0000 mon.a (mon.0) 2216 : cluster [DBG] osdmap e303: 8 total, 8 up, 8 in 2024-03-26T03:52:39.902 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:39 smithi190 bash[23392]: audit 2024-03-26T03:52:39.615357+0000 mon.a (mon.0) 2217 : audit [INF] from='client.? 172.21.15.175:0/1490514685' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi175-64406-57"}]: dispatch 2024-03-26T03:52:39.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:39 smithi175 bash[21527]: cluster 2024-03-26T03:52:38.602199+0000 mon.a (mon.0) 2213 : cluster [DBG] osdmap e302: 8 total, 8 up, 8 in 2024-03-26T03:52:39.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:39 smithi175 bash[21527]: audit 2024-03-26T03:52:38.606340+0000 mon.a (mon.0) 2214 : audit [INF] from='client.? 172.21.15.175:0/1490514685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi175-64406-57"}]: dispatch 2024-03-26T03:52:39.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:39 smithi175 bash[21527]: cluster 2024-03-26T03:52:38.699114+0000 mgr.y (mgr.24431) 333 : cluster [DBG] pgmap v450: 260 pgs: 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:39.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:39 smithi175 bash[21527]: audit 2024-03-26T03:52:39.597299+0000 mon.a (mon.0) 2215 : audit [INF] from='client.? 172.21.15.175:0/1490514685' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi175-64406-57"}]': finished 2024-03-26T03:52:39.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:39 smithi175 bash[21527]: cluster 2024-03-26T03:52:39.605554+0000 mon.a (mon.0) 2216 : cluster [DBG] osdmap e303: 8 total, 8 up, 8 in 2024-03-26T03:52:39.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:39 smithi175 bash[21527]: audit 2024-03-26T03:52:39.615357+0000 mon.a (mon.0) 2217 : audit [INF] from='client.? 172.21.15.175:0/1490514685' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi175-64406-57"}]: dispatch 2024-03-26T03:52:39.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:39 smithi175 bash[29284]: cluster 2024-03-26T03:52:38.602199+0000 mon.a (mon.0) 2213 : cluster [DBG] osdmap e302: 8 total, 8 up, 8 in 2024-03-26T03:52:39.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:39 smithi175 bash[29284]: audit 2024-03-26T03:52:38.606340+0000 mon.a (mon.0) 2214 : audit [INF] from='client.? 172.21.15.175:0/1490514685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi175-64406-57"}]: dispatch 2024-03-26T03:52:39.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:39 smithi175 bash[29284]: cluster 2024-03-26T03:52:38.699114+0000 mgr.y (mgr.24431) 333 : cluster [DBG] pgmap v450: 260 pgs: 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:39.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:39 smithi175 bash[29284]: audit 2024-03-26T03:52:39.597299+0000 mon.a (mon.0) 2215 : audit [INF] from='client.? 172.21.15.175:0/1490514685' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi175-64406-57"}]': finished 2024-03-26T03:52:39.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:39 smithi175 bash[29284]: cluster 2024-03-26T03:52:39.605554+0000 mon.a (mon.0) 2216 : cluster [DBG] osdmap e303: 8 total, 8 up, 8 in 2024-03-26T03:52:39.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:39 smithi175 bash[29284]: audit 2024-03-26T03:52:39.615357+0000 mon.a (mon.0) 2217 : audit [INF] from='client.? 172.21.15.175:0/1490514685' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi175-64406-57"}]: dispatch 2024-03-26T03:52:40.241 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:52:39 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:52:40.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:40 smithi190 bash[23392]: audit 2024-03-26T03:52:39.618230+0000 mon.a (mon.0) 2218 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:52:40.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:40 smithi190 bash[23392]: audit 2024-03-26T03:52:39.903463+0000 mgr.y (mgr.24431) 334 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:52:40.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:40 smithi190 bash[23392]: audit 2024-03-26T03:52:40.601380+0000 mon.a (mon.0) 2219 : audit [INF] from='client.? 172.21.15.175:0/1490514685' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi175-64406-57"}]': finished 2024-03-26T03:52:40.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:40 smithi190 bash[23392]: audit 2024-03-26T03:52:40.601594+0000 mon.a (mon.0) 2220 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:52:40.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:40 smithi190 bash[23392]: cluster 2024-03-26T03:52:40.610944+0000 mon.a (mon.0) 2221 : cluster [DBG] osdmap e304: 8 total, 8 up, 8 in 2024-03-26T03:52:40.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:40 smithi190 bash[23392]: audit 2024-03-26T03:52:40.625571+0000 mon.a (mon.0) 2222 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:52:40.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:40 smithi175 bash[21527]: audit 2024-03-26T03:52:39.618230+0000 mon.a (mon.0) 2218 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:52:40.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:40 smithi175 bash[21527]: audit 2024-03-26T03:52:39.903463+0000 mgr.y (mgr.24431) 334 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:52:40.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:40 smithi175 bash[21527]: audit 2024-03-26T03:52:40.601380+0000 mon.a (mon.0) 2219 : audit [INF] from='client.? 172.21.15.175:0/1490514685' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi175-64406-57"}]': finished 2024-03-26T03:52:40.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:40 smithi175 bash[21527]: audit 2024-03-26T03:52:40.601594+0000 mon.a (mon.0) 2220 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:52:40.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:40 smithi175 bash[21527]: cluster 2024-03-26T03:52:40.610944+0000 mon.a (mon.0) 2221 : cluster [DBG] osdmap e304: 8 total, 8 up, 8 in 2024-03-26T03:52:40.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:40 smithi175 bash[21527]: audit 2024-03-26T03:52:40.625571+0000 mon.a (mon.0) 2222 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:52:40.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:40 smithi175 bash[29284]: audit 2024-03-26T03:52:39.618230+0000 mon.a (mon.0) 2218 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:52:40.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:40 smithi175 bash[29284]: audit 2024-03-26T03:52:39.903463+0000 mgr.y (mgr.24431) 334 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:52:40.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:40 smithi175 bash[29284]: audit 2024-03-26T03:52:40.601380+0000 mon.a (mon.0) 2219 : audit [INF] from='client.? 172.21.15.175:0/1490514685' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi175-64406-57"}]': finished 2024-03-26T03:52:40.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:40 smithi175 bash[29284]: audit 2024-03-26T03:52:40.601594+0000 mon.a (mon.0) 2220 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:52:40.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:40 smithi175 bash[29284]: cluster 2024-03-26T03:52:40.610944+0000 mon.a (mon.0) 2221 : cluster [DBG] osdmap e304: 8 total, 8 up, 8 in 2024-03-26T03:52:40.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:40 smithi175 bash[29284]: audit 2024-03-26T03:52:40.625571+0000 mon.a (mon.0) 2222 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:52:41.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:41 smithi190 bash[23392]: audit 2024-03-26T03:52:40.662423+0000 mon.c (mon.2) 458 : audit [INF] from='client.? 172.21.15.175:0/3109723899' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi175-64406-58"}]: dispatch 2024-03-26T03:52:41.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:41 smithi190 bash[23392]: audit 2024-03-26T03:52:40.662850+0000 mon.a (mon.0) 2223 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi175-64406-58"}]: dispatch 2024-03-26T03:52:41.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:41 smithi190 bash[23392]: audit 2024-03-26T03:52:40.663435+0000 mon.c (mon.2) 459 : audit [INF] from='client.? 172.21.15.175:0/3109723899' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi175-64406-58"}]: dispatch 2024-03-26T03:52:41.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:41 smithi190 bash[23392]: audit 2024-03-26T03:52:40.663697+0000 mon.a (mon.0) 2224 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi175-64406-58"}]: dispatch 2024-03-26T03:52:41.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:41 smithi190 bash[23392]: audit 2024-03-26T03:52:40.664135+0000 mon.c (mon.2) 460 : audit [INF] from='client.? 172.21.15.175:0/3109723899' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi175-64406-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:41.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:41 smithi190 bash[23392]: audit 2024-03-26T03:52:40.664406+0000 mon.a (mon.0) 2225 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi175-64406-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:41.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:41 smithi190 bash[23392]: cluster 2024-03-26T03:52:40.699924+0000 mgr.y (mgr.24431) 335 : cluster [DBG] pgmap v453: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:41.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:41 smithi190 bash[23392]: audit 2024-03-26T03:52:41.604229+0000 mon.a (mon.0) 2226 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-45", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:52:41.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:41 smithi190 bash[23392]: audit 2024-03-26T03:52:41.604419+0000 mon.a (mon.0) 2227 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi175-64406-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:52:41.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:41 smithi190 bash[23392]: cluster 2024-03-26T03:52:41.611948+0000 mon.a (mon.0) 2228 : cluster [DBG] osdmap e305: 8 total, 8 up, 8 in 2024-03-26T03:52:41.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:41 smithi190 bash[23392]: audit 2024-03-26T03:52:41.614535+0000 mon.a (mon.0) 2229 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-45","var": "hit_set_count","val": "3"}]: dispatch 2024-03-26T03:52:41.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:41 smithi190 bash[23392]: audit 2024-03-26T03:52:41.617263+0000 mon.c (mon.2) 461 : audit [INF] from='client.? 172.21.15.175:0/3109723899' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi175-64406-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi175-64406-58"}]: dispatch 2024-03-26T03:52:41.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:41 smithi190 bash[23392]: audit 2024-03-26T03:52:41.618184+0000 mon.a (mon.0) 2230 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi175-64406-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi175-64406-58"}]: dispatch 2024-03-26T03:52:41.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:41 smithi175 bash[21527]: audit 2024-03-26T03:52:40.662423+0000 mon.c (mon.2) 458 : audit [INF] from='client.? 172.21.15.175:0/3109723899' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi175-64406-58"}]: dispatch 2024-03-26T03:52:41.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:41 smithi175 bash[21527]: audit 2024-03-26T03:52:40.662850+0000 mon.a (mon.0) 2223 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi175-64406-58"}]: dispatch 2024-03-26T03:52:41.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:41 smithi175 bash[21527]: audit 2024-03-26T03:52:40.663435+0000 mon.c (mon.2) 459 : audit [INF] from='client.? 172.21.15.175:0/3109723899' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi175-64406-58"}]: dispatch 2024-03-26T03:52:41.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:41 smithi175 bash[21527]: audit 2024-03-26T03:52:40.663697+0000 mon.a (mon.0) 2224 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi175-64406-58"}]: dispatch 2024-03-26T03:52:41.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:41 smithi175 bash[21527]: audit 2024-03-26T03:52:40.664135+0000 mon.c (mon.2) 460 : audit [INF] from='client.? 172.21.15.175:0/3109723899' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi175-64406-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:41.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:41 smithi175 bash[21527]: audit 2024-03-26T03:52:40.664406+0000 mon.a (mon.0) 2225 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi175-64406-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:41.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:41 smithi175 bash[21527]: cluster 2024-03-26T03:52:40.699924+0000 mgr.y (mgr.24431) 335 : cluster [DBG] pgmap v453: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:41.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:41 smithi175 bash[21527]: audit 2024-03-26T03:52:41.604229+0000 mon.a (mon.0) 2226 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-45", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:52:41.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:41 smithi175 bash[21527]: audit 2024-03-26T03:52:41.604419+0000 mon.a (mon.0) 2227 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi175-64406-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:52:41.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:41 smithi175 bash[21527]: cluster 2024-03-26T03:52:41.611948+0000 mon.a (mon.0) 2228 : cluster [DBG] osdmap e305: 8 total, 8 up, 8 in 2024-03-26T03:52:41.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:41 smithi175 bash[21527]: audit 2024-03-26T03:52:41.614535+0000 mon.a (mon.0) 2229 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-45","var": "hit_set_count","val": "3"}]: dispatch 2024-03-26T03:52:41.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:41 smithi175 bash[21527]: audit 2024-03-26T03:52:41.617263+0000 mon.c (mon.2) 461 : audit [INF] from='client.? 172.21.15.175:0/3109723899' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi175-64406-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi175-64406-58"}]: dispatch 2024-03-26T03:52:41.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:41 smithi175 bash[21527]: audit 2024-03-26T03:52:41.618184+0000 mon.a (mon.0) 2230 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi175-64406-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi175-64406-58"}]: dispatch 2024-03-26T03:52:41.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:41 smithi175 bash[29284]: audit 2024-03-26T03:52:40.662423+0000 mon.c (mon.2) 458 : audit [INF] from='client.? 172.21.15.175:0/3109723899' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi175-64406-58"}]: dispatch 2024-03-26T03:52:41.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:41 smithi175 bash[29284]: audit 2024-03-26T03:52:40.662850+0000 mon.a (mon.0) 2223 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi175-64406-58"}]: dispatch 2024-03-26T03:52:41.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:41 smithi175 bash[29284]: audit 2024-03-26T03:52:40.663435+0000 mon.c (mon.2) 459 : audit [INF] from='client.? 172.21.15.175:0/3109723899' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi175-64406-58"}]: dispatch 2024-03-26T03:52:41.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:41 smithi175 bash[29284]: audit 2024-03-26T03:52:40.663697+0000 mon.a (mon.0) 2224 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi175-64406-58"}]: dispatch 2024-03-26T03:52:41.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:41 smithi175 bash[29284]: audit 2024-03-26T03:52:40.664135+0000 mon.c (mon.2) 460 : audit [INF] from='client.? 172.21.15.175:0/3109723899' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi175-64406-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:41.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:41 smithi175 bash[29284]: audit 2024-03-26T03:52:40.664406+0000 mon.a (mon.0) 2225 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi175-64406-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:41.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:41 smithi175 bash[29284]: cluster 2024-03-26T03:52:40.699924+0000 mgr.y (mgr.24431) 335 : cluster [DBG] pgmap v453: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:41.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:41 smithi175 bash[29284]: audit 2024-03-26T03:52:41.604229+0000 mon.a (mon.0) 2226 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-45", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:52:41.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:41 smithi175 bash[29284]: audit 2024-03-26T03:52:41.604419+0000 mon.a (mon.0) 2227 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi175-64406-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:52:41.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:41 smithi175 bash[29284]: cluster 2024-03-26T03:52:41.611948+0000 mon.a (mon.0) 2228 : cluster [DBG] osdmap e305: 8 total, 8 up, 8 in 2024-03-26T03:52:41.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:41 smithi175 bash[29284]: audit 2024-03-26T03:52:41.614535+0000 mon.a (mon.0) 2229 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-45","var": "hit_set_count","val": "3"}]: dispatch 2024-03-26T03:52:41.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:41 smithi175 bash[29284]: audit 2024-03-26T03:52:41.617263+0000 mon.c (mon.2) 461 : audit [INF] from='client.? 172.21.15.175:0/3109723899' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi175-64406-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi175-64406-58"}]: dispatch 2024-03-26T03:52:41.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:41 smithi175 bash[29284]: audit 2024-03-26T03:52:41.618184+0000 mon.a (mon.0) 2230 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi175-64406-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi175-64406-58"}]: dispatch 2024-03-26T03:52:43.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:43 smithi190 bash[23392]: audit 2024-03-26T03:52:42.607846+0000 mon.a (mon.0) 2231 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-45","var": "hit_set_count","val": "3"}]': finished 2024-03-26T03:52:43.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:43 smithi190 bash[23392]: cluster 2024-03-26T03:52:42.629395+0000 mon.a (mon.0) 2232 : cluster [DBG] osdmap e306: 8 total, 8 up, 8 in 2024-03-26T03:52:43.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:43 smithi190 bash[23392]: audit 2024-03-26T03:52:42.633351+0000 mon.a (mon.0) 2233 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-45","var": "hit_set_period","val": "3"}]: dispatch 2024-03-26T03:52:43.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:43 smithi190 bash[23392]: cluster 2024-03-26T03:52:42.700814+0000 mgr.y (mgr.24431) 336 : cluster [DBG] pgmap v456: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:43.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:43 smithi175 bash[29284]: audit 2024-03-26T03:52:42.607846+0000 mon.a (mon.0) 2231 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-45","var": "hit_set_count","val": "3"}]': finished 2024-03-26T03:52:43.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:43 smithi175 bash[29284]: cluster 2024-03-26T03:52:42.629395+0000 mon.a (mon.0) 2232 : cluster [DBG] osdmap e306: 8 total, 8 up, 8 in 2024-03-26T03:52:43.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:43 smithi175 bash[29284]: audit 2024-03-26T03:52:42.633351+0000 mon.a (mon.0) 2233 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-45","var": "hit_set_period","val": "3"}]: dispatch 2024-03-26T03:52:43.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:43 smithi175 bash[29284]: cluster 2024-03-26T03:52:42.700814+0000 mgr.y (mgr.24431) 336 : cluster [DBG] pgmap v456: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:43.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:43 smithi175 bash[21527]: audit 2024-03-26T03:52:42.607846+0000 mon.a (mon.0) 2231 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-45","var": "hit_set_count","val": "3"}]': finished 2024-03-26T03:52:43.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:43 smithi175 bash[21527]: cluster 2024-03-26T03:52:42.629395+0000 mon.a (mon.0) 2232 : cluster [DBG] osdmap e306: 8 total, 8 up, 8 in 2024-03-26T03:52:43.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:43 smithi175 bash[21527]: audit 2024-03-26T03:52:42.633351+0000 mon.a (mon.0) 2233 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-45","var": "hit_set_period","val": "3"}]: dispatch 2024-03-26T03:52:43.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:43 smithi175 bash[21527]: cluster 2024-03-26T03:52:42.700814+0000 mgr.y (mgr.24431) 336 : cluster [DBG] pgmap v456: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:44.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:44 smithi190 bash[23392]: audit 2024-03-26T03:52:43.620217+0000 mon.a (mon.0) 2234 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi175-64406-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi175-64406-58"}]': finished 2024-03-26T03:52:44.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:44 smithi190 bash[23392]: audit 2024-03-26T03:52:43.620493+0000 mon.a (mon.0) 2235 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-45","var": "hit_set_period","val": "3"}]': finished 2024-03-26T03:52:44.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:44 smithi190 bash[23392]: cluster 2024-03-26T03:52:43.631337+0000 mon.a (mon.0) 2236 : cluster [DBG] osdmap e307: 8 total, 8 up, 8 in 2024-03-26T03:52:44.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:44 smithi190 bash[23392]: audit 2024-03-26T03:52:43.635718+0000 mon.a (mon.0) 2237 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-26T03:52:44.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:44 smithi175 bash[29284]: audit 2024-03-26T03:52:43.620217+0000 mon.a (mon.0) 2234 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi175-64406-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi175-64406-58"}]': finished 2024-03-26T03:52:44.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:44 smithi175 bash[29284]: audit 2024-03-26T03:52:43.620493+0000 mon.a (mon.0) 2235 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-45","var": "hit_set_period","val": "3"}]': finished 2024-03-26T03:52:44.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:44 smithi175 bash[29284]: cluster 2024-03-26T03:52:43.631337+0000 mon.a (mon.0) 2236 : cluster [DBG] osdmap e307: 8 total, 8 up, 8 in 2024-03-26T03:52:44.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:44 smithi175 bash[29284]: audit 2024-03-26T03:52:43.635718+0000 mon.a (mon.0) 2237 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-26T03:52:44.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:44 smithi175 bash[21527]: audit 2024-03-26T03:52:43.620217+0000 mon.a (mon.0) 2234 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi175-64406-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi175-64406-58"}]': finished 2024-03-26T03:52:44.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:44 smithi175 bash[21527]: audit 2024-03-26T03:52:43.620493+0000 mon.a (mon.0) 2235 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-45","var": "hit_set_period","val": "3"}]': finished 2024-03-26T03:52:44.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:44 smithi175 bash[21527]: cluster 2024-03-26T03:52:43.631337+0000 mon.a (mon.0) 2236 : cluster [DBG] osdmap e307: 8 total, 8 up, 8 in 2024-03-26T03:52:44.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:44 smithi175 bash[21527]: audit 2024-03-26T03:52:43.635718+0000 mon.a (mon.0) 2237 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-26T03:52:45.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:45 smithi190 bash[23392]: audit 2024-03-26T03:52:44.623093+0000 mon.a (mon.0) 2238 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-45","var": "hit_set_type","val": "bloom"}]': finished 2024-03-26T03:52:45.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:45 smithi190 bash[23392]: cluster 2024-03-26T03:52:44.631722+0000 mon.a (mon.0) 2239 : cluster [DBG] osdmap e308: 8 total, 8 up, 8 in 2024-03-26T03:52:45.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:45 smithi190 bash[23392]: audit 2024-03-26T03:52:44.633252+0000 mon.a (mon.0) 2240 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-26T03:52:45.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:45 smithi190 bash[23392]: cluster 2024-03-26T03:52:44.702930+0000 mgr.y (mgr.24431) 337 : cluster [DBG] pgmap v459: 300 pgs: 2 unknown, 1 creating+activating, 12 creating+peering, 285 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-26T03:52:45.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:45 smithi190 bash[23392]: audit 2024-03-26T03:52:45.625909+0000 mon.a (mon.0) 2241 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-03-26T03:52:45.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:45 smithi190 bash[23392]: cluster 2024-03-26T03:52:45.633832+0000 mon.a (mon.0) 2242 : cluster [DBG] osdmap e309: 8 total, 8 up, 8 in 2024-03-26T03:52:45.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:45 smithi175 bash[21527]: audit 2024-03-26T03:52:44.623093+0000 mon.a (mon.0) 2238 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-45","var": "hit_set_type","val": "bloom"}]': finished 2024-03-26T03:52:45.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:45 smithi175 bash[21527]: cluster 2024-03-26T03:52:44.631722+0000 mon.a (mon.0) 2239 : cluster [DBG] osdmap e308: 8 total, 8 up, 8 in 2024-03-26T03:52:45.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:45 smithi175 bash[21527]: audit 2024-03-26T03:52:44.633252+0000 mon.a (mon.0) 2240 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-26T03:52:45.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:45 smithi175 bash[21527]: cluster 2024-03-26T03:52:44.702930+0000 mgr.y (mgr.24431) 337 : cluster [DBG] pgmap v459: 300 pgs: 2 unknown, 1 creating+activating, 12 creating+peering, 285 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-26T03:52:45.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:45 smithi175 bash[21527]: audit 2024-03-26T03:52:45.625909+0000 mon.a (mon.0) 2241 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-03-26T03:52:45.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:45 smithi175 bash[21527]: cluster 2024-03-26T03:52:45.633832+0000 mon.a (mon.0) 2242 : cluster [DBG] osdmap e309: 8 total, 8 up, 8 in 2024-03-26T03:52:45.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:45 smithi175 bash[29284]: audit 2024-03-26T03:52:44.623093+0000 mon.a (mon.0) 2238 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-45","var": "hit_set_type","val": "bloom"}]': finished 2024-03-26T03:52:45.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:45 smithi175 bash[29284]: cluster 2024-03-26T03:52:44.631722+0000 mon.a (mon.0) 2239 : cluster [DBG] osdmap e308: 8 total, 8 up, 8 in 2024-03-26T03:52:45.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:45 smithi175 bash[29284]: audit 2024-03-26T03:52:44.633252+0000 mon.a (mon.0) 2240 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-26T03:52:45.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:45 smithi175 bash[29284]: cluster 2024-03-26T03:52:44.702930+0000 mgr.y (mgr.24431) 337 : cluster [DBG] pgmap v459: 300 pgs: 2 unknown, 1 creating+activating, 12 creating+peering, 285 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-26T03:52:45.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:45 smithi175 bash[29284]: audit 2024-03-26T03:52:45.625909+0000 mon.a (mon.0) 2241 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-03-26T03:52:45.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:45 smithi175 bash[29284]: cluster 2024-03-26T03:52:45.633832+0000 mon.a (mon.0) 2242 : cluster [DBG] osdmap e309: 8 total, 8 up, 8 in 2024-03-26T03:52:46.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:46 smithi190 bash[23392]: audit 2024-03-26T03:52:45.636148+0000 mon.c (mon.2) 462 : audit [INF] from='client.? 172.21.15.175:0/3109723899' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi175-64406-58"}]: dispatch 2024-03-26T03:52:46.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:46 smithi190 bash[23392]: cluster 2024-03-26T03:52:45.636322+0000 mon.a (mon.0) 2243 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:52:46.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:46 smithi190 bash[23392]: audit 2024-03-26T03:52:45.637829+0000 mon.a (mon.0) 2244 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi175-64406-58"}]: dispatch 2024-03-26T03:52:46.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:46 smithi190 bash[23392]: audit 2024-03-26T03:52:46.628688+0000 mon.a (mon.0) 2245 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi175-64406-58"}]': finished 2024-03-26T03:52:46.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:46 smithi190 bash[23392]: cluster 2024-03-26T03:52:46.635304+0000 mon.a (mon.0) 2246 : cluster [DBG] osdmap e310: 8 total, 8 up, 8 in 2024-03-26T03:52:46.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:46 smithi190 bash[23392]: audit 2024-03-26T03:52:46.637933+0000 mon.c (mon.2) 463 : audit [INF] from='client.? 172.21.15.175:0/3109723899' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi175-64406-58"}]: dispatch 2024-03-26T03:52:46.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:46 smithi190 bash[23392]: audit 2024-03-26T03:52:46.640383+0000 mon.a (mon.0) 2247 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi175-64406-58"}]: dispatch 2024-03-26T03:52:46.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:46 smithi175 bash[29284]: audit 2024-03-26T03:52:45.636148+0000 mon.c (mon.2) 462 : audit [INF] from='client.? 172.21.15.175:0/3109723899' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi175-64406-58"}]: dispatch 2024-03-26T03:52:46.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:46 smithi175 bash[29284]: cluster 2024-03-26T03:52:45.636322+0000 mon.a (mon.0) 2243 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:52:46.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:46 smithi175 bash[29284]: audit 2024-03-26T03:52:45.637829+0000 mon.a (mon.0) 2244 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi175-64406-58"}]: dispatch 2024-03-26T03:52:46.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:46 smithi175 bash[29284]: audit 2024-03-26T03:52:46.628688+0000 mon.a (mon.0) 2245 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi175-64406-58"}]': finished 2024-03-26T03:52:46.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:46 smithi175 bash[29284]: cluster 2024-03-26T03:52:46.635304+0000 mon.a (mon.0) 2246 : cluster [DBG] osdmap e310: 8 total, 8 up, 8 in 2024-03-26T03:52:46.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:46 smithi175 bash[29284]: audit 2024-03-26T03:52:46.637933+0000 mon.c (mon.2) 463 : audit [INF] from='client.? 172.21.15.175:0/3109723899' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi175-64406-58"}]: dispatch 2024-03-26T03:52:46.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:46 smithi175 bash[29284]: audit 2024-03-26T03:52:46.640383+0000 mon.a (mon.0) 2247 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi175-64406-58"}]: dispatch 2024-03-26T03:52:46.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:46 smithi175 bash[21527]: audit 2024-03-26T03:52:45.636148+0000 mon.c (mon.2) 462 : audit [INF] from='client.? 172.21.15.175:0/3109723899' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi175-64406-58"}]: dispatch 2024-03-26T03:52:46.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:46 smithi175 bash[21527]: cluster 2024-03-26T03:52:45.636322+0000 mon.a (mon.0) 2243 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:52:46.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:46 smithi175 bash[21527]: audit 2024-03-26T03:52:45.637829+0000 mon.a (mon.0) 2244 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi175-64406-58"}]: dispatch 2024-03-26T03:52:46.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:46 smithi175 bash[21527]: audit 2024-03-26T03:52:46.628688+0000 mon.a (mon.0) 2245 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi175-64406-58"}]': finished 2024-03-26T03:52:46.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:46 smithi175 bash[21527]: cluster 2024-03-26T03:52:46.635304+0000 mon.a (mon.0) 2246 : cluster [DBG] osdmap e310: 8 total, 8 up, 8 in 2024-03-26T03:52:46.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:46 smithi175 bash[21527]: audit 2024-03-26T03:52:46.637933+0000 mon.c (mon.2) 463 : audit [INF] from='client.? 172.21.15.175:0/3109723899' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi175-64406-58"}]: dispatch 2024-03-26T03:52:46.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:46 smithi175 bash[21527]: audit 2024-03-26T03:52:46.640383+0000 mon.a (mon.0) 2247 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi175-64406-58"}]: dispatch 2024-03-26T03:52:47.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:47 smithi190 bash[23392]: cluster 2024-03-26T03:52:46.703853+0000 mgr.y (mgr.24431) 338 : cluster [DBG] pgmap v462: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:47.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:47 smithi190 bash[23392]: audit 2024-03-26T03:52:47.631975+0000 mon.a (mon.0) 2248 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi175-64406-58"}]': finished 2024-03-26T03:52:47.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:47 smithi190 bash[23392]: cluster 2024-03-26T03:52:47.639994+0000 mon.a (mon.0) 2249 : cluster [DBG] osdmap e311: 8 total, 8 up, 8 in 2024-03-26T03:52:47.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:47 smithi175 bash[21527]: cluster 2024-03-26T03:52:46.703853+0000 mgr.y (mgr.24431) 338 : cluster [DBG] pgmap v462: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:47.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:47 smithi175 bash[21527]: audit 2024-03-26T03:52:47.631975+0000 mon.a (mon.0) 2248 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi175-64406-58"}]': finished 2024-03-26T03:52:47.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:47 smithi175 bash[21527]: cluster 2024-03-26T03:52:47.639994+0000 mon.a (mon.0) 2249 : cluster [DBG] osdmap e311: 8 total, 8 up, 8 in 2024-03-26T03:52:47.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:47 smithi175 bash[29284]: cluster 2024-03-26T03:52:46.703853+0000 mgr.y (mgr.24431) 338 : cluster [DBG] pgmap v462: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:47.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:47 smithi175 bash[29284]: audit 2024-03-26T03:52:47.631975+0000 mon.a (mon.0) 2248 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi175-64406-58"}]': finished 2024-03-26T03:52:47.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:47 smithi175 bash[29284]: cluster 2024-03-26T03:52:47.639994+0000 mon.a (mon.0) 2249 : cluster [DBG] osdmap e311: 8 total, 8 up, 8 in 2024-03-26T03:52:48.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:48 smithi190 bash[23392]: audit 2024-03-26T03:52:47.674128+0000 mon.c (mon.2) 464 : audit [INF] from='client.? 172.21.15.175:0/5468309' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi175-64406-59"}]: dispatch 2024-03-26T03:52:48.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:48 smithi190 bash[23392]: audit 2024-03-26T03:52:47.674506+0000 mon.a (mon.0) 2250 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi175-64406-59"}]: dispatch 2024-03-26T03:52:48.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:48 smithi190 bash[23392]: audit 2024-03-26T03:52:47.675188+0000 mon.c (mon.2) 465 : audit [INF] from='client.? 172.21.15.175:0/5468309' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi175-64406-59"}]: dispatch 2024-03-26T03:52:48.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:48 smithi190 bash[23392]: audit 2024-03-26T03:52:47.675548+0000 mon.a (mon.0) 2251 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi175-64406-59"}]: dispatch 2024-03-26T03:52:48.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:48 smithi190 bash[23392]: audit 2024-03-26T03:52:47.676104+0000 mon.c (mon.2) 466 : audit [INF] from='client.? 172.21.15.175:0/5468309' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi175-64406-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:48.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:48 smithi190 bash[23392]: audit 2024-03-26T03:52:47.676407+0000 mon.a (mon.0) 2252 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi175-64406-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:48.992 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:52:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:52:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:52:48.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:48 smithi175 bash[29284]: audit 2024-03-26T03:52:47.674128+0000 mon.c (mon.2) 464 : audit [INF] from='client.? 172.21.15.175:0/5468309' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi175-64406-59"}]: dispatch 2024-03-26T03:52:48.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:48 smithi175 bash[29284]: audit 2024-03-26T03:52:47.674506+0000 mon.a (mon.0) 2250 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi175-64406-59"}]: dispatch 2024-03-26T03:52:48.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:48 smithi175 bash[29284]: audit 2024-03-26T03:52:47.675188+0000 mon.c (mon.2) 465 : audit [INF] from='client.? 172.21.15.175:0/5468309' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi175-64406-59"}]: dispatch 2024-03-26T03:52:48.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:48 smithi175 bash[29284]: audit 2024-03-26T03:52:47.675548+0000 mon.a (mon.0) 2251 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi175-64406-59"}]: dispatch 2024-03-26T03:52:48.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:48 smithi175 bash[29284]: audit 2024-03-26T03:52:47.676104+0000 mon.c (mon.2) 466 : audit [INF] from='client.? 172.21.15.175:0/5468309' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi175-64406-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:48.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:48 smithi175 bash[29284]: audit 2024-03-26T03:52:47.676407+0000 mon.a (mon.0) 2252 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi175-64406-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:48.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:48 smithi175 bash[21527]: audit 2024-03-26T03:52:47.674128+0000 mon.c (mon.2) 464 : audit [INF] from='client.? 172.21.15.175:0/5468309' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi175-64406-59"}]: dispatch 2024-03-26T03:52:48.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:48 smithi175 bash[21527]: audit 2024-03-26T03:52:47.674506+0000 mon.a (mon.0) 2250 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi175-64406-59"}]: dispatch 2024-03-26T03:52:48.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:48 smithi175 bash[21527]: audit 2024-03-26T03:52:47.675188+0000 mon.c (mon.2) 465 : audit [INF] from='client.? 172.21.15.175:0/5468309' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi175-64406-59"}]: dispatch 2024-03-26T03:52:48.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:48 smithi175 bash[21527]: audit 2024-03-26T03:52:47.675548+0000 mon.a (mon.0) 2251 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi175-64406-59"}]: dispatch 2024-03-26T03:52:48.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:48 smithi175 bash[21527]: audit 2024-03-26T03:52:47.676104+0000 mon.c (mon.2) 466 : audit [INF] from='client.? 172.21.15.175:0/5468309' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi175-64406-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:48.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:48 smithi175 bash[21527]: audit 2024-03-26T03:52:47.676407+0000 mon.a (mon.0) 2252 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi175-64406-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:49.989 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:52:49 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:52:49.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:49 smithi190 bash[23392]: audit 2024-03-26T03:52:48.664577+0000 mon.a (mon.0) 2253 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi175-64406-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:52:49.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:49 smithi190 bash[23392]: cluster 2024-03-26T03:52:48.672089+0000 mon.a (mon.0) 2254 : cluster [DBG] osdmap e312: 8 total, 8 up, 8 in 2024-03-26T03:52:49.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:49 smithi190 bash[23392]: audit 2024-03-26T03:52:48.678124+0000 mon.c (mon.2) 467 : audit [INF] from='client.? 172.21.15.175:0/5468309' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi175-64406-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi175-64406-59"}]: dispatch 2024-03-26T03:52:49.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:49 smithi190 bash[23392]: audit 2024-03-26T03:52:48.686917+0000 mon.a (mon.0) 2255 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi175-64406-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi175-64406-59"}]: dispatch 2024-03-26T03:52:49.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:49 smithi190 bash[23392]: cluster 2024-03-26T03:52:48.704696+0000 mgr.y (mgr.24431) 339 : cluster [DBG] pgmap v465: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:49.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:49 smithi190 bash[23392]: audit 2024-03-26T03:52:49.637314+0000 mon.c (mon.2) 468 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:52:49.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:49 smithi175 bash[21527]: audit 2024-03-26T03:52:48.664577+0000 mon.a (mon.0) 2253 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi175-64406-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:52:49.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:49 smithi175 bash[21527]: cluster 2024-03-26T03:52:48.672089+0000 mon.a (mon.0) 2254 : cluster [DBG] osdmap e312: 8 total, 8 up, 8 in 2024-03-26T03:52:49.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:49 smithi175 bash[21527]: audit 2024-03-26T03:52:48.678124+0000 mon.c (mon.2) 467 : audit [INF] from='client.? 172.21.15.175:0/5468309' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi175-64406-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi175-64406-59"}]: dispatch 2024-03-26T03:52:49.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:49 smithi175 bash[21527]: audit 2024-03-26T03:52:48.686917+0000 mon.a (mon.0) 2255 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi175-64406-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi175-64406-59"}]: dispatch 2024-03-26T03:52:49.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:49 smithi175 bash[21527]: cluster 2024-03-26T03:52:48.704696+0000 mgr.y (mgr.24431) 339 : cluster [DBG] pgmap v465: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:49.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:49 smithi175 bash[21527]: audit 2024-03-26T03:52:49.637314+0000 mon.c (mon.2) 468 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:52:49.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:49 smithi175 bash[29284]: audit 2024-03-26T03:52:48.664577+0000 mon.a (mon.0) 2253 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi175-64406-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:52:49.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:49 smithi175 bash[29284]: cluster 2024-03-26T03:52:48.672089+0000 mon.a (mon.0) 2254 : cluster [DBG] osdmap e312: 8 total, 8 up, 8 in 2024-03-26T03:52:49.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:49 smithi175 bash[29284]: audit 2024-03-26T03:52:48.678124+0000 mon.c (mon.2) 467 : audit [INF] from='client.? 172.21.15.175:0/5468309' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi175-64406-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi175-64406-59"}]: dispatch 2024-03-26T03:52:49.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:49 smithi175 bash[29284]: audit 2024-03-26T03:52:48.686917+0000 mon.a (mon.0) 2255 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi175-64406-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi175-64406-59"}]: dispatch 2024-03-26T03:52:49.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:49 smithi175 bash[29284]: cluster 2024-03-26T03:52:48.704696+0000 mgr.y (mgr.24431) 339 : cluster [DBG] pgmap v465: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:52:49.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:49 smithi175 bash[29284]: audit 2024-03-26T03:52:49.637314+0000 mon.c (mon.2) 468 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:52:50.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:50 smithi190 bash[23392]: cluster 2024-03-26T03:52:49.711107+0000 mon.a (mon.0) 2256 : cluster [DBG] osdmap e313: 8 total, 8 up, 8 in 2024-03-26T03:52:50.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:50 smithi190 bash[23392]: audit 2024-03-26T03:52:49.911334+0000 mgr.y (mgr.24431) 340 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:52:50.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:50 smithi175 bash[21527]: cluster 2024-03-26T03:52:49.711107+0000 mon.a (mon.0) 2256 : cluster [DBG] osdmap e313: 8 total, 8 up, 8 in 2024-03-26T03:52:50.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:50 smithi175 bash[21527]: audit 2024-03-26T03:52:49.911334+0000 mgr.y (mgr.24431) 340 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:52:50.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:50 smithi175 bash[29284]: cluster 2024-03-26T03:52:49.711107+0000 mon.a (mon.0) 2256 : cluster [DBG] osdmap e313: 8 total, 8 up, 8 in 2024-03-26T03:52:50.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:50 smithi175 bash[29284]: audit 2024-03-26T03:52:49.911334+0000 mgr.y (mgr.24431) 340 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:52:51.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:51 smithi190 bash[23392]: cluster 2024-03-26T03:52:50.706409+0000 mgr.y (mgr.24431) 341 : cluster [DBG] pgmap v467: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-26T03:52:51.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:51 smithi190 bash[23392]: audit 2024-03-26T03:52:50.710724+0000 mon.a (mon.0) 2257 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi175-64406-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi175-64406-59"}]': finished 2024-03-26T03:52:51.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:51 smithi190 bash[23392]: cluster 2024-03-26T03:52:50.718088+0000 mon.a (mon.0) 2258 : cluster [DBG] osdmap e314: 8 total, 8 up, 8 in 2024-03-26T03:52:51.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:51 smithi190 bash[23392]: cluster 2024-03-26T03:52:51.486178+0000 mon.a (mon.0) 2259 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:52:51.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:51 smithi175 bash[29284]: cluster 2024-03-26T03:52:50.706409+0000 mgr.y (mgr.24431) 341 : cluster [DBG] pgmap v467: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-26T03:52:51.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:51 smithi175 bash[29284]: audit 2024-03-26T03:52:50.710724+0000 mon.a (mon.0) 2257 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi175-64406-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi175-64406-59"}]': finished 2024-03-26T03:52:51.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:51 smithi175 bash[29284]: cluster 2024-03-26T03:52:50.718088+0000 mon.a (mon.0) 2258 : cluster [DBG] osdmap e314: 8 total, 8 up, 8 in 2024-03-26T03:52:51.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:51 smithi175 bash[29284]: cluster 2024-03-26T03:52:51.486178+0000 mon.a (mon.0) 2259 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:52:51.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:51 smithi175 bash[21527]: cluster 2024-03-26T03:52:50.706409+0000 mgr.y (mgr.24431) 341 : cluster [DBG] pgmap v467: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-26T03:52:51.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:51 smithi175 bash[21527]: audit 2024-03-26T03:52:50.710724+0000 mon.a (mon.0) 2257 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi175-64406-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi175-64406-59"}]': finished 2024-03-26T03:52:51.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:51 smithi175 bash[21527]: cluster 2024-03-26T03:52:50.718088+0000 mon.a (mon.0) 2258 : cluster [DBG] osdmap e314: 8 total, 8 up, 8 in 2024-03-26T03:52:51.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:51 smithi175 bash[21527]: cluster 2024-03-26T03:52:51.486178+0000 mon.a (mon.0) 2259 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:52:52.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:52 smithi190 bash[23392]: cluster 2024-03-26T03:52:51.720992+0000 mon.a (mon.0) 2260 : cluster [DBG] osdmap e315: 8 total, 8 up, 8 in 2024-03-26T03:52:52.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:52 smithi175 bash[29284]: cluster 2024-03-26T03:52:51.720992+0000 mon.a (mon.0) 2260 : cluster [DBG] osdmap e315: 8 total, 8 up, 8 in 2024-03-26T03:52:52.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:52 smithi175 bash[21527]: cluster 2024-03-26T03:52:51.720992+0000 mon.a (mon.0) 2260 : cluster [DBG] osdmap e315: 8 total, 8 up, 8 in 2024-03-26T03:52:54.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:53 smithi190 bash[23392]: cluster 2024-03-26T03:52:52.707364+0000 mgr.y (mgr.24431) 342 : cluster [DBG] pgmap v470: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-26T03:52:54.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:53 smithi190 bash[23392]: cluster 2024-03-26T03:52:52.746496+0000 mon.a (mon.0) 2261 : cluster [DBG] osdmap e316: 8 total, 8 up, 8 in 2024-03-26T03:52:54.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:53 smithi190 bash[23392]: audit 2024-03-26T03:52:52.750853+0000 mon.c (mon.2) 469 : audit [INF] from='client.? 172.21.15.175:0/5468309' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi175-64406-59"}]: dispatch 2024-03-26T03:52:54.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:53 smithi190 bash[23392]: audit 2024-03-26T03:52:52.752479+0000 mon.a (mon.0) 2262 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi175-64406-59"}]: dispatch 2024-03-26T03:52:54.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:53 smithi190 bash[23392]: audit 2024-03-26T03:52:52.818668+0000 mon.c (mon.2) 470 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:52:54.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:53 smithi190 bash[23392]: audit 2024-03-26T03:52:53.603371+0000 mon.c (mon.2) 471 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:52:54.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:53 smithi190 bash[23392]: audit 2024-03-26T03:52:53.604948+0000 mon.c (mon.2) 472 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:52:54.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:53 smithi190 bash[23392]: audit 2024-03-26T03:52:53.614588+0000 mon.a (mon.0) 2263 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:52:54.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:53 smithi175 bash[21527]: cluster 2024-03-26T03:52:52.707364+0000 mgr.y (mgr.24431) 342 : cluster [DBG] pgmap v470: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-26T03:52:54.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:53 smithi175 bash[21527]: cluster 2024-03-26T03:52:52.746496+0000 mon.a (mon.0) 2261 : cluster [DBG] osdmap e316: 8 total, 8 up, 8 in 2024-03-26T03:52:54.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:53 smithi175 bash[21527]: audit 2024-03-26T03:52:52.750853+0000 mon.c (mon.2) 469 : audit [INF] from='client.? 172.21.15.175:0/5468309' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi175-64406-59"}]: dispatch 2024-03-26T03:52:54.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:53 smithi175 bash[21527]: audit 2024-03-26T03:52:52.752479+0000 mon.a (mon.0) 2262 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi175-64406-59"}]: dispatch 2024-03-26T03:52:54.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:53 smithi175 bash[21527]: audit 2024-03-26T03:52:52.818668+0000 mon.c (mon.2) 470 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:52:54.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:53 smithi175 bash[21527]: audit 2024-03-26T03:52:53.603371+0000 mon.c (mon.2) 471 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:52:54.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:53 smithi175 bash[21527]: audit 2024-03-26T03:52:53.604948+0000 mon.c (mon.2) 472 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:52:54.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:53 smithi175 bash[21527]: audit 2024-03-26T03:52:53.614588+0000 mon.a (mon.0) 2263 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:52:54.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:53 smithi175 bash[29284]: cluster 2024-03-26T03:52:52.707364+0000 mgr.y (mgr.24431) 342 : cluster [DBG] pgmap v470: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-26T03:52:54.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:53 smithi175 bash[29284]: cluster 2024-03-26T03:52:52.746496+0000 mon.a (mon.0) 2261 : cluster [DBG] osdmap e316: 8 total, 8 up, 8 in 2024-03-26T03:52:54.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:53 smithi175 bash[29284]: audit 2024-03-26T03:52:52.750853+0000 mon.c (mon.2) 469 : audit [INF] from='client.? 172.21.15.175:0/5468309' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi175-64406-59"}]: dispatch 2024-03-26T03:52:54.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:53 smithi175 bash[29284]: audit 2024-03-26T03:52:52.752479+0000 mon.a (mon.0) 2262 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi175-64406-59"}]: dispatch 2024-03-26T03:52:54.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:53 smithi175 bash[29284]: audit 2024-03-26T03:52:52.818668+0000 mon.c (mon.2) 470 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:52:54.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:53 smithi175 bash[29284]: audit 2024-03-26T03:52:53.603371+0000 mon.c (mon.2) 471 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:52:54.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:53 smithi175 bash[29284]: audit 2024-03-26T03:52:53.604948+0000 mon.c (mon.2) 472 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:52:54.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:53 smithi175 bash[29284]: audit 2024-03-26T03:52:53.614588+0000 mon.a (mon.0) 2263 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:52:54.780 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:7462ddf6:::.RoundTripAppendPP (3019 ms) 2024-03-26T03:52:54.780 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2024-03-26T03:52:54.781 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (3001 ms) 2024-03-26T03:52:54.781 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-03-26T03:52:54.781 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (2968 ms) 2024-03-26T03:52:54.781 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-03-26T03:52:54.781 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (3039 ms) 2024-03-26T03:52:54.781 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolEIOFlag 2024-03-26T03:52:54.781 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: setting pool EIO 2024-03-26T03:52:54.781 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: max_success 102, min_failed 103 2024-03-26T03:52:54.781 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolEIOFlag (4006 ms) 2024-03-26T03:52:54.781 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2024-03-26T03:52:54.781 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (3022 ms) 2024-03-26T03:52:54.781 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio (115320 ms total) 2024-03-26T03:52:54.781 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: 2024-03-26T03:52:54.781 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2024-03-26T03:52:54.781 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-03-26T03:52:54.782 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (2999 ms) 2024-03-26T03:52:54.782 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-03-26T03:52:54.782 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (9077 ms) 2024-03-26T03:52:54.782 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2024-03-26T03:52:54.782 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (14871 ms) 2024-03-26T03:52:54.782 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2024-03-26T03:52:54.782 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (3024 ms) 2024-03-26T03:52:54.782 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (29981 ms total) 2024-03-26T03:52:54.782 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: 2024-03-26T03:52:54.782 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2024-03-26T03:52:54.782 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-03-26T03:52:54.782 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (3014 ms) 2024-03-26T03:52:54.783 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (3014 ms total) 2024-03-26T03:52:54.783 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: 2024-03-26T03:52:54.783 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2024-03-26T03:52:54.783 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2024-03-26T03:52:54.783 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (13959 ms) 2024-03-26T03:52:54.783 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2024-03-26T03:52:54.783 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (6989 ms) 2024-03-26T03:52:54.783 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2024-03-26T03:52:54.783 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (7070 ms) 2024-03-26T03:52:54.783 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2024-03-26T03:52:54.783 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (7082 ms) 2024-03-26T03:52:54.783 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2024-03-26T03:52:54.783 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (3015 ms) 2024-03-26T03:52:54.783 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-03-26T03:52:54.783 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (7040 ms) 2024-03-26T03:52:54.784 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-03-26T03:52:54.784 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (6813 ms) 2024-03-26T03:52:54.784 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2024-03-26T03:52:54.784 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (7055 ms) 2024-03-26T03:52:54.784 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2024-03-26T03:52:54.784 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (7024 ms) 2024-03-26T03:52:54.784 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2024-03-26T03:52:54.784 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (7020 ms) 2024-03-26T03:52:54.784 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2024-03-26T03:52:54.784 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (7038 ms) 2024-03-26T03:52:54.784 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2024-03-26T03:52:54.784 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (7023 ms) 2024-03-26T03:52:54.784 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-03-26T03:52:54.784 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (7137 ms) 2024-03-26T03:52:55.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:54 smithi190 bash[23392]: audit 2024-03-26T03:52:53.739626+0000 mon.a (mon.0) 2264 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi175-64406-59"}]': finished 2024-03-26T03:52:55.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:54 smithi190 bash[23392]: cluster 2024-03-26T03:52:53.747697+0000 mon.a (mon.0) 2265 : cluster [DBG] osdmap e317: 8 total, 8 up, 8 in 2024-03-26T03:52:55.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:54 smithi190 bash[23392]: audit 2024-03-26T03:52:53.751390+0000 mon.c (mon.2) 473 : audit [INF] from='client.? 172.21.15.175:0/5468309' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi175-64406-59"}]: dispatch 2024-03-26T03:52:55.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:54 smithi190 bash[23392]: audit 2024-03-26T03:52:53.768323+0000 mon.a (mon.0) 2266 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi175-64406-59"}]: dispatch 2024-03-26T03:52:55.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:54 smithi175 bash[21527]: audit 2024-03-26T03:52:53.739626+0000 mon.a (mon.0) 2264 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi175-64406-59"}]': finished 2024-03-26T03:52:55.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:54 smithi175 bash[21527]: cluster 2024-03-26T03:52:53.747697+0000 mon.a (mon.0) 2265 : cluster [DBG] osdmap e317: 8 total, 8 up, 8 in 2024-03-26T03:52:55.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:54 smithi175 bash[21527]: audit 2024-03-26T03:52:53.751390+0000 mon.c (mon.2) 473 : audit [INF] from='client.? 172.21.15.175:0/5468309' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi175-64406-59"}]: dispatch 2024-03-26T03:52:55.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:54 smithi175 bash[21527]: audit 2024-03-26T03:52:53.768323+0000 mon.a (mon.0) 2266 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi175-64406-59"}]: dispatch 2024-03-26T03:52:55.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:54 smithi175 bash[29284]: audit 2024-03-26T03:52:53.739626+0000 mon.a (mon.0) 2264 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi175-64406-59"}]': finished 2024-03-26T03:52:55.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:54 smithi175 bash[29284]: cluster 2024-03-26T03:52:53.747697+0000 mon.a (mon.0) 2265 : cluster [DBG] osdmap e317: 8 total, 8 up, 8 in 2024-03-26T03:52:55.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:54 smithi175 bash[29284]: audit 2024-03-26T03:52:53.751390+0000 mon.c (mon.2) 473 : audit [INF] from='client.? 172.21.15.175:0/5468309' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi175-64406-59"}]: dispatch 2024-03-26T03:52:55.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:54 smithi175 bash[29284]: audit 2024-03-26T03:52:53.768323+0000 mon.a (mon.0) 2266 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi175-64406-59"}]: dispatch 2024-03-26T03:52:56.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:55 smithi190 bash[23392]: cluster 2024-03-26T03:52:54.709386+0000 mgr.y (mgr.24431) 343 : cluster [DBG] pgmap v473: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:52:56.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:55 smithi190 bash[23392]: audit 2024-03-26T03:52:54.765081+0000 mon.a (mon.0) 2267 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi175-64406-59"}]': finished 2024-03-26T03:52:56.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:55 smithi190 bash[23392]: cluster 2024-03-26T03:52:54.776141+0000 mon.a (mon.0) 2268 : cluster [DBG] osdmap e318: 8 total, 8 up, 8 in 2024-03-26T03:52:56.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:55 smithi175 bash[29284]: cluster 2024-03-26T03:52:54.709386+0000 mgr.y (mgr.24431) 343 : cluster [DBG] pgmap v473: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:52:56.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:55 smithi175 bash[29284]: audit 2024-03-26T03:52:54.765081+0000 mon.a (mon.0) 2267 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi175-64406-59"}]': finished 2024-03-26T03:52:56.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:55 smithi175 bash[29284]: cluster 2024-03-26T03:52:54.776141+0000 mon.a (mon.0) 2268 : cluster [DBG] osdmap e318: 8 total, 8 up, 8 in 2024-03-26T03:52:56.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:55 smithi175 bash[21527]: cluster 2024-03-26T03:52:54.709386+0000 mgr.y (mgr.24431) 343 : cluster [DBG] pgmap v473: 292 pgs: 292 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:52:56.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:55 smithi175 bash[21527]: audit 2024-03-26T03:52:54.765081+0000 mon.a (mon.0) 2267 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi175-64406-59"}]': finished 2024-03-26T03:52:56.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:55 smithi175 bash[21527]: cluster 2024-03-26T03:52:54.776141+0000 mon.a (mon.0) 2268 : cluster [DBG] osdmap e318: 8 total, 8 up, 8 in 2024-03-26T03:52:57.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:56 smithi190 bash[23392]: cluster 2024-03-26T03:52:55.788055+0000 mon.a (mon.0) 2269 : cluster [DBG] osdmap e319: 8 total, 8 up, 8 in 2024-03-26T03:52:57.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:56 smithi190 bash[23392]: audit 2024-03-26T03:52:55.793952+0000 mon.c (mon.2) 474 : audit [INF] from='client.? 172.21.15.175:0/1113680509' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi175-64406-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:52:57.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:56 smithi190 bash[23392]: audit 2024-03-26T03:52:55.794761+0000 mon.a (mon.0) 2270 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi175-64406-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:52:57.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:56 smithi175 bash[29284]: cluster 2024-03-26T03:52:55.788055+0000 mon.a (mon.0) 2269 : cluster [DBG] osdmap e319: 8 total, 8 up, 8 in 2024-03-26T03:52:57.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:56 smithi175 bash[29284]: audit 2024-03-26T03:52:55.793952+0000 mon.c (mon.2) 474 : audit [INF] from='client.? 172.21.15.175:0/1113680509' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi175-64406-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:52:57.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:56 smithi175 bash[29284]: audit 2024-03-26T03:52:55.794761+0000 mon.a (mon.0) 2270 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi175-64406-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:52:57.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:56 smithi175 bash[21527]: cluster 2024-03-26T03:52:55.788055+0000 mon.a (mon.0) 2269 : cluster [DBG] osdmap e319: 8 total, 8 up, 8 in 2024-03-26T03:52:57.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:56 smithi175 bash[21527]: audit 2024-03-26T03:52:55.793952+0000 mon.c (mon.2) 474 : audit [INF] from='client.? 172.21.15.175:0/1113680509' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi175-64406-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:52:57.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:56 smithi175 bash[21527]: audit 2024-03-26T03:52:55.794761+0000 mon.a (mon.0) 2270 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi175-64406-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:52:58.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:57 smithi190 bash[23392]: cluster 2024-03-26T03:52:56.710327+0000 mgr.y (mgr.24431) 344 : cluster [DBG] pgmap v476: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-03-26T03:52:58.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:57 smithi190 bash[23392]: cluster 2024-03-26T03:52:56.779513+0000 mon.a (mon.0) 2271 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:52:58.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:57 smithi190 bash[23392]: audit 2024-03-26T03:52:56.789173+0000 mon.a (mon.0) 2272 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi175-64406-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:52:58.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:57 smithi190 bash[23392]: cluster 2024-03-26T03:52:56.799020+0000 mon.a (mon.0) 2273 : cluster [DBG] osdmap e320: 8 total, 8 up, 8 in 2024-03-26T03:52:58.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:57 smithi190 bash[23392]: audit 2024-03-26T03:52:57.729714+0000 mon.a (mon.0) 2274 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:52:58.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:57 smithi190 bash[23392]: audit 2024-03-26T03:52:57.730737+0000 mon.a (mon.0) 2275 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-45"}]: dispatch 2024-03-26T03:52:58.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:57 smithi175 bash[21527]: cluster 2024-03-26T03:52:56.710327+0000 mgr.y (mgr.24431) 344 : cluster [DBG] pgmap v476: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-03-26T03:52:58.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:57 smithi175 bash[21527]: cluster 2024-03-26T03:52:56.779513+0000 mon.a (mon.0) 2271 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:52:58.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:57 smithi175 bash[21527]: audit 2024-03-26T03:52:56.789173+0000 mon.a (mon.0) 2272 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi175-64406-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:52:58.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:57 smithi175 bash[21527]: cluster 2024-03-26T03:52:56.799020+0000 mon.a (mon.0) 2273 : cluster [DBG] osdmap e320: 8 total, 8 up, 8 in 2024-03-26T03:52:58.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:57 smithi175 bash[21527]: audit 2024-03-26T03:52:57.729714+0000 mon.a (mon.0) 2274 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:52:58.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:57 smithi175 bash[21527]: audit 2024-03-26T03:52:57.730737+0000 mon.a (mon.0) 2275 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-45"}]: dispatch 2024-03-26T03:52:58.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:57 smithi175 bash[29284]: cluster 2024-03-26T03:52:56.710327+0000 mgr.y (mgr.24431) 344 : cluster [DBG] pgmap v476: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-03-26T03:52:58.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:57 smithi175 bash[29284]: cluster 2024-03-26T03:52:56.779513+0000 mon.a (mon.0) 2271 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:52:58.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:57 smithi175 bash[29284]: audit 2024-03-26T03:52:56.789173+0000 mon.a (mon.0) 2272 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi175-64406-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:52:58.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:57 smithi175 bash[29284]: cluster 2024-03-26T03:52:56.799020+0000 mon.a (mon.0) 2273 : cluster [DBG] osdmap e320: 8 total, 8 up, 8 in 2024-03-26T03:52:58.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:57 smithi175 bash[29284]: audit 2024-03-26T03:52:57.729714+0000 mon.a (mon.0) 2274 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:52:58.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:57 smithi175 bash[29284]: audit 2024-03-26T03:52:57.730737+0000 mon.a (mon.0) 2275 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-45"}]: dispatch 2024-03-26T03:52:59.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:58 smithi190 bash[23392]: audit 2024-03-26T03:52:57.792028+0000 mon.a (mon.0) 2276 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-45"}]': finished 2024-03-26T03:52:59.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:58 smithi190 bash[23392]: cluster 2024-03-26T03:52:57.800857+0000 mon.a (mon.0) 2277 : cluster [DBG] osdmap e321: 8 total, 8 up, 8 in 2024-03-26T03:52:59.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:58 smithi190 bash[23392]: audit 2024-03-26T03:52:57.838404+0000 mon.c (mon.2) 475 : audit [INF] from='client.? 172.21.15.175:0/3322971530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi175-64406-61"}]: dispatch 2024-03-26T03:52:59.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:58 smithi190 bash[23392]: audit 2024-03-26T03:52:57.839005+0000 mon.a (mon.0) 2278 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi175-64406-61"}]: dispatch 2024-03-26T03:52:59.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:58 smithi190 bash[23392]: audit 2024-03-26T03:52:57.840390+0000 mon.c (mon.2) 476 : audit [INF] from='client.? 172.21.15.175:0/3322971530' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi175-64406-61"}]: dispatch 2024-03-26T03:52:59.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:58 smithi190 bash[23392]: audit 2024-03-26T03:52:57.840687+0000 mon.a (mon.0) 2279 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi175-64406-61"}]: dispatch 2024-03-26T03:52:59.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:58 smithi190 bash[23392]: audit 2024-03-26T03:52:57.841390+0000 mon.c (mon.2) 477 : audit [INF] from='client.? 172.21.15.175:0/3322971530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi175-64406-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:59.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:58 smithi190 bash[23392]: audit 2024-03-26T03:52:57.841814+0000 mon.a (mon.0) 2280 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi175-64406-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:59.245 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:52:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:52:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:52:59.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:58 smithi175 bash[29284]: audit 2024-03-26T03:52:57.792028+0000 mon.a (mon.0) 2276 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-45"}]': finished 2024-03-26T03:52:59.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:58 smithi175 bash[29284]: cluster 2024-03-26T03:52:57.800857+0000 mon.a (mon.0) 2277 : cluster [DBG] osdmap e321: 8 total, 8 up, 8 in 2024-03-26T03:52:59.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:58 smithi175 bash[29284]: audit 2024-03-26T03:52:57.838404+0000 mon.c (mon.2) 475 : audit [INF] from='client.? 172.21.15.175:0/3322971530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi175-64406-61"}]: dispatch 2024-03-26T03:52:59.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:58 smithi175 bash[29284]: audit 2024-03-26T03:52:57.839005+0000 mon.a (mon.0) 2278 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi175-64406-61"}]: dispatch 2024-03-26T03:52:59.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:58 smithi175 bash[29284]: audit 2024-03-26T03:52:57.840390+0000 mon.c (mon.2) 476 : audit [INF] from='client.? 172.21.15.175:0/3322971530' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi175-64406-61"}]: dispatch 2024-03-26T03:52:59.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:58 smithi175 bash[29284]: audit 2024-03-26T03:52:57.840687+0000 mon.a (mon.0) 2279 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi175-64406-61"}]: dispatch 2024-03-26T03:52:59.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:58 smithi175 bash[29284]: audit 2024-03-26T03:52:57.841390+0000 mon.c (mon.2) 477 : audit [INF] from='client.? 172.21.15.175:0/3322971530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi175-64406-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:59.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:58 smithi175 bash[29284]: audit 2024-03-26T03:52:57.841814+0000 mon.a (mon.0) 2280 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi175-64406-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:59.246 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:58 smithi175 bash[21527]: audit 2024-03-26T03:52:57.792028+0000 mon.a (mon.0) 2276 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-45"}]': finished 2024-03-26T03:52:59.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:58 smithi175 bash[21527]: cluster 2024-03-26T03:52:57.800857+0000 mon.a (mon.0) 2277 : cluster [DBG] osdmap e321: 8 total, 8 up, 8 in 2024-03-26T03:52:59.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:58 smithi175 bash[21527]: audit 2024-03-26T03:52:57.838404+0000 mon.c (mon.2) 475 : audit [INF] from='client.? 172.21.15.175:0/3322971530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi175-64406-61"}]: dispatch 2024-03-26T03:52:59.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:58 smithi175 bash[21527]: audit 2024-03-26T03:52:57.839005+0000 mon.a (mon.0) 2278 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi175-64406-61"}]: dispatch 2024-03-26T03:52:59.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:58 smithi175 bash[21527]: audit 2024-03-26T03:52:57.840390+0000 mon.c (mon.2) 476 : audit [INF] from='client.? 172.21.15.175:0/3322971530' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi175-64406-61"}]: dispatch 2024-03-26T03:52:59.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:58 smithi175 bash[21527]: audit 2024-03-26T03:52:57.840687+0000 mon.a (mon.0) 2279 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi175-64406-61"}]: dispatch 2024-03-26T03:52:59.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:58 smithi175 bash[21527]: audit 2024-03-26T03:52:57.841390+0000 mon.c (mon.2) 477 : audit [INF] from='client.? 172.21.15.175:0/3322971530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi175-64406-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:52:59.247 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:58 smithi175 bash[21527]: audit 2024-03-26T03:52:57.841814+0000 mon.a (mon.0) 2280 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi175-64406-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:53:00.239 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:52:59 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:53:00.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:59 smithi190 bash[23392]: cluster 2024-03-26T03:52:58.711227+0000 mgr.y (mgr.24431) 345 : cluster [DBG] pgmap v479: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-03-26T03:53:00.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:59 smithi190 bash[23392]: audit 2024-03-26T03:52:58.822967+0000 mon.a (mon.0) 2281 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi175-64406-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:53:00.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:59 smithi190 bash[23392]: cluster 2024-03-26T03:52:58.830635+0000 mon.a (mon.0) 2282 : cluster [DBG] osdmap e322: 8 total, 8 up, 8 in 2024-03-26T03:53:00.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:59 smithi190 bash[23392]: audit 2024-03-26T03:52:58.843546+0000 mon.c (mon.2) 478 : audit [INF] from='client.? 172.21.15.175:0/3322971530' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi175-64406-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi175-64406-61"}]: dispatch 2024-03-26T03:53:00.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:52:59 smithi190 bash[23392]: audit 2024-03-26T03:52:58.844656+0000 mon.a (mon.0) 2283 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi175-64406-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi175-64406-61"}]: dispatch 2024-03-26T03:53:00.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:59 smithi175 bash[21527]: cluster 2024-03-26T03:52:58.711227+0000 mgr.y (mgr.24431) 345 : cluster [DBG] pgmap v479: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-03-26T03:53:00.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:59 smithi175 bash[21527]: audit 2024-03-26T03:52:58.822967+0000 mon.a (mon.0) 2281 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi175-64406-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:53:00.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:59 smithi175 bash[21527]: cluster 2024-03-26T03:52:58.830635+0000 mon.a (mon.0) 2282 : cluster [DBG] osdmap e322: 8 total, 8 up, 8 in 2024-03-26T03:53:00.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:59 smithi175 bash[21527]: audit 2024-03-26T03:52:58.843546+0000 mon.c (mon.2) 478 : audit [INF] from='client.? 172.21.15.175:0/3322971530' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi175-64406-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi175-64406-61"}]: dispatch 2024-03-26T03:53:00.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:52:59 smithi175 bash[21527]: audit 2024-03-26T03:52:58.844656+0000 mon.a (mon.0) 2283 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi175-64406-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi175-64406-61"}]: dispatch 2024-03-26T03:53:00.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:59 smithi175 bash[29284]: cluster 2024-03-26T03:52:58.711227+0000 mgr.y (mgr.24431) 345 : cluster [DBG] pgmap v479: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-03-26T03:53:00.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:59 smithi175 bash[29284]: audit 2024-03-26T03:52:58.822967+0000 mon.a (mon.0) 2281 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi175-64406-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:53:00.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:59 smithi175 bash[29284]: cluster 2024-03-26T03:52:58.830635+0000 mon.a (mon.0) 2282 : cluster [DBG] osdmap e322: 8 total, 8 up, 8 in 2024-03-26T03:53:00.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:59 smithi175 bash[29284]: audit 2024-03-26T03:52:58.843546+0000 mon.c (mon.2) 478 : audit [INF] from='client.? 172.21.15.175:0/3322971530' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi175-64406-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi175-64406-61"}]: dispatch 2024-03-26T03:53:00.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:52:59 smithi175 bash[29284]: audit 2024-03-26T03:52:58.844656+0000 mon.a (mon.0) 2283 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi175-64406-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi175-64406-61"}]: dispatch 2024-03-26T03:53:01.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:00 smithi190 bash[23392]: cluster 2024-03-26T03:52:59.851094+0000 mon.a (mon.0) 2284 : cluster [DBG] osdmap e323: 8 total, 8 up, 8 in 2024-03-26T03:53:01.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:00 smithi190 bash[23392]: audit 2024-03-26T03:52:59.871468+0000 mon.a (mon.0) 2285 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:53:01.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:00 smithi190 bash[23392]: audit 2024-03-26T03:52:59.919228+0000 mgr.y (mgr.24431) 346 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:53:01.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:00 smithi190 bash[23392]: audit 2024-03-26T03:53:00.831572+0000 mon.a (mon.0) 2286 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi175-64406-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi175-64406-61"}]': finished 2024-03-26T03:53:01.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:00 smithi190 bash[23392]: audit 2024-03-26T03:53:00.831830+0000 mon.a (mon.0) 2287 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:53:01.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:00 smithi190 bash[23392]: cluster 2024-03-26T03:53:00.840137+0000 mon.a (mon.0) 2288 : cluster [DBG] osdmap e324: 8 total, 8 up, 8 in 2024-03-26T03:53:01.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:00 smithi175 bash[29284]: cluster 2024-03-26T03:52:59.851094+0000 mon.a (mon.0) 2284 : cluster [DBG] osdmap e323: 8 total, 8 up, 8 in 2024-03-26T03:53:01.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:00 smithi175 bash[29284]: audit 2024-03-26T03:52:59.871468+0000 mon.a (mon.0) 2285 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:53:01.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:00 smithi175 bash[29284]: audit 2024-03-26T03:52:59.919228+0000 mgr.y (mgr.24431) 346 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:53:01.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:00 smithi175 bash[29284]: audit 2024-03-26T03:53:00.831572+0000 mon.a (mon.0) 2286 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi175-64406-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi175-64406-61"}]': finished 2024-03-26T03:53:01.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:00 smithi175 bash[29284]: audit 2024-03-26T03:53:00.831830+0000 mon.a (mon.0) 2287 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:53:01.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:00 smithi175 bash[29284]: cluster 2024-03-26T03:53:00.840137+0000 mon.a (mon.0) 2288 : cluster [DBG] osdmap e324: 8 total, 8 up, 8 in 2024-03-26T03:53:01.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:00 smithi175 bash[21527]: cluster 2024-03-26T03:52:59.851094+0000 mon.a (mon.0) 2284 : cluster [DBG] osdmap e323: 8 total, 8 up, 8 in 2024-03-26T03:53:01.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:00 smithi175 bash[21527]: audit 2024-03-26T03:52:59.871468+0000 mon.a (mon.0) 2285 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:53:01.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:00 smithi175 bash[21527]: audit 2024-03-26T03:52:59.919228+0000 mgr.y (mgr.24431) 346 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:53:01.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:00 smithi175 bash[21527]: audit 2024-03-26T03:53:00.831572+0000 mon.a (mon.0) 2286 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi175-64406-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi175-64406-61"}]': finished 2024-03-26T03:53:01.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:00 smithi175 bash[21527]: audit 2024-03-26T03:53:00.831830+0000 mon.a (mon.0) 2287 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:53:01.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:00 smithi175 bash[21527]: cluster 2024-03-26T03:53:00.840137+0000 mon.a (mon.0) 2288 : cluster [DBG] osdmap e324: 8 total, 8 up, 8 in 2024-03-26T03:53:02.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:01 smithi190 bash[23392]: cluster 2024-03-26T03:53:00.712515+0000 mgr.y (mgr.24431) 347 : cluster [DBG] pgmap v482: 292 pgs: 3 creating+peering, 29 unknown, 260 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:53:02.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:01 smithi190 bash[23392]: audit 2024-03-26T03:53:00.922588+0000 mon.a (mon.0) 2289 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:53:02.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:01 smithi190 bash[23392]: audit 2024-03-26T03:53:01.834325+0000 mon.a (mon.0) 2290 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-47", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:53:02.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:01 smithi190 bash[23392]: cluster 2024-03-26T03:53:01.841169+0000 mon.a (mon.0) 2291 : cluster [DBG] osdmap e325: 8 total, 8 up, 8 in 2024-03-26T03:53:02.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:01 smithi190 bash[23392]: audit 2024-03-26T03:53:01.845820+0000 mon.a (mon.0) 2292 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-47"}]: dispatch 2024-03-26T03:53:02.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:01 smithi175 bash[29284]: cluster 2024-03-26T03:53:00.712515+0000 mgr.y (mgr.24431) 347 : cluster [DBG] pgmap v482: 292 pgs: 3 creating+peering, 29 unknown, 260 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:53:02.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:01 smithi175 bash[29284]: audit 2024-03-26T03:53:00.922588+0000 mon.a (mon.0) 2289 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:53:02.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:01 smithi175 bash[29284]: audit 2024-03-26T03:53:01.834325+0000 mon.a (mon.0) 2290 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-47", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:53:02.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:01 smithi175 bash[29284]: cluster 2024-03-26T03:53:01.841169+0000 mon.a (mon.0) 2291 : cluster [DBG] osdmap e325: 8 total, 8 up, 8 in 2024-03-26T03:53:02.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:01 smithi175 bash[29284]: audit 2024-03-26T03:53:01.845820+0000 mon.a (mon.0) 2292 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-47"}]: dispatch 2024-03-26T03:53:02.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:01 smithi175 bash[21527]: cluster 2024-03-26T03:53:00.712515+0000 mgr.y (mgr.24431) 347 : cluster [DBG] pgmap v482: 292 pgs: 3 creating+peering, 29 unknown, 260 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:53:02.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:01 smithi175 bash[21527]: audit 2024-03-26T03:53:00.922588+0000 mon.a (mon.0) 2289 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:53:02.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:01 smithi175 bash[21527]: audit 2024-03-26T03:53:01.834325+0000 mon.a (mon.0) 2290 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-47", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:53:02.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:01 smithi175 bash[21527]: cluster 2024-03-26T03:53:01.841169+0000 mon.a (mon.0) 2291 : cluster [DBG] osdmap e325: 8 total, 8 up, 8 in 2024-03-26T03:53:02.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:01 smithi175 bash[21527]: audit 2024-03-26T03:53:01.845820+0000 mon.a (mon.0) 2292 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-47"}]: dispatch 2024-03-26T03:53:04.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:03 smithi190 bash[23392]: cluster 2024-03-26T03:53:02.713452+0000 mgr.y (mgr.24431) 348 : cluster [DBG] pgmap v485: 300 pgs: 3 creating+peering, 37 unknown, 260 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:53:04.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:03 smithi190 bash[23392]: audit 2024-03-26T03:53:02.837101+0000 mon.a (mon.0) 2293 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-47"}]': finished 2024-03-26T03:53:04.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:03 smithi190 bash[23392]: cluster 2024-03-26T03:53:02.844106+0000 mon.a (mon.0) 2294 : cluster [DBG] osdmap e326: 8 total, 8 up, 8 in 2024-03-26T03:53:04.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:03 smithi190 bash[23392]: audit 2024-03-26T03:53:02.849147+0000 mon.a (mon.0) 2295 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-47", "mode": "writeback"}]: dispatch 2024-03-26T03:53:04.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:03 smithi190 bash[23392]: audit 2024-03-26T03:53:02.852094+0000 mon.c (mon.2) 479 : audit [INF] from='client.? 172.21.15.175:0/3322971530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi175-64406-61"}]: dispatch 2024-03-26T03:53:04.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:03 smithi190 bash[23392]: audit 2024-03-26T03:53:02.852967+0000 mon.a (mon.0) 2296 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi175-64406-61"}]: dispatch 2024-03-26T03:53:04.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:03 smithi175 bash[21527]: cluster 2024-03-26T03:53:02.713452+0000 mgr.y (mgr.24431) 348 : cluster [DBG] pgmap v485: 300 pgs: 3 creating+peering, 37 unknown, 260 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:53:04.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:03 smithi175 bash[21527]: audit 2024-03-26T03:53:02.837101+0000 mon.a (mon.0) 2293 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-47"}]': finished 2024-03-26T03:53:04.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:03 smithi175 bash[21527]: cluster 2024-03-26T03:53:02.844106+0000 mon.a (mon.0) 2294 : cluster [DBG] osdmap e326: 8 total, 8 up, 8 in 2024-03-26T03:53:04.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:03 smithi175 bash[21527]: audit 2024-03-26T03:53:02.849147+0000 mon.a (mon.0) 2295 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-47", "mode": "writeback"}]: dispatch 2024-03-26T03:53:04.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:03 smithi175 bash[21527]: audit 2024-03-26T03:53:02.852094+0000 mon.c (mon.2) 479 : audit [INF] from='client.? 172.21.15.175:0/3322971530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi175-64406-61"}]: dispatch 2024-03-26T03:53:04.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:03 smithi175 bash[21527]: audit 2024-03-26T03:53:02.852967+0000 mon.a (mon.0) 2296 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi175-64406-61"}]: dispatch 2024-03-26T03:53:04.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:03 smithi175 bash[29284]: cluster 2024-03-26T03:53:02.713452+0000 mgr.y (mgr.24431) 348 : cluster [DBG] pgmap v485: 300 pgs: 3 creating+peering, 37 unknown, 260 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:53:04.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:03 smithi175 bash[29284]: audit 2024-03-26T03:53:02.837101+0000 mon.a (mon.0) 2293 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-47"}]': finished 2024-03-26T03:53:04.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:03 smithi175 bash[29284]: cluster 2024-03-26T03:53:02.844106+0000 mon.a (mon.0) 2294 : cluster [DBG] osdmap e326: 8 total, 8 up, 8 in 2024-03-26T03:53:04.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:03 smithi175 bash[29284]: audit 2024-03-26T03:53:02.849147+0000 mon.a (mon.0) 2295 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-47", "mode": "writeback"}]: dispatch 2024-03-26T03:53:04.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:03 smithi175 bash[29284]: audit 2024-03-26T03:53:02.852094+0000 mon.c (mon.2) 479 : audit [INF] from='client.? 172.21.15.175:0/3322971530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi175-64406-61"}]: dispatch 2024-03-26T03:53:04.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:03 smithi175 bash[29284]: audit 2024-03-26T03:53:02.852967+0000 mon.a (mon.0) 2296 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi175-64406-61"}]: dispatch 2024-03-26T03:53:05.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:04 smithi190 bash[23392]: cluster 2024-03-26T03:53:03.837429+0000 mon.a (mon.0) 2297 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:53:05.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:04 smithi190 bash[23392]: audit 2024-03-26T03:53:03.839951+0000 mon.a (mon.0) 2298 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-47", "mode": "writeback"}]': finished 2024-03-26T03:53:05.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:04 smithi190 bash[23392]: audit 2024-03-26T03:53:03.840184+0000 mon.a (mon.0) 2299 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi175-64406-61"}]': finished 2024-03-26T03:53:05.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:04 smithi190 bash[23392]: cluster 2024-03-26T03:53:03.847147+0000 mon.a (mon.0) 2300 : cluster [DBG] osdmap e327: 8 total, 8 up, 8 in 2024-03-26T03:53:05.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:04 smithi190 bash[23392]: audit 2024-03-26T03:53:03.850844+0000 mon.c (mon.2) 480 : audit [INF] from='client.? 172.21.15.175:0/3322971530' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi175-64406-61"}]: dispatch 2024-03-26T03:53:05.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:04 smithi190 bash[23392]: audit 2024-03-26T03:53:03.852065+0000 mon.a (mon.0) 2301 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T03:53:05.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:04 smithi190 bash[23392]: audit 2024-03-26T03:53:03.853180+0000 mon.a (mon.0) 2302 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi175-64406-61"}]: dispatch 2024-03-26T03:53:05.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:04 smithi190 bash[23392]: audit 2024-03-26T03:53:04.681645+0000 mon.a (mon.0) 2303 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:53:05.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:04 smithi190 bash[23392]: audit 2024-03-26T03:53:04.684527+0000 mon.c (mon.2) 481 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:53:05.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:04 smithi190 bash[23392]: audit 2024-03-26T03:53:04.845044+0000 mon.a (mon.0) 2304 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "hit_set_count","val": "2"}]': finished 2024-03-26T03:53:05.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:04 smithi175 bash[29284]: cluster 2024-03-26T03:53:03.837429+0000 mon.a (mon.0) 2297 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:53:05.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:04 smithi175 bash[29284]: audit 2024-03-26T03:53:03.839951+0000 mon.a (mon.0) 2298 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-47", "mode": "writeback"}]': finished 2024-03-26T03:53:05.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:04 smithi175 bash[29284]: audit 2024-03-26T03:53:03.840184+0000 mon.a (mon.0) 2299 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi175-64406-61"}]': finished 2024-03-26T03:53:05.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:04 smithi175 bash[29284]: cluster 2024-03-26T03:53:03.847147+0000 mon.a (mon.0) 2300 : cluster [DBG] osdmap e327: 8 total, 8 up, 8 in 2024-03-26T03:53:05.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:04 smithi175 bash[29284]: audit 2024-03-26T03:53:03.850844+0000 mon.c (mon.2) 480 : audit [INF] from='client.? 172.21.15.175:0/3322971530' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi175-64406-61"}]: dispatch 2024-03-26T03:53:05.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:04 smithi175 bash[29284]: audit 2024-03-26T03:53:03.852065+0000 mon.a (mon.0) 2301 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T03:53:05.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:04 smithi175 bash[29284]: audit 2024-03-26T03:53:03.853180+0000 mon.a (mon.0) 2302 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi175-64406-61"}]: dispatch 2024-03-26T03:53:05.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:04 smithi175 bash[29284]: audit 2024-03-26T03:53:04.681645+0000 mon.a (mon.0) 2303 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:53:05.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:04 smithi175 bash[29284]: audit 2024-03-26T03:53:04.684527+0000 mon.c (mon.2) 481 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:53:05.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:04 smithi175 bash[29284]: audit 2024-03-26T03:53:04.845044+0000 mon.a (mon.0) 2304 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "hit_set_count","val": "2"}]': finished 2024-03-26T03:53:05.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:04 smithi175 bash[21527]: cluster 2024-03-26T03:53:03.837429+0000 mon.a (mon.0) 2297 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:53:05.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:04 smithi175 bash[21527]: audit 2024-03-26T03:53:03.839951+0000 mon.a (mon.0) 2298 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-47", "mode": "writeback"}]': finished 2024-03-26T03:53:05.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:04 smithi175 bash[21527]: audit 2024-03-26T03:53:03.840184+0000 mon.a (mon.0) 2299 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi175-64406-61"}]': finished 2024-03-26T03:53:05.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:04 smithi175 bash[21527]: cluster 2024-03-26T03:53:03.847147+0000 mon.a (mon.0) 2300 : cluster [DBG] osdmap e327: 8 total, 8 up, 8 in 2024-03-26T03:53:05.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:04 smithi175 bash[21527]: audit 2024-03-26T03:53:03.850844+0000 mon.c (mon.2) 480 : audit [INF] from='client.? 172.21.15.175:0/3322971530' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi175-64406-61"}]: dispatch 2024-03-26T03:53:05.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:04 smithi175 bash[21527]: audit 2024-03-26T03:53:03.852065+0000 mon.a (mon.0) 2301 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T03:53:05.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:04 smithi175 bash[21527]: audit 2024-03-26T03:53:03.853180+0000 mon.a (mon.0) 2302 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi175-64406-61"}]: dispatch 2024-03-26T03:53:05.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:04 smithi175 bash[21527]: audit 2024-03-26T03:53:04.681645+0000 mon.a (mon.0) 2303 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:53:05.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:04 smithi175 bash[21527]: audit 2024-03-26T03:53:04.684527+0000 mon.c (mon.2) 481 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:53:05.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:04 smithi175 bash[21527]: audit 2024-03-26T03:53:04.845044+0000 mon.a (mon.0) 2304 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "hit_set_count","val": "2"}]': finished 2024-03-26T03:53:06.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:05 smithi190 bash[23392]: cluster 2024-03-26T03:53:04.715476+0000 mgr.y (mgr.24431) 349 : cluster [DBG] pgmap v488: 292 pgs: 3 creating+peering, 12 unknown, 277 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s wr, 3 op/s 2024-03-26T03:53:06.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:05 smithi190 bash[23392]: audit 2024-03-26T03:53:04.845194+0000 mon.a (mon.0) 2305 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi175-64406-61"}]': finished 2024-03-26T03:53:06.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:05 smithi190 bash[23392]: cluster 2024-03-26T03:53:04.854849+0000 mon.a (mon.0) 2306 : cluster [DBG] osdmap e328: 8 total, 8 up, 8 in 2024-03-26T03:53:06.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:05 smithi190 bash[23392]: audit 2024-03-26T03:53:04.862234+0000 mon.a (mon.0) 2307 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T03:53:06.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:05 smithi190 bash[23392]: audit 2024-03-26T03:53:04.881947+0000 mon.c (mon.2) 482 : audit [INF] from='client.? 172.21.15.175:0/478890789' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi175-64406-62"}]: dispatch 2024-03-26T03:53:06.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:05 smithi190 bash[23392]: audit 2024-03-26T03:53:04.882250+0000 mon.a (mon.0) 2308 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi175-64406-62"}]: dispatch 2024-03-26T03:53:06.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:05 smithi190 bash[23392]: audit 2024-03-26T03:53:04.883147+0000 mon.c (mon.2) 483 : audit [INF] from='client.? 172.21.15.175:0/478890789' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi175-64406-62"}]: dispatch 2024-03-26T03:53:06.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:05 smithi190 bash[23392]: audit 2024-03-26T03:53:04.883452+0000 mon.a (mon.0) 2309 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi175-64406-62"}]: dispatch 2024-03-26T03:53:06.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:05 smithi190 bash[23392]: audit 2024-03-26T03:53:04.883943+0000 mon.c (mon.2) 484 : audit [INF] from='client.? 172.21.15.175:0/478890789' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi175-64406-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:53:06.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:05 smithi190 bash[23392]: audit 2024-03-26T03:53:04.884203+0000 mon.a (mon.0) 2310 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi175-64406-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:53:06.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:05 smithi190 bash[23392]: cluster 2024-03-26T03:53:05.676745+0000 mon.a (mon.0) 2311 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:53:06.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:05 smithi190 bash[23392]: audit 2024-03-26T03:53:05.847813+0000 mon.a (mon.0) 2312 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "hit_set_period","val": "600"}]': finished 2024-03-26T03:53:06.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:05 smithi190 bash[23392]: audit 2024-03-26T03:53:05.848006+0000 mon.a (mon.0) 2313 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi175-64406-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:53:06.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:05 smithi190 bash[23392]: cluster 2024-03-26T03:53:05.855250+0000 mon.a (mon.0) 2314 : cluster [DBG] osdmap e329: 8 total, 8 up, 8 in 2024-03-26T03:53:06.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:05 smithi190 bash[23392]: audit 2024-03-26T03:53:05.857132+0000 mon.c (mon.2) 485 : audit [INF] from='client.? 172.21.15.175:0/478890789' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi175-64406-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi175-64406-62"}]: dispatch 2024-03-26T03:53:06.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:05 smithi190 bash[23392]: audit 2024-03-26T03:53:05.858320+0000 mon.a (mon.0) 2315 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-26T03:53:06.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:05 smithi190 bash[23392]: audit 2024-03-26T03:53:05.859796+0000 mon.a (mon.0) 2316 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi175-64406-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi175-64406-62"}]: dispatch 2024-03-26T03:53:06.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[21527]: cluster 2024-03-26T03:53:04.715476+0000 mgr.y (mgr.24431) 349 : cluster [DBG] pgmap v488: 292 pgs: 3 creating+peering, 12 unknown, 277 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s wr, 3 op/s 2024-03-26T03:53:06.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[21527]: audit 2024-03-26T03:53:04.845194+0000 mon.a (mon.0) 2305 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi175-64406-61"}]': finished 2024-03-26T03:53:06.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[21527]: cluster 2024-03-26T03:53:04.854849+0000 mon.a (mon.0) 2306 : cluster [DBG] osdmap e328: 8 total, 8 up, 8 in 2024-03-26T03:53:06.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[21527]: audit 2024-03-26T03:53:04.862234+0000 mon.a (mon.0) 2307 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T03:53:06.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[21527]: audit 2024-03-26T03:53:04.881947+0000 mon.c (mon.2) 482 : audit [INF] from='client.? 172.21.15.175:0/478890789' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi175-64406-62"}]: dispatch 2024-03-26T03:53:06.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[21527]: audit 2024-03-26T03:53:04.882250+0000 mon.a (mon.0) 2308 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi175-64406-62"}]: dispatch 2024-03-26T03:53:06.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[21527]: audit 2024-03-26T03:53:04.883147+0000 mon.c (mon.2) 483 : audit [INF] from='client.? 172.21.15.175:0/478890789' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi175-64406-62"}]: dispatch 2024-03-26T03:53:06.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[21527]: audit 2024-03-26T03:53:04.883452+0000 mon.a (mon.0) 2309 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi175-64406-62"}]: dispatch 2024-03-26T03:53:06.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[21527]: audit 2024-03-26T03:53:04.883943+0000 mon.c (mon.2) 484 : audit [INF] from='client.? 172.21.15.175:0/478890789' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi175-64406-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:53:06.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[21527]: audit 2024-03-26T03:53:04.884203+0000 mon.a (mon.0) 2310 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi175-64406-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:53:06.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[21527]: cluster 2024-03-26T03:53:05.676745+0000 mon.a (mon.0) 2311 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:53:06.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[21527]: audit 2024-03-26T03:53:05.847813+0000 mon.a (mon.0) 2312 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "hit_set_period","val": "600"}]': finished 2024-03-26T03:53:06.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[21527]: audit 2024-03-26T03:53:05.848006+0000 mon.a (mon.0) 2313 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi175-64406-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:53:06.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[21527]: cluster 2024-03-26T03:53:05.855250+0000 mon.a (mon.0) 2314 : cluster [DBG] osdmap e329: 8 total, 8 up, 8 in 2024-03-26T03:53:06.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[21527]: audit 2024-03-26T03:53:05.857132+0000 mon.c (mon.2) 485 : audit [INF] from='client.? 172.21.15.175:0/478890789' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi175-64406-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi175-64406-62"}]: dispatch 2024-03-26T03:53:06.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[21527]: audit 2024-03-26T03:53:05.858320+0000 mon.a (mon.0) 2315 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-26T03:53:06.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[21527]: audit 2024-03-26T03:53:05.859796+0000 mon.a (mon.0) 2316 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi175-64406-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi175-64406-62"}]: dispatch 2024-03-26T03:53:06.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[29284]: cluster 2024-03-26T03:53:04.715476+0000 mgr.y (mgr.24431) 349 : cluster [DBG] pgmap v488: 292 pgs: 3 creating+peering, 12 unknown, 277 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s wr, 3 op/s 2024-03-26T03:53:06.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[29284]: audit 2024-03-26T03:53:04.845194+0000 mon.a (mon.0) 2305 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi175-64406-61"}]': finished 2024-03-26T03:53:06.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[29284]: cluster 2024-03-26T03:53:04.854849+0000 mon.a (mon.0) 2306 : cluster [DBG] osdmap e328: 8 total, 8 up, 8 in 2024-03-26T03:53:06.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[29284]: audit 2024-03-26T03:53:04.862234+0000 mon.a (mon.0) 2307 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T03:53:06.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[29284]: audit 2024-03-26T03:53:04.881947+0000 mon.c (mon.2) 482 : audit [INF] from='client.? 172.21.15.175:0/478890789' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi175-64406-62"}]: dispatch 2024-03-26T03:53:06.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[29284]: audit 2024-03-26T03:53:04.882250+0000 mon.a (mon.0) 2308 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi175-64406-62"}]: dispatch 2024-03-26T03:53:06.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[29284]: audit 2024-03-26T03:53:04.883147+0000 mon.c (mon.2) 483 : audit [INF] from='client.? 172.21.15.175:0/478890789' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi175-64406-62"}]: dispatch 2024-03-26T03:53:06.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[29284]: audit 2024-03-26T03:53:04.883452+0000 mon.a (mon.0) 2309 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi175-64406-62"}]: dispatch 2024-03-26T03:53:06.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[29284]: audit 2024-03-26T03:53:04.883943+0000 mon.c (mon.2) 484 : audit [INF] from='client.? 172.21.15.175:0/478890789' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi175-64406-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:53:06.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[29284]: audit 2024-03-26T03:53:04.884203+0000 mon.a (mon.0) 2310 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi175-64406-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:53:06.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[29284]: cluster 2024-03-26T03:53:05.676745+0000 mon.a (mon.0) 2311 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:53:06.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[29284]: audit 2024-03-26T03:53:05.847813+0000 mon.a (mon.0) 2312 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "hit_set_period","val": "600"}]': finished 2024-03-26T03:53:06.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[29284]: audit 2024-03-26T03:53:05.848006+0000 mon.a (mon.0) 2313 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi175-64406-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:53:06.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[29284]: cluster 2024-03-26T03:53:05.855250+0000 mon.a (mon.0) 2314 : cluster [DBG] osdmap e329: 8 total, 8 up, 8 in 2024-03-26T03:53:06.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[29284]: audit 2024-03-26T03:53:05.857132+0000 mon.c (mon.2) 485 : audit [INF] from='client.? 172.21.15.175:0/478890789' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi175-64406-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi175-64406-62"}]: dispatch 2024-03-26T03:53:06.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[29284]: audit 2024-03-26T03:53:05.858320+0000 mon.a (mon.0) 2315 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-26T03:53:06.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:05 smithi175 bash[29284]: audit 2024-03-26T03:53:05.859796+0000 mon.a (mon.0) 2316 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi175-64406-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi175-64406-62"}]: dispatch 2024-03-26T03:53:07.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:06 smithi190 bash[23392]: cluster 2024-03-26T03:53:06.848948+0000 mon.a (mon.0) 2317 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:53:07.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:06 smithi190 bash[23392]: audit 2024-03-26T03:53:06.851525+0000 mon.a (mon.0) 2318 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "hit_set_type","val": "bloom"}]': finished 2024-03-26T03:53:07.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:06 smithi175 bash[21527]: cluster 2024-03-26T03:53:06.848948+0000 mon.a (mon.0) 2317 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:53:07.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:06 smithi175 bash[21527]: audit 2024-03-26T03:53:06.851525+0000 mon.a (mon.0) 2318 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "hit_set_type","val": "bloom"}]': finished 2024-03-26T03:53:07.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:06 smithi175 bash[29284]: cluster 2024-03-26T03:53:06.848948+0000 mon.a (mon.0) 2317 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:53:07.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:06 smithi175 bash[29284]: audit 2024-03-26T03:53:06.851525+0000 mon.a (mon.0) 2318 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "hit_set_type","val": "bloom"}]': finished 2024-03-26T03:53:08.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:07 smithi190 bash[23392]: cluster 2024-03-26T03:53:06.716315+0000 mgr.y (mgr.24431) 350 : cluster [DBG] pgmap v491: 292 pgs: 292 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-26T03:53:08.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:07 smithi190 bash[23392]: cluster 2024-03-26T03:53:06.872290+0000 mon.a (mon.0) 2319 : cluster [DBG] osdmap e330: 8 total, 8 up, 8 in 2024-03-26T03:53:08.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:07 smithi190 bash[23392]: audit 2024-03-26T03:53:06.873485+0000 mon.a (mon.0) 2320 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-26T03:53:08.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:07 smithi190 bash[23392]: audit 2024-03-26T03:53:07.854928+0000 mon.a (mon.0) 2321 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi175-64406-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi175-64406-62"}]': finished 2024-03-26T03:53:08.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:07 smithi190 bash[23392]: audit 2024-03-26T03:53:07.855214+0000 mon.a (mon.0) 2322 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-26T03:53:08.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:07 smithi190 bash[23392]: cluster 2024-03-26T03:53:07.862850+0000 mon.a (mon.0) 2323 : cluster [DBG] osdmap e331: 8 total, 8 up, 8 in 2024-03-26T03:53:08.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:07 smithi190 bash[23392]: audit 2024-03-26T03:53:07.869146+0000 mon.a (mon.0) 2324 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-26T03:53:08.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:07 smithi175 bash[21527]: cluster 2024-03-26T03:53:06.716315+0000 mgr.y (mgr.24431) 350 : cluster [DBG] pgmap v491: 292 pgs: 292 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-26T03:53:08.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:07 smithi175 bash[21527]: cluster 2024-03-26T03:53:06.872290+0000 mon.a (mon.0) 2319 : cluster [DBG] osdmap e330: 8 total, 8 up, 8 in 2024-03-26T03:53:08.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:07 smithi175 bash[21527]: audit 2024-03-26T03:53:06.873485+0000 mon.a (mon.0) 2320 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-26T03:53:08.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:07 smithi175 bash[21527]: audit 2024-03-26T03:53:07.854928+0000 mon.a (mon.0) 2321 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi175-64406-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi175-64406-62"}]': finished 2024-03-26T03:53:08.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:07 smithi175 bash[21527]: audit 2024-03-26T03:53:07.855214+0000 mon.a (mon.0) 2322 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-26T03:53:08.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:07 smithi175 bash[21527]: cluster 2024-03-26T03:53:07.862850+0000 mon.a (mon.0) 2323 : cluster [DBG] osdmap e331: 8 total, 8 up, 8 in 2024-03-26T03:53:08.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:07 smithi175 bash[21527]: audit 2024-03-26T03:53:07.869146+0000 mon.a (mon.0) 2324 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-26T03:53:08.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:07 smithi175 bash[29284]: cluster 2024-03-26T03:53:06.716315+0000 mgr.y (mgr.24431) 350 : cluster [DBG] pgmap v491: 292 pgs: 292 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-26T03:53:08.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:07 smithi175 bash[29284]: cluster 2024-03-26T03:53:06.872290+0000 mon.a (mon.0) 2319 : cluster [DBG] osdmap e330: 8 total, 8 up, 8 in 2024-03-26T03:53:08.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:07 smithi175 bash[29284]: audit 2024-03-26T03:53:06.873485+0000 mon.a (mon.0) 2320 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-26T03:53:08.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:07 smithi175 bash[29284]: audit 2024-03-26T03:53:07.854928+0000 mon.a (mon.0) 2321 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi175-64406-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi175-64406-62"}]': finished 2024-03-26T03:53:08.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:07 smithi175 bash[29284]: audit 2024-03-26T03:53:07.855214+0000 mon.a (mon.0) 2322 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-26T03:53:08.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:07 smithi175 bash[29284]: cluster 2024-03-26T03:53:07.862850+0000 mon.a (mon.0) 2323 : cluster [DBG] osdmap e331: 8 total, 8 up, 8 in 2024-03-26T03:53:08.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:07 smithi175 bash[29284]: audit 2024-03-26T03:53:07.869146+0000 mon.a (mon.0) 2324 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-26T03:53:09.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:53:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:53:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:53:10.239 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:53:09 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:53:10.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:09 smithi190 bash[23392]: cluster 2024-03-26T03:53:08.717264+0000 mgr.y (mgr.24431) 351 : cluster [DBG] pgmap v494: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-26T03:53:10.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:09 smithi190 bash[23392]: audit 2024-03-26T03:53:08.857918+0000 mon.a (mon.0) 2325 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-03-26T03:53:10.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:09 smithi190 bash[23392]: cluster 2024-03-26T03:53:08.865487+0000 mon.a (mon.0) 2326 : cluster [DBG] osdmap e332: 8 total, 8 up, 8 in 2024-03-26T03:53:10.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:09 smithi190 bash[23392]: audit 2024-03-26T03:53:08.869676+0000 mon.a (mon.0) 2327 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-26T03:53:10.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:09 smithi175 bash[21527]: cluster 2024-03-26T03:53:08.717264+0000 mgr.y (mgr.24431) 351 : cluster [DBG] pgmap v494: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-26T03:53:10.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:09 smithi175 bash[21527]: audit 2024-03-26T03:53:08.857918+0000 mon.a (mon.0) 2325 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-03-26T03:53:10.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:09 smithi175 bash[21527]: cluster 2024-03-26T03:53:08.865487+0000 mon.a (mon.0) 2326 : cluster [DBG] osdmap e332: 8 total, 8 up, 8 in 2024-03-26T03:53:10.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:09 smithi175 bash[21527]: audit 2024-03-26T03:53:08.869676+0000 mon.a (mon.0) 2327 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-26T03:53:10.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:09 smithi175 bash[29284]: cluster 2024-03-26T03:53:08.717264+0000 mgr.y (mgr.24431) 351 : cluster [DBG] pgmap v494: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-26T03:53:10.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:09 smithi175 bash[29284]: audit 2024-03-26T03:53:08.857918+0000 mon.a (mon.0) 2325 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-03-26T03:53:10.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:09 smithi175 bash[29284]: cluster 2024-03-26T03:53:08.865487+0000 mon.a (mon.0) 2326 : cluster [DBG] osdmap e332: 8 total, 8 up, 8 in 2024-03-26T03:53:10.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:09 smithi175 bash[29284]: audit 2024-03-26T03:53:08.869676+0000 mon.a (mon.0) 2327 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-26T03:53:11.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:10 smithi190 bash[23392]: audit 2024-03-26T03:53:09.860308+0000 mon.a (mon.0) 2328 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-03-26T03:53:11.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:10 smithi190 bash[23392]: cluster 2024-03-26T03:53:09.871777+0000 mon.a (mon.0) 2329 : cluster [DBG] osdmap e333: 8 total, 8 up, 8 in 2024-03-26T03:53:11.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:10 smithi190 bash[23392]: audit 2024-03-26T03:53:09.876849+0000 mon.c (mon.2) 486 : audit [INF] from='client.? 172.21.15.175:0/478890789' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi175-64406-62"}]: dispatch 2024-03-26T03:53:11.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:10 smithi190 bash[23392]: audit 2024-03-26T03:53:09.890434+0000 mon.a (mon.0) 2330 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi175-64406-62"}]: dispatch 2024-03-26T03:53:11.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:10 smithi190 bash[23392]: audit 2024-03-26T03:53:09.927035+0000 mgr.y (mgr.24431) 352 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:53:11.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:10 smithi190 bash[23392]: audit 2024-03-26T03:53:09.978476+0000 mon.a (mon.0) 2331 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:53:11.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:10 smithi175 bash[21527]: audit 2024-03-26T03:53:09.860308+0000 mon.a (mon.0) 2328 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-03-26T03:53:11.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:10 smithi175 bash[21527]: cluster 2024-03-26T03:53:09.871777+0000 mon.a (mon.0) 2329 : cluster [DBG] osdmap e333: 8 total, 8 up, 8 in 2024-03-26T03:53:11.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:10 smithi175 bash[21527]: audit 2024-03-26T03:53:09.876849+0000 mon.c (mon.2) 486 : audit [INF] from='client.? 172.21.15.175:0/478890789' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi175-64406-62"}]: dispatch 2024-03-26T03:53:11.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:10 smithi175 bash[21527]: audit 2024-03-26T03:53:09.890434+0000 mon.a (mon.0) 2330 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi175-64406-62"}]: dispatch 2024-03-26T03:53:11.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:10 smithi175 bash[21527]: audit 2024-03-26T03:53:09.927035+0000 mgr.y (mgr.24431) 352 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:53:11.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:10 smithi175 bash[21527]: audit 2024-03-26T03:53:09.978476+0000 mon.a (mon.0) 2331 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:53:11.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:10 smithi175 bash[29284]: audit 2024-03-26T03:53:09.860308+0000 mon.a (mon.0) 2328 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-03-26T03:53:11.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:10 smithi175 bash[29284]: cluster 2024-03-26T03:53:09.871777+0000 mon.a (mon.0) 2329 : cluster [DBG] osdmap e333: 8 total, 8 up, 8 in 2024-03-26T03:53:11.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:10 smithi175 bash[29284]: audit 2024-03-26T03:53:09.876849+0000 mon.c (mon.2) 486 : audit [INF] from='client.? 172.21.15.175:0/478890789' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi175-64406-62"}]: dispatch 2024-03-26T03:53:11.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:10 smithi175 bash[29284]: audit 2024-03-26T03:53:09.890434+0000 mon.a (mon.0) 2330 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi175-64406-62"}]: dispatch 2024-03-26T03:53:11.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:10 smithi175 bash[29284]: audit 2024-03-26T03:53:09.927035+0000 mgr.y (mgr.24431) 352 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:53:11.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:10 smithi175 bash[29284]: audit 2024-03-26T03:53:09.978476+0000 mon.a (mon.0) 2331 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:53:12.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:11 smithi190 bash[23392]: cluster 2024-03-26T03:53:10.718466+0000 mgr.y (mgr.24431) 353 : cluster [DBG] pgmap v497: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:53:12.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:11 smithi190 bash[23392]: cluster 2024-03-26T03:53:10.878859+0000 mon.a (mon.0) 2332 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:53:12.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:11 smithi190 bash[23392]: audit 2024-03-26T03:53:10.880562+0000 mon.a (mon.0) 2333 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi175-64406-62"}]': finished 2024-03-26T03:53:12.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:11 smithi190 bash[23392]: audit 2024-03-26T03:53:10.880847+0000 mon.a (mon.0) 2334 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:53:12.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:11 smithi190 bash[23392]: cluster 2024-03-26T03:53:10.888743+0000 mon.a (mon.0) 2335 : cluster [DBG] osdmap e334: 8 total, 8 up, 8 in 2024-03-26T03:53:12.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:11 smithi190 bash[23392]: audit 2024-03-26T03:53:10.890979+0000 mon.a (mon.0) 2336 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-47"}]: dispatch 2024-03-26T03:53:12.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:11 smithi190 bash[23392]: audit 2024-03-26T03:53:10.891893+0000 mon.c (mon.2) 487 : audit [INF] from='client.? 172.21.15.175:0/478890789' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi175-64406-62"}]: dispatch 2024-03-26T03:53:12.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:11 smithi190 bash[23392]: audit 2024-03-26T03:53:10.906162+0000 mon.a (mon.0) 2337 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi175-64406-62"}]: dispatch 2024-03-26T03:53:12.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:11 smithi190 bash[23392]: audit 2024-03-26T03:53:11.884084+0000 mon.a (mon.0) 2338 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-47"}]': finished 2024-03-26T03:53:12.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:11 smithi175 bash[21527]: cluster 2024-03-26T03:53:10.718466+0000 mgr.y (mgr.24431) 353 : cluster [DBG] pgmap v497: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:53:12.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:11 smithi175 bash[21527]: cluster 2024-03-26T03:53:10.878859+0000 mon.a (mon.0) 2332 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:53:12.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:11 smithi175 bash[21527]: audit 2024-03-26T03:53:10.880562+0000 mon.a (mon.0) 2333 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi175-64406-62"}]': finished 2024-03-26T03:53:12.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:11 smithi175 bash[21527]: audit 2024-03-26T03:53:10.880847+0000 mon.a (mon.0) 2334 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:53:12.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:11 smithi175 bash[21527]: cluster 2024-03-26T03:53:10.888743+0000 mon.a (mon.0) 2335 : cluster [DBG] osdmap e334: 8 total, 8 up, 8 in 2024-03-26T03:53:12.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:11 smithi175 bash[21527]: audit 2024-03-26T03:53:10.890979+0000 mon.a (mon.0) 2336 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-47"}]: dispatch 2024-03-26T03:53:12.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:11 smithi175 bash[21527]: audit 2024-03-26T03:53:10.891893+0000 mon.c (mon.2) 487 : audit [INF] from='client.? 172.21.15.175:0/478890789' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi175-64406-62"}]: dispatch 2024-03-26T03:53:12.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:11 smithi175 bash[21527]: audit 2024-03-26T03:53:10.906162+0000 mon.a (mon.0) 2337 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi175-64406-62"}]: dispatch 2024-03-26T03:53:12.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:11 smithi175 bash[21527]: audit 2024-03-26T03:53:11.884084+0000 mon.a (mon.0) 2338 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-47"}]': finished 2024-03-26T03:53:12.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:11 smithi175 bash[29284]: cluster 2024-03-26T03:53:10.718466+0000 mgr.y (mgr.24431) 353 : cluster [DBG] pgmap v497: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:53:12.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:11 smithi175 bash[29284]: cluster 2024-03-26T03:53:10.878859+0000 mon.a (mon.0) 2332 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:53:12.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:11 smithi175 bash[29284]: audit 2024-03-26T03:53:10.880562+0000 mon.a (mon.0) 2333 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi175-64406-62"}]': finished 2024-03-26T03:53:12.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:11 smithi175 bash[29284]: audit 2024-03-26T03:53:10.880847+0000 mon.a (mon.0) 2334 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:53:12.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:11 smithi175 bash[29284]: cluster 2024-03-26T03:53:10.888743+0000 mon.a (mon.0) 2335 : cluster [DBG] osdmap e334: 8 total, 8 up, 8 in 2024-03-26T03:53:12.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:11 smithi175 bash[29284]: audit 2024-03-26T03:53:10.890979+0000 mon.a (mon.0) 2336 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-47"}]: dispatch 2024-03-26T03:53:12.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:11 smithi175 bash[29284]: audit 2024-03-26T03:53:10.891893+0000 mon.c (mon.2) 487 : audit [INF] from='client.? 172.21.15.175:0/478890789' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi175-64406-62"}]: dispatch 2024-03-26T03:53:12.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:11 smithi175 bash[29284]: audit 2024-03-26T03:53:10.906162+0000 mon.a (mon.0) 2337 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi175-64406-62"}]: dispatch 2024-03-26T03:53:12.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:11 smithi175 bash[29284]: audit 2024-03-26T03:53:11.884084+0000 mon.a (mon.0) 2338 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-47"}]': finished 2024-03-26T03:53:13.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:12 smithi190 bash[23392]: audit 2024-03-26T03:53:11.884339+0000 mon.a (mon.0) 2339 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi175-64406-62"}]': finished 2024-03-26T03:53:13.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:12 smithi190 bash[23392]: cluster 2024-03-26T03:53:11.892392+0000 mon.a (mon.0) 2340 : cluster [DBG] osdmap e335: 8 total, 8 up, 8 in 2024-03-26T03:53:13.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:12 smithi190 bash[23392]: audit 2024-03-26T03:53:11.930244+0000 mon.c (mon.2) 488 : audit [INF] from='client.? 172.21.15.175:0/430157263' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi175-64406-63"}]: dispatch 2024-03-26T03:53:13.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:12 smithi190 bash[23392]: audit 2024-03-26T03:53:11.930678+0000 mon.a (mon.0) 2341 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi175-64406-63"}]: dispatch 2024-03-26T03:53:13.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:12 smithi190 bash[23392]: audit 2024-03-26T03:53:11.931249+0000 mon.c (mon.2) 489 : audit [INF] from='client.? 172.21.15.175:0/430157263' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi175-64406-63"}]: dispatch 2024-03-26T03:53:13.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:12 smithi190 bash[23392]: audit 2024-03-26T03:53:11.931555+0000 mon.a (mon.0) 2342 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi175-64406-63"}]: dispatch 2024-03-26T03:53:13.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:12 smithi190 bash[23392]: audit 2024-03-26T03:53:11.932034+0000 mon.c (mon.2) 490 : audit [INF] from='client.? 172.21.15.175:0/430157263' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi175-64406-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:53:13.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:12 smithi190 bash[23392]: audit 2024-03-26T03:53:11.932319+0000 mon.a (mon.0) 2343 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi175-64406-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:53:13.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:12 smithi190 bash[23392]: audit 2024-03-26T03:53:11.956758+0000 mon.a (mon.0) 2344 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:53:13.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:12 smithi190 bash[23392]: audit 2024-03-26T03:53:11.957203+0000 mon.a (mon.0) 2345 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-47"}]: dispatch 2024-03-26T03:53:13.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:12 smithi175 bash[21527]: audit 2024-03-26T03:53:11.884339+0000 mon.a (mon.0) 2339 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi175-64406-62"}]': finished 2024-03-26T03:53:13.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:12 smithi175 bash[21527]: cluster 2024-03-26T03:53:11.892392+0000 mon.a (mon.0) 2340 : cluster [DBG] osdmap e335: 8 total, 8 up, 8 in 2024-03-26T03:53:13.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:12 smithi175 bash[21527]: audit 2024-03-26T03:53:11.930244+0000 mon.c (mon.2) 488 : audit [INF] from='client.? 172.21.15.175:0/430157263' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi175-64406-63"}]: dispatch 2024-03-26T03:53:13.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:12 smithi175 bash[21527]: audit 2024-03-26T03:53:11.930678+0000 mon.a (mon.0) 2341 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi175-64406-63"}]: dispatch 2024-03-26T03:53:13.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:12 smithi175 bash[21527]: audit 2024-03-26T03:53:11.931249+0000 mon.c (mon.2) 489 : audit [INF] from='client.? 172.21.15.175:0/430157263' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi175-64406-63"}]: dispatch 2024-03-26T03:53:13.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:12 smithi175 bash[21527]: audit 2024-03-26T03:53:11.931555+0000 mon.a (mon.0) 2342 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi175-64406-63"}]: dispatch 2024-03-26T03:53:13.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:12 smithi175 bash[21527]: audit 2024-03-26T03:53:11.932034+0000 mon.c (mon.2) 490 : audit [INF] from='client.? 172.21.15.175:0/430157263' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi175-64406-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:53:13.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:12 smithi175 bash[21527]: audit 2024-03-26T03:53:11.932319+0000 mon.a (mon.0) 2343 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi175-64406-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:53:13.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:12 smithi175 bash[21527]: audit 2024-03-26T03:53:11.956758+0000 mon.a (mon.0) 2344 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:53:13.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:12 smithi175 bash[21527]: audit 2024-03-26T03:53:11.957203+0000 mon.a (mon.0) 2345 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-47"}]: dispatch 2024-03-26T03:53:13.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:12 smithi175 bash[29284]: audit 2024-03-26T03:53:11.884339+0000 mon.a (mon.0) 2339 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi175-64406-62"}]': finished 2024-03-26T03:53:13.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:12 smithi175 bash[29284]: cluster 2024-03-26T03:53:11.892392+0000 mon.a (mon.0) 2340 : cluster [DBG] osdmap e335: 8 total, 8 up, 8 in 2024-03-26T03:53:13.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:12 smithi175 bash[29284]: audit 2024-03-26T03:53:11.930244+0000 mon.c (mon.2) 488 : audit [INF] from='client.? 172.21.15.175:0/430157263' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi175-64406-63"}]: dispatch 2024-03-26T03:53:13.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:12 smithi175 bash[29284]: audit 2024-03-26T03:53:11.930678+0000 mon.a (mon.0) 2341 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi175-64406-63"}]: dispatch 2024-03-26T03:53:13.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:12 smithi175 bash[29284]: audit 2024-03-26T03:53:11.931249+0000 mon.c (mon.2) 489 : audit [INF] from='client.? 172.21.15.175:0/430157263' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi175-64406-63"}]: dispatch 2024-03-26T03:53:13.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:12 smithi175 bash[29284]: audit 2024-03-26T03:53:11.931555+0000 mon.a (mon.0) 2342 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi175-64406-63"}]: dispatch 2024-03-26T03:53:13.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:12 smithi175 bash[29284]: audit 2024-03-26T03:53:11.932034+0000 mon.c (mon.2) 490 : audit [INF] from='client.? 172.21.15.175:0/430157263' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi175-64406-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:53:13.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:12 smithi175 bash[29284]: audit 2024-03-26T03:53:11.932319+0000 mon.a (mon.0) 2343 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi175-64406-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:53:13.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:12 smithi175 bash[29284]: audit 2024-03-26T03:53:11.956758+0000 mon.a (mon.0) 2344 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:53:13.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:12 smithi175 bash[29284]: audit 2024-03-26T03:53:11.957203+0000 mon.a (mon.0) 2345 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-47"}]: dispatch 2024-03-26T03:53:14.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:13 smithi190 bash[23392]: cluster 2024-03-26T03:53:12.719344+0000 mgr.y (mgr.24431) 354 : cluster [DBG] pgmap v500: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:53:14.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:13 smithi190 bash[23392]: audit 2024-03-26T03:53:12.908330+0000 mon.a (mon.0) 2346 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi175-64406-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:53:14.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:13 smithi190 bash[23392]: cluster 2024-03-26T03:53:12.915757+0000 mon.a (mon.0) 2347 : cluster [DBG] osdmap e336: 8 total, 8 up, 8 in 2024-03-26T03:53:14.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:13 smithi190 bash[23392]: audit 2024-03-26T03:53:12.918149+0000 mon.c (mon.2) 491 : audit [INF] from='client.? 172.21.15.175:0/430157263' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi175-64406-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi175-64406-63"}]: dispatch 2024-03-26T03:53:14.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:13 smithi190 bash[23392]: audit 2024-03-26T03:53:12.919390+0000 mon.a (mon.0) 2348 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi175-64406-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi175-64406-63"}]: dispatch 2024-03-26T03:53:14.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:13 smithi175 bash[21527]: cluster 2024-03-26T03:53:12.719344+0000 mgr.y (mgr.24431) 354 : cluster [DBG] pgmap v500: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:53:14.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:13 smithi175 bash[21527]: audit 2024-03-26T03:53:12.908330+0000 mon.a (mon.0) 2346 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi175-64406-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:53:14.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:13 smithi175 bash[21527]: cluster 2024-03-26T03:53:12.915757+0000 mon.a (mon.0) 2347 : cluster [DBG] osdmap e336: 8 total, 8 up, 8 in 2024-03-26T03:53:14.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:13 smithi175 bash[21527]: audit 2024-03-26T03:53:12.918149+0000 mon.c (mon.2) 491 : audit [INF] from='client.? 172.21.15.175:0/430157263' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi175-64406-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi175-64406-63"}]: dispatch 2024-03-26T03:53:14.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:13 smithi175 bash[21527]: audit 2024-03-26T03:53:12.919390+0000 mon.a (mon.0) 2348 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi175-64406-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi175-64406-63"}]: dispatch 2024-03-26T03:53:14.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:13 smithi175 bash[29284]: cluster 2024-03-26T03:53:12.719344+0000 mgr.y (mgr.24431) 354 : cluster [DBG] pgmap v500: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:53:14.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:13 smithi175 bash[29284]: audit 2024-03-26T03:53:12.908330+0000 mon.a (mon.0) 2346 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi175-64406-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:53:14.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:13 smithi175 bash[29284]: cluster 2024-03-26T03:53:12.915757+0000 mon.a (mon.0) 2347 : cluster [DBG] osdmap e336: 8 total, 8 up, 8 in 2024-03-26T03:53:14.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:13 smithi175 bash[29284]: audit 2024-03-26T03:53:12.918149+0000 mon.c (mon.2) 491 : audit [INF] from='client.? 172.21.15.175:0/430157263' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi175-64406-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi175-64406-63"}]: dispatch 2024-03-26T03:53:14.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:13 smithi175 bash[29284]: audit 2024-03-26T03:53:12.919390+0000 mon.a (mon.0) 2348 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi175-64406-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi175-64406-63"}]: dispatch 2024-03-26T03:53:15.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:14 smithi190 bash[23392]: cluster 2024-03-26T03:53:13.935164+0000 mon.a (mon.0) 2349 : cluster [DBG] osdmap e337: 8 total, 8 up, 8 in 2024-03-26T03:53:15.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:14 smithi190 bash[23392]: audit 2024-03-26T03:53:13.940693+0000 mon.a (mon.0) 2350 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:53:15.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:14 smithi190 bash[23392]: audit 2024-03-26T03:53:14.916288+0000 mon.a (mon.0) 2351 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi175-64406-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi175-64406-63"}]': finished 2024-03-26T03:53:15.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:14 smithi190 bash[23392]: audit 2024-03-26T03:53:14.916560+0000 mon.a (mon.0) 2352 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:53:15.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:14 smithi190 bash[23392]: cluster 2024-03-26T03:53:14.924885+0000 mon.a (mon.0) 2353 : cluster [DBG] osdmap e338: 8 total, 8 up, 8 in 2024-03-26T03:53:15.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:14 smithi190 bash[23392]: audit 2024-03-26T03:53:14.940229+0000 mon.a (mon.0) 2354 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:53:15.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:14 smithi175 bash[21527]: cluster 2024-03-26T03:53:13.935164+0000 mon.a (mon.0) 2349 : cluster [DBG] osdmap e337: 8 total, 8 up, 8 in 2024-03-26T03:53:15.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:14 smithi175 bash[21527]: audit 2024-03-26T03:53:13.940693+0000 mon.a (mon.0) 2350 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:53:15.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:14 smithi175 bash[21527]: audit 2024-03-26T03:53:14.916288+0000 mon.a (mon.0) 2351 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi175-64406-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi175-64406-63"}]': finished 2024-03-26T03:53:15.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:14 smithi175 bash[21527]: audit 2024-03-26T03:53:14.916560+0000 mon.a (mon.0) 2352 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:53:15.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:14 smithi175 bash[21527]: cluster 2024-03-26T03:53:14.924885+0000 mon.a (mon.0) 2353 : cluster [DBG] osdmap e338: 8 total, 8 up, 8 in 2024-03-26T03:53:15.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:14 smithi175 bash[21527]: audit 2024-03-26T03:53:14.940229+0000 mon.a (mon.0) 2354 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:53:15.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:14 smithi175 bash[29284]: cluster 2024-03-26T03:53:13.935164+0000 mon.a (mon.0) 2349 : cluster [DBG] osdmap e337: 8 total, 8 up, 8 in 2024-03-26T03:53:15.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:14 smithi175 bash[29284]: audit 2024-03-26T03:53:13.940693+0000 mon.a (mon.0) 2350 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:53:15.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:14 smithi175 bash[29284]: audit 2024-03-26T03:53:14.916288+0000 mon.a (mon.0) 2351 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi175-64406-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi175-64406-63"}]': finished 2024-03-26T03:53:15.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:14 smithi175 bash[29284]: audit 2024-03-26T03:53:14.916560+0000 mon.a (mon.0) 2352 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:53:15.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:14 smithi175 bash[29284]: cluster 2024-03-26T03:53:14.924885+0000 mon.a (mon.0) 2353 : cluster [DBG] osdmap e338: 8 total, 8 up, 8 in 2024-03-26T03:53:15.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:14 smithi175 bash[29284]: audit 2024-03-26T03:53:14.940229+0000 mon.a (mon.0) 2354 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:53:16.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:15 smithi190 bash[23392]: cluster 2024-03-26T03:53:14.721245+0000 mgr.y (mgr.24431) 355 : cluster [DBG] pgmap v503: 292 pgs: 20 creating+peering, 12 unknown, 260 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:53:16.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:15 smithi190 bash[23392]: audit 2024-03-26T03:53:15.919256+0000 mon.a (mon.0) 2355 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-49", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:53:16.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:15 smithi190 bash[23392]: cluster 2024-03-26T03:53:15.928620+0000 mon.a (mon.0) 2356 : cluster [DBG] osdmap e339: 8 total, 8 up, 8 in 2024-03-26T03:53:16.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:15 smithi190 bash[23392]: audit 2024-03-26T03:53:15.932273+0000 mon.a (mon.0) 2357 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-49"}]: dispatch 2024-03-26T03:53:16.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:15 smithi175 bash[29284]: cluster 2024-03-26T03:53:14.721245+0000 mgr.y (mgr.24431) 355 : cluster [DBG] pgmap v503: 292 pgs: 20 creating+peering, 12 unknown, 260 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:53:16.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:15 smithi175 bash[29284]: audit 2024-03-26T03:53:15.919256+0000 mon.a (mon.0) 2355 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-49", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:53:16.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:15 smithi175 bash[29284]: cluster 2024-03-26T03:53:15.928620+0000 mon.a (mon.0) 2356 : cluster [DBG] osdmap e339: 8 total, 8 up, 8 in 2024-03-26T03:53:16.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:15 smithi175 bash[29284]: audit 2024-03-26T03:53:15.932273+0000 mon.a (mon.0) 2357 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-49"}]: dispatch 2024-03-26T03:53:16.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:15 smithi175 bash[21527]: cluster 2024-03-26T03:53:14.721245+0000 mgr.y (mgr.24431) 355 : cluster [DBG] pgmap v503: 292 pgs: 20 creating+peering, 12 unknown, 260 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:53:16.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:15 smithi175 bash[21527]: audit 2024-03-26T03:53:15.919256+0000 mon.a (mon.0) 2355 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-49", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:53:16.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:15 smithi175 bash[21527]: cluster 2024-03-26T03:53:15.928620+0000 mon.a (mon.0) 2356 : cluster [DBG] osdmap e339: 8 total, 8 up, 8 in 2024-03-26T03:53:16.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:15 smithi175 bash[21527]: audit 2024-03-26T03:53:15.932273+0000 mon.a (mon.0) 2357 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-49"}]: dispatch 2024-03-26T03:53:17.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:16 smithi190 bash[23392]: cluster 2024-03-26T03:53:16.492807+0000 mon.a (mon.0) 2358 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:53:17.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:16 smithi190 bash[23392]: audit 2024-03-26T03:53:16.922140+0000 mon.a (mon.0) 2359 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-49"}]': finished 2024-03-26T03:53:17.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:16 smithi190 bash[23392]: cluster 2024-03-26T03:53:16.930190+0000 mon.a (mon.0) 2360 : cluster [DBG] osdmap e340: 8 total, 8 up, 8 in 2024-03-26T03:53:17.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:16 smithi190 bash[23392]: audit 2024-03-26T03:53:16.931546+0000 mon.a (mon.0) 2361 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-49", "mode": "readproxy"}]: dispatch 2024-03-26T03:53:17.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:16 smithi190 bash[23392]: audit 2024-03-26T03:53:16.934825+0000 mon.c (mon.2) 492 : audit [INF] from='client.? 172.21.15.175:0/430157263' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi175-64406-63"}]: dispatch 2024-03-26T03:53:17.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:16 smithi190 bash[23392]: audit 2024-03-26T03:53:16.935724+0000 mon.a (mon.0) 2362 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi175-64406-63"}]: dispatch 2024-03-26T03:53:17.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:16 smithi175 bash[29284]: cluster 2024-03-26T03:53:16.492807+0000 mon.a (mon.0) 2358 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:53:17.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:16 smithi175 bash[29284]: audit 2024-03-26T03:53:16.922140+0000 mon.a (mon.0) 2359 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-49"}]': finished 2024-03-26T03:53:17.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:16 smithi175 bash[29284]: cluster 2024-03-26T03:53:16.930190+0000 mon.a (mon.0) 2360 : cluster [DBG] osdmap e340: 8 total, 8 up, 8 in 2024-03-26T03:53:17.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:16 smithi175 bash[29284]: audit 2024-03-26T03:53:16.931546+0000 mon.a (mon.0) 2361 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-49", "mode": "readproxy"}]: dispatch 2024-03-26T03:53:17.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:16 smithi175 bash[29284]: audit 2024-03-26T03:53:16.934825+0000 mon.c (mon.2) 492 : audit [INF] from='client.? 172.21.15.175:0/430157263' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi175-64406-63"}]: dispatch 2024-03-26T03:53:17.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:16 smithi175 bash[29284]: audit 2024-03-26T03:53:16.935724+0000 mon.a (mon.0) 2362 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi175-64406-63"}]: dispatch 2024-03-26T03:53:17.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:16 smithi175 bash[21527]: cluster 2024-03-26T03:53:16.492807+0000 mon.a (mon.0) 2358 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:53:17.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:16 smithi175 bash[21527]: audit 2024-03-26T03:53:16.922140+0000 mon.a (mon.0) 2359 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-49"}]': finished 2024-03-26T03:53:17.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:16 smithi175 bash[21527]: cluster 2024-03-26T03:53:16.930190+0000 mon.a (mon.0) 2360 : cluster [DBG] osdmap e340: 8 total, 8 up, 8 in 2024-03-26T03:53:17.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:16 smithi175 bash[21527]: audit 2024-03-26T03:53:16.931546+0000 mon.a (mon.0) 2361 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-49", "mode": "readproxy"}]: dispatch 2024-03-26T03:53:17.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:16 smithi175 bash[21527]: audit 2024-03-26T03:53:16.934825+0000 mon.c (mon.2) 492 : audit [INF] from='client.? 172.21.15.175:0/430157263' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi175-64406-63"}]: dispatch 2024-03-26T03:53:17.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:16 smithi175 bash[21527]: audit 2024-03-26T03:53:16.935724+0000 mon.a (mon.0) 2362 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi175-64406-63"}]: dispatch 2024-03-26T03:53:18.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:17 smithi190 bash[23392]: cluster 2024-03-26T03:53:16.722279+0000 mgr.y (mgr.24431) 356 : cluster [DBG] pgmap v506: 300 pgs: 8 unknown, 32 creating+peering, 260 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-26T03:53:18.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:17 smithi190 bash[23392]: cluster 2024-03-26T03:53:17.922486+0000 mon.a (mon.0) 2363 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:53:18.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:17 smithi190 bash[23392]: audit 2024-03-26T03:53:17.924933+0000 mon.a (mon.0) 2364 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-49", "mode": "readproxy"}]': finished 2024-03-26T03:53:18.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:17 smithi190 bash[23392]: audit 2024-03-26T03:53:17.925147+0000 mon.a (mon.0) 2365 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi175-64406-63"}]': finished 2024-03-26T03:53:18.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:17 smithi190 bash[23392]: audit 2024-03-26T03:53:17.935105+0000 mon.c (mon.2) 493 : audit [INF] from='client.? 172.21.15.175:0/430157263' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi175-64406-63"}]: dispatch 2024-03-26T03:53:18.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:17 smithi190 bash[23392]: cluster 2024-03-26T03:53:17.936866+0000 mon.a (mon.0) 2366 : cluster [DBG] osdmap e341: 8 total, 8 up, 8 in 2024-03-26T03:53:18.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:17 smithi190 bash[23392]: audit 2024-03-26T03:53:17.940017+0000 mon.a (mon.0) 2367 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi175-64406-63"}]: dispatch 2024-03-26T03:53:18.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:17 smithi175 bash[29284]: cluster 2024-03-26T03:53:16.722279+0000 mgr.y (mgr.24431) 356 : cluster [DBG] pgmap v506: 300 pgs: 8 unknown, 32 creating+peering, 260 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-26T03:53:18.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:17 smithi175 bash[29284]: cluster 2024-03-26T03:53:17.922486+0000 mon.a (mon.0) 2363 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:53:18.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:17 smithi175 bash[29284]: audit 2024-03-26T03:53:17.924933+0000 mon.a (mon.0) 2364 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-49", "mode": "readproxy"}]': finished 2024-03-26T03:53:18.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:17 smithi175 bash[29284]: audit 2024-03-26T03:53:17.925147+0000 mon.a (mon.0) 2365 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi175-64406-63"}]': finished 2024-03-26T03:53:18.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:17 smithi175 bash[29284]: audit 2024-03-26T03:53:17.935105+0000 mon.c (mon.2) 493 : audit [INF] from='client.? 172.21.15.175:0/430157263' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi175-64406-63"}]: dispatch 2024-03-26T03:53:18.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:17 smithi175 bash[29284]: cluster 2024-03-26T03:53:17.936866+0000 mon.a (mon.0) 2366 : cluster [DBG] osdmap e341: 8 total, 8 up, 8 in 2024-03-26T03:53:18.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:17 smithi175 bash[29284]: audit 2024-03-26T03:53:17.940017+0000 mon.a (mon.0) 2367 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi175-64406-63"}]: dispatch 2024-03-26T03:53:18.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:17 smithi175 bash[21527]: cluster 2024-03-26T03:53:16.722279+0000 mgr.y (mgr.24431) 356 : cluster [DBG] pgmap v506: 300 pgs: 8 unknown, 32 creating+peering, 260 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-26T03:53:18.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:17 smithi175 bash[21527]: cluster 2024-03-26T03:53:17.922486+0000 mon.a (mon.0) 2363 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:53:18.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:17 smithi175 bash[21527]: audit 2024-03-26T03:53:17.924933+0000 mon.a (mon.0) 2364 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-49", "mode": "readproxy"}]': finished 2024-03-26T03:53:18.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:17 smithi175 bash[21527]: audit 2024-03-26T03:53:17.925147+0000 mon.a (mon.0) 2365 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi175-64406-63"}]': finished 2024-03-26T03:53:18.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:17 smithi175 bash[21527]: audit 2024-03-26T03:53:17.935105+0000 mon.c (mon.2) 493 : audit [INF] from='client.? 172.21.15.175:0/430157263' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi175-64406-63"}]: dispatch 2024-03-26T03:53:18.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:17 smithi175 bash[21527]: cluster 2024-03-26T03:53:17.936866+0000 mon.a (mon.0) 2366 : cluster [DBG] osdmap e341: 8 total, 8 up, 8 in 2024-03-26T03:53:18.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:17 smithi175 bash[21527]: audit 2024-03-26T03:53:17.940017+0000 mon.a (mon.0) 2367 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi175-64406-63"}]: dispatch 2024-03-26T03:53:19.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:53:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:53:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:53:20.239 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:53:19 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:53:20.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:19 smithi190 bash[23392]: cluster 2024-03-26T03:53:18.723186+0000 mgr.y (mgr.24431) 357 : cluster [DBG] pgmap v509: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-26T03:53:20.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:19 smithi190 bash[23392]: audit 2024-03-26T03:53:18.928378+0000 mon.a (mon.0) 2368 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi175-64406-63"}]': finished 2024-03-26T03:53:20.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:19 smithi190 bash[23392]: cluster 2024-03-26T03:53:18.936839+0000 mon.a (mon.0) 2369 : cluster [DBG] osdmap e342: 8 total, 8 up, 8 in 2024-03-26T03:53:20.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:19 smithi190 bash[23392]: audit 2024-03-26T03:53:18.991430+0000 mon.c (mon.2) 494 : audit [INF] from='client.? 172.21.15.175:0/3120570652' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi175-64406-64"}]: dispatch 2024-03-26T03:53:20.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:19 smithi190 bash[23392]: audit 2024-03-26T03:53:18.991821+0000 mon.a (mon.0) 2370 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi175-64406-64"}]: dispatch 2024-03-26T03:53:20.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:19 smithi190 bash[23392]: audit 2024-03-26T03:53:18.992469+0000 mon.c (mon.2) 495 : audit [INF] from='client.? 172.21.15.175:0/3120570652' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi175-64406-64"}]: dispatch 2024-03-26T03:53:20.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:19 smithi190 bash[23392]: audit 2024-03-26T03:53:18.992817+0000 mon.a (mon.0) 2371 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi175-64406-64"}]: dispatch 2024-03-26T03:53:20.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:19 smithi190 bash[23392]: audit 2024-03-26T03:53:18.993378+0000 mon.c (mon.2) 496 : audit [INF] from='client.? 172.21.15.175:0/3120570652' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi175-64406-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:53:20.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:19 smithi190 bash[23392]: audit 2024-03-26T03:53:18.993672+0000 mon.a (mon.0) 2372 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi175-64406-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:53:20.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:19 smithi190 bash[23392]: audit 2024-03-26T03:53:19.719712+0000 mon.c (mon.2) 497 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:53:20.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:19 smithi175 bash[21527]: cluster 2024-03-26T03:53:18.723186+0000 mgr.y (mgr.24431) 357 : cluster [DBG] pgmap v509: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-26T03:53:20.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:19 smithi175 bash[21527]: audit 2024-03-26T03:53:18.928378+0000 mon.a (mon.0) 2368 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi175-64406-63"}]': finished 2024-03-26T03:53:20.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:19 smithi175 bash[21527]: cluster 2024-03-26T03:53:18.936839+0000 mon.a (mon.0) 2369 : cluster [DBG] osdmap e342: 8 total, 8 up, 8 in 2024-03-26T03:53:20.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:19 smithi175 bash[21527]: audit 2024-03-26T03:53:18.991430+0000 mon.c (mon.2) 494 : audit [INF] from='client.? 172.21.15.175:0/3120570652' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi175-64406-64"}]: dispatch 2024-03-26T03:53:20.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:19 smithi175 bash[21527]: audit 2024-03-26T03:53:18.991821+0000 mon.a (mon.0) 2370 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi175-64406-64"}]: dispatch 2024-03-26T03:53:20.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:19 smithi175 bash[21527]: audit 2024-03-26T03:53:18.992469+0000 mon.c (mon.2) 495 : audit [INF] from='client.? 172.21.15.175:0/3120570652' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi175-64406-64"}]: dispatch 2024-03-26T03:53:20.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:19 smithi175 bash[21527]: audit 2024-03-26T03:53:18.992817+0000 mon.a (mon.0) 2371 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi175-64406-64"}]: dispatch 2024-03-26T03:53:20.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:19 smithi175 bash[21527]: audit 2024-03-26T03:53:18.993378+0000 mon.c (mon.2) 496 : audit [INF] from='client.? 172.21.15.175:0/3120570652' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi175-64406-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:53:20.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:19 smithi175 bash[21527]: audit 2024-03-26T03:53:18.993672+0000 mon.a (mon.0) 2372 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi175-64406-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:53:20.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:19 smithi175 bash[21527]: audit 2024-03-26T03:53:19.719712+0000 mon.c (mon.2) 497 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:53:20.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:19 smithi175 bash[29284]: cluster 2024-03-26T03:53:18.723186+0000 mgr.y (mgr.24431) 357 : cluster [DBG] pgmap v509: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-26T03:53:20.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:19 smithi175 bash[29284]: audit 2024-03-26T03:53:18.928378+0000 mon.a (mon.0) 2368 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi175-64406-63"}]': finished 2024-03-26T03:53:20.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:19 smithi175 bash[29284]: cluster 2024-03-26T03:53:18.936839+0000 mon.a (mon.0) 2369 : cluster [DBG] osdmap e342: 8 total, 8 up, 8 in 2024-03-26T03:53:20.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:19 smithi175 bash[29284]: audit 2024-03-26T03:53:18.991430+0000 mon.c (mon.2) 494 : audit [INF] from='client.? 172.21.15.175:0/3120570652' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi175-64406-64"}]: dispatch 2024-03-26T03:53:20.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:19 smithi175 bash[29284]: audit 2024-03-26T03:53:18.991821+0000 mon.a (mon.0) 2370 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi175-64406-64"}]: dispatch 2024-03-26T03:53:20.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:19 smithi175 bash[29284]: audit 2024-03-26T03:53:18.992469+0000 mon.c (mon.2) 495 : audit [INF] from='client.? 172.21.15.175:0/3120570652' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi175-64406-64"}]: dispatch 2024-03-26T03:53:20.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:19 smithi175 bash[29284]: audit 2024-03-26T03:53:18.992817+0000 mon.a (mon.0) 2371 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi175-64406-64"}]: dispatch 2024-03-26T03:53:20.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:19 smithi175 bash[29284]: audit 2024-03-26T03:53:18.993378+0000 mon.c (mon.2) 496 : audit [INF] from='client.? 172.21.15.175:0/3120570652' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi175-64406-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:53:20.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:19 smithi175 bash[29284]: audit 2024-03-26T03:53:18.993672+0000 mon.a (mon.0) 2372 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi175-64406-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:53:20.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:19 smithi175 bash[29284]: audit 2024-03-26T03:53:19.719712+0000 mon.c (mon.2) 497 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:53:21.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:21 smithi190 bash[23392]: audit 2024-03-26T03:53:19.938258+0000 mgr.y (mgr.24431) 358 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:53:21.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:21 smithi190 bash[23392]: audit 2024-03-26T03:53:19.986886+0000 mon.a (mon.0) 2373 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi175-64406-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:53:21.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:21 smithi190 bash[23392]: cluster 2024-03-26T03:53:19.994514+0000 mon.a (mon.0) 2374 : cluster [DBG] osdmap e343: 8 total, 8 up, 8 in 2024-03-26T03:53:21.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:21 smithi190 bash[23392]: audit 2024-03-26T03:53:19.997274+0000 mon.c (mon.2) 498 : audit [INF] from='client.? 172.21.15.175:0/3120570652' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi175-64406-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi175-64406-64"}]: dispatch 2024-03-26T03:53:21.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:21 smithi190 bash[23392]: audit 2024-03-26T03:53:19.999688+0000 mon.a (mon.0) 2375 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi175-64406-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi175-64406-64"}]: dispatch 2024-03-26T03:53:21.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:20 smithi175 bash[21527]: audit 2024-03-26T03:53:19.938258+0000 mgr.y (mgr.24431) 358 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:53:21.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:20 smithi175 bash[21527]: audit 2024-03-26T03:53:19.986886+0000 mon.a (mon.0) 2373 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi175-64406-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:53:21.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:20 smithi175 bash[21527]: cluster 2024-03-26T03:53:19.994514+0000 mon.a (mon.0) 2374 : cluster [DBG] osdmap e343: 8 total, 8 up, 8 in 2024-03-26T03:53:21.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:20 smithi175 bash[21527]: audit 2024-03-26T03:53:19.997274+0000 mon.c (mon.2) 498 : audit [INF] from='client.? 172.21.15.175:0/3120570652' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi175-64406-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi175-64406-64"}]: dispatch 2024-03-26T03:53:21.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:20 smithi175 bash[21527]: audit 2024-03-26T03:53:19.999688+0000 mon.a (mon.0) 2375 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi175-64406-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi175-64406-64"}]: dispatch 2024-03-26T03:53:21.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:20 smithi175 bash[29284]: audit 2024-03-26T03:53:19.938258+0000 mgr.y (mgr.24431) 358 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:53:21.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:20 smithi175 bash[29284]: audit 2024-03-26T03:53:19.986886+0000 mon.a (mon.0) 2373 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi175-64406-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:53:21.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:20 smithi175 bash[29284]: cluster 2024-03-26T03:53:19.994514+0000 mon.a (mon.0) 2374 : cluster [DBG] osdmap e343: 8 total, 8 up, 8 in 2024-03-26T03:53:21.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:20 smithi175 bash[29284]: audit 2024-03-26T03:53:19.997274+0000 mon.c (mon.2) 498 : audit [INF] from='client.? 172.21.15.175:0/3120570652' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi175-64406-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi175-64406-64"}]: dispatch 2024-03-26T03:53:21.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:20 smithi175 bash[29284]: audit 2024-03-26T03:53:19.999688+0000 mon.a (mon.0) 2375 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi175-64406-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi175-64406-64"}]: dispatch 2024-03-26T03:53:22.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:22 smithi190 bash[23392]: cluster 2024-03-26T03:53:20.724092+0000 mgr.y (mgr.24431) 359 : cluster [DBG] pgmap v512: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:53:22.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:22 smithi190 bash[23392]: cluster 2024-03-26T03:53:21.019369+0000 mon.a (mon.0) 2376 : cluster [DBG] osdmap e344: 8 total, 8 up, 8 in 2024-03-26T03:53:22.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:22 smithi190 bash[23392]: cluster 2024-03-26T03:53:21.494355+0000 mon.a (mon.0) 2377 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:53:22.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:21 smithi175 bash[21527]: cluster 2024-03-26T03:53:20.724092+0000 mgr.y (mgr.24431) 359 : cluster [DBG] pgmap v512: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:53:22.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:21 smithi175 bash[21527]: cluster 2024-03-26T03:53:21.019369+0000 mon.a (mon.0) 2376 : cluster [DBG] osdmap e344: 8 total, 8 up, 8 in 2024-03-26T03:53:22.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:21 smithi175 bash[21527]: cluster 2024-03-26T03:53:21.494355+0000 mon.a (mon.0) 2377 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:53:22.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:22 smithi175 bash[29284]: cluster 2024-03-26T03:53:20.724092+0000 mgr.y (mgr.24431) 359 : cluster [DBG] pgmap v512: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:53:22.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:22 smithi175 bash[29284]: cluster 2024-03-26T03:53:21.019369+0000 mon.a (mon.0) 2376 : cluster [DBG] osdmap e344: 8 total, 8 up, 8 in 2024-03-26T03:53:22.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:22 smithi175 bash[29284]: cluster 2024-03-26T03:53:21.494355+0000 mon.a (mon.0) 2377 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:53:23.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:23 smithi190 bash[23392]: audit 2024-03-26T03:53:22.009422+0000 mon.a (mon.0) 2378 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi175-64406-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi175-64406-64"}]': finished 2024-03-26T03:53:23.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:23 smithi190 bash[23392]: cluster 2024-03-26T03:53:22.016456+0000 mon.a (mon.0) 2379 : cluster [DBG] osdmap e345: 8 total, 8 up, 8 in 2024-03-26T03:53:23.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:23 smithi175 bash[21527]: audit 2024-03-26T03:53:22.009422+0000 mon.a (mon.0) 2378 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi175-64406-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi175-64406-64"}]': finished 2024-03-26T03:53:23.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:23 smithi175 bash[21527]: cluster 2024-03-26T03:53:22.016456+0000 mon.a (mon.0) 2379 : cluster [DBG] osdmap e345: 8 total, 8 up, 8 in 2024-03-26T03:53:23.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:23 smithi175 bash[29284]: audit 2024-03-26T03:53:22.009422+0000 mon.a (mon.0) 2378 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi175-64406-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi175-64406-64"}]': finished 2024-03-26T03:53:23.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:23 smithi175 bash[29284]: cluster 2024-03-26T03:53:22.016456+0000 mon.a (mon.0) 2379 : cluster [DBG] osdmap e345: 8 total, 8 up, 8 in 2024-03-26T03:53:24.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:24 smithi190 bash[23392]: cluster 2024-03-26T03:53:22.724996+0000 mgr.y (mgr.24431) 360 : cluster [DBG] pgmap v515: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:53:24.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:24 smithi190 bash[23392]: cluster 2024-03-26T03:53:23.027309+0000 mon.a (mon.0) 2380 : cluster [DBG] osdmap e346: 8 total, 8 up, 8 in 2024-03-26T03:53:24.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:24 smithi175 bash[21527]: cluster 2024-03-26T03:53:22.724996+0000 mgr.y (mgr.24431) 360 : cluster [DBG] pgmap v515: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:53:24.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:24 smithi175 bash[21527]: cluster 2024-03-26T03:53:23.027309+0000 mon.a (mon.0) 2380 : cluster [DBG] osdmap e346: 8 total, 8 up, 8 in 2024-03-26T03:53:24.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:24 smithi175 bash[29284]: cluster 2024-03-26T03:53:22.724996+0000 mgr.y (mgr.24431) 360 : cluster [DBG] pgmap v515: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:53:24.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:24 smithi175 bash[29284]: cluster 2024-03-26T03:53:23.027309+0000 mon.a (mon.0) 2380 : cluster [DBG] osdmap e346: 8 total, 8 up, 8 in 2024-03-26T03:53:25.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:25 smithi190 bash[23392]: cluster 2024-03-26T03:53:24.035639+0000 mon.a (mon.0) 2381 : cluster [DBG] osdmap e347: 8 total, 8 up, 8 in 2024-03-26T03:53:25.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:25 smithi190 bash[23392]: audit 2024-03-26T03:53:24.039676+0000 mon.c (mon.2) 499 : audit [INF] from='client.? 172.21.15.175:0/3120570652' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi175-64406-64"}]: dispatch 2024-03-26T03:53:25.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:25 smithi190 bash[23392]: audit 2024-03-26T03:53:24.041002+0000 mon.a (mon.0) 2382 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi175-64406-64"}]: dispatch 2024-03-26T03:53:25.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:25 smithi175 bash[21527]: cluster 2024-03-26T03:53:24.035639+0000 mon.a (mon.0) 2381 : cluster [DBG] osdmap e347: 8 total, 8 up, 8 in 2024-03-26T03:53:25.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:25 smithi175 bash[21527]: audit 2024-03-26T03:53:24.039676+0000 mon.c (mon.2) 499 : audit [INF] from='client.? 172.21.15.175:0/3120570652' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi175-64406-64"}]: dispatch 2024-03-26T03:53:25.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:25 smithi175 bash[21527]: audit 2024-03-26T03:53:24.041002+0000 mon.a (mon.0) 2382 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi175-64406-64"}]: dispatch 2024-03-26T03:53:25.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:25 smithi175 bash[29284]: cluster 2024-03-26T03:53:24.035639+0000 mon.a (mon.0) 2381 : cluster [DBG] osdmap e347: 8 total, 8 up, 8 in 2024-03-26T03:53:25.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:25 smithi175 bash[29284]: audit 2024-03-26T03:53:24.039676+0000 mon.c (mon.2) 499 : audit [INF] from='client.? 172.21.15.175:0/3120570652' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi175-64406-64"}]: dispatch 2024-03-26T03:53:25.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:25 smithi175 bash[29284]: audit 2024-03-26T03:53:24.041002+0000 mon.a (mon.0) 2382 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi175-64406-64"}]: dispatch 2024-03-26T03:53:26.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:26 smithi190 bash[23392]: cluster 2024-03-26T03:53:24.726926+0000 mgr.y (mgr.24431) 361 : cluster [DBG] pgmap v518: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:53:26.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:26 smithi190 bash[23392]: audit 2024-03-26T03:53:25.039907+0000 mon.a (mon.0) 2383 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi175-64406-64"}]': finished 2024-03-26T03:53:26.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:26 smithi190 bash[23392]: cluster 2024-03-26T03:53:25.056448+0000 mon.a (mon.0) 2384 : cluster [DBG] osdmap e348: 8 total, 8 up, 8 in 2024-03-26T03:53:26.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:26 smithi190 bash[23392]: audit 2024-03-26T03:53:25.059771+0000 mon.c (mon.2) 500 : audit [INF] from='client.? 172.21.15.175:0/3120570652' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi175-64406-64"}]: dispatch 2024-03-26T03:53:26.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:26 smithi190 bash[23392]: audit 2024-03-26T03:53:25.060753+0000 mon.a (mon.0) 2385 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi175-64406-64"}]: dispatch 2024-03-26T03:53:26.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:26 smithi175 bash[21527]: cluster 2024-03-26T03:53:24.726926+0000 mgr.y (mgr.24431) 361 : cluster [DBG] pgmap v518: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:53:26.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:26 smithi175 bash[21527]: audit 2024-03-26T03:53:25.039907+0000 mon.a (mon.0) 2383 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi175-64406-64"}]': finished 2024-03-26T03:53:26.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:26 smithi175 bash[21527]: cluster 2024-03-26T03:53:25.056448+0000 mon.a (mon.0) 2384 : cluster [DBG] osdmap e348: 8 total, 8 up, 8 in 2024-03-26T03:53:26.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:26 smithi175 bash[21527]: audit 2024-03-26T03:53:25.059771+0000 mon.c (mon.2) 500 : audit [INF] from='client.? 172.21.15.175:0/3120570652' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi175-64406-64"}]: dispatch 2024-03-26T03:53:26.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:26 smithi175 bash[21527]: audit 2024-03-26T03:53:25.060753+0000 mon.a (mon.0) 2385 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi175-64406-64"}]: dispatch 2024-03-26T03:53:26.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:26 smithi175 bash[29284]: cluster 2024-03-26T03:53:24.726926+0000 mgr.y (mgr.24431) 361 : cluster [DBG] pgmap v518: 292 pgs: 292 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:53:26.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:26 smithi175 bash[29284]: audit 2024-03-26T03:53:25.039907+0000 mon.a (mon.0) 2383 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi175-64406-64"}]': finished 2024-03-26T03:53:26.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:26 smithi175 bash[29284]: cluster 2024-03-26T03:53:25.056448+0000 mon.a (mon.0) 2384 : cluster [DBG] osdmap e348: 8 total, 8 up, 8 in 2024-03-26T03:53:26.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:26 smithi175 bash[29284]: audit 2024-03-26T03:53:25.059771+0000 mon.c (mon.2) 500 : audit [INF] from='client.? 172.21.15.175:0/3120570652' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi175-64406-64"}]: dispatch 2024-03-26T03:53:26.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:26 smithi175 bash[29284]: audit 2024-03-26T03:53:25.060753+0000 mon.a (mon.0) 2385 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi175-64406-64"}]: dispatch 2024-03-26T03:53:27.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:27 smithi190 bash[23392]: audit 2024-03-26T03:53:26.050818+0000 mon.a (mon.0) 2386 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi175-64406-64"}]': finished 2024-03-26T03:53:27.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:27 smithi190 bash[23392]: cluster 2024-03-26T03:53:26.058131+0000 mon.a (mon.0) 2387 : cluster [DBG] osdmap e349: 8 total, 8 up, 8 in 2024-03-26T03:53:27.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:27 smithi190 bash[23392]: audit 2024-03-26T03:53:26.086440+0000 mon.a (mon.0) 2388 : audit [INF] from='client.? 172.21.15.175:0/2606461361' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi175-64406-65"}]: dispatch 2024-03-26T03:53:27.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:27 smithi190 bash[23392]: audit 2024-03-26T03:53:26.087508+0000 mon.a (mon.0) 2389 : audit [INF] from='client.? 172.21.15.175:0/2606461361' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi175-64406-65"}]: dispatch 2024-03-26T03:53:27.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:27 smithi190 bash[23392]: audit 2024-03-26T03:53:26.087917+0000 mon.a (mon.0) 2390 : audit [INF] from='client.? 172.21.15.175:0/2606461361' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi175-64406-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:53:27.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:27 smithi175 bash[21527]: audit 2024-03-26T03:53:26.050818+0000 mon.a (mon.0) 2386 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi175-64406-64"}]': finished 2024-03-26T03:53:27.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:27 smithi175 bash[21527]: cluster 2024-03-26T03:53:26.058131+0000 mon.a (mon.0) 2387 : cluster [DBG] osdmap e349: 8 total, 8 up, 8 in 2024-03-26T03:53:27.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:27 smithi175 bash[21527]: audit 2024-03-26T03:53:26.086440+0000 mon.a (mon.0) 2388 : audit [INF] from='client.? 172.21.15.175:0/2606461361' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi175-64406-65"}]: dispatch 2024-03-26T03:53:27.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:27 smithi175 bash[21527]: audit 2024-03-26T03:53:26.087508+0000 mon.a (mon.0) 2389 : audit [INF] from='client.? 172.21.15.175:0/2606461361' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi175-64406-65"}]: dispatch 2024-03-26T03:53:27.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:27 smithi175 bash[21527]: audit 2024-03-26T03:53:26.087917+0000 mon.a (mon.0) 2390 : audit [INF] from='client.? 172.21.15.175:0/2606461361' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi175-64406-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:53:27.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:27 smithi175 bash[29284]: audit 2024-03-26T03:53:26.050818+0000 mon.a (mon.0) 2386 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi175-64406-64"}]': finished 2024-03-26T03:53:27.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:27 smithi175 bash[29284]: cluster 2024-03-26T03:53:26.058131+0000 mon.a (mon.0) 2387 : cluster [DBG] osdmap e349: 8 total, 8 up, 8 in 2024-03-26T03:53:27.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:27 smithi175 bash[29284]: audit 2024-03-26T03:53:26.086440+0000 mon.a (mon.0) 2388 : audit [INF] from='client.? 172.21.15.175:0/2606461361' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi175-64406-65"}]: dispatch 2024-03-26T03:53:27.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:27 smithi175 bash[29284]: audit 2024-03-26T03:53:26.087508+0000 mon.a (mon.0) 2389 : audit [INF] from='client.? 172.21.15.175:0/2606461361' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi175-64406-65"}]: dispatch 2024-03-26T03:53:27.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:27 smithi175 bash[29284]: audit 2024-03-26T03:53:26.087917+0000 mon.a (mon.0) 2390 : audit [INF] from='client.? 172.21.15.175:0/2606461361' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi175-64406-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:53:28.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:28 smithi190 bash[23392]: cluster 2024-03-26T03:53:26.727805+0000 mgr.y (mgr.24431) 362 : cluster [DBG] pgmap v521: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:53:28.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:28 smithi190 bash[23392]: audit 2024-03-26T03:53:27.060163+0000 mon.a (mon.0) 2391 : audit [INF] from='client.? 172.21.15.175:0/2606461361' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi175-64406-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:53:28.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:28 smithi190 bash[23392]: cluster 2024-03-26T03:53:27.069517+0000 mon.a (mon.0) 2392 : cluster [DBG] osdmap e350: 8 total, 8 up, 8 in 2024-03-26T03:53:28.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:28 smithi190 bash[23392]: audit 2024-03-26T03:53:27.071243+0000 mon.a (mon.0) 2393 : audit [INF] from='client.? 172.21.15.175:0/2606461361' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi175-64406-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi175-64406-65"}]: dispatch 2024-03-26T03:53:28.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:28 smithi175 bash[29284]: cluster 2024-03-26T03:53:26.727805+0000 mgr.y (mgr.24431) 362 : cluster [DBG] pgmap v521: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:53:28.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:28 smithi175 bash[29284]: audit 2024-03-26T03:53:27.060163+0000 mon.a (mon.0) 2391 : audit [INF] from='client.? 172.21.15.175:0/2606461361' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi175-64406-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:53:28.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:28 smithi175 bash[29284]: cluster 2024-03-26T03:53:27.069517+0000 mon.a (mon.0) 2392 : cluster [DBG] osdmap e350: 8 total, 8 up, 8 in 2024-03-26T03:53:28.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:28 smithi175 bash[29284]: audit 2024-03-26T03:53:27.071243+0000 mon.a (mon.0) 2393 : audit [INF] from='client.? 172.21.15.175:0/2606461361' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi175-64406-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi175-64406-65"}]: dispatch 2024-03-26T03:53:28.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:28 smithi175 bash[21527]: cluster 2024-03-26T03:53:26.727805+0000 mgr.y (mgr.24431) 362 : cluster [DBG] pgmap v521: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:53:28.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:28 smithi175 bash[21527]: audit 2024-03-26T03:53:27.060163+0000 mon.a (mon.0) 2391 : audit [INF] from='client.? 172.21.15.175:0/2606461361' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi175-64406-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:53:28.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:28 smithi175 bash[21527]: cluster 2024-03-26T03:53:27.069517+0000 mon.a (mon.0) 2392 : cluster [DBG] osdmap e350: 8 total, 8 up, 8 in 2024-03-26T03:53:28.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:28 smithi175 bash[21527]: audit 2024-03-26T03:53:27.071243+0000 mon.a (mon.0) 2393 : audit [INF] from='client.? 172.21.15.175:0/2606461361' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi175-64406-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi175-64406-65"}]: dispatch 2024-03-26T03:53:29.092 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:53:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:53:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:53:29.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:29 smithi190 bash[23392]: cluster 2024-03-26T03:53:28.091683+0000 mon.a (mon.0) 2394 : cluster [DBG] osdmap e351: 8 total, 8 up, 8 in 2024-03-26T03:53:29.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:29 smithi190 bash[23392]: audit 2024-03-26T03:53:28.477332+0000 mon.a (mon.0) 2395 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:53:29.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:29 smithi175 bash[29284]: cluster 2024-03-26T03:53:28.091683+0000 mon.a (mon.0) 2394 : cluster [DBG] osdmap e351: 8 total, 8 up, 8 in 2024-03-26T03:53:29.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:29 smithi175 bash[29284]: audit 2024-03-26T03:53:28.477332+0000 mon.a (mon.0) 2395 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:53:29.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:29 smithi175 bash[21527]: cluster 2024-03-26T03:53:28.091683+0000 mon.a (mon.0) 2394 : cluster [DBG] osdmap e351: 8 total, 8 up, 8 in 2024-03-26T03:53:29.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:29 smithi175 bash[21527]: audit 2024-03-26T03:53:28.477332+0000 mon.a (mon.0) 2395 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:53:30.239 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:53:29 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:53:30.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:30 smithi190 bash[23392]: cluster 2024-03-26T03:53:28.728628+0000 mgr.y (mgr.24431) 363 : cluster [DBG] pgmap v524: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:53:30.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:30 smithi190 bash[23392]: audit 2024-03-26T03:53:29.077027+0000 mon.a (mon.0) 2396 : audit [INF] from='client.? 172.21.15.175:0/2606461361' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi175-64406-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi175-64406-65"}]': finished 2024-03-26T03:53:30.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:30 smithi190 bash[23392]: audit 2024-03-26T03:53:29.077234+0000 mon.a (mon.0) 2397 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:53:30.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:30 smithi190 bash[23392]: cluster 2024-03-26T03:53:29.087491+0000 mon.a (mon.0) 2398 : cluster [DBG] osdmap e352: 8 total, 8 up, 8 in 2024-03-26T03:53:30.239 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:30 smithi190 bash[23392]: audit 2024-03-26T03:53:29.093020+0000 mon.a (mon.0) 2399 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-49"}]: dispatch 2024-03-26T03:53:30.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:30 smithi190 bash[23392]: cluster 2024-03-26T03:53:30.077695+0000 mon.a (mon.0) 2400 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:53:30.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:30 smithi190 bash[23392]: audit 2024-03-26T03:53:30.080116+0000 mon.a (mon.0) 2401 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-49"}]': finished 2024-03-26T03:53:30.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:30 smithi190 bash[23392]: cluster 2024-03-26T03:53:30.087448+0000 mon.a (mon.0) 2402 : cluster [DBG] osdmap e353: 8 total, 8 up, 8 in 2024-03-26T03:53:30.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:30 smithi175 bash[21527]: cluster 2024-03-26T03:53:28.728628+0000 mgr.y (mgr.24431) 363 : cluster [DBG] pgmap v524: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:53:30.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:30 smithi175 bash[21527]: audit 2024-03-26T03:53:29.077027+0000 mon.a (mon.0) 2396 : audit [INF] from='client.? 172.21.15.175:0/2606461361' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi175-64406-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi175-64406-65"}]': finished 2024-03-26T03:53:30.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:30 smithi175 bash[21527]: audit 2024-03-26T03:53:29.077234+0000 mon.a (mon.0) 2397 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:53:30.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:30 smithi175 bash[21527]: cluster 2024-03-26T03:53:29.087491+0000 mon.a (mon.0) 2398 : cluster [DBG] osdmap e352: 8 total, 8 up, 8 in 2024-03-26T03:53:30.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:30 smithi175 bash[21527]: audit 2024-03-26T03:53:29.093020+0000 mon.a (mon.0) 2399 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-49"}]: dispatch 2024-03-26T03:53:30.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:30 smithi175 bash[21527]: cluster 2024-03-26T03:53:30.077695+0000 mon.a (mon.0) 2400 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:53:30.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:30 smithi175 bash[21527]: audit 2024-03-26T03:53:30.080116+0000 mon.a (mon.0) 2401 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-49"}]': finished 2024-03-26T03:53:30.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:30 smithi175 bash[21527]: cluster 2024-03-26T03:53:30.087448+0000 mon.a (mon.0) 2402 : cluster [DBG] osdmap e353: 8 total, 8 up, 8 in 2024-03-26T03:53:30.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:30 smithi175 bash[29284]: cluster 2024-03-26T03:53:28.728628+0000 mgr.y (mgr.24431) 363 : cluster [DBG] pgmap v524: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:53:30.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:30 smithi175 bash[29284]: audit 2024-03-26T03:53:29.077027+0000 mon.a (mon.0) 2396 : audit [INF] from='client.? 172.21.15.175:0/2606461361' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi175-64406-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi175-64406-65"}]': finished 2024-03-26T03:53:30.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:30 smithi175 bash[29284]: audit 2024-03-26T03:53:29.077234+0000 mon.a (mon.0) 2397 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:53:30.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:30 smithi175 bash[29284]: cluster 2024-03-26T03:53:29.087491+0000 mon.a (mon.0) 2398 : cluster [DBG] osdmap e352: 8 total, 8 up, 8 in 2024-03-26T03:53:30.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:30 smithi175 bash[29284]: audit 2024-03-26T03:53:29.093020+0000 mon.a (mon.0) 2399 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-49"}]: dispatch 2024-03-26T03:53:30.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:30 smithi175 bash[29284]: cluster 2024-03-26T03:53:30.077695+0000 mon.a (mon.0) 2400 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:53:30.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:30 smithi175 bash[29284]: audit 2024-03-26T03:53:30.080116+0000 mon.a (mon.0) 2401 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-49"}]': finished 2024-03-26T03:53:30.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:30 smithi175 bash[29284]: cluster 2024-03-26T03:53:30.087448+0000 mon.a (mon.0) 2402 : cluster [DBG] osdmap e353: 8 total, 8 up, 8 in 2024-03-26T03:53:31.126 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP2163:head 2024-03-26T03:53:31.126 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:5d165639:::164:head 2024-03-26T03:53:31.126 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:f43765fc:::165:head 2024-03-26T03:53:31.126 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:b4c720e9:::166:head 2024-03-26T03:53:31.126 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:e694b040:::167:head 2024-03-26T03:53:31.126 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:afa38db2:::168:head 2024-03-26T03:53:31.126 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:77ba9f53:::169:head 2024-03-26T03:53:31.126 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:87495034:::170:head 2024-03-26T03:53:31.126 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:7c96bf0e:::171:head 2024-03-26T03:53:31.127 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:dbe346cc:::172:head 2024-03-26T03:53:31.127 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:e943ec24:::173:head 2024-03-26T03:53:31.127 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:f97a9c0c:::174:head 2024-03-26T03:53:31.127 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:6f26e74d:::175:head 2024-03-26T03:53:31.127 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:4f95e106:::176:head 2024-03-26T03:53:31.127 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:0e6f2f8f:::177:head 2024-03-26T03:53:31.127 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:05db05f1:::178:head 2024-03-26T03:53:31.127 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:38a78d66:::179:head 2024-03-26T03:53:31.127 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:d095610b:::180:head 2024-03-26T03:53:31.127 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:a1a9d709:::181:head 2024-03-26T03:53:31.127 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:1e5d39db:::182:head 2024-03-26T03:53:31.127 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:f7df4fb9:::183:head 2024-03-26T03:53:31.127 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:03a7f161:::184:head 2024-03-26T03:53:31.127 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:ba70721e:::185:head 2024-03-26T03:53:31.127 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:28e5662d:::186:head 2024-03-26T03:53:31.127 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:973d52de:::187:head 2024-03-26T03:53:31.127 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:4303eb1c:::188:head 2024-03-26T03:53:31.128 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:b990b48e:::189:head 2024-03-26T03:53:31.128 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:29b8165b:::190:head 2024-03-26T03:53:31.128 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:3547f197:::191:head 2024-03-26T03:53:31.128 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:7e260936:::192:head 2024-03-26T03:53:31.128 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:1abec7b1:::193:head 2024-03-26T03:53:31.128 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:10fdda93:::194:head 2024-03-26T03:53:31.128 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:15817eea:::195:head 2024-03-26T03:53:31.128 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:770bab57:::196:head 2024-03-26T03:53:31.128 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:ed9e13e7:::197:head 2024-03-26T03:53:31.128 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:71471a8f:::198:head 2024-03-26T03:53:31.128 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: checking for 270:10fb1d02:::199:head 2024-03-26T03:53:31.128 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (8031 ms) 2024-03-26T03:53:31.128 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-03-26T03:53:31.128 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: got ls 1711425166,0 2024-03-26T03:53:31.128 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: first is 1711425166 2024-03-26T03:53:31.128 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: got ls 1711425166,0 2024-03-26T03:53:31.128 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: got ls 1711425166,0 2024-03-26T03:53:31.129 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: got ls 1711425166,1711425168,1711425169,0 2024-03-26T03:53:31.129 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: got ls 1711425166,1711425168,1711425169,0 2024-03-26T03:53:31.129 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: got ls 1711425166,1711425168,1711425169,0 2024-03-26T03:53:31.129 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: got ls 1711425166,1711425168,1711425169,1711425171,1711425172,0 2024-03-26T03:53:31.129 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: got ls 1711425166,1711425168,1711425169,1711425171,1711425172,0 2024-03-26T03:53:31.129 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: got ls 1711425166,1711425168,1711425169,1711425171,1711425172,0 2024-03-26T03:53:31.129 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: got ls 1711425166,1711425168,1711425169,1711425171,1711425172,1711425174,1711425175,0 2024-03-26T03:53:31.129 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: got ls 1711425166,1711425168,1711425169,1711425171,1711425172,1711425174,1711425175,0 2024-03-26T03:53:31.129 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: got ls 1711425166,1711425168,1711425169,1711425171,1711425172,1711425174,1711425175,0 2024-03-26T03:53:31.129 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: got ls 1711425169,1711425171,1711425172,1711425174,1711425175,1711425177,1711425178,0 2024-03-26T03:53:31.129 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: first now 1711425169, trimmed 2024-03-26T03:53:31.129 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (20227 ms) 2024-03-26T03:53:31.129 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-03-26T03:53:31.129 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: foo0 2024-03-26T03:53:31.129 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-03-26T03:53:31.129 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (14086 ms) 2024-03-26T03:53:31.130 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-03-26T03:53:31.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:31 smithi190 bash[23392]: audit 2024-03-26T03:53:29.949979+0000 mgr.y (mgr.24431) 364 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:53:31.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:31 smithi190 bash[23392]: audit 2024-03-26T03:53:30.145723+0000 mon.a (mon.0) 2403 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:53:31.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:31 smithi190 bash[23392]: audit 2024-03-26T03:53:30.146421+0000 mon.a (mon.0) 2404 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-49"}]: dispatch 2024-03-26T03:53:31.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:31 smithi175 bash[21527]: audit 2024-03-26T03:53:29.949979+0000 mgr.y (mgr.24431) 364 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:53:31.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:31 smithi175 bash[21527]: audit 2024-03-26T03:53:30.145723+0000 mon.a (mon.0) 2403 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:53:31.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:31 smithi175 bash[21527]: audit 2024-03-26T03:53:30.146421+0000 mon.a (mon.0) 2404 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-49"}]: dispatch 2024-03-26T03:53:31.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:31 smithi175 bash[29284]: audit 2024-03-26T03:53:29.949979+0000 mgr.y (mgr.24431) 364 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:53:31.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:31 smithi175 bash[29284]: audit 2024-03-26T03:53:30.145723+0000 mon.a (mon.0) 2403 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:53:31.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:31 smithi175 bash[29284]: audit 2024-03-26T03:53:30.146421+0000 mon.a (mon.0) 2404 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-49"}]: dispatch 2024-03-26T03:53:32.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:32 smithi190 bash[23392]: cluster 2024-03-26T03:53:30.729540+0000 mgr.y (mgr.24431) 365 : cluster [DBG] pgmap v527: 300 pgs: 8 creating+peering, 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:53:32.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:32 smithi190 bash[23392]: cluster 2024-03-26T03:53:31.103074+0000 mon.a (mon.0) 2405 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:53:32.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:32 smithi190 bash[23392]: cluster 2024-03-26T03:53:31.124307+0000 mon.a (mon.0) 2406 : cluster [DBG] osdmap e354: 8 total, 8 up, 8 in 2024-03-26T03:53:32.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:32 smithi190 bash[23392]: audit 2024-03-26T03:53:31.125989+0000 mon.a (mon.0) 2407 : audit [INF] from='client.? 172.21.15.175:0/2606461361' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi175-64406-65"}]: dispatch 2024-03-26T03:53:32.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:32 smithi175 bash[29284]: cluster 2024-03-26T03:53:30.729540+0000 mgr.y (mgr.24431) 365 : cluster [DBG] pgmap v527: 300 pgs: 8 creating+peering, 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:53:32.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:32 smithi175 bash[29284]: cluster 2024-03-26T03:53:31.103074+0000 mon.a (mon.0) 2405 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:53:32.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:32 smithi175 bash[29284]: cluster 2024-03-26T03:53:31.124307+0000 mon.a (mon.0) 2406 : cluster [DBG] osdmap e354: 8 total, 8 up, 8 in 2024-03-26T03:53:32.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:32 smithi175 bash[29284]: audit 2024-03-26T03:53:31.125989+0000 mon.a (mon.0) 2407 : audit [INF] from='client.? 172.21.15.175:0/2606461361' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi175-64406-65"}]: dispatch 2024-03-26T03:53:32.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:32 smithi175 bash[21527]: cluster 2024-03-26T03:53:30.729540+0000 mgr.y (mgr.24431) 365 : cluster [DBG] pgmap v527: 300 pgs: 8 creating+peering, 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:53:32.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:32 smithi175 bash[21527]: cluster 2024-03-26T03:53:31.103074+0000 mon.a (mon.0) 2405 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:53:32.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:32 smithi175 bash[21527]: cluster 2024-03-26T03:53:31.124307+0000 mon.a (mon.0) 2406 : cluster [DBG] osdmap e354: 8 total, 8 up, 8 in 2024-03-26T03:53:32.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:32 smithi175 bash[21527]: audit 2024-03-26T03:53:31.125989+0000 mon.a (mon.0) 2407 : audit [INF] from='client.? 172.21.15.175:0/2606461361' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi175-64406-65"}]: dispatch 2024-03-26T03:53:33.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:33 smithi190 bash[23392]: audit 2024-03-26T03:53:32.118141+0000 mon.a (mon.0) 2408 : audit [INF] from='client.? 172.21.15.175:0/2606461361' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi175-64406-65"}]': finished 2024-03-26T03:53:33.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:33 smithi190 bash[23392]: cluster 2024-03-26T03:53:32.129485+0000 mon.a (mon.0) 2409 : cluster [DBG] osdmap e355: 8 total, 8 up, 8 in 2024-03-26T03:53:33.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:33 smithi190 bash[23392]: audit 2024-03-26T03:53:32.134378+0000 mon.a (mon.0) 2410 : audit [INF] from='client.? 172.21.15.175:0/2606461361' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi175-64406-65"}]: dispatch 2024-03-26T03:53:33.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:33 smithi190 bash[23392]: audit 2024-03-26T03:53:32.138365+0000 mon.a (mon.0) 2411 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:53:33.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:33 smithi190 bash[23392]: audit 2024-03-26T03:53:33.122267+0000 mon.a (mon.0) 2412 : audit [INF] from='client.? 172.21.15.175:0/2606461361' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi175-64406-65"}]': finished 2024-03-26T03:53:33.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:33 smithi190 bash[23392]: audit 2024-03-26T03:53:33.122461+0000 mon.a (mon.0) 2413 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:53:33.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:33 smithi190 bash[23392]: cluster 2024-03-26T03:53:33.130909+0000 mon.a (mon.0) 2414 : cluster [DBG] osdmap e356: 8 total, 8 up, 8 in 2024-03-26T03:53:33.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:33 smithi175 bash[29284]: audit 2024-03-26T03:53:32.118141+0000 mon.a (mon.0) 2408 : audit [INF] from='client.? 172.21.15.175:0/2606461361' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi175-64406-65"}]': finished 2024-03-26T03:53:33.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:33 smithi175 bash[29284]: cluster 2024-03-26T03:53:32.129485+0000 mon.a (mon.0) 2409 : cluster [DBG] osdmap e355: 8 total, 8 up, 8 in 2024-03-26T03:53:33.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:33 smithi175 bash[29284]: audit 2024-03-26T03:53:32.134378+0000 mon.a (mon.0) 2410 : audit [INF] from='client.? 172.21.15.175:0/2606461361' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi175-64406-65"}]: dispatch 2024-03-26T03:53:33.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:33 smithi175 bash[29284]: audit 2024-03-26T03:53:32.138365+0000 mon.a (mon.0) 2411 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:53:33.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:33 smithi175 bash[29284]: audit 2024-03-26T03:53:33.122267+0000 mon.a (mon.0) 2412 : audit [INF] from='client.? 172.21.15.175:0/2606461361' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi175-64406-65"}]': finished 2024-03-26T03:53:33.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:33 smithi175 bash[29284]: audit 2024-03-26T03:53:33.122461+0000 mon.a (mon.0) 2413 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:53:33.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:33 smithi175 bash[29284]: cluster 2024-03-26T03:53:33.130909+0000 mon.a (mon.0) 2414 : cluster [DBG] osdmap e356: 8 total, 8 up, 8 in 2024-03-26T03:53:33.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:33 smithi175 bash[21527]: audit 2024-03-26T03:53:32.118141+0000 mon.a (mon.0) 2408 : audit [INF] from='client.? 172.21.15.175:0/2606461361' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi175-64406-65"}]': finished 2024-03-26T03:53:33.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:33 smithi175 bash[21527]: cluster 2024-03-26T03:53:32.129485+0000 mon.a (mon.0) 2409 : cluster [DBG] osdmap e355: 8 total, 8 up, 8 in 2024-03-26T03:53:33.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:33 smithi175 bash[21527]: audit 2024-03-26T03:53:32.134378+0000 mon.a (mon.0) 2410 : audit [INF] from='client.? 172.21.15.175:0/2606461361' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi175-64406-65"}]: dispatch 2024-03-26T03:53:33.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:33 smithi175 bash[21527]: audit 2024-03-26T03:53:32.138365+0000 mon.a (mon.0) 2411 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:53:33.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:33 smithi175 bash[21527]: audit 2024-03-26T03:53:33.122267+0000 mon.a (mon.0) 2412 : audit [INF] from='client.? 172.21.15.175:0/2606461361' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi175-64406-65"}]': finished 2024-03-26T03:53:33.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:33 smithi175 bash[21527]: audit 2024-03-26T03:53:33.122461+0000 mon.a (mon.0) 2413 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:53:33.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:33 smithi175 bash[21527]: cluster 2024-03-26T03:53:33.130909+0000 mon.a (mon.0) 2414 : cluster [DBG] osdmap e356: 8 total, 8 up, 8 in 2024-03-26T03:53:34.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:34 smithi190 bash[23392]: cluster 2024-03-26T03:53:32.730402+0000 mgr.y (mgr.24431) 366 : cluster [DBG] pgmap v530: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:53:34.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:34 smithi190 bash[23392]: audit 2024-03-26T03:53:33.186462+0000 mon.a (mon.0) 2415 : audit [INF] from='client.? 172.21.15.175:0/3458683690' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi175-64406-66"}]: dispatch 2024-03-26T03:53:34.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:34 smithi190 bash[23392]: audit 2024-03-26T03:53:33.187188+0000 mon.a (mon.0) 2416 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:53:34.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:34 smithi190 bash[23392]: audit 2024-03-26T03:53:33.188020+0000 mon.a (mon.0) 2417 : audit [INF] from='client.? 172.21.15.175:0/3458683690' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi175-64406-66"}]: dispatch 2024-03-26T03:53:34.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:34 smithi190 bash[23392]: audit 2024-03-26T03:53:33.188427+0000 mon.a (mon.0) 2418 : audit [INF] from='client.? 172.21.15.175:0/3458683690' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi175-64406-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:53:34.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:34 smithi190 bash[23392]: audit 2024-03-26T03:53:34.157772+0000 mon.a (mon.0) 2419 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-51", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:53:34.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:34 smithi190 bash[23392]: audit 2024-03-26T03:53:34.157967+0000 mon.a (mon.0) 2420 : audit [INF] from='client.? 172.21.15.175:0/3458683690' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi175-64406-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:53:34.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:34 smithi190 bash[23392]: cluster 2024-03-26T03:53:34.167415+0000 mon.a (mon.0) 2421 : cluster [DBG] osdmap e357: 8 total, 8 up, 8 in 2024-03-26T03:53:34.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:34 smithi175 bash[21527]: cluster 2024-03-26T03:53:32.730402+0000 mgr.y (mgr.24431) 366 : cluster [DBG] pgmap v530: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:53:34.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:34 smithi175 bash[21527]: audit 2024-03-26T03:53:33.186462+0000 mon.a (mon.0) 2415 : audit [INF] from='client.? 172.21.15.175:0/3458683690' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi175-64406-66"}]: dispatch 2024-03-26T03:53:34.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:34 smithi175 bash[21527]: audit 2024-03-26T03:53:33.187188+0000 mon.a (mon.0) 2416 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:53:34.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:34 smithi175 bash[21527]: audit 2024-03-26T03:53:33.188020+0000 mon.a (mon.0) 2417 : audit [INF] from='client.? 172.21.15.175:0/3458683690' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi175-64406-66"}]: dispatch 2024-03-26T03:53:34.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:34 smithi175 bash[21527]: audit 2024-03-26T03:53:33.188427+0000 mon.a (mon.0) 2418 : audit [INF] from='client.? 172.21.15.175:0/3458683690' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi175-64406-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:53:34.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:34 smithi175 bash[21527]: audit 2024-03-26T03:53:34.157772+0000 mon.a (mon.0) 2419 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-51", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:53:34.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:34 smithi175 bash[21527]: audit 2024-03-26T03:53:34.157967+0000 mon.a (mon.0) 2420 : audit [INF] from='client.? 172.21.15.175:0/3458683690' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi175-64406-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:53:34.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:34 smithi175 bash[21527]: cluster 2024-03-26T03:53:34.167415+0000 mon.a (mon.0) 2421 : cluster [DBG] osdmap e357: 8 total, 8 up, 8 in 2024-03-26T03:53:34.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:34 smithi175 bash[29284]: cluster 2024-03-26T03:53:32.730402+0000 mgr.y (mgr.24431) 366 : cluster [DBG] pgmap v530: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:53:34.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:34 smithi175 bash[29284]: audit 2024-03-26T03:53:33.186462+0000 mon.a (mon.0) 2415 : audit [INF] from='client.? 172.21.15.175:0/3458683690' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi175-64406-66"}]: dispatch 2024-03-26T03:53:34.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:34 smithi175 bash[29284]: audit 2024-03-26T03:53:33.187188+0000 mon.a (mon.0) 2416 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:53:34.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:34 smithi175 bash[29284]: audit 2024-03-26T03:53:33.188020+0000 mon.a (mon.0) 2417 : audit [INF] from='client.? 172.21.15.175:0/3458683690' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi175-64406-66"}]: dispatch 2024-03-26T03:53:34.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:34 smithi175 bash[29284]: audit 2024-03-26T03:53:33.188427+0000 mon.a (mon.0) 2418 : audit [INF] from='client.? 172.21.15.175:0/3458683690' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi175-64406-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:53:34.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:34 smithi175 bash[29284]: audit 2024-03-26T03:53:34.157772+0000 mon.a (mon.0) 2419 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-51", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:53:34.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:34 smithi175 bash[29284]: audit 2024-03-26T03:53:34.157967+0000 mon.a (mon.0) 2420 : audit [INF] from='client.? 172.21.15.175:0/3458683690' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi175-64406-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:53:34.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:34 smithi175 bash[29284]: cluster 2024-03-26T03:53:34.167415+0000 mon.a (mon.0) 2421 : cluster [DBG] osdmap e357: 8 total, 8 up, 8 in 2024-03-26T03:53:35.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:35 smithi190 bash[23392]: audit 2024-03-26T03:53:34.169384+0000 mon.a (mon.0) 2422 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-51"}]: dispatch 2024-03-26T03:53:35.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:35 smithi190 bash[23392]: audit 2024-03-26T03:53:34.170215+0000 mon.a (mon.0) 2423 : audit [INF] from='client.? 172.21.15.175:0/3458683690' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi175-64406-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi175-64406-66"}]: dispatch 2024-03-26T03:53:35.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:35 smithi190 bash[23392]: audit 2024-03-26T03:53:34.767679+0000 mon.a (mon.0) 2424 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:53:35.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:35 smithi190 bash[23392]: audit 2024-03-26T03:53:34.770782+0000 mon.c (mon.2) 501 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:53:35.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:35 smithi190 bash[23392]: audit 2024-03-26T03:53:35.161190+0000 mon.a (mon.0) 2425 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-51"}]': finished 2024-03-26T03:53:35.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:35 smithi175 bash[21527]: audit 2024-03-26T03:53:34.169384+0000 mon.a (mon.0) 2422 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-51"}]: dispatch 2024-03-26T03:53:35.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:35 smithi175 bash[21527]: audit 2024-03-26T03:53:34.170215+0000 mon.a (mon.0) 2423 : audit [INF] from='client.? 172.21.15.175:0/3458683690' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi175-64406-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi175-64406-66"}]: dispatch 2024-03-26T03:53:35.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:35 smithi175 bash[21527]: audit 2024-03-26T03:53:34.767679+0000 mon.a (mon.0) 2424 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:53:35.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:35 smithi175 bash[21527]: audit 2024-03-26T03:53:34.770782+0000 mon.c (mon.2) 501 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:53:35.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:35 smithi175 bash[21527]: audit 2024-03-26T03:53:35.161190+0000 mon.a (mon.0) 2425 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-51"}]': finished 2024-03-26T03:53:35.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:35 smithi175 bash[29284]: audit 2024-03-26T03:53:34.169384+0000 mon.a (mon.0) 2422 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-51"}]: dispatch 2024-03-26T03:53:35.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:35 smithi175 bash[29284]: audit 2024-03-26T03:53:34.170215+0000 mon.a (mon.0) 2423 : audit [INF] from='client.? 172.21.15.175:0/3458683690' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi175-64406-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi175-64406-66"}]: dispatch 2024-03-26T03:53:35.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:35 smithi175 bash[29284]: audit 2024-03-26T03:53:34.767679+0000 mon.a (mon.0) 2424 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:53:35.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:35 smithi175 bash[29284]: audit 2024-03-26T03:53:34.770782+0000 mon.c (mon.2) 501 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:53:35.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:35 smithi175 bash[29284]: audit 2024-03-26T03:53:35.161190+0000 mon.a (mon.0) 2425 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-51"}]': finished 2024-03-26T03:53:36.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:36 smithi190 bash[23392]: cluster 2024-03-26T03:53:34.732515+0000 mgr.y (mgr.24431) 367 : cluster [DBG] pgmap v533: 292 pgs: 15 unknown, 277 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 511 B/s wr, 0 op/s 2024-03-26T03:53:36.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:36 smithi190 bash[23392]: cluster 2024-03-26T03:53:35.179631+0000 mon.a (mon.0) 2426 : cluster [DBG] osdmap e358: 8 total, 8 up, 8 in 2024-03-26T03:53:36.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:36 smithi190 bash[23392]: audit 2024-03-26T03:53:35.182128+0000 mon.a (mon.0) 2427 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-51", "mode": "writeback"}]: dispatch 2024-03-26T03:53:36.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:36 smithi190 bash[23392]: cluster 2024-03-26T03:53:36.161847+0000 mon.a (mon.0) 2428 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:53:36.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:36 smithi190 bash[23392]: audit 2024-03-26T03:53:36.164685+0000 mon.a (mon.0) 2429 : audit [INF] from='client.? 172.21.15.175:0/3458683690' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi175-64406-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi175-64406-66"}]': finished 2024-03-26T03:53:36.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:36 smithi190 bash[23392]: audit 2024-03-26T03:53:36.164859+0000 mon.a (mon.0) 2430 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-51", "mode": "writeback"}]': finished 2024-03-26T03:53:36.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:36 smithi190 bash[23392]: cluster 2024-03-26T03:53:36.170763+0000 mon.a (mon.0) 2431 : cluster [DBG] osdmap e359: 8 total, 8 up, 8 in 2024-03-26T03:53:36.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:36 smithi175 bash[21527]: cluster 2024-03-26T03:53:34.732515+0000 mgr.y (mgr.24431) 367 : cluster [DBG] pgmap v533: 292 pgs: 15 unknown, 277 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 511 B/s wr, 0 op/s 2024-03-26T03:53:36.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:36 smithi175 bash[21527]: cluster 2024-03-26T03:53:35.179631+0000 mon.a (mon.0) 2426 : cluster [DBG] osdmap e358: 8 total, 8 up, 8 in 2024-03-26T03:53:36.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:36 smithi175 bash[21527]: audit 2024-03-26T03:53:35.182128+0000 mon.a (mon.0) 2427 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-51", "mode": "writeback"}]: dispatch 2024-03-26T03:53:36.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:36 smithi175 bash[21527]: cluster 2024-03-26T03:53:36.161847+0000 mon.a (mon.0) 2428 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:53:36.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:36 smithi175 bash[21527]: audit 2024-03-26T03:53:36.164685+0000 mon.a (mon.0) 2429 : audit [INF] from='client.? 172.21.15.175:0/3458683690' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi175-64406-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi175-64406-66"}]': finished 2024-03-26T03:53:36.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:36 smithi175 bash[21527]: audit 2024-03-26T03:53:36.164859+0000 mon.a (mon.0) 2430 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-51", "mode": "writeback"}]': finished 2024-03-26T03:53:36.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:36 smithi175 bash[21527]: cluster 2024-03-26T03:53:36.170763+0000 mon.a (mon.0) 2431 : cluster [DBG] osdmap e359: 8 total, 8 up, 8 in 2024-03-26T03:53:36.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:36 smithi175 bash[29284]: cluster 2024-03-26T03:53:34.732515+0000 mgr.y (mgr.24431) 367 : cluster [DBG] pgmap v533: 292 pgs: 15 unknown, 277 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 511 B/s wr, 0 op/s 2024-03-26T03:53:36.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:36 smithi175 bash[29284]: cluster 2024-03-26T03:53:35.179631+0000 mon.a (mon.0) 2426 : cluster [DBG] osdmap e358: 8 total, 8 up, 8 in 2024-03-26T03:53:36.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:36 smithi175 bash[29284]: audit 2024-03-26T03:53:35.182128+0000 mon.a (mon.0) 2427 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-51", "mode": "writeback"}]: dispatch 2024-03-26T03:53:36.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:36 smithi175 bash[29284]: cluster 2024-03-26T03:53:36.161847+0000 mon.a (mon.0) 2428 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:53:36.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:36 smithi175 bash[29284]: audit 2024-03-26T03:53:36.164685+0000 mon.a (mon.0) 2429 : audit [INF] from='client.? 172.21.15.175:0/3458683690' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi175-64406-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi175-64406-66"}]': finished 2024-03-26T03:53:36.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:36 smithi175 bash[29284]: audit 2024-03-26T03:53:36.164859+0000 mon.a (mon.0) 2430 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-51", "mode": "writeback"}]': finished 2024-03-26T03:53:36.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:36 smithi175 bash[29284]: cluster 2024-03-26T03:53:36.170763+0000 mon.a (mon.0) 2431 : cluster [DBG] osdmap e359: 8 total, 8 up, 8 in 2024-03-26T03:53:37.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:37 smithi190 bash[23392]: audit 2024-03-26T03:53:36.267220+0000 mon.a (mon.0) 2432 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-51","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T03:53:37.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:37 smithi190 bash[23392]: cluster 2024-03-26T03:53:36.500379+0000 mon.a (mon.0) 2433 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:53:37.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:37 smithi190 bash[23392]: audit 2024-03-26T03:53:37.167542+0000 mon.a (mon.0) 2434 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-51","var": "hit_set_count","val": "2"}]': finished 2024-03-26T03:53:37.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:37 smithi190 bash[23392]: cluster 2024-03-26T03:53:37.175008+0000 mon.a (mon.0) 2435 : cluster [DBG] osdmap e360: 8 total, 8 up, 8 in 2024-03-26T03:53:37.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:37 smithi190 bash[23392]: audit 2024-03-26T03:53:37.180838+0000 mon.a (mon.0) 2436 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-51","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T03:53:37.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:37 smithi175 bash[21527]: audit 2024-03-26T03:53:36.267220+0000 mon.a (mon.0) 2432 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-51","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T03:53:37.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:37 smithi175 bash[21527]: cluster 2024-03-26T03:53:36.500379+0000 mon.a (mon.0) 2433 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:53:37.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:37 smithi175 bash[21527]: audit 2024-03-26T03:53:37.167542+0000 mon.a (mon.0) 2434 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-51","var": "hit_set_count","val": "2"}]': finished 2024-03-26T03:53:37.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:37 smithi175 bash[21527]: cluster 2024-03-26T03:53:37.175008+0000 mon.a (mon.0) 2435 : cluster [DBG] osdmap e360: 8 total, 8 up, 8 in 2024-03-26T03:53:37.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:37 smithi175 bash[21527]: audit 2024-03-26T03:53:37.180838+0000 mon.a (mon.0) 2436 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-51","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T03:53:37.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:37 smithi175 bash[29284]: audit 2024-03-26T03:53:36.267220+0000 mon.a (mon.0) 2432 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-51","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T03:53:37.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:37 smithi175 bash[29284]: cluster 2024-03-26T03:53:36.500379+0000 mon.a (mon.0) 2433 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:53:37.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:37 smithi175 bash[29284]: audit 2024-03-26T03:53:37.167542+0000 mon.a (mon.0) 2434 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-51","var": "hit_set_count","val": "2"}]': finished 2024-03-26T03:53:37.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:37 smithi175 bash[29284]: cluster 2024-03-26T03:53:37.175008+0000 mon.a (mon.0) 2435 : cluster [DBG] osdmap e360: 8 total, 8 up, 8 in 2024-03-26T03:53:37.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:37 smithi175 bash[29284]: audit 2024-03-26T03:53:37.180838+0000 mon.a (mon.0) 2436 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-51","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T03:53:38.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:38 smithi190 bash[23392]: cluster 2024-03-26T03:53:36.733460+0000 mgr.y (mgr.24431) 368 : cluster [DBG] pgmap v536: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-26T03:53:38.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:38 smithi190 bash[23392]: audit 2024-03-26T03:53:38.170457+0000 mon.a (mon.0) 2437 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-51","var": "hit_set_period","val": "600"}]': finished 2024-03-26T03:53:38.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:38 smithi190 bash[23392]: cluster 2024-03-26T03:53:38.182408+0000 mon.a (mon.0) 2438 : cluster [DBG] osdmap e361: 8 total, 8 up, 8 in 2024-03-26T03:53:38.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:38 smithi190 bash[23392]: audit 2024-03-26T03:53:38.186376+0000 mon.a (mon.0) 2439 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-26T03:53:38.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:38 smithi190 bash[23392]: audit 2024-03-26T03:53:38.187797+0000 mon.a (mon.0) 2440 : audit [INF] from='client.? 172.21.15.175:0/3458683690' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi175-64406-66"}]: dispatch 2024-03-26T03:53:38.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:38 smithi175 bash[29284]: cluster 2024-03-26T03:53:36.733460+0000 mgr.y (mgr.24431) 368 : cluster [DBG] pgmap v536: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-26T03:53:38.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:38 smithi175 bash[29284]: audit 2024-03-26T03:53:38.170457+0000 mon.a (mon.0) 2437 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-51","var": "hit_set_period","val": "600"}]': finished 2024-03-26T03:53:38.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:38 smithi175 bash[29284]: cluster 2024-03-26T03:53:38.182408+0000 mon.a (mon.0) 2438 : cluster [DBG] osdmap e361: 8 total, 8 up, 8 in 2024-03-26T03:53:38.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:38 smithi175 bash[29284]: audit 2024-03-26T03:53:38.186376+0000 mon.a (mon.0) 2439 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-26T03:53:38.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:38 smithi175 bash[29284]: audit 2024-03-26T03:53:38.187797+0000 mon.a (mon.0) 2440 : audit [INF] from='client.? 172.21.15.175:0/3458683690' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi175-64406-66"}]: dispatch 2024-03-26T03:53:38.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:38 smithi175 bash[21527]: cluster 2024-03-26T03:53:36.733460+0000 mgr.y (mgr.24431) 368 : cluster [DBG] pgmap v536: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-26T03:53:38.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:38 smithi175 bash[21527]: audit 2024-03-26T03:53:38.170457+0000 mon.a (mon.0) 2437 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-51","var": "hit_set_period","val": "600"}]': finished 2024-03-26T03:53:38.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:38 smithi175 bash[21527]: cluster 2024-03-26T03:53:38.182408+0000 mon.a (mon.0) 2438 : cluster [DBG] osdmap e361: 8 total, 8 up, 8 in 2024-03-26T03:53:38.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:38 smithi175 bash[21527]: audit 2024-03-26T03:53:38.186376+0000 mon.a (mon.0) 2439 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-26T03:53:38.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:38 smithi175 bash[21527]: audit 2024-03-26T03:53:38.187797+0000 mon.a (mon.0) 2440 : audit [INF] from='client.? 172.21.15.175:0/3458683690' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi175-64406-66"}]: dispatch 2024-03-26T03:53:39.237 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:53:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:53:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:53:39.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:39 smithi190 bash[23392]: cluster 2024-03-26T03:53:39.170666+0000 mon.a (mon.0) 2441 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:53:39.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:39 smithi190 bash[23392]: audit 2024-03-26T03:53:39.174654+0000 mon.a (mon.0) 2442 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-51","var": "hit_set_type","val": "bloom"}]': finished 2024-03-26T03:53:39.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:39 smithi190 bash[23392]: audit 2024-03-26T03:53:39.174866+0000 mon.a (mon.0) 2443 : audit [INF] from='client.? 172.21.15.175:0/3458683690' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi175-64406-66"}]': finished 2024-03-26T03:53:39.489 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:39 smithi190 bash[23392]: cluster 2024-03-26T03:53:39.186864+0000 mon.a (mon.0) 2444 : cluster [DBG] osdmap e362: 8 total, 8 up, 8 in 2024-03-26T03:53:39.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:39 smithi190 bash[23392]: audit 2024-03-26T03:53:39.190494+0000 mon.a (mon.0) 2445 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-26T03:53:39.490 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:39 smithi190 bash[23392]: audit 2024-03-26T03:53:39.191027+0000 mon.a (mon.0) 2446 : audit [INF] from='client.? 172.21.15.175:0/3458683690' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi175-64406-66"}]: dispatch 2024-03-26T03:53:39.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:39 smithi175 bash[21527]: cluster 2024-03-26T03:53:39.170666+0000 mon.a (mon.0) 2441 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:53:39.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:39 smithi175 bash[21527]: audit 2024-03-26T03:53:39.174654+0000 mon.a (mon.0) 2442 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-51","var": "hit_set_type","val": "bloom"}]': finished 2024-03-26T03:53:39.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:39 smithi175 bash[21527]: audit 2024-03-26T03:53:39.174866+0000 mon.a (mon.0) 2443 : audit [INF] from='client.? 172.21.15.175:0/3458683690' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi175-64406-66"}]': finished 2024-03-26T03:53:39.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:39 smithi175 bash[21527]: cluster 2024-03-26T03:53:39.186864+0000 mon.a (mon.0) 2444 : cluster [DBG] osdmap e362: 8 total, 8 up, 8 in 2024-03-26T03:53:39.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:39 smithi175 bash[21527]: audit 2024-03-26T03:53:39.190494+0000 mon.a (mon.0) 2445 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-26T03:53:39.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:39 smithi175 bash[21527]: audit 2024-03-26T03:53:39.191027+0000 mon.a (mon.0) 2446 : audit [INF] from='client.? 172.21.15.175:0/3458683690' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi175-64406-66"}]: dispatch 2024-03-26T03:53:39.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:39 smithi175 bash[29284]: cluster 2024-03-26T03:53:39.170666+0000 mon.a (mon.0) 2441 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:53:39.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:39 smithi175 bash[29284]: audit 2024-03-26T03:53:39.174654+0000 mon.a (mon.0) 2442 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-51","var": "hit_set_type","val": "bloom"}]': finished 2024-03-26T03:53:39.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:39 smithi175 bash[29284]: audit 2024-03-26T03:53:39.174866+0000 mon.a (mon.0) 2443 : audit [INF] from='client.? 172.21.15.175:0/3458683690' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi175-64406-66"}]': finished 2024-03-26T03:53:39.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:39 smithi175 bash[29284]: cluster 2024-03-26T03:53:39.186864+0000 mon.a (mon.0) 2444 : cluster [DBG] osdmap e362: 8 total, 8 up, 8 in 2024-03-26T03:53:39.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:39 smithi175 bash[29284]: audit 2024-03-26T03:53:39.190494+0000 mon.a (mon.0) 2445 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-26T03:53:39.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:39 smithi175 bash[29284]: audit 2024-03-26T03:53:39.191027+0000 mon.a (mon.0) 2446 : audit [INF] from='client.? 172.21.15.175:0/3458683690' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi175-64406-66"}]: dispatch 2024-03-26T03:53:40.199 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ 2024-03-26T03:53:40.199 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3023 ms) 2024-03-26T03:53:40.199 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2024-03-26T03:53:40.199 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (7053 ms) 2024-03-26T03:53:40.200 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-03-26T03:53:40.200 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (7043 ms) 2024-03-26T03:53:40.200 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2024-03-26T03:53:40.200 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (7044 ms) 2024-03-26T03:53:40.200 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2024-03-26T03:53:40.200 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (7118 ms) 2024-03-26T03:53:40.200 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2024-03-26T03:53:40.200 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (7074 ms) 2024-03-26T03:53:40.200 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2024-03-26T03:53:40.200 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (7055 ms) 2024-03-26T03:53:40.200 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (139676 ms total) 2024-03-26T03:53:40.200 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: 2024-03-26T03:53:40.200 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [----------] Global test environment tear-down 2024-03-26T03:53:40.200 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [==========] 57 tests from 4 test suites ran. (287992 ms total) 2024-03-26T03:53:40.200 INFO:tasks.workunit.client.0.smithi175.stdout: api_aio_pp: [ PASSED ] 57 tests. 2024-03-26T03:53:40.239 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:53:39 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:53:40.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:40 smithi175 bash[21527]: cluster 2024-03-26T03:53:38.734403+0000 mgr.y (mgr.24431) 369 : cluster [DBG] pgmap v539: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-26T03:53:40.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:40 smithi175 bash[21527]: audit 2024-03-26T03:53:40.178427+0000 mon.a (mon.0) 2447 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-26T03:53:40.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:40 smithi175 bash[21527]: audit 2024-03-26T03:53:40.178560+0000 mon.a (mon.0) 2448 : audit [INF] from='client.? 172.21.15.175:0/3458683690' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi175-64406-66"}]': finished 2024-03-26T03:53:40.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:40 smithi175 bash[21527]: cluster 2024-03-26T03:53:40.186982+0000 mon.a (mon.0) 2449 : cluster [DBG] osdmap e363: 8 total, 8 up, 8 in 2024-03-26T03:53:40.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:40 smithi175 bash[21527]: audit 2024-03-26T03:53:40.189332+0000 mon.a (mon.0) 2450 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-51","var": "target_max_objects","val": "1"}]: dispatch 2024-03-26T03:53:40.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:40 smithi175 bash[29284]: cluster 2024-03-26T03:53:38.734403+0000 mgr.y (mgr.24431) 369 : cluster [DBG] pgmap v539: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-26T03:53:40.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:40 smithi175 bash[29284]: audit 2024-03-26T03:53:40.178427+0000 mon.a (mon.0) 2447 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-26T03:53:40.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:40 smithi175 bash[29284]: audit 2024-03-26T03:53:40.178560+0000 mon.a (mon.0) 2448 : audit [INF] from='client.? 172.21.15.175:0/3458683690' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi175-64406-66"}]': finished 2024-03-26T03:53:40.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:40 smithi175 bash[29284]: cluster 2024-03-26T03:53:40.186982+0000 mon.a (mon.0) 2449 : cluster [DBG] osdmap e363: 8 total, 8 up, 8 in 2024-03-26T03:53:40.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:40 smithi175 bash[29284]: audit 2024-03-26T03:53:40.189332+0000 mon.a (mon.0) 2450 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-51","var": "target_max_objects","val": "1"}]: dispatch 2024-03-26T03:53:40.739 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:40 smithi190 bash[23392]: cluster 2024-03-26T03:53:38.734403+0000 mgr.y (mgr.24431) 369 : cluster [DBG] pgmap v539: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-26T03:53:40.739 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:40 smithi190 bash[23392]: audit 2024-03-26T03:53:40.178427+0000 mon.a (mon.0) 2447 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-26T03:53:40.739 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:40 smithi190 bash[23392]: audit 2024-03-26T03:53:40.178560+0000 mon.a (mon.0) 2448 : audit [INF] from='client.? 172.21.15.175:0/3458683690' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi175-64406-66"}]': finished 2024-03-26T03:53:40.739 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:40 smithi190 bash[23392]: cluster 2024-03-26T03:53:40.186982+0000 mon.a (mon.0) 2449 : cluster [DBG] osdmap e363: 8 total, 8 up, 8 in 2024-03-26T03:53:40.739 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:40 smithi190 bash[23392]: audit 2024-03-26T03:53:40.189332+0000 mon.a (mon.0) 2450 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-51","var": "target_max_objects","val": "1"}]: dispatch 2024-03-26T03:53:41.739 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:41 smithi190 bash[23392]: audit 2024-03-26T03:53:39.961764+0000 mgr.y (mgr.24431) 370 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:53:41.739 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:41 smithi190 bash[23392]: audit 2024-03-26T03:53:41.181307+0000 mon.a (mon.0) 2451 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-51","var": "target_max_objects","val": "1"}]': finished 2024-03-26T03:53:41.739 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:41 smithi190 bash[23392]: cluster 2024-03-26T03:53:41.193417+0000 mon.a (mon.0) 2452 : cluster [DBG] osdmap e364: 8 total, 8 up, 8 in 2024-03-26T03:53:41.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:41 smithi175 bash[29284]: audit 2024-03-26T03:53:39.961764+0000 mgr.y (mgr.24431) 370 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:53:41.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:41 smithi175 bash[29284]: audit 2024-03-26T03:53:41.181307+0000 mon.a (mon.0) 2451 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-51","var": "target_max_objects","val": "1"}]': finished 2024-03-26T03:53:41.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:41 smithi175 bash[29284]: cluster 2024-03-26T03:53:41.193417+0000 mon.a (mon.0) 2452 : cluster [DBG] osdmap e364: 8 total, 8 up, 8 in 2024-03-26T03:53:41.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:41 smithi175 bash[21527]: audit 2024-03-26T03:53:39.961764+0000 mgr.y (mgr.24431) 370 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:53:41.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:41 smithi175 bash[21527]: audit 2024-03-26T03:53:41.181307+0000 mon.a (mon.0) 2451 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-51","var": "target_max_objects","val": "1"}]': finished 2024-03-26T03:53:41.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:41 smithi175 bash[21527]: cluster 2024-03-26T03:53:41.193417+0000 mon.a (mon.0) 2452 : cluster [DBG] osdmap e364: 8 total, 8 up, 8 in 2024-03-26T03:53:42.739 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:42 smithi190 bash[23392]: cluster 2024-03-26T03:53:40.735292+0000 mgr.y (mgr.24431) 371 : cluster [DBG] pgmap v542: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-26T03:53:42.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:42 smithi175 bash[29284]: cluster 2024-03-26T03:53:40.735292+0000 mgr.y (mgr.24431) 371 : cluster [DBG] pgmap v542: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-26T03:53:42.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:42 smithi175 bash[21527]: cluster 2024-03-26T03:53:40.735292+0000 mgr.y (mgr.24431) 371 : cluster [DBG] pgmap v542: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-26T03:53:43.739 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:43 smithi190 bash[23392]: cluster 2024-03-26T03:53:43.251942+0000 mon.a (mon.0) 2453 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-03-26T03:53:43.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:43 smithi175 bash[29284]: cluster 2024-03-26T03:53:43.251942+0000 mon.a (mon.0) 2453 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-03-26T03:53:43.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:43 smithi175 bash[21527]: cluster 2024-03-26T03:53:43.251942+0000 mon.a (mon.0) 2453 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-03-26T03:53:44.739 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:44 smithi190 bash[23392]: cluster 2024-03-26T03:53:42.736168+0000 mgr.y (mgr.24431) 372 : cluster [DBG] pgmap v544: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-26T03:53:44.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:44 smithi175 bash[21527]: cluster 2024-03-26T03:53:42.736168+0000 mgr.y (mgr.24431) 372 : cluster [DBG] pgmap v544: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-26T03:53:44.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:44 smithi175 bash[29284]: cluster 2024-03-26T03:53:42.736168+0000 mgr.y (mgr.24431) 372 : cluster [DBG] pgmap v544: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-26T03:53:46.739 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:46 smithi190 bash[23392]: cluster 2024-03-26T03:53:44.738275+0000 mgr.y (mgr.24431) 373 : cluster [DBG] pgmap v545: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-26T03:53:46.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:46 smithi175 bash[29284]: cluster 2024-03-26T03:53:44.738275+0000 mgr.y (mgr.24431) 373 : cluster [DBG] pgmap v545: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-26T03:53:46.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:46 smithi175 bash[21527]: cluster 2024-03-26T03:53:44.738275+0000 mgr.y (mgr.24431) 373 : cluster [DBG] pgmap v545: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-26T03:53:48.767 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:48 smithi175 bash[21527]: cluster 2024-03-26T03:53:46.740020+0000 mgr.y (mgr.24431) 374 : cluster [DBG] pgmap v546: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-26T03:53:48.767 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:48 smithi175 bash[29284]: cluster 2024-03-26T03:53:46.740020+0000 mgr.y (mgr.24431) 374 : cluster [DBG] pgmap v546: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-26T03:53:48.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:48 smithi190 bash[23392]: cluster 2024-03-26T03:53:46.740020+0000 mgr.y (mgr.24431) 374 : cluster [DBG] pgmap v546: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-26T03:53:49.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:53:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:53:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:53:50.239 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:53:49 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:53:50.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:50 smithi190 bash[23392]: cluster 2024-03-26T03:53:48.740949+0000 mgr.y (mgr.24431) 375 : cluster [DBG] pgmap v547: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 598 B/s rd, 0 B/s wr, 0 op/s 2024-03-26T03:53:50.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:50 smithi190 bash[23392]: audit 2024-03-26T03:53:49.813029+0000 mon.a (mon.0) 2454 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:53:50.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:50 smithi190 bash[23392]: audit 2024-03-26T03:53:49.816236+0000 mon.c (mon.2) 502 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:53:50.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:50 smithi175 bash[29284]: cluster 2024-03-26T03:53:48.740949+0000 mgr.y (mgr.24431) 375 : cluster [DBG] pgmap v547: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 598 B/s rd, 0 B/s wr, 0 op/s 2024-03-26T03:53:50.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:50 smithi175 bash[29284]: audit 2024-03-26T03:53:49.813029+0000 mon.a (mon.0) 2454 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:53:50.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:50 smithi175 bash[29284]: audit 2024-03-26T03:53:49.816236+0000 mon.c (mon.2) 502 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:53:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:50 smithi175 bash[21527]: cluster 2024-03-26T03:53:48.740949+0000 mgr.y (mgr.24431) 375 : cluster [DBG] pgmap v547: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 598 B/s rd, 0 B/s wr, 0 op/s 2024-03-26T03:53:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:50 smithi175 bash[21527]: audit 2024-03-26T03:53:49.813029+0000 mon.a (mon.0) 2454 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:53:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:50 smithi175 bash[21527]: audit 2024-03-26T03:53:49.816236+0000 mon.c (mon.2) 502 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:53:51.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:51 smithi190 bash[23392]: audit 2024-03-26T03:53:49.973484+0000 mgr.y (mgr.24431) 376 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:53:51.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:51 smithi190 bash[23392]: audit 2024-03-26T03:53:51.212872+0000 mon.a (mon.0) 2455 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:53:51.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:51 smithi175 bash[29284]: audit 2024-03-26T03:53:49.973484+0000 mgr.y (mgr.24431) 376 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:53:51.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:51 smithi175 bash[29284]: audit 2024-03-26T03:53:51.212872+0000 mon.a (mon.0) 2455 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:53:51.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:51 smithi175 bash[21527]: audit 2024-03-26T03:53:49.973484+0000 mgr.y (mgr.24431) 376 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:53:51.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:51 smithi175 bash[21527]: audit 2024-03-26T03:53:51.212872+0000 mon.a (mon.0) 2455 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:53:52.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:52 smithi190 bash[23392]: cluster 2024-03-26T03:53:50.743731+0000 mgr.y (mgr.24431) 377 : cluster [DBG] pgmap v548: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-26T03:53:52.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:52 smithi190 bash[23392]: audit 2024-03-26T03:53:51.528111+0000 mon.a (mon.0) 2456 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:53:52.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:52 smithi190 bash[23392]: cluster 2024-03-26T03:53:51.538649+0000 mon.a (mon.0) 2457 : cluster [DBG] osdmap e365: 8 total, 8 up, 8 in 2024-03-26T03:53:52.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:52 smithi190 bash[23392]: audit 2024-03-26T03:53:51.541672+0000 mon.a (mon.0) 2458 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-51"}]: dispatch 2024-03-26T03:53:52.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:52 smithi175 bash[21527]: cluster 2024-03-26T03:53:50.743731+0000 mgr.y (mgr.24431) 377 : cluster [DBG] pgmap v548: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-26T03:53:52.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:52 smithi175 bash[21527]: audit 2024-03-26T03:53:51.528111+0000 mon.a (mon.0) 2456 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:53:52.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:52 smithi175 bash[21527]: cluster 2024-03-26T03:53:51.538649+0000 mon.a (mon.0) 2457 : cluster [DBG] osdmap e365: 8 total, 8 up, 8 in 2024-03-26T03:53:52.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:52 smithi175 bash[21527]: audit 2024-03-26T03:53:51.541672+0000 mon.a (mon.0) 2458 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-51"}]: dispatch 2024-03-26T03:53:52.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:52 smithi175 bash[29284]: cluster 2024-03-26T03:53:50.743731+0000 mgr.y (mgr.24431) 377 : cluster [DBG] pgmap v548: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-26T03:53:52.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:52 smithi175 bash[29284]: audit 2024-03-26T03:53:51.528111+0000 mon.a (mon.0) 2456 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:53:52.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:52 smithi175 bash[29284]: cluster 2024-03-26T03:53:51.538649+0000 mon.a (mon.0) 2457 : cluster [DBG] osdmap e365: 8 total, 8 up, 8 in 2024-03-26T03:53:52.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:52 smithi175 bash[29284]: audit 2024-03-26T03:53:51.541672+0000 mon.a (mon.0) 2458 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-51"}]: dispatch 2024-03-26T03:53:53.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:53 smithi190 bash[23392]: audit 2024-03-26T03:53:52.530991+0000 mon.a (mon.0) 2459 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-51"}]': finished 2024-03-26T03:53:53.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:53 smithi190 bash[23392]: cluster 2024-03-26T03:53:52.540306+0000 mon.a (mon.0) 2460 : cluster [DBG] osdmap e366: 8 total, 8 up, 8 in 2024-03-26T03:53:53.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:53 smithi190 bash[23392]: audit 2024-03-26T03:53:52.624450+0000 mon.a (mon.0) 2461 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:53:53.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:53 smithi190 bash[23392]: audit 2024-03-26T03:53:52.625263+0000 mon.a (mon.0) 2462 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-51"}]: dispatch 2024-03-26T03:53:53.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:53 smithi190 bash[23392]: cluster 2024-03-26T03:53:52.744625+0000 mgr.y (mgr.24431) 378 : cluster [DBG] pgmap v551: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:53:53.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:53 smithi175 bash[21527]: audit 2024-03-26T03:53:52.530991+0000 mon.a (mon.0) 2459 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-51"}]': finished 2024-03-26T03:53:53.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:53 smithi175 bash[21527]: cluster 2024-03-26T03:53:52.540306+0000 mon.a (mon.0) 2460 : cluster [DBG] osdmap e366: 8 total, 8 up, 8 in 2024-03-26T03:53:53.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:53 smithi175 bash[21527]: audit 2024-03-26T03:53:52.624450+0000 mon.a (mon.0) 2461 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:53:53.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:53 smithi175 bash[21527]: audit 2024-03-26T03:53:52.625263+0000 mon.a (mon.0) 2462 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-51"}]: dispatch 2024-03-26T03:53:53.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:53 smithi175 bash[21527]: cluster 2024-03-26T03:53:52.744625+0000 mgr.y (mgr.24431) 378 : cluster [DBG] pgmap v551: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:53:53.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:53 smithi175 bash[29284]: audit 2024-03-26T03:53:52.530991+0000 mon.a (mon.0) 2459 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-51"}]': finished 2024-03-26T03:53:53.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:53 smithi175 bash[29284]: cluster 2024-03-26T03:53:52.540306+0000 mon.a (mon.0) 2460 : cluster [DBG] osdmap e366: 8 total, 8 up, 8 in 2024-03-26T03:53:53.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:53 smithi175 bash[29284]: audit 2024-03-26T03:53:52.624450+0000 mon.a (mon.0) 2461 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:53:53.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:53 smithi175 bash[29284]: audit 2024-03-26T03:53:52.625263+0000 mon.a (mon.0) 2462 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-51"}]: dispatch 2024-03-26T03:53:53.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:53 smithi175 bash[29284]: cluster 2024-03-26T03:53:52.744625+0000 mgr.y (mgr.24431) 378 : cluster [DBG] pgmap v551: 292 pgs: 292 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:53:54.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:54 smithi190 bash[23392]: cluster 2024-03-26T03:53:53.545955+0000 mon.a (mon.0) 2463 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-03-26T03:53:54.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:54 smithi190 bash[23392]: cluster 2024-03-26T03:53:53.564617+0000 mon.a (mon.0) 2464 : cluster [DBG] osdmap e367: 8 total, 8 up, 8 in 2024-03-26T03:53:54.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:54 smithi190 bash[23392]: audit 2024-03-26T03:53:53.708777+0000 mon.c (mon.2) 503 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:53:54.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:54 smithi190 bash[23392]: audit 2024-03-26T03:53:54.439168+0000 mon.c (mon.2) 504 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:53:54.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:54 smithi190 bash[23392]: audit 2024-03-26T03:53:54.440815+0000 mon.c (mon.2) 505 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:53:54.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:54 smithi190 bash[23392]: audit 2024-03-26T03:53:54.454492+0000 mon.a (mon.0) 2465 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:53:54.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:54 smithi175 bash[21527]: cluster 2024-03-26T03:53:53.545955+0000 mon.a (mon.0) 2463 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-03-26T03:53:54.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:54 smithi175 bash[21527]: cluster 2024-03-26T03:53:53.564617+0000 mon.a (mon.0) 2464 : cluster [DBG] osdmap e367: 8 total, 8 up, 8 in 2024-03-26T03:53:54.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:54 smithi175 bash[21527]: audit 2024-03-26T03:53:53.708777+0000 mon.c (mon.2) 503 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:53:54.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:54 smithi175 bash[21527]: audit 2024-03-26T03:53:54.439168+0000 mon.c (mon.2) 504 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:53:54.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:54 smithi175 bash[21527]: audit 2024-03-26T03:53:54.440815+0000 mon.c (mon.2) 505 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:53:54.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:54 smithi175 bash[21527]: audit 2024-03-26T03:53:54.454492+0000 mon.a (mon.0) 2465 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:53:54.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:54 smithi175 bash[29284]: cluster 2024-03-26T03:53:53.545955+0000 mon.a (mon.0) 2463 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-03-26T03:53:54.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:54 smithi175 bash[29284]: cluster 2024-03-26T03:53:53.564617+0000 mon.a (mon.0) 2464 : cluster [DBG] osdmap e367: 8 total, 8 up, 8 in 2024-03-26T03:53:54.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:54 smithi175 bash[29284]: audit 2024-03-26T03:53:53.708777+0000 mon.c (mon.2) 503 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:53:54.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:54 smithi175 bash[29284]: audit 2024-03-26T03:53:54.439168+0000 mon.c (mon.2) 504 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:53:54.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:54 smithi175 bash[29284]: audit 2024-03-26T03:53:54.440815+0000 mon.c (mon.2) 505 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:53:54.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:54 smithi175 bash[29284]: audit 2024-03-26T03:53:54.454492+0000 mon.a (mon.0) 2465 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:53:55.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:55 smithi190 bash[23392]: cluster 2024-03-26T03:53:54.576645+0000 mon.a (mon.0) 2466 : cluster [DBG] osdmap e368: 8 total, 8 up, 8 in 2024-03-26T03:53:55.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:55 smithi190 bash[23392]: audit 2024-03-26T03:53:54.579196+0000 mon.a (mon.0) 2467 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:53:55.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:55 smithi190 bash[23392]: cluster 2024-03-26T03:53:54.746757+0000 mgr.y (mgr.24431) 379 : cluster [DBG] pgmap v554: 292 pgs: 20 creating+peering, 12 unknown, 260 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:53:55.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:55 smithi175 bash[29284]: cluster 2024-03-26T03:53:54.576645+0000 mon.a (mon.0) 2466 : cluster [DBG] osdmap e368: 8 total, 8 up, 8 in 2024-03-26T03:53:55.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:55 smithi175 bash[29284]: audit 2024-03-26T03:53:54.579196+0000 mon.a (mon.0) 2467 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:53:55.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:55 smithi175 bash[29284]: cluster 2024-03-26T03:53:54.746757+0000 mgr.y (mgr.24431) 379 : cluster [DBG] pgmap v554: 292 pgs: 20 creating+peering, 12 unknown, 260 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:53:55.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:55 smithi175 bash[21527]: cluster 2024-03-26T03:53:54.576645+0000 mon.a (mon.0) 2466 : cluster [DBG] osdmap e368: 8 total, 8 up, 8 in 2024-03-26T03:53:55.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:55 smithi175 bash[21527]: audit 2024-03-26T03:53:54.579196+0000 mon.a (mon.0) 2467 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:53:55.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:55 smithi175 bash[21527]: cluster 2024-03-26T03:53:54.746757+0000 mgr.y (mgr.24431) 379 : cluster [DBG] pgmap v554: 292 pgs: 20 creating+peering, 12 unknown, 260 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:53:56.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:56 smithi190 bash[23392]: cluster 2024-03-26T03:53:55.571308+0000 mon.a (mon.0) 2468 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:53:56.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:56 smithi190 bash[23392]: audit 2024-03-26T03:53:55.573034+0000 mon.a (mon.0) 2469 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:53:56.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:56 smithi190 bash[23392]: cluster 2024-03-26T03:53:55.583311+0000 mon.a (mon.0) 2470 : cluster [DBG] osdmap e369: 8 total, 8 up, 8 in 2024-03-26T03:53:56.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:56 smithi190 bash[23392]: audit 2024-03-26T03:53:55.665734+0000 mon.a (mon.0) 2471 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:53:56.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:56 smithi190 bash[23392]: audit 2024-03-26T03:53:55.666370+0000 mon.a (mon.0) 2472 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-53"}]: dispatch 2024-03-26T03:53:56.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:56 smithi175 bash[21527]: cluster 2024-03-26T03:53:55.571308+0000 mon.a (mon.0) 2468 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:53:56.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:56 smithi175 bash[21527]: audit 2024-03-26T03:53:55.573034+0000 mon.a (mon.0) 2469 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:53:56.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:56 smithi175 bash[21527]: cluster 2024-03-26T03:53:55.583311+0000 mon.a (mon.0) 2470 : cluster [DBG] osdmap e369: 8 total, 8 up, 8 in 2024-03-26T03:53:56.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:56 smithi175 bash[21527]: audit 2024-03-26T03:53:55.665734+0000 mon.a (mon.0) 2471 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:53:56.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:56 smithi175 bash[21527]: audit 2024-03-26T03:53:55.666370+0000 mon.a (mon.0) 2472 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-53"}]: dispatch 2024-03-26T03:53:56.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:56 smithi175 bash[29284]: cluster 2024-03-26T03:53:55.571308+0000 mon.a (mon.0) 2468 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:53:56.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:56 smithi175 bash[29284]: audit 2024-03-26T03:53:55.573034+0000 mon.a (mon.0) 2469 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:53:56.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:56 smithi175 bash[29284]: cluster 2024-03-26T03:53:55.583311+0000 mon.a (mon.0) 2470 : cluster [DBG] osdmap e369: 8 total, 8 up, 8 in 2024-03-26T03:53:56.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:56 smithi175 bash[29284]: audit 2024-03-26T03:53:55.665734+0000 mon.a (mon.0) 2471 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:53:56.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:56 smithi175 bash[29284]: audit 2024-03-26T03:53:55.666370+0000 mon.a (mon.0) 2472 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-53"}]: dispatch 2024-03-26T03:53:57.989 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:57 smithi190 bash[23392]: cluster 2024-03-26T03:53:56.603588+0000 mon.a (mon.0) 2473 : cluster [DBG] osdmap e370: 8 total, 8 up, 8 in 2024-03-26T03:53:57.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:57 smithi190 bash[23392]: cluster 2024-03-26T03:53:56.747604+0000 mgr.y (mgr.24431) 380 : cluster [DBG] pgmap v557: 260 pgs: 260 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-26T03:53:57.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:57 smithi190 bash[23392]: cluster 2024-03-26T03:53:57.613165+0000 mon.a (mon.0) 2474 : cluster [DBG] osdmap e371: 8 total, 8 up, 8 in 2024-03-26T03:53:57.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:57 smithi190 bash[23392]: audit 2024-03-26T03:53:57.619640+0000 mon.a (mon.0) 2475 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:53:57.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:57 smithi175 bash[21527]: cluster 2024-03-26T03:53:56.603588+0000 mon.a (mon.0) 2473 : cluster [DBG] osdmap e370: 8 total, 8 up, 8 in 2024-03-26T03:53:57.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:57 smithi175 bash[21527]: cluster 2024-03-26T03:53:56.747604+0000 mgr.y (mgr.24431) 380 : cluster [DBG] pgmap v557: 260 pgs: 260 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-26T03:53:57.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:57 smithi175 bash[21527]: cluster 2024-03-26T03:53:57.613165+0000 mon.a (mon.0) 2474 : cluster [DBG] osdmap e371: 8 total, 8 up, 8 in 2024-03-26T03:53:57.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:57 smithi175 bash[21527]: audit 2024-03-26T03:53:57.619640+0000 mon.a (mon.0) 2475 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:53:57.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:57 smithi175 bash[29284]: cluster 2024-03-26T03:53:56.603588+0000 mon.a (mon.0) 2473 : cluster [DBG] osdmap e370: 8 total, 8 up, 8 in 2024-03-26T03:53:57.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:57 smithi175 bash[29284]: cluster 2024-03-26T03:53:56.747604+0000 mgr.y (mgr.24431) 380 : cluster [DBG] pgmap v557: 260 pgs: 260 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-26T03:53:57.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:57 smithi175 bash[29284]: cluster 2024-03-26T03:53:57.613165+0000 mon.a (mon.0) 2474 : cluster [DBG] osdmap e371: 8 total, 8 up, 8 in 2024-03-26T03:53:57.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:57 smithi175 bash[29284]: audit 2024-03-26T03:53:57.619640+0000 mon.a (mon.0) 2475 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:53:59.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:53:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:53:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:53:59.984 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:59 smithi190 bash[23392]: audit 2024-03-26T03:53:58.605578+0000 mon.a (mon.0) 2476 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:53:59.984 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:59 smithi190 bash[23392]: cluster 2024-03-26T03:53:58.613751+0000 mon.a (mon.0) 2477 : cluster [DBG] osdmap e372: 8 total, 8 up, 8 in 2024-03-26T03:53:59.984 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:59 smithi190 bash[23392]: audit 2024-03-26T03:53:58.620630+0000 mon.a (mon.0) 2478 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:53:59.984 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:59 smithi190 bash[23392]: audit 2024-03-26T03:53:58.717152+0000 mon.a (mon.0) 2479 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:53:59.984 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:59 smithi190 bash[23392]: audit 2024-03-26T03:53:58.717839+0000 mon.a (mon.0) 2480 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-55"}]: dispatch 2024-03-26T03:53:59.984 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:53:59 smithi190 bash[23392]: cluster 2024-03-26T03:53:58.748344+0000 mgr.y (mgr.24431) 381 : cluster [DBG] pgmap v560: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:53:59.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:59 smithi175 bash[21527]: audit 2024-03-26T03:53:58.605578+0000 mon.a (mon.0) 2476 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:53:59.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:59 smithi175 bash[21527]: cluster 2024-03-26T03:53:58.613751+0000 mon.a (mon.0) 2477 : cluster [DBG] osdmap e372: 8 total, 8 up, 8 in 2024-03-26T03:53:59.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:59 smithi175 bash[21527]: audit 2024-03-26T03:53:58.620630+0000 mon.a (mon.0) 2478 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:53:59.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:59 smithi175 bash[21527]: audit 2024-03-26T03:53:58.717152+0000 mon.a (mon.0) 2479 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:53:59.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:59 smithi175 bash[21527]: audit 2024-03-26T03:53:58.717839+0000 mon.a (mon.0) 2480 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-55"}]: dispatch 2024-03-26T03:53:59.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:53:59 smithi175 bash[21527]: cluster 2024-03-26T03:53:58.748344+0000 mgr.y (mgr.24431) 381 : cluster [DBG] pgmap v560: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:53:59.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:59 smithi175 bash[29284]: audit 2024-03-26T03:53:58.605578+0000 mon.a (mon.0) 2476 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:53:59.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:59 smithi175 bash[29284]: cluster 2024-03-26T03:53:58.613751+0000 mon.a (mon.0) 2477 : cluster [DBG] osdmap e372: 8 total, 8 up, 8 in 2024-03-26T03:53:59.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:59 smithi175 bash[29284]: audit 2024-03-26T03:53:58.620630+0000 mon.a (mon.0) 2478 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:53:59.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:59 smithi175 bash[29284]: audit 2024-03-26T03:53:58.717152+0000 mon.a (mon.0) 2479 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:53:59.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:59 smithi175 bash[29284]: audit 2024-03-26T03:53:58.717839+0000 mon.a (mon.0) 2480 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-55"}]: dispatch 2024-03-26T03:53:59.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:53:59 smithi175 bash[29284]: cluster 2024-03-26T03:53:58.748344+0000 mgr.y (mgr.24431) 381 : cluster [DBG] pgmap v560: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:54:00.239 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:53:59 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:54:00.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:00 smithi190 bash[23392]: cluster 2024-03-26T03:53:59.635596+0000 mon.a (mon.0) 2481 : cluster [DBG] osdmap e373: 8 total, 8 up, 8 in 2024-03-26T03:54:00.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:00 smithi190 bash[23392]: audit 2024-03-26T03:53:59.985124+0000 mgr.y (mgr.24431) 382 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:54:00.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:00 smithi175 bash[21527]: cluster 2024-03-26T03:53:59.635596+0000 mon.a (mon.0) 2481 : cluster [DBG] osdmap e373: 8 total, 8 up, 8 in 2024-03-26T03:54:00.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:00 smithi175 bash[21527]: audit 2024-03-26T03:53:59.985124+0000 mgr.y (mgr.24431) 382 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:54:00.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:00 smithi175 bash[29284]: cluster 2024-03-26T03:53:59.635596+0000 mon.a (mon.0) 2481 : cluster [DBG] osdmap e373: 8 total, 8 up, 8 in 2024-03-26T03:54:00.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:00 smithi175 bash[29284]: audit 2024-03-26T03:53:59.985124+0000 mgr.y (mgr.24431) 382 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:54:01.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:01 smithi190 bash[23392]: cluster 2024-03-26T03:54:00.634962+0000 mon.a (mon.0) 2482 : cluster [DBG] osdmap e374: 8 total, 8 up, 8 in 2024-03-26T03:54:01.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:01 smithi190 bash[23392]: audit 2024-03-26T03:54:00.649888+0000 mon.a (mon.0) 2483 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:54:01.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:01 smithi190 bash[23392]: cluster 2024-03-26T03:54:00.749184+0000 mgr.y (mgr.24431) 383 : cluster [DBG] pgmap v563: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-03-26T03:54:01.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:01 smithi190 bash[23392]: cluster 2024-03-26T03:54:01.508740+0000 mon.a (mon.0) 2484 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:54:01.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:01 smithi190 bash[23392]: audit 2024-03-26T03:54:01.624171+0000 mon.a (mon.0) 2485 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:54:01.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:01 smithi190 bash[23392]: cluster 2024-03-26T03:54:01.631776+0000 mon.a (mon.0) 2486 : cluster [DBG] osdmap e375: 8 total, 8 up, 8 in 2024-03-26T03:54:01.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:01 smithi175 bash[29284]: cluster 2024-03-26T03:54:00.634962+0000 mon.a (mon.0) 2482 : cluster [DBG] osdmap e374: 8 total, 8 up, 8 in 2024-03-26T03:54:01.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:01 smithi175 bash[29284]: audit 2024-03-26T03:54:00.649888+0000 mon.a (mon.0) 2483 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:54:01.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:01 smithi175 bash[29284]: cluster 2024-03-26T03:54:00.749184+0000 mgr.y (mgr.24431) 383 : cluster [DBG] pgmap v563: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-03-26T03:54:01.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:01 smithi175 bash[29284]: cluster 2024-03-26T03:54:01.508740+0000 mon.a (mon.0) 2484 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:54:01.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:01 smithi175 bash[29284]: audit 2024-03-26T03:54:01.624171+0000 mon.a (mon.0) 2485 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:54:01.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:01 smithi175 bash[29284]: cluster 2024-03-26T03:54:01.631776+0000 mon.a (mon.0) 2486 : cluster [DBG] osdmap e375: 8 total, 8 up, 8 in 2024-03-26T03:54:01.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:01 smithi175 bash[21527]: cluster 2024-03-26T03:54:00.634962+0000 mon.a (mon.0) 2482 : cluster [DBG] osdmap e374: 8 total, 8 up, 8 in 2024-03-26T03:54:01.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:01 smithi175 bash[21527]: audit 2024-03-26T03:54:00.649888+0000 mon.a (mon.0) 2483 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:54:01.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:01 smithi175 bash[21527]: cluster 2024-03-26T03:54:00.749184+0000 mgr.y (mgr.24431) 383 : cluster [DBG] pgmap v563: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-03-26T03:54:01.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:01 smithi175 bash[21527]: cluster 2024-03-26T03:54:01.508740+0000 mon.a (mon.0) 2484 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:54:01.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:01 smithi175 bash[21527]: audit 2024-03-26T03:54:01.624171+0000 mon.a (mon.0) 2485 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:54:01.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:01 smithi175 bash[21527]: cluster 2024-03-26T03:54:01.631776+0000 mon.a (mon.0) 2486 : cluster [DBG] osdmap e375: 8 total, 8 up, 8 in 2024-03-26T03:54:02.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:02 smithi190 bash[23392]: audit 2024-03-26T03:54:01.721910+0000 mon.a (mon.0) 2487 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:54:02.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:02 smithi190 bash[23392]: audit 2024-03-26T03:54:01.722435+0000 mon.a (mon.0) 2488 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-57"}]: dispatch 2024-03-26T03:54:02.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:02 smithi190 bash[23392]: cluster 2024-03-26T03:54:02.633810+0000 mon.a (mon.0) 2489 : cluster [DBG] osdmap e376: 8 total, 8 up, 8 in 2024-03-26T03:54:02.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:02 smithi175 bash[21527]: audit 2024-03-26T03:54:01.721910+0000 mon.a (mon.0) 2487 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:54:02.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:02 smithi175 bash[21527]: audit 2024-03-26T03:54:01.722435+0000 mon.a (mon.0) 2488 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-57"}]: dispatch 2024-03-26T03:54:02.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:02 smithi175 bash[21527]: cluster 2024-03-26T03:54:02.633810+0000 mon.a (mon.0) 2489 : cluster [DBG] osdmap e376: 8 total, 8 up, 8 in 2024-03-26T03:54:02.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:02 smithi175 bash[29284]: audit 2024-03-26T03:54:01.721910+0000 mon.a (mon.0) 2487 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:54:02.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:02 smithi175 bash[29284]: audit 2024-03-26T03:54:01.722435+0000 mon.a (mon.0) 2488 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-57"}]: dispatch 2024-03-26T03:54:02.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:02 smithi175 bash[29284]: cluster 2024-03-26T03:54:02.633810+0000 mon.a (mon.0) 2489 : cluster [DBG] osdmap e376: 8 total, 8 up, 8 in 2024-03-26T03:54:03.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:03 smithi190 bash[23392]: cluster 2024-03-26T03:54:02.750044+0000 mgr.y (mgr.24431) 384 : cluster [DBG] pgmap v566: 260 pgs: 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-03-26T03:54:03.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:03 smithi190 bash[23392]: cluster 2024-03-26T03:54:03.639248+0000 mon.a (mon.0) 2490 : cluster [DBG] osdmap e377: 8 total, 8 up, 8 in 2024-03-26T03:54:03.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:03 smithi190 bash[23392]: audit 2024-03-26T03:54:03.642817+0000 mon.a (mon.0) 2491 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:54:03.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:03 smithi175 bash[21527]: cluster 2024-03-26T03:54:02.750044+0000 mgr.y (mgr.24431) 384 : cluster [DBG] pgmap v566: 260 pgs: 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-03-26T03:54:03.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:03 smithi175 bash[21527]: cluster 2024-03-26T03:54:03.639248+0000 mon.a (mon.0) 2490 : cluster [DBG] osdmap e377: 8 total, 8 up, 8 in 2024-03-26T03:54:03.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:03 smithi175 bash[21527]: audit 2024-03-26T03:54:03.642817+0000 mon.a (mon.0) 2491 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:54:03.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:03 smithi175 bash[29284]: cluster 2024-03-26T03:54:02.750044+0000 mgr.y (mgr.24431) 384 : cluster [DBG] pgmap v566: 260 pgs: 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-03-26T03:54:03.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:03 smithi175 bash[29284]: cluster 2024-03-26T03:54:03.639248+0000 mon.a (mon.0) 2490 : cluster [DBG] osdmap e377: 8 total, 8 up, 8 in 2024-03-26T03:54:03.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:03 smithi175 bash[29284]: audit 2024-03-26T03:54:03.642817+0000 mon.a (mon.0) 2491 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:54:05.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:05 smithi190 bash[23392]: audit 2024-03-26T03:54:04.634009+0000 mon.a (mon.0) 2492 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:54:05.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:05 smithi190 bash[23392]: cluster 2024-03-26T03:54:04.642804+0000 mon.a (mon.0) 2493 : cluster [DBG] osdmap e378: 8 total, 8 up, 8 in 2024-03-26T03:54:05.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:05 smithi190 bash[23392]: audit 2024-03-26T03:54:04.648226+0000 mon.a (mon.0) 2494 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:54:05.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:05 smithi190 bash[23392]: audit 2024-03-26T03:54:04.743816+0000 mon.a (mon.0) 2495 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:54:05.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:05 smithi190 bash[23392]: audit 2024-03-26T03:54:04.744345+0000 mon.a (mon.0) 2496 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-59"}]: dispatch 2024-03-26T03:54:05.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:05 smithi190 bash[23392]: cluster 2024-03-26T03:54:04.751472+0000 mgr.y (mgr.24431) 385 : cluster [DBG] pgmap v569: 292 pgs: 2 creating+peering, 7 creating+activating, 16 unknown, 267 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 767 B/s wr, 0 op/s 2024-03-26T03:54:05.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:05 smithi190 bash[23392]: audit 2024-03-26T03:54:04.858786+0000 mon.a (mon.0) 2497 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:54:05.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:05 smithi190 bash[23392]: audit 2024-03-26T03:54:04.861755+0000 mon.c (mon.2) 506 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:54:05.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:05 smithi175 bash[21527]: audit 2024-03-26T03:54:04.634009+0000 mon.a (mon.0) 2492 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:54:05.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:05 smithi175 bash[21527]: cluster 2024-03-26T03:54:04.642804+0000 mon.a (mon.0) 2493 : cluster [DBG] osdmap e378: 8 total, 8 up, 8 in 2024-03-26T03:54:05.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:05 smithi175 bash[21527]: audit 2024-03-26T03:54:04.648226+0000 mon.a (mon.0) 2494 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:54:05.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:05 smithi175 bash[21527]: audit 2024-03-26T03:54:04.743816+0000 mon.a (mon.0) 2495 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:54:05.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:05 smithi175 bash[21527]: audit 2024-03-26T03:54:04.744345+0000 mon.a (mon.0) 2496 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-59"}]: dispatch 2024-03-26T03:54:05.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:05 smithi175 bash[21527]: cluster 2024-03-26T03:54:04.751472+0000 mgr.y (mgr.24431) 385 : cluster [DBG] pgmap v569: 292 pgs: 2 creating+peering, 7 creating+activating, 16 unknown, 267 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 767 B/s wr, 0 op/s 2024-03-26T03:54:05.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:05 smithi175 bash[21527]: audit 2024-03-26T03:54:04.858786+0000 mon.a (mon.0) 2497 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:54:05.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:05 smithi175 bash[21527]: audit 2024-03-26T03:54:04.861755+0000 mon.c (mon.2) 506 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:54:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:05 smithi175 bash[29284]: audit 2024-03-26T03:54:04.634009+0000 mon.a (mon.0) 2492 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:54:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:05 smithi175 bash[29284]: cluster 2024-03-26T03:54:04.642804+0000 mon.a (mon.0) 2493 : cluster [DBG] osdmap e378: 8 total, 8 up, 8 in 2024-03-26T03:54:05.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:05 smithi175 bash[29284]: audit 2024-03-26T03:54:04.648226+0000 mon.a (mon.0) 2494 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:54:05.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:05 smithi175 bash[29284]: audit 2024-03-26T03:54:04.743816+0000 mon.a (mon.0) 2495 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:54:05.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:05 smithi175 bash[29284]: audit 2024-03-26T03:54:04.744345+0000 mon.a (mon.0) 2496 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-59"}]: dispatch 2024-03-26T03:54:05.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:05 smithi175 bash[29284]: cluster 2024-03-26T03:54:04.751472+0000 mgr.y (mgr.24431) 385 : cluster [DBG] pgmap v569: 292 pgs: 2 creating+peering, 7 creating+activating, 16 unknown, 267 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 767 B/s wr, 0 op/s 2024-03-26T03:54:05.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:05 smithi175 bash[29284]: audit 2024-03-26T03:54:04.858786+0000 mon.a (mon.0) 2497 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:54:05.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:05 smithi175 bash[29284]: audit 2024-03-26T03:54:04.861755+0000 mon.c (mon.2) 506 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:54:06.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:06 smithi190 bash[23392]: cluster 2024-03-26T03:54:05.655549+0000 mon.a (mon.0) 2498 : cluster [DBG] osdmap e379: 8 total, 8 up, 8 in 2024-03-26T03:54:06.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:06 smithi175 bash[29284]: cluster 2024-03-26T03:54:05.655549+0000 mon.a (mon.0) 2498 : cluster [DBG] osdmap e379: 8 total, 8 up, 8 in 2024-03-26T03:54:06.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:06 smithi175 bash[21527]: cluster 2024-03-26T03:54:05.655549+0000 mon.a (mon.0) 2498 : cluster [DBG] osdmap e379: 8 total, 8 up, 8 in 2024-03-26T03:54:07.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:07 smithi190 bash[23392]: cluster 2024-03-26T03:54:06.670829+0000 mon.a (mon.0) 2499 : cluster [DBG] osdmap e380: 8 total, 8 up, 8 in 2024-03-26T03:54:07.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:07 smithi190 bash[23392]: audit 2024-03-26T03:54:06.673133+0000 mon.a (mon.0) 2500 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:54:07.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:07 smithi190 bash[23392]: cluster 2024-03-26T03:54:06.752265+0000 mgr.y (mgr.24431) 386 : cluster [DBG] pgmap v572: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-03-26T03:54:07.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:07 smithi175 bash[29284]: cluster 2024-03-26T03:54:06.670829+0000 mon.a (mon.0) 2499 : cluster [DBG] osdmap e380: 8 total, 8 up, 8 in 2024-03-26T03:54:07.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:07 smithi175 bash[29284]: audit 2024-03-26T03:54:06.673133+0000 mon.a (mon.0) 2500 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:54:07.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:07 smithi175 bash[29284]: cluster 2024-03-26T03:54:06.752265+0000 mgr.y (mgr.24431) 386 : cluster [DBG] pgmap v572: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-03-26T03:54:07.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:07 smithi175 bash[21527]: cluster 2024-03-26T03:54:06.670829+0000 mon.a (mon.0) 2499 : cluster [DBG] osdmap e380: 8 total, 8 up, 8 in 2024-03-26T03:54:07.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:07 smithi175 bash[21527]: audit 2024-03-26T03:54:06.673133+0000 mon.a (mon.0) 2500 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:54:07.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:07 smithi175 bash[21527]: cluster 2024-03-26T03:54:06.752265+0000 mgr.y (mgr.24431) 386 : cluster [DBG] pgmap v572: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-03-26T03:54:08.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:08 smithi190 bash[23392]: cluster 2024-03-26T03:54:07.661515+0000 mon.a (mon.0) 2501 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:54:08.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:08 smithi190 bash[23392]: audit 2024-03-26T03:54:07.671219+0000 mon.a (mon.0) 2502 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:54:08.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:08 smithi190 bash[23392]: cluster 2024-03-26T03:54:07.681937+0000 mon.a (mon.0) 2503 : cluster [DBG] osdmap e381: 8 total, 8 up, 8 in 2024-03-26T03:54:08.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:08 smithi190 bash[23392]: audit 2024-03-26T03:54:07.687791+0000 mon.a (mon.0) 2504 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:54:08.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:08 smithi190 bash[23392]: audit 2024-03-26T03:54:07.712768+0000 mon.a (mon.0) 2505 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-26T03:54:08.992 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:54:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:54:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:54:08.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:08 smithi175 bash[21527]: cluster 2024-03-26T03:54:07.661515+0000 mon.a (mon.0) 2501 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:54:08.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:08 smithi175 bash[21527]: audit 2024-03-26T03:54:07.671219+0000 mon.a (mon.0) 2502 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:54:08.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:08 smithi175 bash[21527]: cluster 2024-03-26T03:54:07.681937+0000 mon.a (mon.0) 2503 : cluster [DBG] osdmap e381: 8 total, 8 up, 8 in 2024-03-26T03:54:08.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:08 smithi175 bash[21527]: audit 2024-03-26T03:54:07.687791+0000 mon.a (mon.0) 2504 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:54:08.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:08 smithi175 bash[21527]: audit 2024-03-26T03:54:07.712768+0000 mon.a (mon.0) 2505 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-26T03:54:08.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:08 smithi175 bash[29284]: cluster 2024-03-26T03:54:07.661515+0000 mon.a (mon.0) 2501 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:54:08.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:08 smithi175 bash[29284]: audit 2024-03-26T03:54:07.671219+0000 mon.a (mon.0) 2502 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:54:08.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:08 smithi175 bash[29284]: cluster 2024-03-26T03:54:07.681937+0000 mon.a (mon.0) 2503 : cluster [DBG] osdmap e381: 8 total, 8 up, 8 in 2024-03-26T03:54:08.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:08 smithi175 bash[29284]: audit 2024-03-26T03:54:07.687791+0000 mon.a (mon.0) 2504 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:54:08.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:08 smithi175 bash[29284]: audit 2024-03-26T03:54:07.712768+0000 mon.a (mon.0) 2505 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-26T03:54:09.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:09 smithi190 bash[23392]: audit 2024-03-26T03:54:08.683158+0000 mon.a (mon.0) 2506 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-26T03:54:09.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:09 smithi190 bash[23392]: cluster 2024-03-26T03:54:08.689911+0000 mon.a (mon.0) 2507 : cluster [DBG] osdmap e382: 8 total, 8 up, 8 in 2024-03-26T03:54:09.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:09 smithi190 bash[23392]: cluster 2024-03-26T03:54:08.753132+0000 mgr.y (mgr.24431) 387 : cluster [DBG] pgmap v575: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:54:09.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:09 smithi190 bash[23392]: audit 2024-03-26T03:54:08.772883+0000 mon.a (mon.0) 2508 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:54:09.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:09 smithi190 bash[23392]: audit 2024-03-26T03:54:08.773965+0000 mon.a (mon.0) 2509 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-61"}]: dispatch 2024-03-26T03:54:09.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:09 smithi175 bash[21527]: audit 2024-03-26T03:54:08.683158+0000 mon.a (mon.0) 2506 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-26T03:54:09.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:09 smithi175 bash[21527]: cluster 2024-03-26T03:54:08.689911+0000 mon.a (mon.0) 2507 : cluster [DBG] osdmap e382: 8 total, 8 up, 8 in 2024-03-26T03:54:09.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:09 smithi175 bash[21527]: cluster 2024-03-26T03:54:08.753132+0000 mgr.y (mgr.24431) 387 : cluster [DBG] pgmap v575: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:54:09.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:09 smithi175 bash[21527]: audit 2024-03-26T03:54:08.772883+0000 mon.a (mon.0) 2508 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:54:09.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:09 smithi175 bash[21527]: audit 2024-03-26T03:54:08.773965+0000 mon.a (mon.0) 2509 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-61"}]: dispatch 2024-03-26T03:54:09.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:09 smithi175 bash[29284]: audit 2024-03-26T03:54:08.683158+0000 mon.a (mon.0) 2506 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-26T03:54:09.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:09 smithi175 bash[29284]: cluster 2024-03-26T03:54:08.689911+0000 mon.a (mon.0) 2507 : cluster [DBG] osdmap e382: 8 total, 8 up, 8 in 2024-03-26T03:54:09.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:09 smithi175 bash[29284]: cluster 2024-03-26T03:54:08.753132+0000 mgr.y (mgr.24431) 387 : cluster [DBG] pgmap v575: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:54:09.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:09 smithi175 bash[29284]: audit 2024-03-26T03:54:08.772883+0000 mon.a (mon.0) 2508 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:54:09.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:09 smithi175 bash[29284]: audit 2024-03-26T03:54:08.773965+0000 mon.a (mon.0) 2509 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-61"}]: dispatch 2024-03-26T03:54:10.490 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:54:10 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:54:10.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:10 smithi190 bash[23392]: cluster 2024-03-26T03:54:09.702453+0000 mon.a (mon.0) 2510 : cluster [DBG] osdmap e383: 8 total, 8 up, 8 in 2024-03-26T03:54:10.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:10 smithi190 bash[23392]: audit 2024-03-26T03:54:09.995765+0000 mgr.y (mgr.24431) 388 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:54:10.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:10 smithi175 bash[21527]: cluster 2024-03-26T03:54:09.702453+0000 mon.a (mon.0) 2510 : cluster [DBG] osdmap e383: 8 total, 8 up, 8 in 2024-03-26T03:54:10.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:10 smithi175 bash[21527]: audit 2024-03-26T03:54:09.995765+0000 mgr.y (mgr.24431) 388 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:54:10.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:10 smithi175 bash[29284]: cluster 2024-03-26T03:54:09.702453+0000 mon.a (mon.0) 2510 : cluster [DBG] osdmap e383: 8 total, 8 up, 8 in 2024-03-26T03:54:10.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:10 smithi175 bash[29284]: audit 2024-03-26T03:54:09.995765+0000 mgr.y (mgr.24431) 388 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:54:11.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:11 smithi190 bash[23392]: cluster 2024-03-26T03:54:10.714414+0000 mon.a (mon.0) 2511 : cluster [DBG] osdmap e384: 8 total, 8 up, 8 in 2024-03-26T03:54:11.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:11 smithi190 bash[23392]: audit 2024-03-26T03:54:10.717642+0000 mon.a (mon.0) 2512 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:54:11.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:11 smithi190 bash[23392]: cluster 2024-03-26T03:54:10.753893+0000 mgr.y (mgr.24431) 389 : cluster [DBG] pgmap v578: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-26T03:54:11.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:11 smithi175 bash[29284]: cluster 2024-03-26T03:54:10.714414+0000 mon.a (mon.0) 2511 : cluster [DBG] osdmap e384: 8 total, 8 up, 8 in 2024-03-26T03:54:11.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:11 smithi175 bash[29284]: audit 2024-03-26T03:54:10.717642+0000 mon.a (mon.0) 2512 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:54:11.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:11 smithi175 bash[29284]: cluster 2024-03-26T03:54:10.753893+0000 mgr.y (mgr.24431) 389 : cluster [DBG] pgmap v578: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-26T03:54:11.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:11 smithi175 bash[21527]: cluster 2024-03-26T03:54:10.714414+0000 mon.a (mon.0) 2511 : cluster [DBG] osdmap e384: 8 total, 8 up, 8 in 2024-03-26T03:54:11.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:11 smithi175 bash[21527]: audit 2024-03-26T03:54:10.717642+0000 mon.a (mon.0) 2512 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:54:11.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:11 smithi175 bash[21527]: cluster 2024-03-26T03:54:10.753893+0000 mgr.y (mgr.24431) 389 : cluster [DBG] pgmap v578: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-26T03:54:12.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:12 smithi190 bash[23392]: audit 2024-03-26T03:54:11.716061+0000 mon.a (mon.0) 2513 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:54:12.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:12 smithi190 bash[23392]: cluster 2024-03-26T03:54:11.725744+0000 mon.a (mon.0) 2514 : cluster [DBG] osdmap e385: 8 total, 8 up, 8 in 2024-03-26T03:54:12.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:12 smithi190 bash[23392]: audit 2024-03-26T03:54:11.729858+0000 mon.a (mon.0) 2515 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:54:12.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:12 smithi190 bash[23392]: audit 2024-03-26T03:54:11.751649+0000 mon.a (mon.0) 2516 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-26T03:54:12.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:12 smithi175 bash[21527]: audit 2024-03-26T03:54:11.716061+0000 mon.a (mon.0) 2513 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:54:12.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:12 smithi175 bash[21527]: cluster 2024-03-26T03:54:11.725744+0000 mon.a (mon.0) 2514 : cluster [DBG] osdmap e385: 8 total, 8 up, 8 in 2024-03-26T03:54:12.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:12 smithi175 bash[21527]: audit 2024-03-26T03:54:11.729858+0000 mon.a (mon.0) 2515 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:54:12.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:12 smithi175 bash[21527]: audit 2024-03-26T03:54:11.751649+0000 mon.a (mon.0) 2516 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-26T03:54:12.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:12 smithi175 bash[29284]: audit 2024-03-26T03:54:11.716061+0000 mon.a (mon.0) 2513 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:54:12.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:12 smithi175 bash[29284]: cluster 2024-03-26T03:54:11.725744+0000 mon.a (mon.0) 2514 : cluster [DBG] osdmap e385: 8 total, 8 up, 8 in 2024-03-26T03:54:12.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:12 smithi175 bash[29284]: audit 2024-03-26T03:54:11.729858+0000 mon.a (mon.0) 2515 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:54:12.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:12 smithi175 bash[29284]: audit 2024-03-26T03:54:11.751649+0000 mon.a (mon.0) 2516 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-26T03:54:13.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:13 smithi190 bash[23392]: audit 2024-03-26T03:54:12.718895+0000 mon.a (mon.0) 2517 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-26T03:54:13.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:13 smithi190 bash[23392]: cluster 2024-03-26T03:54:12.726462+0000 mon.a (mon.0) 2518 : cluster [DBG] osdmap e386: 8 total, 8 up, 8 in 2024-03-26T03:54:13.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:13 smithi190 bash[23392]: cluster 2024-03-26T03:54:12.754739+0000 mgr.y (mgr.24431) 390 : cluster [DBG] pgmap v581: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-26T03:54:13.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:13 smithi175 bash[21527]: audit 2024-03-26T03:54:12.718895+0000 mon.a (mon.0) 2517 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-26T03:54:13.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:13 smithi175 bash[21527]: cluster 2024-03-26T03:54:12.726462+0000 mon.a (mon.0) 2518 : cluster [DBG] osdmap e386: 8 total, 8 up, 8 in 2024-03-26T03:54:13.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:13 smithi175 bash[21527]: cluster 2024-03-26T03:54:12.754739+0000 mgr.y (mgr.24431) 390 : cluster [DBG] pgmap v581: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-26T03:54:13.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:13 smithi175 bash[29284]: audit 2024-03-26T03:54:12.718895+0000 mon.a (mon.0) 2517 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-26T03:54:13.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:13 smithi175 bash[29284]: cluster 2024-03-26T03:54:12.726462+0000 mon.a (mon.0) 2518 : cluster [DBG] osdmap e386: 8 total, 8 up, 8 in 2024-03-26T03:54:13.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:13 smithi175 bash[29284]: cluster 2024-03-26T03:54:12.754739+0000 mgr.y (mgr.24431) 390 : cluster [DBG] pgmap v581: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-26T03:54:14.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:14 smithi190 bash[23392]: cluster 2024-03-26T03:54:13.729818+0000 mon.a (mon.0) 2519 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:54:14.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:14 smithi190 bash[23392]: cluster 2024-03-26T03:54:13.748386+0000 mon.a (mon.0) 2520 : cluster [DBG] osdmap e387: 8 total, 8 up, 8 in 2024-03-26T03:54:14.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:14 smithi175 bash[21527]: cluster 2024-03-26T03:54:13.729818+0000 mon.a (mon.0) 2519 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:54:14.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:14 smithi175 bash[21527]: cluster 2024-03-26T03:54:13.748386+0000 mon.a (mon.0) 2520 : cluster [DBG] osdmap e387: 8 total, 8 up, 8 in 2024-03-26T03:54:14.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:14 smithi175 bash[29284]: cluster 2024-03-26T03:54:13.729818+0000 mon.a (mon.0) 2519 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:54:14.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:14 smithi175 bash[29284]: cluster 2024-03-26T03:54:13.748386+0000 mon.a (mon.0) 2520 : cluster [DBG] osdmap e387: 8 total, 8 up, 8 in 2024-03-26T03:54:16.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:15 smithi190 bash[23392]: cluster 2024-03-26T03:54:14.755731+0000 mgr.y (mgr.24431) 391 : cluster [DBG] pgmap v584: 292 pgs: 12 unknown, 280 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 767 B/s wr, 2 op/s 2024-03-26T03:54:16.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:15 smithi190 bash[23392]: cluster 2024-03-26T03:54:14.758494+0000 mon.a (mon.0) 2521 : cluster [DBG] osdmap e388: 8 total, 8 up, 8 in 2024-03-26T03:54:16.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:15 smithi175 bash[29284]: cluster 2024-03-26T03:54:14.755731+0000 mgr.y (mgr.24431) 391 : cluster [DBG] pgmap v584: 292 pgs: 12 unknown, 280 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 767 B/s wr, 2 op/s 2024-03-26T03:54:16.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:15 smithi175 bash[29284]: cluster 2024-03-26T03:54:14.758494+0000 mon.a (mon.0) 2521 : cluster [DBG] osdmap e388: 8 total, 8 up, 8 in 2024-03-26T03:54:16.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:15 smithi175 bash[21527]: cluster 2024-03-26T03:54:14.755731+0000 mgr.y (mgr.24431) 391 : cluster [DBG] pgmap v584: 292 pgs: 12 unknown, 280 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 767 B/s wr, 2 op/s 2024-03-26T03:54:16.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:15 smithi175 bash[21527]: cluster 2024-03-26T03:54:14.758494+0000 mon.a (mon.0) 2521 : cluster [DBG] osdmap e388: 8 total, 8 up, 8 in 2024-03-26T03:54:17.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:16 smithi190 bash[23392]: cluster 2024-03-26T03:54:15.763412+0000 mon.a (mon.0) 2522 : cluster [DBG] osdmap e389: 8 total, 8 up, 8 in 2024-03-26T03:54:17.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:16 smithi175 bash[21527]: cluster 2024-03-26T03:54:15.763412+0000 mon.a (mon.0) 2522 : cluster [DBG] osdmap e389: 8 total, 8 up, 8 in 2024-03-26T03:54:17.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:16 smithi175 bash[29284]: cluster 2024-03-26T03:54:15.763412+0000 mon.a (mon.0) 2522 : cluster [DBG] osdmap e389: 8 total, 8 up, 8 in 2024-03-26T03:54:18.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:17 smithi190 bash[23392]: cluster 2024-03-26T03:54:16.756589+0000 mgr.y (mgr.24431) 392 : cluster [DBG] pgmap v586: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-03-26T03:54:18.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:17 smithi190 bash[23392]: cluster 2024-03-26T03:54:16.789391+0000 mon.a (mon.0) 2523 : cluster [DBG] osdmap e390: 8 total, 8 up, 8 in 2024-03-26T03:54:18.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:17 smithi175 bash[29284]: cluster 2024-03-26T03:54:16.756589+0000 mgr.y (mgr.24431) 392 : cluster [DBG] pgmap v586: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-03-26T03:54:18.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:17 smithi175 bash[29284]: cluster 2024-03-26T03:54:16.789391+0000 mon.a (mon.0) 2523 : cluster [DBG] osdmap e390: 8 total, 8 up, 8 in 2024-03-26T03:54:18.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:17 smithi175 bash[21527]: cluster 2024-03-26T03:54:16.756589+0000 mgr.y (mgr.24431) 392 : cluster [DBG] pgmap v586: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-03-26T03:54:18.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:17 smithi175 bash[21527]: cluster 2024-03-26T03:54:16.789391+0000 mon.a (mon.0) 2523 : cluster [DBG] osdmap e390: 8 total, 8 up, 8 in 2024-03-26T03:54:19.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:54:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:54:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:54:20.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:19 smithi190 bash[23392]: cluster 2024-03-26T03:54:18.757407+0000 mgr.y (mgr.24431) 393 : cluster [DBG] pgmap v588: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-03-26T03:54:20.240 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:54:20 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:54:20.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:19 smithi175 bash[21527]: cluster 2024-03-26T03:54:18.757407+0000 mgr.y (mgr.24431) 393 : cluster [DBG] pgmap v588: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-03-26T03:54:20.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:19 smithi175 bash[29284]: cluster 2024-03-26T03:54:18.757407+0000 mgr.y (mgr.24431) 393 : cluster [DBG] pgmap v588: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-03-26T03:54:21.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:20 smithi190 bash[23392]: audit 2024-03-26T03:54:19.897434+0000 mon.c (mon.2) 507 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:54:21.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:20 smithi190 bash[23392]: audit 2024-03-26T03:54:20.004039+0000 mgr.y (mgr.24431) 394 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:54:21.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:20 smithi175 bash[21527]: audit 2024-03-26T03:54:19.897434+0000 mon.c (mon.2) 507 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:54:21.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:20 smithi175 bash[21527]: audit 2024-03-26T03:54:20.004039+0000 mgr.y (mgr.24431) 394 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:54:21.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:20 smithi175 bash[29284]: audit 2024-03-26T03:54:19.897434+0000 mon.c (mon.2) 507 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:54:21.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:20 smithi175 bash[29284]: audit 2024-03-26T03:54:20.004039+0000 mgr.y (mgr.24431) 394 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:54:22.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:22 smithi190 bash[23392]: cluster 2024-03-26T03:54:20.760260+0000 mgr.y (mgr.24431) 395 : cluster [DBG] pgmap v589: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-03-26T03:54:22.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:22 smithi190 bash[23392]: cluster 2024-03-26T03:54:21.525196+0000 mon.a (mon.0) 2524 : cluster [DBG] osdmap e391: 8 total, 8 up, 8 in 2024-03-26T03:54:22.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:22 smithi175 bash[29284]: cluster 2024-03-26T03:54:20.760260+0000 mgr.y (mgr.24431) 395 : cluster [DBG] pgmap v589: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-03-26T03:54:22.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:22 smithi175 bash[29284]: cluster 2024-03-26T03:54:21.525196+0000 mon.a (mon.0) 2524 : cluster [DBG] osdmap e391: 8 total, 8 up, 8 in 2024-03-26T03:54:22.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:22 smithi175 bash[21527]: cluster 2024-03-26T03:54:20.760260+0000 mgr.y (mgr.24431) 395 : cluster [DBG] pgmap v589: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-03-26T03:54:22.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:22 smithi175 bash[21527]: cluster 2024-03-26T03:54:21.525196+0000 mon.a (mon.0) 2524 : cluster [DBG] osdmap e391: 8 total, 8 up, 8 in 2024-03-26T03:54:24.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:24 smithi190 bash[23392]: cluster 2024-03-26T03:54:22.761136+0000 mgr.y (mgr.24431) 396 : cluster [DBG] pgmap v591: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 292 B/s wr, 2 op/s 2024-03-26T03:54:24.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:24 smithi175 bash[21527]: cluster 2024-03-26T03:54:22.761136+0000 mgr.y (mgr.24431) 396 : cluster [DBG] pgmap v591: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 292 B/s wr, 2 op/s 2024-03-26T03:54:24.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:24 smithi175 bash[29284]: cluster 2024-03-26T03:54:22.761136+0000 mgr.y (mgr.24431) 396 : cluster [DBG] pgmap v591: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 292 B/s wr, 2 op/s 2024-03-26T03:54:26.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:26 smithi190 bash[23392]: cluster 2024-03-26T03:54:24.763233+0000 mgr.y (mgr.24431) 397 : cluster [DBG] pgmap v592: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 37 KiB/s rd, 255 B/s wr, 60 op/s 2024-03-26T03:54:26.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:26 smithi175 bash[21527]: cluster 2024-03-26T03:54:24.763233+0000 mgr.y (mgr.24431) 397 : cluster [DBG] pgmap v592: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 37 KiB/s rd, 255 B/s wr, 60 op/s 2024-03-26T03:54:26.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:26 smithi175 bash[29284]: cluster 2024-03-26T03:54:24.763233+0000 mgr.y (mgr.24431) 397 : cluster [DBG] pgmap v592: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 37 KiB/s rd, 255 B/s wr, 60 op/s 2024-03-26T03:54:28.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:28 smithi190 bash[23392]: cluster 2024-03-26T03:54:26.764787+0000 mgr.y (mgr.24431) 398 : cluster [DBG] pgmap v593: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 44 KiB/s rd, 205 B/s wr, 73 op/s 2024-03-26T03:54:28.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:28 smithi190 bash[23392]: cluster 2024-03-26T03:54:27.544969+0000 mon.a (mon.0) 2525 : cluster [DBG] osdmap e392: 8 total, 8 up, 8 in 2024-03-26T03:54:28.992 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:54:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:54:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:54:28.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:28 smithi175 bash[29284]: cluster 2024-03-26T03:54:26.764787+0000 mgr.y (mgr.24431) 398 : cluster [DBG] pgmap v593: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 44 KiB/s rd, 205 B/s wr, 73 op/s 2024-03-26T03:54:28.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:28 smithi175 bash[29284]: cluster 2024-03-26T03:54:27.544969+0000 mon.a (mon.0) 2525 : cluster [DBG] osdmap e392: 8 total, 8 up, 8 in 2024-03-26T03:54:28.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:28 smithi175 bash[21527]: cluster 2024-03-26T03:54:26.764787+0000 mgr.y (mgr.24431) 398 : cluster [DBG] pgmap v593: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 44 KiB/s rd, 205 B/s wr, 73 op/s 2024-03-26T03:54:28.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:28 smithi175 bash[21527]: cluster 2024-03-26T03:54:27.544969+0000 mon.a (mon.0) 2525 : cluster [DBG] osdmap e392: 8 total, 8 up, 8 in 2024-03-26T03:54:30.490 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:54:30 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:54:30.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:30 smithi190 bash[23392]: cluster 2024-03-26T03:54:28.765669+0000 mgr.y (mgr.24431) 399 : cluster [DBG] pgmap v595: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s rd, 0 B/s wr, 89 op/s 2024-03-26T03:54:30.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:30 smithi190 bash[23392]: audit 2024-03-26T03:54:30.015682+0000 mgr.y (mgr.24431) 400 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:54:30.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:30 smithi175 bash[29284]: cluster 2024-03-26T03:54:28.765669+0000 mgr.y (mgr.24431) 399 : cluster [DBG] pgmap v595: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s rd, 0 B/s wr, 89 op/s 2024-03-26T03:54:30.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:30 smithi175 bash[29284]: audit 2024-03-26T03:54:30.015682+0000 mgr.y (mgr.24431) 400 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:54:30.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:30 smithi175 bash[21527]: cluster 2024-03-26T03:54:28.765669+0000 mgr.y (mgr.24431) 399 : cluster [DBG] pgmap v595: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s rd, 0 B/s wr, 89 op/s 2024-03-26T03:54:30.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:30 smithi175 bash[21527]: audit 2024-03-26T03:54:30.015682+0000 mgr.y (mgr.24431) 400 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:54:31.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:31 smithi190 bash[23392]: cluster 2024-03-26T03:54:30.768588+0000 mgr.y (mgr.24431) 401 : cluster [DBG] pgmap v596: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s rd, 0 B/s wr, 78 op/s 2024-03-26T03:54:31.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:31 smithi175 bash[29284]: cluster 2024-03-26T03:54:30.768588+0000 mgr.y (mgr.24431) 401 : cluster [DBG] pgmap v596: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s rd, 0 B/s wr, 78 op/s 2024-03-26T03:54:31.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:31 smithi175 bash[21527]: cluster 2024-03-26T03:54:30.768588+0000 mgr.y (mgr.24431) 401 : cluster [DBG] pgmap v596: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s rd, 0 B/s wr, 78 op/s 2024-03-26T03:54:34.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:33 smithi190 bash[23392]: cluster 2024-03-26T03:54:32.769648+0000 mgr.y (mgr.24431) 402 : cluster [DBG] pgmap v597: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 72 op/s 2024-03-26T03:54:34.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:33 smithi175 bash[21527]: cluster 2024-03-26T03:54:32.769648+0000 mgr.y (mgr.24431) 402 : cluster [DBG] pgmap v597: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 72 op/s 2024-03-26T03:54:34.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:33 smithi175 bash[29284]: cluster 2024-03-26T03:54:32.769648+0000 mgr.y (mgr.24431) 402 : cluster [DBG] pgmap v597: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 72 op/s 2024-03-26T03:54:36.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:35 smithi190 bash[23392]: cluster 2024-03-26T03:54:34.771825+0000 mgr.y (mgr.24431) 403 : cluster [DBG] pgmap v598: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 25 op/s 2024-03-26T03:54:36.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:35 smithi190 bash[23392]: audit 2024-03-26T03:54:34.931655+0000 mon.c (mon.2) 508 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:54:36.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:35 smithi175 bash[21527]: cluster 2024-03-26T03:54:34.771825+0000 mgr.y (mgr.24431) 403 : cluster [DBG] pgmap v598: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 25 op/s 2024-03-26T03:54:36.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:35 smithi175 bash[21527]: audit 2024-03-26T03:54:34.931655+0000 mon.c (mon.2) 508 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:54:36.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:35 smithi175 bash[29284]: cluster 2024-03-26T03:54:34.771825+0000 mgr.y (mgr.24431) 403 : cluster [DBG] pgmap v598: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 25 op/s 2024-03-26T03:54:36.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:35 smithi175 bash[29284]: audit 2024-03-26T03:54:34.931655+0000 mon.c (mon.2) 508 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:54:37.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:37 smithi190 bash[23392]: cluster 2024-03-26T03:54:36.536893+0000 mon.a (mon.0) 2526 : cluster [DBG] osdmap e393: 8 total, 8 up, 8 in 2024-03-26T03:54:37.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:37 smithi175 bash[29284]: cluster 2024-03-26T03:54:36.536893+0000 mon.a (mon.0) 2526 : cluster [DBG] osdmap e393: 8 total, 8 up, 8 in 2024-03-26T03:54:37.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:37 smithi175 bash[21527]: cluster 2024-03-26T03:54:36.536893+0000 mon.a (mon.0) 2526 : cluster [DBG] osdmap e393: 8 total, 8 up, 8 in 2024-03-26T03:54:38.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:38 smithi190 bash[23392]: cluster 2024-03-26T03:54:36.772733+0000 mgr.y (mgr.24431) 404 : cluster [DBG] pgmap v600: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-26T03:54:38.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:38 smithi175 bash[29284]: cluster 2024-03-26T03:54:36.772733+0000 mgr.y (mgr.24431) 404 : cluster [DBG] pgmap v600: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-26T03:54:38.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:38 smithi175 bash[21527]: cluster 2024-03-26T03:54:36.772733+0000 mgr.y (mgr.24431) 404 : cluster [DBG] pgmap v600: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-26T03:54:38.993 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:54:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:54:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:54:39.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:39 smithi190 bash[23392]: cluster 2024-03-26T03:54:38.554045+0000 mon.a (mon.0) 2527 : cluster [DBG] osdmap e394: 8 total, 8 up, 8 in 2024-03-26T03:54:39.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:39 smithi190 bash[23392]: cluster 2024-03-26T03:54:38.773740+0000 mgr.y (mgr.24431) 405 : cluster [DBG] pgmap v602: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-26T03:54:39.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:39 smithi175 bash[29284]: cluster 2024-03-26T03:54:38.554045+0000 mon.a (mon.0) 2527 : cluster [DBG] osdmap e394: 8 total, 8 up, 8 in 2024-03-26T03:54:39.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:39 smithi175 bash[29284]: cluster 2024-03-26T03:54:38.773740+0000 mgr.y (mgr.24431) 405 : cluster [DBG] pgmap v602: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-26T03:54:39.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:39 smithi175 bash[21527]: cluster 2024-03-26T03:54:38.554045+0000 mon.a (mon.0) 2527 : cluster [DBG] osdmap e394: 8 total, 8 up, 8 in 2024-03-26T03:54:39.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:39 smithi175 bash[21527]: cluster 2024-03-26T03:54:38.773740+0000 mgr.y (mgr.24431) 405 : cluster [DBG] pgmap v602: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-26T03:54:40.490 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:54:40 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:54:40.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:40 smithi190 bash[23392]: audit 2024-03-26T03:54:40.024355+0000 mgr.y (mgr.24431) 406 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:54:40.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:40 smithi175 bash[29284]: audit 2024-03-26T03:54:40.024355+0000 mgr.y (mgr.24431) 406 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:54:40.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:40 smithi175 bash[21527]: audit 2024-03-26T03:54:40.024355+0000 mgr.y (mgr.24431) 406 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:54:41.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:41 smithi190 bash[23392]: cluster 2024-03-26T03:54:40.776581+0000 mgr.y (mgr.24431) 407 : cluster [DBG] pgmap v603: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-26T03:54:41.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:41 smithi175 bash[29284]: cluster 2024-03-26T03:54:40.776581+0000 mgr.y (mgr.24431) 407 : cluster [DBG] pgmap v603: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-26T03:54:41.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:41 smithi175 bash[21527]: cluster 2024-03-26T03:54:40.776581+0000 mgr.y (mgr.24431) 407 : cluster [DBG] pgmap v603: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-26T03:54:44.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:43 smithi190 bash[23392]: cluster 2024-03-26T03:54:42.777625+0000 mgr.y (mgr.24431) 408 : cluster [DBG] pgmap v604: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-26T03:54:44.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:43 smithi175 bash[29284]: cluster 2024-03-26T03:54:42.777625+0000 mgr.y (mgr.24431) 408 : cluster [DBG] pgmap v604: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-26T03:54:44.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:43 smithi175 bash[21527]: cluster 2024-03-26T03:54:42.777625+0000 mgr.y (mgr.24431) 408 : cluster [DBG] pgmap v604: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-26T03:54:46.240 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:45 smithi190 bash[23392]: cluster 2024-03-26T03:54:44.779776+0000 mgr.y (mgr.24431) 409 : cluster [DBG] pgmap v605: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 744 B/s rd, 0 op/s 2024-03-26T03:54:46.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:45 smithi175 bash[21527]: cluster 2024-03-26T03:54:44.779776+0000 mgr.y (mgr.24431) 409 : cluster [DBG] pgmap v605: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 744 B/s rd, 0 op/s 2024-03-26T03:54:46.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:45 smithi175 bash[29284]: cluster 2024-03-26T03:54:44.779776+0000 mgr.y (mgr.24431) 409 : cluster [DBG] pgmap v605: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 744 B/s rd, 0 op/s 2024-03-26T03:54:47.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:47 smithi190 bash[23392]: cluster 2024-03-26T03:54:46.534953+0000 mon.a (mon.0) 2528 : cluster [DBG] osdmap e395: 8 total, 8 up, 8 in 2024-03-26T03:54:47.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:47 smithi175 bash[29284]: cluster 2024-03-26T03:54:46.534953+0000 mon.a (mon.0) 2528 : cluster [DBG] osdmap e395: 8 total, 8 up, 8 in 2024-03-26T03:54:47.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:47 smithi175 bash[21527]: cluster 2024-03-26T03:54:46.534953+0000 mon.a (mon.0) 2528 : cluster [DBG] osdmap e395: 8 total, 8 up, 8 in 2024-03-26T03:54:48.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:48 smithi190 bash[23392]: cluster 2024-03-26T03:54:46.780918+0000 mgr.y (mgr.24431) 410 : cluster [DBG] pgmap v607: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-26T03:54:48.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:48 smithi175 bash[21527]: cluster 2024-03-26T03:54:46.780918+0000 mgr.y (mgr.24431) 410 : cluster [DBG] pgmap v607: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-26T03:54:48.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:48 smithi175 bash[29284]: cluster 2024-03-26T03:54:46.780918+0000 mgr.y (mgr.24431) 410 : cluster [DBG] pgmap v607: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-26T03:54:48.993 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:54:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:54:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:54:49.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:49 smithi190 bash[23392]: audit 2024-03-26T03:54:48.596828+0000 mon.a (mon.0) 2529 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:54:49.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:49 smithi190 bash[23392]: audit 2024-03-26T03:54:48.597725+0000 mon.a (mon.0) 2530 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-63"}]: dispatch 2024-03-26T03:54:49.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:49 smithi175 bash[21527]: audit 2024-03-26T03:54:48.596828+0000 mon.a (mon.0) 2529 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:54:49.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:49 smithi175 bash[21527]: audit 2024-03-26T03:54:48.597725+0000 mon.a (mon.0) 2530 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-63"}]: dispatch 2024-03-26T03:54:49.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:49 smithi175 bash[29284]: audit 2024-03-26T03:54:48.596828+0000 mon.a (mon.0) 2529 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:54:49.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:49 smithi175 bash[29284]: audit 2024-03-26T03:54:48.597725+0000 mon.a (mon.0) 2530 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-63"}]: dispatch 2024-03-26T03:54:50.490 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:54:50 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:54:50.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:50 smithi190 bash[23392]: cluster 2024-03-26T03:54:48.781858+0000 mgr.y (mgr.24431) 411 : cluster [DBG] pgmap v608: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-26T03:54:50.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:50 smithi190 bash[23392]: cluster 2024-03-26T03:54:49.562989+0000 mon.a (mon.0) 2531 : cluster [DBG] osdmap e396: 8 total, 8 up, 8 in 2024-03-26T03:54:50.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:50 smithi190 bash[23392]: audit 2024-03-26T03:54:49.966908+0000 mon.c (mon.2) 509 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:54:50.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:50 smithi190 bash[23392]: audit 2024-03-26T03:54:50.032560+0000 mgr.y (mgr.24431) 412 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:54:50.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:50 smithi175 bash[21527]: cluster 2024-03-26T03:54:48.781858+0000 mgr.y (mgr.24431) 411 : cluster [DBG] pgmap v608: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-26T03:54:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:50 smithi175 bash[21527]: cluster 2024-03-26T03:54:49.562989+0000 mon.a (mon.0) 2531 : cluster [DBG] osdmap e396: 8 total, 8 up, 8 in 2024-03-26T03:54:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:50 smithi175 bash[21527]: audit 2024-03-26T03:54:49.966908+0000 mon.c (mon.2) 509 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:54:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:50 smithi175 bash[21527]: audit 2024-03-26T03:54:50.032560+0000 mgr.y (mgr.24431) 412 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:54:50.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:50 smithi175 bash[29284]: cluster 2024-03-26T03:54:48.781858+0000 mgr.y (mgr.24431) 411 : cluster [DBG] pgmap v608: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-26T03:54:50.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:50 smithi175 bash[29284]: cluster 2024-03-26T03:54:49.562989+0000 mon.a (mon.0) 2531 : cluster [DBG] osdmap e396: 8 total, 8 up, 8 in 2024-03-26T03:54:50.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:50 smithi175 bash[29284]: audit 2024-03-26T03:54:49.966908+0000 mon.c (mon.2) 509 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:54:50.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:50 smithi175 bash[29284]: audit 2024-03-26T03:54:50.032560+0000 mgr.y (mgr.24431) 412 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:54:51.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:51 smithi190 bash[23392]: cluster 2024-03-26T03:54:50.575035+0000 mon.a (mon.0) 2532 : cluster [DBG] osdmap e397: 8 total, 8 up, 8 in 2024-03-26T03:54:51.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:51 smithi190 bash[23392]: audit 2024-03-26T03:54:50.577179+0000 mon.a (mon.0) 2533 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:54:51.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:51 smithi190 bash[23392]: cluster 2024-03-26T03:54:50.782737+0000 mgr.y (mgr.24431) 413 : cluster [DBG] pgmap v611: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:54:51.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:51 smithi175 bash[21527]: cluster 2024-03-26T03:54:50.575035+0000 mon.a (mon.0) 2532 : cluster [DBG] osdmap e397: 8 total, 8 up, 8 in 2024-03-26T03:54:51.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:51 smithi175 bash[21527]: audit 2024-03-26T03:54:50.577179+0000 mon.a (mon.0) 2533 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:54:51.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:51 smithi175 bash[21527]: cluster 2024-03-26T03:54:50.782737+0000 mgr.y (mgr.24431) 413 : cluster [DBG] pgmap v611: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:54:51.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:51 smithi175 bash[29284]: cluster 2024-03-26T03:54:50.575035+0000 mon.a (mon.0) 2532 : cluster [DBG] osdmap e397: 8 total, 8 up, 8 in 2024-03-26T03:54:51.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:51 smithi175 bash[29284]: audit 2024-03-26T03:54:50.577179+0000 mon.a (mon.0) 2533 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:54:51.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:51 smithi175 bash[29284]: cluster 2024-03-26T03:54:50.782737+0000 mgr.y (mgr.24431) 413 : cluster [DBG] pgmap v611: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:54:52.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:52 smithi190 bash[23392]: cluster 2024-03-26T03:54:51.566076+0000 mon.a (mon.0) 2534 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:54:52.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:52 smithi190 bash[23392]: audit 2024-03-26T03:54:51.575740+0000 mon.a (mon.0) 2535 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:54:52.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:52 smithi190 bash[23392]: cluster 2024-03-26T03:54:51.585798+0000 mon.a (mon.0) 2536 : cluster [DBG] osdmap e398: 8 total, 8 up, 8 in 2024-03-26T03:54:52.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:52 smithi190 bash[23392]: audit 2024-03-26T03:54:51.590701+0000 mon.a (mon.0) 2537 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:54:52.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:52 smithi190 bash[23392]: audit 2024-03-26T03:54:51.607881+0000 mon.a (mon.0) 2538 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-26T03:54:52.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:52 smithi175 bash[21527]: cluster 2024-03-26T03:54:51.566076+0000 mon.a (mon.0) 2534 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:54:52.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:52 smithi175 bash[21527]: audit 2024-03-26T03:54:51.575740+0000 mon.a (mon.0) 2535 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:54:52.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:52 smithi175 bash[21527]: cluster 2024-03-26T03:54:51.585798+0000 mon.a (mon.0) 2536 : cluster [DBG] osdmap e398: 8 total, 8 up, 8 in 2024-03-26T03:54:52.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:52 smithi175 bash[21527]: audit 2024-03-26T03:54:51.590701+0000 mon.a (mon.0) 2537 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:54:52.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:52 smithi175 bash[21527]: audit 2024-03-26T03:54:51.607881+0000 mon.a (mon.0) 2538 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-26T03:54:52.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:52 smithi175 bash[29284]: cluster 2024-03-26T03:54:51.566076+0000 mon.a (mon.0) 2534 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:54:52.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:52 smithi175 bash[29284]: audit 2024-03-26T03:54:51.575740+0000 mon.a (mon.0) 2535 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:54:52.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:52 smithi175 bash[29284]: cluster 2024-03-26T03:54:51.585798+0000 mon.a (mon.0) 2536 : cluster [DBG] osdmap e398: 8 total, 8 up, 8 in 2024-03-26T03:54:52.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:52 smithi175 bash[29284]: audit 2024-03-26T03:54:51.590701+0000 mon.a (mon.0) 2537 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:54:52.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:52 smithi175 bash[29284]: audit 2024-03-26T03:54:51.607881+0000 mon.a (mon.0) 2538 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-26T03:54:53.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:53 smithi190 bash[23392]: audit 2024-03-26T03:54:52.586489+0000 mon.a (mon.0) 2539 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-26T03:54:53.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:53 smithi190 bash[23392]: cluster 2024-03-26T03:54:52.593472+0000 mon.a (mon.0) 2540 : cluster [DBG] osdmap e399: 8 total, 8 up, 8 in 2024-03-26T03:54:53.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:53 smithi190 bash[23392]: cluster 2024-03-26T03:54:52.783701+0000 mgr.y (mgr.24431) 414 : cluster [DBG] pgmap v614: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:54:53.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:53 smithi175 bash[29284]: audit 2024-03-26T03:54:52.586489+0000 mon.a (mon.0) 2539 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-26T03:54:53.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:53 smithi175 bash[29284]: cluster 2024-03-26T03:54:52.593472+0000 mon.a (mon.0) 2540 : cluster [DBG] osdmap e399: 8 total, 8 up, 8 in 2024-03-26T03:54:53.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:53 smithi175 bash[29284]: cluster 2024-03-26T03:54:52.783701+0000 mgr.y (mgr.24431) 414 : cluster [DBG] pgmap v614: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:54:53.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:53 smithi175 bash[21527]: audit 2024-03-26T03:54:52.586489+0000 mon.a (mon.0) 2539 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-26T03:54:53.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:53 smithi175 bash[21527]: cluster 2024-03-26T03:54:52.593472+0000 mon.a (mon.0) 2540 : cluster [DBG] osdmap e399: 8 total, 8 up, 8 in 2024-03-26T03:54:53.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:53 smithi175 bash[21527]: cluster 2024-03-26T03:54:52.783701+0000 mgr.y (mgr.24431) 414 : cluster [DBG] pgmap v614: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:54:54.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:54 smithi190 bash[23392]: cluster 2024-03-26T03:54:53.604564+0000 mon.a (mon.0) 2541 : cluster [DBG] osdmap e400: 8 total, 8 up, 8 in 2024-03-26T03:54:54.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:54 smithi190 bash[23392]: audit 2024-03-26T03:54:54.547209+0000 mon.c (mon.2) 510 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:54:54.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:54 smithi175 bash[21527]: cluster 2024-03-26T03:54:53.604564+0000 mon.a (mon.0) 2541 : cluster [DBG] osdmap e400: 8 total, 8 up, 8 in 2024-03-26T03:54:54.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:54 smithi175 bash[21527]: audit 2024-03-26T03:54:54.547209+0000 mon.c (mon.2) 510 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:54:54.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:54 smithi175 bash[29284]: cluster 2024-03-26T03:54:53.604564+0000 mon.a (mon.0) 2541 : cluster [DBG] osdmap e400: 8 total, 8 up, 8 in 2024-03-26T03:54:54.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:54 smithi175 bash[29284]: audit 2024-03-26T03:54:54.547209+0000 mon.c (mon.2) 510 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:54:55.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:55 smithi190 bash[23392]: cluster 2024-03-26T03:54:54.606509+0000 mon.a (mon.0) 2542 : cluster [DBG] osdmap e401: 8 total, 8 up, 8 in 2024-03-26T03:54:55.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:55 smithi190 bash[23392]: cluster 2024-03-26T03:54:54.785888+0000 mgr.y (mgr.24431) 415 : cluster [DBG] pgmap v617: 292 pgs: 6 unknown, 286 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-26T03:54:55.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:55 smithi190 bash[23392]: audit 2024-03-26T03:54:55.073727+0000 mon.a (mon.0) 2543 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:54:55.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:55 smithi190 bash[23392]: audit 2024-03-26T03:54:55.080777+0000 mon.a (mon.0) 2544 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:54:55.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:55 smithi175 bash[21527]: cluster 2024-03-26T03:54:54.606509+0000 mon.a (mon.0) 2542 : cluster [DBG] osdmap e401: 8 total, 8 up, 8 in 2024-03-26T03:54:55.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:55 smithi175 bash[21527]: cluster 2024-03-26T03:54:54.785888+0000 mgr.y (mgr.24431) 415 : cluster [DBG] pgmap v617: 292 pgs: 6 unknown, 286 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-26T03:54:55.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:55 smithi175 bash[21527]: audit 2024-03-26T03:54:55.073727+0000 mon.a (mon.0) 2543 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:54:55.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:55 smithi175 bash[21527]: audit 2024-03-26T03:54:55.080777+0000 mon.a (mon.0) 2544 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:54:55.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:55 smithi175 bash[29284]: cluster 2024-03-26T03:54:54.606509+0000 mon.a (mon.0) 2542 : cluster [DBG] osdmap e401: 8 total, 8 up, 8 in 2024-03-26T03:54:55.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:55 smithi175 bash[29284]: cluster 2024-03-26T03:54:54.785888+0000 mgr.y (mgr.24431) 415 : cluster [DBG] pgmap v617: 292 pgs: 6 unknown, 286 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-26T03:54:55.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:55 smithi175 bash[29284]: audit 2024-03-26T03:54:55.073727+0000 mon.a (mon.0) 2543 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:54:55.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:55 smithi175 bash[29284]: audit 2024-03-26T03:54:55.080777+0000 mon.a (mon.0) 2544 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:54:56.990 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:56 smithi190 bash[23392]: cluster 2024-03-26T03:54:55.625140+0000 mon.a (mon.0) 2545 : cluster [DBG] osdmap e402: 8 total, 8 up, 8 in 2024-03-26T03:54:56.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:56 smithi190 bash[23392]: audit 2024-03-26T03:54:55.669272+0000 mon.c (mon.2) 511 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:54:56.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:56 smithi190 bash[23392]: audit 2024-03-26T03:54:55.670398+0000 mon.c (mon.2) 512 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:54:56.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:56 smithi190 bash[23392]: audit 2024-03-26T03:54:55.677782+0000 mon.a (mon.0) 2546 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:54:56.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:56 smithi175 bash[29284]: cluster 2024-03-26T03:54:55.625140+0000 mon.a (mon.0) 2545 : cluster [DBG] osdmap e402: 8 total, 8 up, 8 in 2024-03-26T03:54:56.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:56 smithi175 bash[29284]: audit 2024-03-26T03:54:55.669272+0000 mon.c (mon.2) 511 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:54:56.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:56 smithi175 bash[29284]: audit 2024-03-26T03:54:55.670398+0000 mon.c (mon.2) 512 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:54:56.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:56 smithi175 bash[29284]: audit 2024-03-26T03:54:55.677782+0000 mon.a (mon.0) 2546 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:54:56.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:56 smithi175 bash[21527]: cluster 2024-03-26T03:54:55.625140+0000 mon.a (mon.0) 2545 : cluster [DBG] osdmap e402: 8 total, 8 up, 8 in 2024-03-26T03:54:56.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:56 smithi175 bash[21527]: audit 2024-03-26T03:54:55.669272+0000 mon.c (mon.2) 511 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:54:56.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:56 smithi175 bash[21527]: audit 2024-03-26T03:54:55.670398+0000 mon.c (mon.2) 512 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:54:56.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:56 smithi175 bash[21527]: audit 2024-03-26T03:54:55.677782+0000 mon.a (mon.0) 2546 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:54:57.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:57 smithi190 bash[23392]: cluster 2024-03-26T03:54:56.786942+0000 mgr.y (mgr.24431) 416 : cluster [DBG] pgmap v619: 292 pgs: 292 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 2.6 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2024-03-26T03:54:57.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:57 smithi175 bash[29284]: cluster 2024-03-26T03:54:56.786942+0000 mgr.y (mgr.24431) 416 : cluster [DBG] pgmap v619: 292 pgs: 292 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 2.6 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2024-03-26T03:54:57.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:57 smithi175 bash[21527]: cluster 2024-03-26T03:54:56.786942+0000 mgr.y (mgr.24431) 416 : cluster [DBG] pgmap v619: 292 pgs: 292 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 2.6 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2024-03-26T03:54:59.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:54:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:54:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:55:00.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:54:59 smithi190 bash[23392]: cluster 2024-03-26T03:54:58.787833+0000 mgr.y (mgr.24431) 417 : cluster [DBG] pgmap v620: 292 pgs: 292 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-03-26T03:55:00.241 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:55:00 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:55:00.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:54:59 smithi175 bash[21527]: cluster 2024-03-26T03:54:58.787833+0000 mgr.y (mgr.24431) 417 : cluster [DBG] pgmap v620: 292 pgs: 292 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-03-26T03:55:00.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:54:59 smithi175 bash[29284]: cluster 2024-03-26T03:54:58.787833+0000 mgr.y (mgr.24431) 417 : cluster [DBG] pgmap v620: 292 pgs: 292 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-03-26T03:55:01.132 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:00 smithi175 bash[21527]: audit 2024-03-26T03:55:00.044136+0000 mgr.y (mgr.24431) 418 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:55:01.132 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:00 smithi175 bash[29284]: audit 2024-03-26T03:55:00.044136+0000 mgr.y (mgr.24431) 418 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:55:01.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:00 smithi190 bash[23392]: audit 2024-03-26T03:55:00.044136+0000 mgr.y (mgr.24431) 418 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:55:02.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:01 smithi190 bash[23392]: cluster 2024-03-26T03:55:00.790513+0000 mgr.y (mgr.24431) 419 : cluster [DBG] pgmap v621: 292 pgs: 292 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 2.8 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-03-26T03:55:02.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:01 smithi190 bash[23392]: cluster 2024-03-26T03:55:01.533487+0000 mon.a (mon.0) 2547 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:55:02.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:01 smithi190 bash[23392]: cluster 2024-03-26T03:55:01.545781+0000 mon.a (mon.0) 2548 : cluster [DBG] osdmap e403: 8 total, 8 up, 8 in 2024-03-26T03:55:02.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:01 smithi175 bash[21527]: cluster 2024-03-26T03:55:00.790513+0000 mgr.y (mgr.24431) 419 : cluster [DBG] pgmap v621: 292 pgs: 292 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 2.8 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-03-26T03:55:02.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:01 smithi175 bash[21527]: cluster 2024-03-26T03:55:01.533487+0000 mon.a (mon.0) 2547 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:55:02.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:01 smithi175 bash[21527]: cluster 2024-03-26T03:55:01.545781+0000 mon.a (mon.0) 2548 : cluster [DBG] osdmap e403: 8 total, 8 up, 8 in 2024-03-26T03:55:02.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:01 smithi175 bash[29284]: cluster 2024-03-26T03:55:00.790513+0000 mgr.y (mgr.24431) 419 : cluster [DBG] pgmap v621: 292 pgs: 292 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 2.8 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-03-26T03:55:02.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:01 smithi175 bash[29284]: cluster 2024-03-26T03:55:01.533487+0000 mon.a (mon.0) 2547 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:55:02.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:01 smithi175 bash[29284]: cluster 2024-03-26T03:55:01.545781+0000 mon.a (mon.0) 2548 : cluster [DBG] osdmap e403: 8 total, 8 up, 8 in 2024-03-26T03:55:04.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:03 smithi190 bash[23392]: cluster 2024-03-26T03:55:02.791532+0000 mgr.y (mgr.24431) 420 : cluster [DBG] pgmap v623: 292 pgs: 292 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 639 B/s wr, 2 op/s 2024-03-26T03:55:04.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:03 smithi175 bash[29284]: cluster 2024-03-26T03:55:02.791532+0000 mgr.y (mgr.24431) 420 : cluster [DBG] pgmap v623: 292 pgs: 292 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 639 B/s wr, 2 op/s 2024-03-26T03:55:04.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:03 smithi175 bash[21527]: cluster 2024-03-26T03:55:02.791532+0000 mgr.y (mgr.24431) 420 : cluster [DBG] pgmap v623: 292 pgs: 292 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 639 B/s wr, 2 op/s 2024-03-26T03:55:06.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:05 smithi190 bash[23392]: cluster 2024-03-26T03:55:04.793632+0000 mgr.y (mgr.24431) 421 : cluster [DBG] pgmap v624: 292 pgs: 292 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1005 B/s rd, 0 B/s wr, 1 op/s 2024-03-26T03:55:06.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:05 smithi190 bash[23392]: audit 2024-03-26T03:55:05.000834+0000 mon.c (mon.2) 513 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:55:06.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:05 smithi190 bash[23392]: audit 2024-03-26T03:55:05.667309+0000 mon.a (mon.0) 2549 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:55:06.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:05 smithi190 bash[23392]: audit 2024-03-26T03:55:05.668265+0000 mon.a (mon.0) 2550 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-65"}]: dispatch 2024-03-26T03:55:06.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:05 smithi175 bash[29284]: cluster 2024-03-26T03:55:04.793632+0000 mgr.y (mgr.24431) 421 : cluster [DBG] pgmap v624: 292 pgs: 292 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1005 B/s rd, 0 B/s wr, 1 op/s 2024-03-26T03:55:06.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:05 smithi175 bash[29284]: audit 2024-03-26T03:55:05.000834+0000 mon.c (mon.2) 513 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:55:06.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:05 smithi175 bash[29284]: audit 2024-03-26T03:55:05.667309+0000 mon.a (mon.0) 2549 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:55:06.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:05 smithi175 bash[29284]: audit 2024-03-26T03:55:05.668265+0000 mon.a (mon.0) 2550 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-65"}]: dispatch 2024-03-26T03:55:06.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:05 smithi175 bash[21527]: cluster 2024-03-26T03:55:04.793632+0000 mgr.y (mgr.24431) 421 : cluster [DBG] pgmap v624: 292 pgs: 292 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1005 B/s rd, 0 B/s wr, 1 op/s 2024-03-26T03:55:06.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:05 smithi175 bash[21527]: audit 2024-03-26T03:55:05.000834+0000 mon.c (mon.2) 513 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:55:06.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:05 smithi175 bash[21527]: audit 2024-03-26T03:55:05.667309+0000 mon.a (mon.0) 2549 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:55:06.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:05 smithi175 bash[21527]: audit 2024-03-26T03:55:05.668265+0000 mon.a (mon.0) 2550 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-65"}]: dispatch 2024-03-26T03:55:07.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:06 smithi190 bash[23392]: cluster 2024-03-26T03:55:05.876194+0000 mon.a (mon.0) 2551 : cluster [DBG] osdmap e404: 8 total, 8 up, 8 in 2024-03-26T03:55:07.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:06 smithi190 bash[23392]: cluster 2024-03-26T03:55:06.885389+0000 mon.a (mon.0) 2552 : cluster [DBG] osdmap e405: 8 total, 8 up, 8 in 2024-03-26T03:55:07.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:06 smithi190 bash[23392]: audit 2024-03-26T03:55:06.889998+0000 mon.a (mon.0) 2553 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:55:07.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:06 smithi175 bash[29284]: cluster 2024-03-26T03:55:05.876194+0000 mon.a (mon.0) 2551 : cluster [DBG] osdmap e404: 8 total, 8 up, 8 in 2024-03-26T03:55:07.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:06 smithi175 bash[29284]: cluster 2024-03-26T03:55:06.885389+0000 mon.a (mon.0) 2552 : cluster [DBG] osdmap e405: 8 total, 8 up, 8 in 2024-03-26T03:55:07.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:06 smithi175 bash[29284]: audit 2024-03-26T03:55:06.889998+0000 mon.a (mon.0) 2553 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:55:07.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:06 smithi175 bash[21527]: cluster 2024-03-26T03:55:05.876194+0000 mon.a (mon.0) 2551 : cluster [DBG] osdmap e404: 8 total, 8 up, 8 in 2024-03-26T03:55:07.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:06 smithi175 bash[21527]: cluster 2024-03-26T03:55:06.885389+0000 mon.a (mon.0) 2552 : cluster [DBG] osdmap e405: 8 total, 8 up, 8 in 2024-03-26T03:55:07.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:06 smithi175 bash[21527]: audit 2024-03-26T03:55:06.889998+0000 mon.a (mon.0) 2553 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:55:08.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:07 smithi190 bash[23392]: cluster 2024-03-26T03:55:06.794473+0000 mgr.y (mgr.24431) 422 : cluster [DBG] pgmap v626: 260 pgs: 260 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:55:08.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:07 smithi190 bash[23392]: audit 2024-03-26T03:55:07.876137+0000 mon.a (mon.0) 2554 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:55:08.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:07 smithi190 bash[23392]: cluster 2024-03-26T03:55:07.884404+0000 mon.a (mon.0) 2555 : cluster [DBG] osdmap e406: 8 total, 8 up, 8 in 2024-03-26T03:55:08.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:07 smithi190 bash[23392]: audit 2024-03-26T03:55:07.886974+0000 mon.a (mon.0) 2556 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:55:08.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:07 smithi175 bash[21527]: cluster 2024-03-26T03:55:06.794473+0000 mgr.y (mgr.24431) 422 : cluster [DBG] pgmap v626: 260 pgs: 260 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:55:08.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:07 smithi175 bash[21527]: audit 2024-03-26T03:55:07.876137+0000 mon.a (mon.0) 2554 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:55:08.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:07 smithi175 bash[21527]: cluster 2024-03-26T03:55:07.884404+0000 mon.a (mon.0) 2555 : cluster [DBG] osdmap e406: 8 total, 8 up, 8 in 2024-03-26T03:55:08.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:07 smithi175 bash[21527]: audit 2024-03-26T03:55:07.886974+0000 mon.a (mon.0) 2556 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:55:08.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:07 smithi175 bash[29284]: cluster 2024-03-26T03:55:06.794473+0000 mgr.y (mgr.24431) 422 : cluster [DBG] pgmap v626: 260 pgs: 260 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:55:08.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:07 smithi175 bash[29284]: audit 2024-03-26T03:55:07.876137+0000 mon.a (mon.0) 2554 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:55:08.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:07 smithi175 bash[29284]: cluster 2024-03-26T03:55:07.884404+0000 mon.a (mon.0) 2555 : cluster [DBG] osdmap e406: 8 total, 8 up, 8 in 2024-03-26T03:55:08.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:07 smithi175 bash[29284]: audit 2024-03-26T03:55:07.886974+0000 mon.a (mon.0) 2556 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:55:09.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:55:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:55:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:55:10.241 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:55:10 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:55:10.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:09 smithi190 bash[23392]: cluster 2024-03-26T03:55:08.795349+0000 mgr.y (mgr.24431) 423 : cluster [DBG] pgmap v629: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:55:10.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:09 smithi190 bash[23392]: cluster 2024-03-26T03:55:08.886580+0000 mon.a (mon.0) 2557 : cluster [DBG] osdmap e407: 8 total, 8 up, 8 in 2024-03-26T03:55:10.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:09 smithi190 bash[23392]: audit 2024-03-26T03:55:08.960767+0000 mon.a (mon.0) 2558 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:55:10.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:09 smithi190 bash[23392]: audit 2024-03-26T03:55:08.961727+0000 mon.a (mon.0) 2559 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-67"}]: dispatch 2024-03-26T03:55:10.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:09 smithi175 bash[21527]: cluster 2024-03-26T03:55:08.795349+0000 mgr.y (mgr.24431) 423 : cluster [DBG] pgmap v629: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:55:10.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:09 smithi175 bash[21527]: cluster 2024-03-26T03:55:08.886580+0000 mon.a (mon.0) 2557 : cluster [DBG] osdmap e407: 8 total, 8 up, 8 in 2024-03-26T03:55:10.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:09 smithi175 bash[21527]: audit 2024-03-26T03:55:08.960767+0000 mon.a (mon.0) 2558 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:55:10.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:09 smithi175 bash[21527]: audit 2024-03-26T03:55:08.961727+0000 mon.a (mon.0) 2559 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-67"}]: dispatch 2024-03-26T03:55:10.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:09 smithi175 bash[29284]: cluster 2024-03-26T03:55:08.795349+0000 mgr.y (mgr.24431) 423 : cluster [DBG] pgmap v629: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:55:10.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:09 smithi175 bash[29284]: cluster 2024-03-26T03:55:08.886580+0000 mon.a (mon.0) 2557 : cluster [DBG] osdmap e407: 8 total, 8 up, 8 in 2024-03-26T03:55:10.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:09 smithi175 bash[29284]: audit 2024-03-26T03:55:08.960767+0000 mon.a (mon.0) 2558 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:55:10.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:09 smithi175 bash[29284]: audit 2024-03-26T03:55:08.961727+0000 mon.a (mon.0) 2559 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-67"}]: dispatch 2024-03-26T03:55:11.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:10 smithi190 bash[23392]: cluster 2024-03-26T03:55:09.945599+0000 mon.a (mon.0) 2560 : cluster [DBG] osdmap e408: 8 total, 8 up, 8 in 2024-03-26T03:55:11.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:10 smithi190 bash[23392]: audit 2024-03-26T03:55:10.052890+0000 mgr.y (mgr.24431) 424 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:55:11.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:10 smithi175 bash[21527]: cluster 2024-03-26T03:55:09.945599+0000 mon.a (mon.0) 2560 : cluster [DBG] osdmap e408: 8 total, 8 up, 8 in 2024-03-26T03:55:11.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:10 smithi175 bash[21527]: audit 2024-03-26T03:55:10.052890+0000 mgr.y (mgr.24431) 424 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:55:11.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:10 smithi175 bash[29284]: cluster 2024-03-26T03:55:09.945599+0000 mon.a (mon.0) 2560 : cluster [DBG] osdmap e408: 8 total, 8 up, 8 in 2024-03-26T03:55:11.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:10 smithi175 bash[29284]: audit 2024-03-26T03:55:10.052890+0000 mgr.y (mgr.24431) 424 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:55:12.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:11 smithi190 bash[23392]: cluster 2024-03-26T03:55:10.796224+0000 mgr.y (mgr.24431) 425 : cluster [DBG] pgmap v632: 260 pgs: 260 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-26T03:55:12.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:11 smithi190 bash[23392]: cluster 2024-03-26T03:55:10.966295+0000 mon.a (mon.0) 2561 : cluster [DBG] osdmap e409: 8 total, 8 up, 8 in 2024-03-26T03:55:12.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:11 smithi190 bash[23392]: audit 2024-03-26T03:55:10.972383+0000 mon.a (mon.0) 2562 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:55:12.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:11 smithi175 bash[21527]: cluster 2024-03-26T03:55:10.796224+0000 mgr.y (mgr.24431) 425 : cluster [DBG] pgmap v632: 260 pgs: 260 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-26T03:55:12.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:11 smithi175 bash[21527]: cluster 2024-03-26T03:55:10.966295+0000 mon.a (mon.0) 2561 : cluster [DBG] osdmap e409: 8 total, 8 up, 8 in 2024-03-26T03:55:12.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:11 smithi175 bash[21527]: audit 2024-03-26T03:55:10.972383+0000 mon.a (mon.0) 2562 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:55:12.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:11 smithi175 bash[29284]: cluster 2024-03-26T03:55:10.796224+0000 mgr.y (mgr.24431) 425 : cluster [DBG] pgmap v632: 260 pgs: 260 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-26T03:55:12.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:11 smithi175 bash[29284]: cluster 2024-03-26T03:55:10.966295+0000 mon.a (mon.0) 2561 : cluster [DBG] osdmap e409: 8 total, 8 up, 8 in 2024-03-26T03:55:12.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:11 smithi175 bash[29284]: audit 2024-03-26T03:55:10.972383+0000 mon.a (mon.0) 2562 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:55:13.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:12 smithi190 bash[23392]: audit 2024-03-26T03:55:11.955945+0000 mon.a (mon.0) 2563 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:55:13.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:12 smithi190 bash[23392]: cluster 2024-03-26T03:55:11.964250+0000 mon.a (mon.0) 2564 : cluster [DBG] osdmap e410: 8 total, 8 up, 8 in 2024-03-26T03:55:13.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:12 smithi190 bash[23392]: audit 2024-03-26T03:55:11.967758+0000 mon.a (mon.0) 2565 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:55:13.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:12 smithi190 bash[23392]: audit 2024-03-26T03:55:12.049624+0000 mon.a (mon.0) 2566 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:55:13.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:12 smithi190 bash[23392]: audit 2024-03-26T03:55:12.050046+0000 mon.a (mon.0) 2567 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-69"}]: dispatch 2024-03-26T03:55:13.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:12 smithi175 bash[29284]: audit 2024-03-26T03:55:11.955945+0000 mon.a (mon.0) 2563 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:55:13.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:12 smithi175 bash[29284]: cluster 2024-03-26T03:55:11.964250+0000 mon.a (mon.0) 2564 : cluster [DBG] osdmap e410: 8 total, 8 up, 8 in 2024-03-26T03:55:13.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:12 smithi175 bash[29284]: audit 2024-03-26T03:55:11.967758+0000 mon.a (mon.0) 2565 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:55:13.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:12 smithi175 bash[29284]: audit 2024-03-26T03:55:12.049624+0000 mon.a (mon.0) 2566 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:55:13.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:12 smithi175 bash[29284]: audit 2024-03-26T03:55:12.050046+0000 mon.a (mon.0) 2567 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-69"}]: dispatch 2024-03-26T03:55:13.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:12 smithi175 bash[21527]: audit 2024-03-26T03:55:11.955945+0000 mon.a (mon.0) 2563 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:55:13.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:12 smithi175 bash[21527]: cluster 2024-03-26T03:55:11.964250+0000 mon.a (mon.0) 2564 : cluster [DBG] osdmap e410: 8 total, 8 up, 8 in 2024-03-26T03:55:13.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:12 smithi175 bash[21527]: audit 2024-03-26T03:55:11.967758+0000 mon.a (mon.0) 2565 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:55:13.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:12 smithi175 bash[21527]: audit 2024-03-26T03:55:12.049624+0000 mon.a (mon.0) 2566 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:55:13.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:12 smithi175 bash[21527]: audit 2024-03-26T03:55:12.050046+0000 mon.a (mon.0) 2567 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-69"}]: dispatch 2024-03-26T03:55:14.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:14 smithi190 bash[23392]: cluster 2024-03-26T03:55:12.797132+0000 mgr.y (mgr.24431) 426 : cluster [DBG] pgmap v635: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-26T03:55:14.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:14 smithi190 bash[23392]: cluster 2024-03-26T03:55:12.999549+0000 mon.a (mon.0) 2568 : cluster [DBG] osdmap e411: 8 total, 8 up, 8 in 2024-03-26T03:55:14.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:14 smithi175 bash[29284]: cluster 2024-03-26T03:55:12.797132+0000 mgr.y (mgr.24431) 426 : cluster [DBG] pgmap v635: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-26T03:55:14.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:14 smithi175 bash[29284]: cluster 2024-03-26T03:55:12.999549+0000 mon.a (mon.0) 2568 : cluster [DBG] osdmap e411: 8 total, 8 up, 8 in 2024-03-26T03:55:14.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:14 smithi175 bash[21527]: cluster 2024-03-26T03:55:12.797132+0000 mgr.y (mgr.24431) 426 : cluster [DBG] pgmap v635: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-26T03:55:14.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:14 smithi175 bash[21527]: cluster 2024-03-26T03:55:12.999549+0000 mon.a (mon.0) 2568 : cluster [DBG] osdmap e411: 8 total, 8 up, 8 in 2024-03-26T03:55:15.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:15 smithi190 bash[23392]: cluster 2024-03-26T03:55:14.007333+0000 mon.a (mon.0) 2569 : cluster [DBG] osdmap e412: 8 total, 8 up, 8 in 2024-03-26T03:55:15.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:15 smithi190 bash[23392]: audit 2024-03-26T03:55:14.020790+0000 mon.a (mon.0) 2570 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:55:15.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:15 smithi190 bash[23392]: audit 2024-03-26T03:55:14.997714+0000 mon.a (mon.0) 2571 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:55:15.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:15 smithi190 bash[23392]: cluster 2024-03-26T03:55:15.010693+0000 mon.a (mon.0) 2572 : cluster [DBG] osdmap e413: 8 total, 8 up, 8 in 2024-03-26T03:55:15.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:15 smithi175 bash[21527]: cluster 2024-03-26T03:55:14.007333+0000 mon.a (mon.0) 2569 : cluster [DBG] osdmap e412: 8 total, 8 up, 8 in 2024-03-26T03:55:15.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:15 smithi175 bash[21527]: audit 2024-03-26T03:55:14.020790+0000 mon.a (mon.0) 2570 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:55:15.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:15 smithi175 bash[21527]: audit 2024-03-26T03:55:14.997714+0000 mon.a (mon.0) 2571 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:55:15.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:15 smithi175 bash[21527]: cluster 2024-03-26T03:55:15.010693+0000 mon.a (mon.0) 2572 : cluster [DBG] osdmap e413: 8 total, 8 up, 8 in 2024-03-26T03:55:15.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:15 smithi175 bash[29284]: cluster 2024-03-26T03:55:14.007333+0000 mon.a (mon.0) 2569 : cluster [DBG] osdmap e412: 8 total, 8 up, 8 in 2024-03-26T03:55:15.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:15 smithi175 bash[29284]: audit 2024-03-26T03:55:14.020790+0000 mon.a (mon.0) 2570 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:55:15.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:15 smithi175 bash[29284]: audit 2024-03-26T03:55:14.997714+0000 mon.a (mon.0) 2571 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:55:15.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:15 smithi175 bash[29284]: cluster 2024-03-26T03:55:15.010693+0000 mon.a (mon.0) 2572 : cluster [DBG] osdmap e413: 8 total, 8 up, 8 in 2024-03-26T03:55:16.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:16 smithi190 bash[23392]: cluster 2024-03-26T03:55:14.799427+0000 mgr.y (mgr.24431) 427 : cluster [DBG] pgmap v638: 292 pgs: 27 creating+peering, 5 unknown, 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 767 B/s wr, 0 op/s 2024-03-26T03:55:16.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:16 smithi190 bash[23392]: cluster 2024-03-26T03:55:15.015642+0000 mon.a (mon.0) 2573 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:55:16.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:16 smithi190 bash[23392]: audit 2024-03-26T03:55:15.020978+0000 mon.a (mon.0) 2574 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:55:16.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:16 smithi190 bash[23392]: audit 2024-03-26T03:55:15.048814+0000 mon.a (mon.0) 2575 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-26T03:55:16.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:16 smithi190 bash[23392]: audit 2024-03-26T03:55:16.040489+0000 mon.a (mon.0) 2576 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-26T03:55:16.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:16 smithi190 bash[23392]: cluster 2024-03-26T03:55:16.050237+0000 mon.a (mon.0) 2577 : cluster [DBG] osdmap e414: 8 total, 8 up, 8 in 2024-03-26T03:55:16.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:16 smithi175 bash[21527]: cluster 2024-03-26T03:55:14.799427+0000 mgr.y (mgr.24431) 427 : cluster [DBG] pgmap v638: 292 pgs: 27 creating+peering, 5 unknown, 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 767 B/s wr, 0 op/s 2024-03-26T03:55:16.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:16 smithi175 bash[21527]: cluster 2024-03-26T03:55:15.015642+0000 mon.a (mon.0) 2573 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:55:16.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:16 smithi175 bash[21527]: audit 2024-03-26T03:55:15.020978+0000 mon.a (mon.0) 2574 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:55:16.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:16 smithi175 bash[21527]: audit 2024-03-26T03:55:15.048814+0000 mon.a (mon.0) 2575 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-26T03:55:16.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:16 smithi175 bash[21527]: audit 2024-03-26T03:55:16.040489+0000 mon.a (mon.0) 2576 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-26T03:55:16.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:16 smithi175 bash[21527]: cluster 2024-03-26T03:55:16.050237+0000 mon.a (mon.0) 2577 : cluster [DBG] osdmap e414: 8 total, 8 up, 8 in 2024-03-26T03:55:16.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:16 smithi175 bash[29284]: cluster 2024-03-26T03:55:14.799427+0000 mgr.y (mgr.24431) 427 : cluster [DBG] pgmap v638: 292 pgs: 27 creating+peering, 5 unknown, 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 767 B/s wr, 0 op/s 2024-03-26T03:55:16.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:16 smithi175 bash[29284]: cluster 2024-03-26T03:55:15.015642+0000 mon.a (mon.0) 2573 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:55:16.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:16 smithi175 bash[29284]: audit 2024-03-26T03:55:15.020978+0000 mon.a (mon.0) 2574 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:55:16.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:16 smithi175 bash[29284]: audit 2024-03-26T03:55:15.048814+0000 mon.a (mon.0) 2575 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-26T03:55:16.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:16 smithi175 bash[29284]: audit 2024-03-26T03:55:16.040489+0000 mon.a (mon.0) 2576 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-26T03:55:16.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:16 smithi175 bash[29284]: cluster 2024-03-26T03:55:16.050237+0000 mon.a (mon.0) 2577 : cluster [DBG] osdmap e414: 8 total, 8 up, 8 in 2024-03-26T03:55:18.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:18 smithi190 bash[23392]: cluster 2024-03-26T03:55:16.800278+0000 mgr.y (mgr.24431) 428 : cluster [DBG] pgmap v641: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-26T03:55:18.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:18 smithi190 bash[23392]: cluster 2024-03-26T03:55:17.083920+0000 mon.a (mon.0) 2578 : cluster [DBG] osdmap e415: 8 total, 8 up, 8 in 2024-03-26T03:55:18.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:18 smithi175 bash[29284]: cluster 2024-03-26T03:55:16.800278+0000 mgr.y (mgr.24431) 428 : cluster [DBG] pgmap v641: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-26T03:55:18.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:18 smithi175 bash[29284]: cluster 2024-03-26T03:55:17.083920+0000 mon.a (mon.0) 2578 : cluster [DBG] osdmap e415: 8 total, 8 up, 8 in 2024-03-26T03:55:18.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:18 smithi175 bash[21527]: cluster 2024-03-26T03:55:16.800278+0000 mgr.y (mgr.24431) 428 : cluster [DBG] pgmap v641: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-26T03:55:18.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:18 smithi175 bash[21527]: cluster 2024-03-26T03:55:17.083920+0000 mon.a (mon.0) 2578 : cluster [DBG] osdmap e415: 8 total, 8 up, 8 in 2024-03-26T03:55:19.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:55:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:55:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:55:20.491 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:55:20 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:55:20.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:20 smithi190 bash[23392]: cluster 2024-03-26T03:55:18.801147+0000 mgr.y (mgr.24431) 429 : cluster [DBG] pgmap v643: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 639 B/s wr, 1 op/s 2024-03-26T03:55:20.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:20 smithi190 bash[23392]: audit 2024-03-26T03:55:20.035468+0000 mon.c (mon.2) 514 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:55:20.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:20 smithi175 bash[21527]: cluster 2024-03-26T03:55:18.801147+0000 mgr.y (mgr.24431) 429 : cluster [DBG] pgmap v643: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 639 B/s wr, 1 op/s 2024-03-26T03:55:20.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:20 smithi175 bash[21527]: audit 2024-03-26T03:55:20.035468+0000 mon.c (mon.2) 514 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:55:20.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:20 smithi175 bash[29284]: cluster 2024-03-26T03:55:18.801147+0000 mgr.y (mgr.24431) 429 : cluster [DBG] pgmap v643: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 639 B/s wr, 1 op/s 2024-03-26T03:55:20.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:20 smithi175 bash[29284]: audit 2024-03-26T03:55:20.035468+0000 mon.c (mon.2) 514 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:55:21.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:21 smithi190 bash[23392]: audit 2024-03-26T03:55:20.062066+0000 mgr.y (mgr.24431) 430 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:55:21.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:21 smithi175 bash[21527]: audit 2024-03-26T03:55:20.062066+0000 mgr.y (mgr.24431) 430 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:55:21.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:21 smithi175 bash[29284]: audit 2024-03-26T03:55:20.062066+0000 mgr.y (mgr.24431) 430 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:55:22.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:22 smithi190 bash[23392]: cluster 2024-03-26T03:55:20.803938+0000 mgr.y (mgr.24431) 431 : cluster [DBG] pgmap v644: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-03-26T03:55:22.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:22 smithi190 bash[23392]: cluster 2024-03-26T03:55:21.540800+0000 mon.a (mon.0) 2579 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:55:22.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:22 smithi175 bash[21527]: cluster 2024-03-26T03:55:20.803938+0000 mgr.y (mgr.24431) 431 : cluster [DBG] pgmap v644: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-03-26T03:55:22.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:22 smithi175 bash[21527]: cluster 2024-03-26T03:55:21.540800+0000 mon.a (mon.0) 2579 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:55:22.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:22 smithi175 bash[29284]: cluster 2024-03-26T03:55:20.803938+0000 mgr.y (mgr.24431) 431 : cluster [DBG] pgmap v644: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-03-26T03:55:22.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:22 smithi175 bash[29284]: cluster 2024-03-26T03:55:21.540800+0000 mon.a (mon.0) 2579 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:55:24.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:24 smithi175 bash[29284]: cluster 2024-03-26T03:55:22.804840+0000 mgr.y (mgr.24431) 432 : cluster [DBG] pgmap v645: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 787 B/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-26T03:55:24.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:24 smithi175 bash[21527]: cluster 2024-03-26T03:55:22.804840+0000 mgr.y (mgr.24431) 432 : cluster [DBG] pgmap v645: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 787 B/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-26T03:55:24.495 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:24 smithi190 bash[23392]: cluster 2024-03-26T03:55:22.804840+0000 mgr.y (mgr.24431) 432 : cluster [DBG] pgmap v645: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 787 B/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-26T03:55:26.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:26 smithi190 bash[23392]: cluster 2024-03-26T03:55:24.807158+0000 mgr.y (mgr.24431) 433 : cluster [DBG] pgmap v646: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 701 B/s rd, 1.5 KiB/s wr, 3 op/s 2024-03-26T03:55:26.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:26 smithi175 bash[21527]: cluster 2024-03-26T03:55:24.807158+0000 mgr.y (mgr.24431) 433 : cluster [DBG] pgmap v646: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 701 B/s rd, 1.5 KiB/s wr, 3 op/s 2024-03-26T03:55:26.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:26 smithi175 bash[29284]: cluster 2024-03-26T03:55:24.807158+0000 mgr.y (mgr.24431) 433 : cluster [DBG] pgmap v646: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 701 B/s rd, 1.5 KiB/s wr, 3 op/s 2024-03-26T03:55:28.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:28 smithi190 bash[23392]: cluster 2024-03-26T03:55:26.808450+0000 mgr.y (mgr.24431) 434 : cluster [DBG] pgmap v647: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-03-26T03:55:28.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:28 smithi175 bash[21527]: cluster 2024-03-26T03:55:26.808450+0000 mgr.y (mgr.24431) 434 : cluster [DBG] pgmap v647: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-03-26T03:55:28.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:28 smithi175 bash[29284]: cluster 2024-03-26T03:55:26.808450+0000 mgr.y (mgr.24431) 434 : cluster [DBG] pgmap v647: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-03-26T03:55:29.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:55:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:55:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:55:30.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:30 smithi190 bash[23392]: cluster 2024-03-26T03:55:28.809316+0000 mgr.y (mgr.24431) 435 : cluster [DBG] pgmap v648: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 960 B/s rd, 1.1 KiB/s wr, 2 op/s 2024-03-26T03:55:30.491 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:55:30 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:55:30.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:30 smithi175 bash[21527]: cluster 2024-03-26T03:55:28.809316+0000 mgr.y (mgr.24431) 435 : cluster [DBG] pgmap v648: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 960 B/s rd, 1.1 KiB/s wr, 2 op/s 2024-03-26T03:55:30.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:30 smithi175 bash[29284]: cluster 2024-03-26T03:55:28.809316+0000 mgr.y (mgr.24431) 435 : cluster [DBG] pgmap v648: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 960 B/s rd, 1.1 KiB/s wr, 2 op/s 2024-03-26T03:55:31.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:31 smithi190 bash[23392]: audit 2024-03-26T03:55:30.072955+0000 mgr.y (mgr.24431) 436 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:55:31.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:31 smithi175 bash[21527]: audit 2024-03-26T03:55:30.072955+0000 mgr.y (mgr.24431) 436 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:55:31.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:31 smithi175 bash[29284]: audit 2024-03-26T03:55:30.072955+0000 mgr.y (mgr.24431) 436 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:55:32.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:32 smithi190 bash[23392]: cluster 2024-03-26T03:55:30.812101+0000 mgr.y (mgr.24431) 437 : cluster [DBG] pgmap v649: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-03-26T03:55:32.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:32 smithi175 bash[21527]: cluster 2024-03-26T03:55:30.812101+0000 mgr.y (mgr.24431) 437 : cluster [DBG] pgmap v649: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-03-26T03:55:32.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:32 smithi175 bash[29284]: cluster 2024-03-26T03:55:30.812101+0000 mgr.y (mgr.24431) 437 : cluster [DBG] pgmap v649: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-03-26T03:55:34.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:34 smithi190 bash[23392]: cluster 2024-03-26T03:55:32.813021+0000 mgr.y (mgr.24431) 438 : cluster [DBG] pgmap v650: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T03:55:34.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:34 smithi175 bash[21527]: cluster 2024-03-26T03:55:32.813021+0000 mgr.y (mgr.24431) 438 : cluster [DBG] pgmap v650: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T03:55:34.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:34 smithi175 bash[29284]: cluster 2024-03-26T03:55:32.813021+0000 mgr.y (mgr.24431) 438 : cluster [DBG] pgmap v650: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T03:55:35.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:35 smithi190 bash[23392]: audit 2024-03-26T03:55:35.069269+0000 mon.c (mon.2) 515 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:55:35.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:35 smithi175 bash[21527]: audit 2024-03-26T03:55:35.069269+0000 mon.c (mon.2) 515 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:55:35.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:35 smithi175 bash[29284]: audit 2024-03-26T03:55:35.069269+0000 mon.c (mon.2) 515 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:55:36.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:36 smithi190 bash[23392]: cluster 2024-03-26T03:55:34.815366+0000 mgr.y (mgr.24431) 439 : cluster [DBG] pgmap v651: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T03:55:36.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:36 smithi175 bash[21527]: cluster 2024-03-26T03:55:34.815366+0000 mgr.y (mgr.24431) 439 : cluster [DBG] pgmap v651: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T03:55:36.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:36 smithi175 bash[29284]: cluster 2024-03-26T03:55:34.815366+0000 mgr.y (mgr.24431) 439 : cluster [DBG] pgmap v651: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T03:55:38.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:38 smithi190 bash[23392]: cluster 2024-03-26T03:55:36.816649+0000 mgr.y (mgr.24431) 440 : cluster [DBG] pgmap v652: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T03:55:38.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:38 smithi190 bash[23392]: audit 2024-03-26T03:55:37.203879+0000 mon.a (mon.0) 2580 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:55:38.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:38 smithi190 bash[23392]: audit 2024-03-26T03:55:37.204807+0000 mon.a (mon.0) 2581 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-71"}]: dispatch 2024-03-26T03:55:38.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:38 smithi175 bash[21527]: cluster 2024-03-26T03:55:36.816649+0000 mgr.y (mgr.24431) 440 : cluster [DBG] pgmap v652: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T03:55:38.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:38 smithi175 bash[21527]: audit 2024-03-26T03:55:37.203879+0000 mon.a (mon.0) 2580 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:55:38.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:38 smithi175 bash[21527]: audit 2024-03-26T03:55:37.204807+0000 mon.a (mon.0) 2581 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-71"}]: dispatch 2024-03-26T03:55:38.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:38 smithi175 bash[29284]: cluster 2024-03-26T03:55:36.816649+0000 mgr.y (mgr.24431) 440 : cluster [DBG] pgmap v652: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T03:55:38.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:38 smithi175 bash[29284]: audit 2024-03-26T03:55:37.203879+0000 mon.a (mon.0) 2580 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:55:38.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:38 smithi175 bash[29284]: audit 2024-03-26T03:55:37.204807+0000 mon.a (mon.0) 2581 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-71"}]: dispatch 2024-03-26T03:55:39.184 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:55:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:55:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:55:39.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:39 smithi190 bash[23392]: cluster 2024-03-26T03:55:38.152339+0000 mon.a (mon.0) 2582 : cluster [DBG] osdmap e416: 8 total, 8 up, 8 in 2024-03-26T03:55:39.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:39 smithi190 bash[23392]: cluster 2024-03-26T03:55:39.157562+0000 mon.a (mon.0) 2583 : cluster [DBG] osdmap e417: 8 total, 8 up, 8 in 2024-03-26T03:55:39.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:39 smithi190 bash[23392]: audit 2024-03-26T03:55:39.164099+0000 mon.a (mon.0) 2584 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:55:39.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:39 smithi175 bash[29284]: cluster 2024-03-26T03:55:38.152339+0000 mon.a (mon.0) 2582 : cluster [DBG] osdmap e416: 8 total, 8 up, 8 in 2024-03-26T03:55:39.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:39 smithi175 bash[29284]: cluster 2024-03-26T03:55:39.157562+0000 mon.a (mon.0) 2583 : cluster [DBG] osdmap e417: 8 total, 8 up, 8 in 2024-03-26T03:55:39.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:39 smithi175 bash[29284]: audit 2024-03-26T03:55:39.164099+0000 mon.a (mon.0) 2584 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:55:39.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:39 smithi175 bash[21527]: cluster 2024-03-26T03:55:38.152339+0000 mon.a (mon.0) 2582 : cluster [DBG] osdmap e416: 8 total, 8 up, 8 in 2024-03-26T03:55:39.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:39 smithi175 bash[21527]: cluster 2024-03-26T03:55:39.157562+0000 mon.a (mon.0) 2583 : cluster [DBG] osdmap e417: 8 total, 8 up, 8 in 2024-03-26T03:55:39.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:39 smithi175 bash[21527]: audit 2024-03-26T03:55:39.164099+0000 mon.a (mon.0) 2584 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:55:40.491 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:55:40 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:55:40.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:40 smithi190 bash[23392]: cluster 2024-03-26T03:55:38.817761+0000 mgr.y (mgr.24431) 441 : cluster [DBG] pgmap v654: 260 pgs: 260 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 920 B/s rd, 0 op/s 2024-03-26T03:55:40.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:40 smithi190 bash[23392]: audit 2024-03-26T03:55:40.153292+0000 mon.a (mon.0) 2585 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:55:40.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:40 smithi190 bash[23392]: cluster 2024-03-26T03:55:40.162547+0000 mon.a (mon.0) 2586 : cluster [DBG] osdmap e418: 8 total, 8 up, 8 in 2024-03-26T03:55:40.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:40 smithi190 bash[23392]: audit 2024-03-26T03:55:40.167257+0000 mon.a (mon.0) 2587 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:55:40.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:40 smithi175 bash[21527]: cluster 2024-03-26T03:55:38.817761+0000 mgr.y (mgr.24431) 441 : cluster [DBG] pgmap v654: 260 pgs: 260 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 920 B/s rd, 0 op/s 2024-03-26T03:55:40.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:40 smithi175 bash[21527]: audit 2024-03-26T03:55:40.153292+0000 mon.a (mon.0) 2585 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:55:40.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:40 smithi175 bash[21527]: cluster 2024-03-26T03:55:40.162547+0000 mon.a (mon.0) 2586 : cluster [DBG] osdmap e418: 8 total, 8 up, 8 in 2024-03-26T03:55:40.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:40 smithi175 bash[21527]: audit 2024-03-26T03:55:40.167257+0000 mon.a (mon.0) 2587 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:55:40.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:40 smithi175 bash[29284]: cluster 2024-03-26T03:55:38.817761+0000 mgr.y (mgr.24431) 441 : cluster [DBG] pgmap v654: 260 pgs: 260 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 920 B/s rd, 0 op/s 2024-03-26T03:55:40.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:40 smithi175 bash[29284]: audit 2024-03-26T03:55:40.153292+0000 mon.a (mon.0) 2585 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:55:40.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:40 smithi175 bash[29284]: cluster 2024-03-26T03:55:40.162547+0000 mon.a (mon.0) 2586 : cluster [DBG] osdmap e418: 8 total, 8 up, 8 in 2024-03-26T03:55:40.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:40 smithi175 bash[29284]: audit 2024-03-26T03:55:40.167257+0000 mon.a (mon.0) 2587 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:55:41.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:41 smithi190 bash[23392]: audit 2024-03-26T03:55:40.081212+0000 mgr.y (mgr.24431) 442 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:55:41.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:41 smithi190 bash[23392]: cluster 2024-03-26T03:55:41.163746+0000 mon.a (mon.0) 2588 : cluster [DBG] osdmap e419: 8 total, 8 up, 8 in 2024-03-26T03:55:41.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:41 smithi175 bash[21527]: audit 2024-03-26T03:55:40.081212+0000 mgr.y (mgr.24431) 442 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:55:41.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:41 smithi175 bash[21527]: cluster 2024-03-26T03:55:41.163746+0000 mon.a (mon.0) 2588 : cluster [DBG] osdmap e419: 8 total, 8 up, 8 in 2024-03-26T03:55:41.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:41 smithi175 bash[29284]: audit 2024-03-26T03:55:40.081212+0000 mgr.y (mgr.24431) 442 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:55:41.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:41 smithi175 bash[29284]: cluster 2024-03-26T03:55:41.163746+0000 mon.a (mon.0) 2588 : cluster [DBG] osdmap e419: 8 total, 8 up, 8 in 2024-03-26T03:55:42.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:42 smithi190 bash[23392]: cluster 2024-03-26T03:55:40.818654+0000 mgr.y (mgr.24431) 443 : cluster [DBG] pgmap v657: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:55:42.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:42 smithi190 bash[23392]: cluster 2024-03-26T03:55:42.167107+0000 mon.a (mon.0) 2589 : cluster [DBG] osdmap e420: 8 total, 8 up, 8 in 2024-03-26T03:55:42.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:42 smithi175 bash[21527]: cluster 2024-03-26T03:55:40.818654+0000 mgr.y (mgr.24431) 443 : cluster [DBG] pgmap v657: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:55:42.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:42 smithi175 bash[21527]: cluster 2024-03-26T03:55:42.167107+0000 mon.a (mon.0) 2589 : cluster [DBG] osdmap e420: 8 total, 8 up, 8 in 2024-03-26T03:55:42.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:42 smithi175 bash[29284]: cluster 2024-03-26T03:55:40.818654+0000 mgr.y (mgr.24431) 443 : cluster [DBG] pgmap v657: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:55:42.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:42 smithi175 bash[29284]: cluster 2024-03-26T03:55:42.167107+0000 mon.a (mon.0) 2589 : cluster [DBG] osdmap e420: 8 total, 8 up, 8 in 2024-03-26T03:55:44.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:44 smithi190 bash[23392]: cluster 2024-03-26T03:55:42.819580+0000 mgr.y (mgr.24431) 444 : cluster [DBG] pgmap v660: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:55:44.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:44 smithi175 bash[29284]: cluster 2024-03-26T03:55:42.819580+0000 mgr.y (mgr.24431) 444 : cluster [DBG] pgmap v660: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:55:44.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:44 smithi175 bash[21527]: cluster 2024-03-26T03:55:42.819580+0000 mgr.y (mgr.24431) 444 : cluster [DBG] pgmap v660: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:55:46.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:46 smithi190 bash[23392]: cluster 2024-03-26T03:55:44.822074+0000 mgr.y (mgr.24431) 445 : cluster [DBG] pgmap v661: 292 pgs: 6 creating+peering, 286 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-03-26T03:55:46.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:46 smithi175 bash[21527]: cluster 2024-03-26T03:55:44.822074+0000 mgr.y (mgr.24431) 445 : cluster [DBG] pgmap v661: 292 pgs: 6 creating+peering, 286 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-03-26T03:55:46.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:46 smithi175 bash[29284]: cluster 2024-03-26T03:55:44.822074+0000 mgr.y (mgr.24431) 445 : cluster [DBG] pgmap v661: 292 pgs: 6 creating+peering, 286 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-03-26T03:55:48.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:48 smithi190 bash[23392]: cluster 2024-03-26T03:55:46.823450+0000 mgr.y (mgr.24431) 446 : cluster [DBG] pgmap v662: 292 pgs: 292 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-03-26T03:55:48.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:48 smithi175 bash[29284]: cluster 2024-03-26T03:55:46.823450+0000 mgr.y (mgr.24431) 446 : cluster [DBG] pgmap v662: 292 pgs: 292 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-03-26T03:55:48.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:48 smithi175 bash[21527]: cluster 2024-03-26T03:55:46.823450+0000 mgr.y (mgr.24431) 446 : cluster [DBG] pgmap v662: 292 pgs: 292 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-03-26T03:55:48.993 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:55:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:55:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:55:50.491 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:55:50 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:55:50.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:50 smithi190 bash[23392]: cluster 2024-03-26T03:55:48.824442+0000 mgr.y (mgr.24431) 447 : cluster [DBG] pgmap v663: 292 pgs: 292 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 895 B/s wr, 3 op/s 2024-03-26T03:55:50.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:50 smithi190 bash[23392]: audit 2024-03-26T03:55:50.104104+0000 mon.c (mon.2) 516 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:55:50.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:50 smithi175 bash[29284]: cluster 2024-03-26T03:55:48.824442+0000 mgr.y (mgr.24431) 447 : cluster [DBG] pgmap v663: 292 pgs: 292 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 895 B/s wr, 3 op/s 2024-03-26T03:55:50.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:50 smithi175 bash[29284]: audit 2024-03-26T03:55:50.104104+0000 mon.c (mon.2) 516 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:55:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:50 smithi175 bash[21527]: cluster 2024-03-26T03:55:48.824442+0000 mgr.y (mgr.24431) 447 : cluster [DBG] pgmap v663: 292 pgs: 292 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 895 B/s wr, 3 op/s 2024-03-26T03:55:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:50 smithi175 bash[21527]: audit 2024-03-26T03:55:50.104104+0000 mon.c (mon.2) 516 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:55:51.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:51 smithi190 bash[23392]: audit 2024-03-26T03:55:50.092689+0000 mgr.y (mgr.24431) 448 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:55:51.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:51 smithi190 bash[23392]: cluster 2024-03-26T03:55:50.827130+0000 mgr.y (mgr.24431) 449 : cluster [DBG] pgmap v664: 292 pgs: 292 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 741 B/s wr, 3 op/s 2024-03-26T03:55:51.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:51 smithi175 bash[29284]: audit 2024-03-26T03:55:50.092689+0000 mgr.y (mgr.24431) 448 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:55:51.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:51 smithi175 bash[29284]: cluster 2024-03-26T03:55:50.827130+0000 mgr.y (mgr.24431) 449 : cluster [DBG] pgmap v664: 292 pgs: 292 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 741 B/s wr, 3 op/s 2024-03-26T03:55:51.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:51 smithi175 bash[21527]: audit 2024-03-26T03:55:50.092689+0000 mgr.y (mgr.24431) 448 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:55:51.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:51 smithi175 bash[21527]: cluster 2024-03-26T03:55:50.827130+0000 mgr.y (mgr.24431) 449 : cluster [DBG] pgmap v664: 292 pgs: 292 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 741 B/s wr, 3 op/s 2024-03-26T03:55:52.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:52 smithi190 bash[23392]: audit 2024-03-26T03:55:52.280073+0000 mon.a (mon.0) 2590 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:55:52.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:52 smithi190 bash[23392]: audit 2024-03-26T03:55:52.281000+0000 mon.a (mon.0) 2591 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-73"}]: dispatch 2024-03-26T03:55:52.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:52 smithi175 bash[29284]: audit 2024-03-26T03:55:52.280073+0000 mon.a (mon.0) 2590 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:55:52.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:52 smithi175 bash[29284]: audit 2024-03-26T03:55:52.281000+0000 mon.a (mon.0) 2591 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-73"}]: dispatch 2024-03-26T03:55:52.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:52 smithi175 bash[21527]: audit 2024-03-26T03:55:52.280073+0000 mon.a (mon.0) 2590 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:55:52.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:52 smithi175 bash[21527]: audit 2024-03-26T03:55:52.281000+0000 mon.a (mon.0) 2591 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-73"}]: dispatch 2024-03-26T03:55:53.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:53 smithi190 bash[23392]: cluster 2024-03-26T03:55:52.594243+0000 mon.a (mon.0) 2592 : cluster [DBG] osdmap e421: 8 total, 8 up, 8 in 2024-03-26T03:55:53.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:53 smithi190 bash[23392]: cluster 2024-03-26T03:55:52.828005+0000 mgr.y (mgr.24431) 450 : cluster [DBG] pgmap v666: 260 pgs: 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 306 B/s wr, 1 op/s 2024-03-26T03:55:53.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:53 smithi175 bash[29284]: cluster 2024-03-26T03:55:52.594243+0000 mon.a (mon.0) 2592 : cluster [DBG] osdmap e421: 8 total, 8 up, 8 in 2024-03-26T03:55:53.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:53 smithi175 bash[29284]: cluster 2024-03-26T03:55:52.828005+0000 mgr.y (mgr.24431) 450 : cluster [DBG] pgmap v666: 260 pgs: 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 306 B/s wr, 1 op/s 2024-03-26T03:55:53.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:53 smithi175 bash[21527]: cluster 2024-03-26T03:55:52.594243+0000 mon.a (mon.0) 2592 : cluster [DBG] osdmap e421: 8 total, 8 up, 8 in 2024-03-26T03:55:53.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:53 smithi175 bash[21527]: cluster 2024-03-26T03:55:52.828005+0000 mgr.y (mgr.24431) 450 : cluster [DBG] pgmap v666: 260 pgs: 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 306 B/s wr, 1 op/s 2024-03-26T03:55:54.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:54 smithi190 bash[23392]: cluster 2024-03-26T03:55:53.607921+0000 mon.a (mon.0) 2593 : cluster [DBG] osdmap e422: 8 total, 8 up, 8 in 2024-03-26T03:55:54.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:54 smithi190 bash[23392]: audit 2024-03-26T03:55:53.623549+0000 mon.a (mon.0) 2594 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:55:54.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:54 smithi175 bash[29284]: cluster 2024-03-26T03:55:53.607921+0000 mon.a (mon.0) 2593 : cluster [DBG] osdmap e422: 8 total, 8 up, 8 in 2024-03-26T03:55:54.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:54 smithi175 bash[29284]: audit 2024-03-26T03:55:53.623549+0000 mon.a (mon.0) 2594 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:55:54.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:54 smithi175 bash[21527]: cluster 2024-03-26T03:55:53.607921+0000 mon.a (mon.0) 2593 : cluster [DBG] osdmap e422: 8 total, 8 up, 8 in 2024-03-26T03:55:54.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:54 smithi175 bash[21527]: audit 2024-03-26T03:55:53.623549+0000 mon.a (mon.0) 2594 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:55:55.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:55 smithi190 bash[23392]: audit 2024-03-26T03:55:54.600109+0000 mon.a (mon.0) 2595 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:55:55.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:55 smithi190 bash[23392]: cluster 2024-03-26T03:55:54.610014+0000 mon.a (mon.0) 2596 : cluster [DBG] osdmap e423: 8 total, 8 up, 8 in 2024-03-26T03:55:55.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:55 smithi190 bash[23392]: audit 2024-03-26T03:55:54.617912+0000 mon.a (mon.0) 2597 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:55:55.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:55 smithi190 bash[23392]: cluster 2024-03-26T03:55:54.830271+0000 mgr.y (mgr.24431) 451 : cluster [DBG] pgmap v669: 292 pgs: 3 unknown, 289 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:55:55.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:55 smithi175 bash[29284]: audit 2024-03-26T03:55:54.600109+0000 mon.a (mon.0) 2595 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:55:55.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:55 smithi175 bash[29284]: cluster 2024-03-26T03:55:54.610014+0000 mon.a (mon.0) 2596 : cluster [DBG] osdmap e423: 8 total, 8 up, 8 in 2024-03-26T03:55:55.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:55 smithi175 bash[29284]: audit 2024-03-26T03:55:54.617912+0000 mon.a (mon.0) 2597 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:55:55.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:55 smithi175 bash[29284]: cluster 2024-03-26T03:55:54.830271+0000 mgr.y (mgr.24431) 451 : cluster [DBG] pgmap v669: 292 pgs: 3 unknown, 289 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:55:55.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:55 smithi175 bash[21527]: audit 2024-03-26T03:55:54.600109+0000 mon.a (mon.0) 2595 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:55:55.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:55 smithi175 bash[21527]: cluster 2024-03-26T03:55:54.610014+0000 mon.a (mon.0) 2596 : cluster [DBG] osdmap e423: 8 total, 8 up, 8 in 2024-03-26T03:55:55.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:55 smithi175 bash[21527]: audit 2024-03-26T03:55:54.617912+0000 mon.a (mon.0) 2597 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:55:55.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:55 smithi175 bash[21527]: cluster 2024-03-26T03:55:54.830271+0000 mgr.y (mgr.24431) 451 : cluster [DBG] pgmap v669: 292 pgs: 3 unknown, 289 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:55:56.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:56 smithi190 bash[23392]: cluster 2024-03-26T03:55:55.651315+0000 mon.a (mon.0) 2598 : cluster [DBG] osdmap e424: 8 total, 8 up, 8 in 2024-03-26T03:55:56.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:56 smithi190 bash[23392]: audit 2024-03-26T03:55:55.765059+0000 mon.c (mon.2) 517 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:55:56.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:56 smithi190 bash[23392]: audit 2024-03-26T03:55:56.294534+0000 mon.a (mon.0) 2599 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:55:56.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:56 smithi190 bash[23392]: audit 2024-03-26T03:55:56.303892+0000 mon.a (mon.0) 2600 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:55:56.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:56 smithi190 bash[23392]: audit 2024-03-26T03:55:56.444125+0000 mon.c (mon.2) 518 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:56.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:56 smithi190 bash[23392]: audit 2024-03-26T03:55:56.444893+0000 mon.a (mon.0) 2601 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:56.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:56 smithi190 bash[23392]: audit 2024-03-26T03:55:56.446930+0000 mon.c (mon.2) 519 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:56.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:56 smithi190 bash[23392]: audit 2024-03-26T03:55:56.447666+0000 mon.a (mon.0) 2602 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:56.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:56 smithi190 bash[23392]: audit 2024-03-26T03:55:56.449736+0000 mon.c (mon.2) 520 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:56.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:56 smithi190 bash[23392]: audit 2024-03-26T03:55:56.450395+0000 mon.a (mon.0) 2603 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:56.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:56 smithi190 bash[23392]: audit 2024-03-26T03:55:56.452372+0000 mon.c (mon.2) 521 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:56.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:56 smithi190 bash[23392]: audit 2024-03-26T03:55:56.453056+0000 mon.a (mon.0) 2604 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:56.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:56 smithi190 bash[23392]: cephadm 2024-03-26T03:55:56.454666+0000 mgr.y (mgr.24431) 452 : cephadm [INF] Adjusting osd_memory_target on smithi175 to 3292M 2024-03-26T03:55:56.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:56 smithi190 bash[23392]: audit 2024-03-26T03:55:56.463989+0000 mon.a (mon.0) 2605 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:55:56.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:56 smithi175 bash[21527]: cluster 2024-03-26T03:55:55.651315+0000 mon.a (mon.0) 2598 : cluster [DBG] osdmap e424: 8 total, 8 up, 8 in 2024-03-26T03:55:56.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:56 smithi175 bash[21527]: audit 2024-03-26T03:55:55.765059+0000 mon.c (mon.2) 517 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:55:56.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:56 smithi175 bash[21527]: audit 2024-03-26T03:55:56.294534+0000 mon.a (mon.0) 2599 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:55:56.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:56 smithi175 bash[21527]: audit 2024-03-26T03:55:56.303892+0000 mon.a (mon.0) 2600 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:55:56.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:56 smithi175 bash[21527]: audit 2024-03-26T03:55:56.444125+0000 mon.c (mon.2) 518 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:56.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:56 smithi175 bash[21527]: audit 2024-03-26T03:55:56.444893+0000 mon.a (mon.0) 2601 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:56.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:56 smithi175 bash[21527]: audit 2024-03-26T03:55:56.446930+0000 mon.c (mon.2) 519 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:56.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:56 smithi175 bash[21527]: audit 2024-03-26T03:55:56.447666+0000 mon.a (mon.0) 2602 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:56.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:56 smithi175 bash[21527]: audit 2024-03-26T03:55:56.449736+0000 mon.c (mon.2) 520 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:56.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:56 smithi175 bash[21527]: audit 2024-03-26T03:55:56.450395+0000 mon.a (mon.0) 2603 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:56.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:56 smithi175 bash[21527]: audit 2024-03-26T03:55:56.452372+0000 mon.c (mon.2) 521 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:56.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:56 smithi175 bash[21527]: audit 2024-03-26T03:55:56.453056+0000 mon.a (mon.0) 2604 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:56.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:56 smithi175 bash[21527]: cephadm 2024-03-26T03:55:56.454666+0000 mgr.y (mgr.24431) 452 : cephadm [INF] Adjusting osd_memory_target on smithi175 to 3292M 2024-03-26T03:55:56.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:56 smithi175 bash[21527]: audit 2024-03-26T03:55:56.463989+0000 mon.a (mon.0) 2605 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:55:56.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:56 smithi175 bash[29284]: cluster 2024-03-26T03:55:55.651315+0000 mon.a (mon.0) 2598 : cluster [DBG] osdmap e424: 8 total, 8 up, 8 in 2024-03-26T03:55:56.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:56 smithi175 bash[29284]: audit 2024-03-26T03:55:55.765059+0000 mon.c (mon.2) 517 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:55:56.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:56 smithi175 bash[29284]: audit 2024-03-26T03:55:56.294534+0000 mon.a (mon.0) 2599 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:55:56.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:56 smithi175 bash[29284]: audit 2024-03-26T03:55:56.303892+0000 mon.a (mon.0) 2600 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:55:56.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:56 smithi175 bash[29284]: audit 2024-03-26T03:55:56.444125+0000 mon.c (mon.2) 518 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:56.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:56 smithi175 bash[29284]: audit 2024-03-26T03:55:56.444893+0000 mon.a (mon.0) 2601 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:56.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:56 smithi175 bash[29284]: audit 2024-03-26T03:55:56.446930+0000 mon.c (mon.2) 519 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:56.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:56 smithi175 bash[29284]: audit 2024-03-26T03:55:56.447666+0000 mon.a (mon.0) 2602 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:56.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:56 smithi175 bash[29284]: audit 2024-03-26T03:55:56.449736+0000 mon.c (mon.2) 520 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:56.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:56 smithi175 bash[29284]: audit 2024-03-26T03:55:56.450395+0000 mon.a (mon.0) 2603 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:56.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:56 smithi175 bash[29284]: audit 2024-03-26T03:55:56.452372+0000 mon.c (mon.2) 521 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:56.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:56 smithi175 bash[29284]: audit 2024-03-26T03:55:56.453056+0000 mon.a (mon.0) 2604 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:56.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:56 smithi175 bash[29284]: cephadm 2024-03-26T03:55:56.454666+0000 mgr.y (mgr.24431) 452 : cephadm [INF] Adjusting osd_memory_target on smithi175 to 3292M 2024-03-26T03:55:56.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:56 smithi175 bash[29284]: audit 2024-03-26T03:55:56.463989+0000 mon.a (mon.0) 2605 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:55:57.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:57 smithi190 bash[23392]: cluster 2024-03-26T03:55:56.664096+0000 mon.a (mon.0) 2606 : cluster [DBG] osdmap e425: 8 total, 8 up, 8 in 2024-03-26T03:55:57.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:57 smithi190 bash[23392]: audit 2024-03-26T03:55:56.765455+0000 mon.a (mon.0) 2607 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:55:57.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:57 smithi190 bash[23392]: audit 2024-03-26T03:55:56.766299+0000 mon.a (mon.0) 2608 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-75"}]: dispatch 2024-03-26T03:55:57.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:57 smithi190 bash[23392]: cluster 2024-03-26T03:55:56.830939+0000 mgr.y (mgr.24431) 453 : cluster [DBG] pgmap v672: 292 pgs: 292 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-26T03:55:57.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:57 smithi190 bash[23392]: audit 2024-03-26T03:55:56.938391+0000 mon.c (mon.2) 522 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:57.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:57 smithi190 bash[23392]: audit 2024-03-26T03:55:56.939047+0000 mon.a (mon.0) 2609 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:57.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:57 smithi190 bash[23392]: audit 2024-03-26T03:55:56.940928+0000 mon.c (mon.2) 523 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:57.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:57 smithi190 bash[23392]: audit 2024-03-26T03:55:56.941458+0000 mon.a (mon.0) 2610 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:57.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:57 smithi190 bash[23392]: audit 2024-03-26T03:55:56.943178+0000 mon.c (mon.2) 524 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:57.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:57 smithi190 bash[23392]: audit 2024-03-26T03:55:56.943724+0000 mon.a (mon.0) 2611 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:57.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:57 smithi190 bash[23392]: audit 2024-03-26T03:55:56.945445+0000 mon.c (mon.2) 525 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:57.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:57 smithi190 bash[23392]: audit 2024-03-26T03:55:56.945986+0000 mon.a (mon.0) 2612 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:57.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:57 smithi190 bash[23392]: cephadm 2024-03-26T03:55:56.947474+0000 mgr.y (mgr.24431) 454 : cephadm [INF] Adjusting osd_memory_target on smithi190 to 3292M 2024-03-26T03:55:57.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:57 smithi190 bash[23392]: audit 2024-03-26T03:55:56.955905+0000 mon.a (mon.0) 2613 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:55:57.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:57 smithi190 bash[23392]: audit 2024-03-26T03:55:56.961322+0000 mon.c (mon.2) 526 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:55:57.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:57 smithi190 bash[23392]: audit 2024-03-26T03:55:56.963064+0000 mon.c (mon.2) 527 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:55:57.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:57 smithi190 bash[23392]: audit 2024-03-26T03:55:56.973622+0000 mon.a (mon.0) 2614 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:55:57.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[21527]: cluster 2024-03-26T03:55:56.664096+0000 mon.a (mon.0) 2606 : cluster [DBG] osdmap e425: 8 total, 8 up, 8 in 2024-03-26T03:55:57.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[21527]: audit 2024-03-26T03:55:56.765455+0000 mon.a (mon.0) 2607 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:55:57.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[21527]: audit 2024-03-26T03:55:56.766299+0000 mon.a (mon.0) 2608 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-75"}]: dispatch 2024-03-26T03:55:57.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[21527]: cluster 2024-03-26T03:55:56.830939+0000 mgr.y (mgr.24431) 453 : cluster [DBG] pgmap v672: 292 pgs: 292 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-26T03:55:57.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[21527]: audit 2024-03-26T03:55:56.938391+0000 mon.c (mon.2) 522 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:57.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[21527]: audit 2024-03-26T03:55:56.939047+0000 mon.a (mon.0) 2609 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:57.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[21527]: audit 2024-03-26T03:55:56.940928+0000 mon.c (mon.2) 523 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:57.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[21527]: audit 2024-03-26T03:55:56.941458+0000 mon.a (mon.0) 2610 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:57.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[21527]: audit 2024-03-26T03:55:56.943178+0000 mon.c (mon.2) 524 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:57.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[21527]: audit 2024-03-26T03:55:56.943724+0000 mon.a (mon.0) 2611 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:57.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[21527]: audit 2024-03-26T03:55:56.945445+0000 mon.c (mon.2) 525 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:57.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[21527]: audit 2024-03-26T03:55:56.945986+0000 mon.a (mon.0) 2612 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:57.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[21527]: cephadm 2024-03-26T03:55:56.947474+0000 mgr.y (mgr.24431) 454 : cephadm [INF] Adjusting osd_memory_target on smithi190 to 3292M 2024-03-26T03:55:57.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[21527]: audit 2024-03-26T03:55:56.955905+0000 mon.a (mon.0) 2613 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:55:57.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[21527]: audit 2024-03-26T03:55:56.961322+0000 mon.c (mon.2) 526 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:55:57.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[21527]: audit 2024-03-26T03:55:56.963064+0000 mon.c (mon.2) 527 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:55:57.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[21527]: audit 2024-03-26T03:55:56.973622+0000 mon.a (mon.0) 2614 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:55:57.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[29284]: cluster 2024-03-26T03:55:56.664096+0000 mon.a (mon.0) 2606 : cluster [DBG] osdmap e425: 8 total, 8 up, 8 in 2024-03-26T03:55:57.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[29284]: audit 2024-03-26T03:55:56.765455+0000 mon.a (mon.0) 2607 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:55:57.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[29284]: audit 2024-03-26T03:55:56.766299+0000 mon.a (mon.0) 2608 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-75"}]: dispatch 2024-03-26T03:55:57.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[29284]: cluster 2024-03-26T03:55:56.830939+0000 mgr.y (mgr.24431) 453 : cluster [DBG] pgmap v672: 292 pgs: 292 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-26T03:55:57.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[29284]: audit 2024-03-26T03:55:56.938391+0000 mon.c (mon.2) 522 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:57.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[29284]: audit 2024-03-26T03:55:56.939047+0000 mon.a (mon.0) 2609 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:57.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[29284]: audit 2024-03-26T03:55:56.940928+0000 mon.c (mon.2) 523 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:57.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[29284]: audit 2024-03-26T03:55:56.941458+0000 mon.a (mon.0) 2610 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:57.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[29284]: audit 2024-03-26T03:55:56.943178+0000 mon.c (mon.2) 524 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:57.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[29284]: audit 2024-03-26T03:55:56.943724+0000 mon.a (mon.0) 2611 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:57.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[29284]: audit 2024-03-26T03:55:56.945445+0000 mon.c (mon.2) 525 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:57.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[29284]: audit 2024-03-26T03:55:56.945986+0000 mon.a (mon.0) 2612 : audit [INF] from='mgr.24431 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T03:55:57.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[29284]: cephadm 2024-03-26T03:55:56.947474+0000 mgr.y (mgr.24431) 454 : cephadm [INF] Adjusting osd_memory_target on smithi190 to 3292M 2024-03-26T03:55:57.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[29284]: audit 2024-03-26T03:55:56.955905+0000 mon.a (mon.0) 2613 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:55:57.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[29284]: audit 2024-03-26T03:55:56.961322+0000 mon.c (mon.2) 526 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:55:57.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[29284]: audit 2024-03-26T03:55:56.963064+0000 mon.c (mon.2) 527 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:55:57.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:57 smithi175 bash[29284]: audit 2024-03-26T03:55:56.973622+0000 mon.a (mon.0) 2614 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:55:58.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:58 smithi190 bash[23392]: cluster 2024-03-26T03:55:57.676294+0000 mon.a (mon.0) 2615 : cluster [DBG] osdmap e426: 8 total, 8 up, 8 in 2024-03-26T03:55:58.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:58 smithi175 bash[21527]: cluster 2024-03-26T03:55:57.676294+0000 mon.a (mon.0) 2615 : cluster [DBG] osdmap e426: 8 total, 8 up, 8 in 2024-03-26T03:55:58.992 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:55:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:55:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:55:58.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:58 smithi175 bash[29284]: cluster 2024-03-26T03:55:57.676294+0000 mon.a (mon.0) 2615 : cluster [DBG] osdmap e426: 8 total, 8 up, 8 in 2024-03-26T03:56:00.102 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:59 smithi190 bash[23392]: cluster 2024-03-26T03:55:58.682077+0000 mon.a (mon.0) 2616 : cluster [DBG] osdmap e427: 8 total, 8 up, 8 in 2024-03-26T03:56:00.103 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:59 smithi190 bash[23392]: audit 2024-03-26T03:55:58.688082+0000 mon.a (mon.0) 2617 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:56:00.103 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:59 smithi190 bash[23392]: cluster 2024-03-26T03:55:58.831839+0000 mgr.y (mgr.24431) 455 : cluster [DBG] pgmap v675: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:56:00.103 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:59 smithi190 bash[23392]: audit 2024-03-26T03:55:59.672612+0000 mon.a (mon.0) 2618 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:56:00.103 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:55:59 smithi190 bash[23392]: cluster 2024-03-26T03:55:59.684967+0000 mon.a (mon.0) 2619 : cluster [DBG] osdmap e428: 8 total, 8 up, 8 in 2024-03-26T03:56:00.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:59 smithi175 bash[21527]: cluster 2024-03-26T03:55:58.682077+0000 mon.a (mon.0) 2616 : cluster [DBG] osdmap e427: 8 total, 8 up, 8 in 2024-03-26T03:56:00.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:59 smithi175 bash[21527]: audit 2024-03-26T03:55:58.688082+0000 mon.a (mon.0) 2617 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:56:00.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:59 smithi175 bash[21527]: cluster 2024-03-26T03:55:58.831839+0000 mgr.y (mgr.24431) 455 : cluster [DBG] pgmap v675: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:56:00.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:59 smithi175 bash[21527]: audit 2024-03-26T03:55:59.672612+0000 mon.a (mon.0) 2618 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:56:00.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:55:59 smithi175 bash[21527]: cluster 2024-03-26T03:55:59.684967+0000 mon.a (mon.0) 2619 : cluster [DBG] osdmap e428: 8 total, 8 up, 8 in 2024-03-26T03:56:00.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:59 smithi175 bash[29284]: cluster 2024-03-26T03:55:58.682077+0000 mon.a (mon.0) 2616 : cluster [DBG] osdmap e427: 8 total, 8 up, 8 in 2024-03-26T03:56:00.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:59 smithi175 bash[29284]: audit 2024-03-26T03:55:58.688082+0000 mon.a (mon.0) 2617 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:56:00.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:59 smithi175 bash[29284]: cluster 2024-03-26T03:55:58.831839+0000 mgr.y (mgr.24431) 455 : cluster [DBG] pgmap v675: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:56:00.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:59 smithi175 bash[29284]: audit 2024-03-26T03:55:59.672612+0000 mon.a (mon.0) 2618 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:56:00.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:55:59 smithi175 bash[29284]: cluster 2024-03-26T03:55:59.684967+0000 mon.a (mon.0) 2619 : cluster [DBG] osdmap e428: 8 total, 8 up, 8 in 2024-03-26T03:56:00.491 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:56:00 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:56:01.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:00 smithi190 bash[23392]: cluster 2024-03-26T03:55:59.689432+0000 mon.a (mon.0) 2620 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:56:01.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:00 smithi190 bash[23392]: audit 2024-03-26T03:55:59.701840+0000 mon.a (mon.0) 2621 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:56:01.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:00 smithi190 bash[23392]: audit 2024-03-26T03:56:00.104069+0000 mgr.y (mgr.24431) 456 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:56:01.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:00 smithi190 bash[23392]: cluster 2024-03-26T03:56:00.685053+0000 mon.a (mon.0) 2622 : cluster [DBG] osdmap e429: 8 total, 8 up, 8 in 2024-03-26T03:56:01.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:00 smithi175 bash[21527]: cluster 2024-03-26T03:55:59.689432+0000 mon.a (mon.0) 2620 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:56:01.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:00 smithi175 bash[21527]: audit 2024-03-26T03:55:59.701840+0000 mon.a (mon.0) 2621 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:56:01.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:00 smithi175 bash[21527]: audit 2024-03-26T03:56:00.104069+0000 mgr.y (mgr.24431) 456 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:56:01.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:00 smithi175 bash[21527]: cluster 2024-03-26T03:56:00.685053+0000 mon.a (mon.0) 2622 : cluster [DBG] osdmap e429: 8 total, 8 up, 8 in 2024-03-26T03:56:01.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:00 smithi175 bash[29284]: cluster 2024-03-26T03:55:59.689432+0000 mon.a (mon.0) 2620 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:56:01.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:00 smithi175 bash[29284]: audit 2024-03-26T03:55:59.701840+0000 mon.a (mon.0) 2621 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:56:01.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:00 smithi175 bash[29284]: audit 2024-03-26T03:56:00.104069+0000 mgr.y (mgr.24431) 456 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:56:01.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:00 smithi175 bash[29284]: cluster 2024-03-26T03:56:00.685053+0000 mon.a (mon.0) 2622 : cluster [DBG] osdmap e429: 8 total, 8 up, 8 in 2024-03-26T03:56:02.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:01 smithi190 bash[23392]: audit 2024-03-26T03:56:00.787729+0000 mon.a (mon.0) 2623 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:56:02.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:01 smithi190 bash[23392]: audit 2024-03-26T03:56:00.788652+0000 mon.a (mon.0) 2624 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-77"}]: dispatch 2024-03-26T03:56:02.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:01 smithi190 bash[23392]: cluster 2024-03-26T03:56:00.832708+0000 mgr.y (mgr.24431) 457 : cluster [DBG] pgmap v678: 292 pgs: 4 creating+peering, 18 creating+activating, 270 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-26T03:56:02.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:01 smithi175 bash[21527]: audit 2024-03-26T03:56:00.787729+0000 mon.a (mon.0) 2623 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:56:02.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:01 smithi175 bash[21527]: audit 2024-03-26T03:56:00.788652+0000 mon.a (mon.0) 2624 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-77"}]: dispatch 2024-03-26T03:56:02.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:01 smithi175 bash[21527]: cluster 2024-03-26T03:56:00.832708+0000 mgr.y (mgr.24431) 457 : cluster [DBG] pgmap v678: 292 pgs: 4 creating+peering, 18 creating+activating, 270 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-26T03:56:02.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:01 smithi175 bash[29284]: audit 2024-03-26T03:56:00.787729+0000 mon.a (mon.0) 2623 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:56:02.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:01 smithi175 bash[29284]: audit 2024-03-26T03:56:00.788652+0000 mon.a (mon.0) 2624 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-77"}]: dispatch 2024-03-26T03:56:02.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:01 smithi175 bash[29284]: cluster 2024-03-26T03:56:00.832708+0000 mgr.y (mgr.24431) 457 : cluster [DBG] pgmap v678: 292 pgs: 4 creating+peering, 18 creating+activating, 270 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-26T03:56:03.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:02 smithi190 bash[23392]: cluster 2024-03-26T03:56:01.757563+0000 mon.a (mon.0) 2625 : cluster [DBG] osdmap e430: 8 total, 8 up, 8 in 2024-03-26T03:56:03.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:02 smithi190 bash[23392]: cluster 2024-03-26T03:56:02.761559+0000 mon.a (mon.0) 2626 : cluster [DBG] osdmap e431: 8 total, 8 up, 8 in 2024-03-26T03:56:03.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:02 smithi190 bash[23392]: audit 2024-03-26T03:56:02.764749+0000 mon.a (mon.0) 2627 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:56:03.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:02 smithi175 bash[21527]: cluster 2024-03-26T03:56:01.757563+0000 mon.a (mon.0) 2625 : cluster [DBG] osdmap e430: 8 total, 8 up, 8 in 2024-03-26T03:56:03.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:02 smithi175 bash[21527]: cluster 2024-03-26T03:56:02.761559+0000 mon.a (mon.0) 2626 : cluster [DBG] osdmap e431: 8 total, 8 up, 8 in 2024-03-26T03:56:03.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:02 smithi175 bash[21527]: audit 2024-03-26T03:56:02.764749+0000 mon.a (mon.0) 2627 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:56:03.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:02 smithi175 bash[29284]: cluster 2024-03-26T03:56:01.757563+0000 mon.a (mon.0) 2625 : cluster [DBG] osdmap e430: 8 total, 8 up, 8 in 2024-03-26T03:56:03.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:02 smithi175 bash[29284]: cluster 2024-03-26T03:56:02.761559+0000 mon.a (mon.0) 2626 : cluster [DBG] osdmap e431: 8 total, 8 up, 8 in 2024-03-26T03:56:03.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:02 smithi175 bash[29284]: audit 2024-03-26T03:56:02.764749+0000 mon.a (mon.0) 2627 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:56:04.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:03 smithi190 bash[23392]: cluster 2024-03-26T03:56:02.833660+0000 mgr.y (mgr.24431) 458 : cluster [DBG] pgmap v681: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-26T03:56:04.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:03 smithi190 bash[23392]: audit 2024-03-26T03:56:03.758378+0000 mon.a (mon.0) 2628 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:56:04.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:03 smithi190 bash[23392]: cluster 2024-03-26T03:56:03.767299+0000 mon.a (mon.0) 2629 : cluster [DBG] osdmap e432: 8 total, 8 up, 8 in 2024-03-26T03:56:04.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:03 smithi190 bash[23392]: audit 2024-03-26T03:56:03.780889+0000 mon.a (mon.0) 2630 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:56:04.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:03 smithi175 bash[21527]: cluster 2024-03-26T03:56:02.833660+0000 mgr.y (mgr.24431) 458 : cluster [DBG] pgmap v681: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-26T03:56:04.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:03 smithi175 bash[21527]: audit 2024-03-26T03:56:03.758378+0000 mon.a (mon.0) 2628 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:56:04.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:03 smithi175 bash[21527]: cluster 2024-03-26T03:56:03.767299+0000 mon.a (mon.0) 2629 : cluster [DBG] osdmap e432: 8 total, 8 up, 8 in 2024-03-26T03:56:04.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:03 smithi175 bash[21527]: audit 2024-03-26T03:56:03.780889+0000 mon.a (mon.0) 2630 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:56:04.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:03 smithi175 bash[29284]: cluster 2024-03-26T03:56:02.833660+0000 mgr.y (mgr.24431) 458 : cluster [DBG] pgmap v681: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-26T03:56:04.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:03 smithi175 bash[29284]: audit 2024-03-26T03:56:03.758378+0000 mon.a (mon.0) 2628 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:56:04.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:03 smithi175 bash[29284]: cluster 2024-03-26T03:56:03.767299+0000 mon.a (mon.0) 2629 : cluster [DBG] osdmap e432: 8 total, 8 up, 8 in 2024-03-26T03:56:04.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:03 smithi175 bash[29284]: audit 2024-03-26T03:56:03.780889+0000 mon.a (mon.0) 2630 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:56:06.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:05 smithi190 bash[23392]: cluster 2024-03-26T03:56:04.806423+0000 mon.a (mon.0) 2631 : cluster [DBG] osdmap e433: 8 total, 8 up, 8 in 2024-03-26T03:56:06.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:05 smithi190 bash[23392]: cluster 2024-03-26T03:56:04.834869+0000 mgr.y (mgr.24431) 459 : cluster [DBG] pgmap v684: 292 pgs: 2 unknown, 290 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 767 B/s wr, 1 op/s 2024-03-26T03:56:06.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:05 smithi190 bash[23392]: audit 2024-03-26T03:56:05.139263+0000 mon.c (mon.2) 528 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:56:06.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:05 smithi175 bash[21527]: cluster 2024-03-26T03:56:04.806423+0000 mon.a (mon.0) 2631 : cluster [DBG] osdmap e433: 8 total, 8 up, 8 in 2024-03-26T03:56:06.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:05 smithi175 bash[21527]: cluster 2024-03-26T03:56:04.834869+0000 mgr.y (mgr.24431) 459 : cluster [DBG] pgmap v684: 292 pgs: 2 unknown, 290 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 767 B/s wr, 1 op/s 2024-03-26T03:56:06.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:05 smithi175 bash[21527]: audit 2024-03-26T03:56:05.139263+0000 mon.c (mon.2) 528 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:56:06.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:05 smithi175 bash[29284]: cluster 2024-03-26T03:56:04.806423+0000 mon.a (mon.0) 2631 : cluster [DBG] osdmap e433: 8 total, 8 up, 8 in 2024-03-26T03:56:06.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:05 smithi175 bash[29284]: cluster 2024-03-26T03:56:04.834869+0000 mgr.y (mgr.24431) 459 : cluster [DBG] pgmap v684: 292 pgs: 2 unknown, 290 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 767 B/s wr, 1 op/s 2024-03-26T03:56:06.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:05 smithi175 bash[29284]: audit 2024-03-26T03:56:05.139263+0000 mon.c (mon.2) 528 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:56:07.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:06 smithi190 bash[23392]: cluster 2024-03-26T03:56:05.797700+0000 mon.a (mon.0) 2632 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:56:07.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:06 smithi190 bash[23392]: cluster 2024-03-26T03:56:05.820862+0000 mon.a (mon.0) 2633 : cluster [DBG] osdmap e434: 8 total, 8 up, 8 in 2024-03-26T03:56:07.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:06 smithi190 bash[23392]: audit 2024-03-26T03:56:05.866826+0000 mon.a (mon.0) 2634 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "298.1d"}]: dispatch 2024-03-26T03:56:07.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:06 smithi190 bash[23392]: audit 2024-03-26T03:56:05.867226+0000 mgr.y (mgr.24431) 460 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "298.1d"}]: dispatch 2024-03-26T03:56:07.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:06 smithi190 bash[23392]: cluster 2024-03-26T03:56:06.042479+0000 osd.1 (osd.1) 9 : cluster [DBG] 298.1d deep-scrub starts 2024-03-26T03:56:07.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:06 smithi190 bash[23392]: cluster 2024-03-26T03:56:06.047190+0000 osd.1 (osd.1) 10 : cluster [DBG] 298.1d deep-scrub ok 2024-03-26T03:56:07.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:06 smithi175 bash[21527]: cluster 2024-03-26T03:56:05.797700+0000 mon.a (mon.0) 2632 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:56:07.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:06 smithi175 bash[21527]: cluster 2024-03-26T03:56:05.820862+0000 mon.a (mon.0) 2633 : cluster [DBG] osdmap e434: 8 total, 8 up, 8 in 2024-03-26T03:56:07.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:06 smithi175 bash[21527]: audit 2024-03-26T03:56:05.866826+0000 mon.a (mon.0) 2634 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "298.1d"}]: dispatch 2024-03-26T03:56:07.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:06 smithi175 bash[21527]: audit 2024-03-26T03:56:05.867226+0000 mgr.y (mgr.24431) 460 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "298.1d"}]: dispatch 2024-03-26T03:56:07.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:06 smithi175 bash[21527]: cluster 2024-03-26T03:56:06.042479+0000 osd.1 (osd.1) 9 : cluster [DBG] 298.1d deep-scrub starts 2024-03-26T03:56:07.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:06 smithi175 bash[21527]: cluster 2024-03-26T03:56:06.047190+0000 osd.1 (osd.1) 10 : cluster [DBG] 298.1d deep-scrub ok 2024-03-26T03:56:07.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:06 smithi175 bash[29284]: cluster 2024-03-26T03:56:05.797700+0000 mon.a (mon.0) 2632 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:56:07.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:06 smithi175 bash[29284]: cluster 2024-03-26T03:56:05.820862+0000 mon.a (mon.0) 2633 : cluster [DBG] osdmap e434: 8 total, 8 up, 8 in 2024-03-26T03:56:07.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:06 smithi175 bash[29284]: audit 2024-03-26T03:56:05.866826+0000 mon.a (mon.0) 2634 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "298.1d"}]: dispatch 2024-03-26T03:56:07.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:06 smithi175 bash[29284]: audit 2024-03-26T03:56:05.867226+0000 mgr.y (mgr.24431) 460 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "298.1d"}]: dispatch 2024-03-26T03:56:07.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:06 smithi175 bash[29284]: cluster 2024-03-26T03:56:06.042479+0000 osd.1 (osd.1) 9 : cluster [DBG] 298.1d deep-scrub starts 2024-03-26T03:56:07.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:06 smithi175 bash[29284]: cluster 2024-03-26T03:56:06.047190+0000 osd.1 (osd.1) 10 : cluster [DBG] 298.1d deep-scrub ok 2024-03-26T03:56:08.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:07 smithi190 bash[23392]: cluster 2024-03-26T03:56:06.835794+0000 mgr.y (mgr.24431) 461 : cluster [DBG] pgmap v686: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-26T03:56:08.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:07 smithi175 bash[21527]: cluster 2024-03-26T03:56:06.835794+0000 mgr.y (mgr.24431) 461 : cluster [DBG] pgmap v686: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-26T03:56:08.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:07 smithi175 bash[29284]: cluster 2024-03-26T03:56:06.835794+0000 mgr.y (mgr.24431) 461 : cluster [DBG] pgmap v686: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-26T03:56:09.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:56:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:56:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:56:10.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:09 smithi190 bash[23392]: cluster 2024-03-26T03:56:08.836680+0000 mgr.y (mgr.24431) 462 : cluster [DBG] pgmap v687: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 852 B/s wr, 2 op/s 2024-03-26T03:56:10.242 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:56:10 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:56:10.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:09 smithi175 bash[21527]: cluster 2024-03-26T03:56:08.836680+0000 mgr.y (mgr.24431) 462 : cluster [DBG] pgmap v687: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 852 B/s wr, 2 op/s 2024-03-26T03:56:10.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:09 smithi175 bash[29284]: cluster 2024-03-26T03:56:08.836680+0000 mgr.y (mgr.24431) 462 : cluster [DBG] pgmap v687: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 852 B/s wr, 2 op/s 2024-03-26T03:56:11.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:10 smithi190 bash[23392]: audit 2024-03-26T03:56:10.115696+0000 mgr.y (mgr.24431) 463 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:56:11.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:10 smithi175 bash[29284]: audit 2024-03-26T03:56:10.115696+0000 mgr.y (mgr.24431) 463 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:56:11.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:10 smithi175 bash[21527]: audit 2024-03-26T03:56:10.115696+0000 mgr.y (mgr.24431) 463 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:56:12.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:11 smithi190 bash[23392]: cluster 2024-03-26T03:56:10.839413+0000 mgr.y (mgr.24431) 464 : cluster [DBG] pgmap v688: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1013 B/s wr, 3 op/s 2024-03-26T03:56:12.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:11 smithi175 bash[21527]: cluster 2024-03-26T03:56:10.839413+0000 mgr.y (mgr.24431) 464 : cluster [DBG] pgmap v688: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1013 B/s wr, 3 op/s 2024-03-26T03:56:12.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:11 smithi175 bash[29284]: cluster 2024-03-26T03:56:10.839413+0000 mgr.y (mgr.24431) 464 : cluster [DBG] pgmap v688: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1013 B/s wr, 3 op/s 2024-03-26T03:56:14.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:13 smithi190 bash[23392]: cluster 2024-03-26T03:56:12.840322+0000 mgr.y (mgr.24431) 465 : cluster [DBG] pgmap v689: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 382 B/s wr, 2 op/s 2024-03-26T03:56:14.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:13 smithi175 bash[21527]: cluster 2024-03-26T03:56:12.840322+0000 mgr.y (mgr.24431) 465 : cluster [DBG] pgmap v689: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 382 B/s wr, 2 op/s 2024-03-26T03:56:14.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:13 smithi175 bash[29284]: cluster 2024-03-26T03:56:12.840322+0000 mgr.y (mgr.24431) 465 : cluster [DBG] pgmap v689: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 382 B/s wr, 2 op/s 2024-03-26T03:56:16.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:15 smithi190 bash[23392]: cluster 2024-03-26T03:56:14.842640+0000 mgr.y (mgr.24431) 466 : cluster [DBG] pgmap v690: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 204 B/s wr, 1 op/s 2024-03-26T03:56:16.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:15 smithi175 bash[29284]: cluster 2024-03-26T03:56:14.842640+0000 mgr.y (mgr.24431) 466 : cluster [DBG] pgmap v690: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 204 B/s wr, 1 op/s 2024-03-26T03:56:16.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:15 smithi175 bash[21527]: cluster 2024-03-26T03:56:14.842640+0000 mgr.y (mgr.24431) 466 : cluster [DBG] pgmap v690: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 204 B/s wr, 1 op/s 2024-03-26T03:56:18.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:17 smithi190 bash[23392]: cluster 2024-03-26T03:56:16.843917+0000 mgr.y (mgr.24431) 467 : cluster [DBG] pgmap v691: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 928 B/s rd, 185 B/s wr, 1 op/s 2024-03-26T03:56:18.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:17 smithi175 bash[29284]: cluster 2024-03-26T03:56:16.843917+0000 mgr.y (mgr.24431) 467 : cluster [DBG] pgmap v691: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 928 B/s rd, 185 B/s wr, 1 op/s 2024-03-26T03:56:18.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:17 smithi175 bash[21527]: cluster 2024-03-26T03:56:16.843917+0000 mgr.y (mgr.24431) 467 : cluster [DBG] pgmap v691: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 928 B/s rd, 185 B/s wr, 1 op/s 2024-03-26T03:56:19.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:56:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:56:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:56:20.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:19 smithi190 bash[23392]: cluster 2024-03-26T03:56:18.844718+0000 mgr.y (mgr.24431) 468 : cluster [DBG] pgmap v692: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 170 B/s wr, 1 op/s 2024-03-26T03:56:20.242 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:56:20 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:56:20.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:19 smithi175 bash[29284]: cluster 2024-03-26T03:56:18.844718+0000 mgr.y (mgr.24431) 468 : cluster [DBG] pgmap v692: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 170 B/s wr, 1 op/s 2024-03-26T03:56:20.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:19 smithi175 bash[21527]: cluster 2024-03-26T03:56:18.844718+0000 mgr.y (mgr.24431) 468 : cluster [DBG] pgmap v692: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 170 B/s wr, 1 op/s 2024-03-26T03:56:21.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:20 smithi190 bash[23392]: audit 2024-03-26T03:56:20.120069+0000 mgr.y (mgr.24431) 469 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:56:21.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:20 smithi190 bash[23392]: audit 2024-03-26T03:56:20.174197+0000 mon.c (mon.2) 529 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:56:21.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:20 smithi175 bash[21527]: audit 2024-03-26T03:56:20.120069+0000 mgr.y (mgr.24431) 469 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:56:21.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:20 smithi175 bash[21527]: audit 2024-03-26T03:56:20.174197+0000 mon.c (mon.2) 529 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:56:21.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:20 smithi175 bash[29284]: audit 2024-03-26T03:56:20.120069+0000 mgr.y (mgr.24431) 469 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:56:21.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:20 smithi175 bash[29284]: audit 2024-03-26T03:56:20.174197+0000 mon.c (mon.2) 529 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:56:22.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:21 smithi190 bash[23392]: cluster 2024-03-26T03:56:20.847506+0000 mgr.y (mgr.24431) 470 : cluster [DBG] pgmap v693: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-03-26T03:56:22.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:21 smithi175 bash[21527]: cluster 2024-03-26T03:56:20.847506+0000 mgr.y (mgr.24431) 470 : cluster [DBG] pgmap v693: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-03-26T03:56:22.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:21 smithi175 bash[29284]: cluster 2024-03-26T03:56:20.847506+0000 mgr.y (mgr.24431) 470 : cluster [DBG] pgmap v693: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-03-26T03:56:24.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:23 smithi190 bash[23392]: cluster 2024-03-26T03:56:22.848341+0000 mgr.y (mgr.24431) 471 : cluster [DBG] pgmap v694: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:56:24.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:23 smithi175 bash[29284]: cluster 2024-03-26T03:56:22.848341+0000 mgr.y (mgr.24431) 471 : cluster [DBG] pgmap v694: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:56:24.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:23 smithi175 bash[21527]: cluster 2024-03-26T03:56:22.848341+0000 mgr.y (mgr.24431) 471 : cluster [DBG] pgmap v694: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:56:26.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:25 smithi190 bash[23392]: cluster 2024-03-26T03:56:24.850711+0000 mgr.y (mgr.24431) 472 : cluster [DBG] pgmap v695: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:56:26.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:25 smithi190 bash[23392]: audit 2024-03-26T03:56:25.905823+0000 mon.a (mon.0) 2635 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:56:26.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:25 smithi190 bash[23392]: audit 2024-03-26T03:56:25.906634+0000 mon.a (mon.0) 2636 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-79"}]: dispatch 2024-03-26T03:56:26.242 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 03:56:25 smithi190 bash[43655]: debug 2024-03-26T03:56:25.905+0000 7fef9737a700 -1 snap_mapper.add_oid found existing snaps mapped on 298:bc3819d3:test-rados-api-smithi175-64594-80::foo:2, removing 2024-03-26T03:56:26.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:25 smithi175 bash[21527]: cluster 2024-03-26T03:56:24.850711+0000 mgr.y (mgr.24431) 472 : cluster [DBG] pgmap v695: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:56:26.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:25 smithi175 bash[21527]: audit 2024-03-26T03:56:25.905823+0000 mon.a (mon.0) 2635 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:56:26.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:25 smithi175 bash[21527]: audit 2024-03-26T03:56:25.906634+0000 mon.a (mon.0) 2636 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-79"}]: dispatch 2024-03-26T03:56:26.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:25 smithi175 bash[29284]: cluster 2024-03-26T03:56:24.850711+0000 mgr.y (mgr.24431) 472 : cluster [DBG] pgmap v695: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T03:56:26.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:25 smithi175 bash[29284]: audit 2024-03-26T03:56:25.905823+0000 mon.a (mon.0) 2635 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:56:26.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:25 smithi175 bash[29284]: audit 2024-03-26T03:56:25.906634+0000 mon.a (mon.0) 2636 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-79"}]: dispatch 2024-03-26T03:56:26.244 INFO:journalctl@ceph.osd.1.smithi175.stdout:Mar 26 03:56:25 smithi175 bash[37791]: debug 2024-03-26T03:56:25.898+0000 7ffa3f302700 -1 snap_mapper.add_oid found existing snaps mapped on 298:bc3819d3:test-rados-api-smithi175-64594-80::foo:2, removing 2024-03-26T03:56:26.244 INFO:journalctl@ceph.osd.0.smithi175.stdout:Mar 26 03:56:25 smithi175 bash[32304]: debug 2024-03-26T03:56:25.902+0000 7ff3b1926700 -1 snap_mapper.add_oid found existing snaps mapped on 298:bc3819d3:test-rados-api-smithi175-64594-80::foo:2, removing 2024-03-26T03:56:28.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:27 smithi190 bash[23392]: cluster 2024-03-26T03:56:26.851972+0000 mgr.y (mgr.24431) 473 : cluster [DBG] pgmap v696: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:56:28.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:27 smithi190 bash[23392]: cluster 2024-03-26T03:56:26.960498+0000 mon.a (mon.0) 2637 : cluster [DBG] osdmap e435: 8 total, 8 up, 8 in 2024-03-26T03:56:28.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:27 smithi175 bash[21527]: cluster 2024-03-26T03:56:26.851972+0000 mgr.y (mgr.24431) 473 : cluster [DBG] pgmap v696: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:56:28.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:27 smithi175 bash[21527]: cluster 2024-03-26T03:56:26.960498+0000 mon.a (mon.0) 2637 : cluster [DBG] osdmap e435: 8 total, 8 up, 8 in 2024-03-26T03:56:28.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:27 smithi175 bash[29284]: cluster 2024-03-26T03:56:26.851972+0000 mgr.y (mgr.24431) 473 : cluster [DBG] pgmap v696: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:56:28.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:27 smithi175 bash[29284]: cluster 2024-03-26T03:56:26.960498+0000 mon.a (mon.0) 2637 : cluster [DBG] osdmap e435: 8 total, 8 up, 8 in 2024-03-26T03:56:29.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:56:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:56:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:56:29.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:29 smithi175 bash[29284]: cluster 2024-03-26T03:56:27.971411+0000 mon.a (mon.0) 2638 : cluster [DBG] osdmap e436: 8 total, 8 up, 8 in 2024-03-26T03:56:29.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:29 smithi175 bash[29284]: audit 2024-03-26T03:56:27.974969+0000 mon.a (mon.0) 2639 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:56:29.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:29 smithi175 bash[29284]: audit 2024-03-26T03:56:28.962660+0000 mon.a (mon.0) 2640 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:56:29.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:29 smithi175 bash[29284]: cluster 2024-03-26T03:56:28.974270+0000 mon.a (mon.0) 2641 : cluster [DBG] osdmap e437: 8 total, 8 up, 8 in 2024-03-26T03:56:29.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:29 smithi175 bash[21527]: cluster 2024-03-26T03:56:27.971411+0000 mon.a (mon.0) 2638 : cluster [DBG] osdmap e436: 8 total, 8 up, 8 in 2024-03-26T03:56:29.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:29 smithi175 bash[21527]: audit 2024-03-26T03:56:27.974969+0000 mon.a (mon.0) 2639 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:56:29.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:29 smithi175 bash[21527]: audit 2024-03-26T03:56:28.962660+0000 mon.a (mon.0) 2640 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:56:29.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:29 smithi175 bash[21527]: cluster 2024-03-26T03:56:28.974270+0000 mon.a (mon.0) 2641 : cluster [DBG] osdmap e437: 8 total, 8 up, 8 in 2024-03-26T03:56:29.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:29 smithi190 bash[23392]: cluster 2024-03-26T03:56:27.971411+0000 mon.a (mon.0) 2638 : cluster [DBG] osdmap e436: 8 total, 8 up, 8 in 2024-03-26T03:56:29.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:29 smithi190 bash[23392]: audit 2024-03-26T03:56:27.974969+0000 mon.a (mon.0) 2639 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:56:29.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:29 smithi190 bash[23392]: audit 2024-03-26T03:56:28.962660+0000 mon.a (mon.0) 2640 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:56:29.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:29 smithi190 bash[23392]: cluster 2024-03-26T03:56:28.974270+0000 mon.a (mon.0) 2641 : cluster [DBG] osdmap e437: 8 total, 8 up, 8 in 2024-03-26T03:56:30.491 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:56:30 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:56:30.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:30 smithi190 bash[23392]: cluster 2024-03-26T03:56:28.852963+0000 mgr.y (mgr.24431) 474 : cluster [DBG] pgmap v699: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-26T03:56:30.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:30 smithi190 bash[23392]: cluster 2024-03-26T03:56:28.978119+0000 mon.a (mon.0) 2642 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:56:30.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:30 smithi190 bash[23392]: audit 2024-03-26T03:56:28.983471+0000 mon.a (mon.0) 2643 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:56:30.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:30 smithi190 bash[23392]: audit 2024-03-26T03:56:29.008428+0000 mon.a (mon.0) 2644 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-26T03:56:30.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:30 smithi190 bash[23392]: audit 2024-03-26T03:56:30.000262+0000 mon.a (mon.0) 2645 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-26T03:56:30.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:30 smithi190 bash[23392]: cluster 2024-03-26T03:56:30.007317+0000 mon.a (mon.0) 2646 : cluster [DBG] osdmap e438: 8 total, 8 up, 8 in 2024-03-26T03:56:30.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:30 smithi190 bash[23392]: audit 2024-03-26T03:56:30.009945+0000 mon.a (mon.0) 2647 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_tier","val": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:56:30.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:30 smithi175 bash[21527]: cluster 2024-03-26T03:56:28.852963+0000 mgr.y (mgr.24431) 474 : cluster [DBG] pgmap v699: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-26T03:56:30.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:30 smithi175 bash[21527]: cluster 2024-03-26T03:56:28.978119+0000 mon.a (mon.0) 2642 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:56:30.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:30 smithi175 bash[21527]: audit 2024-03-26T03:56:28.983471+0000 mon.a (mon.0) 2643 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:56:30.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:30 smithi175 bash[21527]: audit 2024-03-26T03:56:29.008428+0000 mon.a (mon.0) 2644 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-26T03:56:30.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:30 smithi175 bash[21527]: audit 2024-03-26T03:56:30.000262+0000 mon.a (mon.0) 2645 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-26T03:56:30.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:30 smithi175 bash[21527]: cluster 2024-03-26T03:56:30.007317+0000 mon.a (mon.0) 2646 : cluster [DBG] osdmap e438: 8 total, 8 up, 8 in 2024-03-26T03:56:30.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:30 smithi175 bash[21527]: audit 2024-03-26T03:56:30.009945+0000 mon.a (mon.0) 2647 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_tier","val": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:56:30.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:30 smithi175 bash[29284]: cluster 2024-03-26T03:56:28.852963+0000 mgr.y (mgr.24431) 474 : cluster [DBG] pgmap v699: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-26T03:56:30.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:30 smithi175 bash[29284]: cluster 2024-03-26T03:56:28.978119+0000 mon.a (mon.0) 2642 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:56:30.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:30 smithi175 bash[29284]: audit 2024-03-26T03:56:28.983471+0000 mon.a (mon.0) 2643 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:56:30.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:30 smithi175 bash[29284]: audit 2024-03-26T03:56:29.008428+0000 mon.a (mon.0) 2644 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-26T03:56:30.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:30 smithi175 bash[29284]: audit 2024-03-26T03:56:30.000262+0000 mon.a (mon.0) 2645 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-26T03:56:30.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:30 smithi175 bash[29284]: cluster 2024-03-26T03:56:30.007317+0000 mon.a (mon.0) 2646 : cluster [DBG] osdmap e438: 8 total, 8 up, 8 in 2024-03-26T03:56:30.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:30 smithi175 bash[29284]: audit 2024-03-26T03:56:30.009945+0000 mon.a (mon.0) 2647 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_tier","val": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:56:31.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:31 smithi190 bash[23392]: audit 2024-03-26T03:56:30.131284+0000 mgr.y (mgr.24431) 475 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:56:31.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:31 smithi190 bash[23392]: audit 2024-03-26T03:56:31.002979+0000 mon.a (mon.0) 2648 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_tier","val": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:56:31.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:31 smithi190 bash[23392]: cluster 2024-03-26T03:56:31.011015+0000 mon.a (mon.0) 2649 : cluster [DBG] osdmap e439: 8 total, 8 up, 8 in 2024-03-26T03:56:31.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:31 smithi190 bash[23392]: audit 2024-03-26T03:56:31.012961+0000 mon.a (mon.0) 2650 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-26T03:56:31.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:31 smithi175 bash[21527]: audit 2024-03-26T03:56:30.131284+0000 mgr.y (mgr.24431) 475 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:56:31.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:31 smithi175 bash[21527]: audit 2024-03-26T03:56:31.002979+0000 mon.a (mon.0) 2648 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_tier","val": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:56:31.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:31 smithi175 bash[21527]: cluster 2024-03-26T03:56:31.011015+0000 mon.a (mon.0) 2649 : cluster [DBG] osdmap e439: 8 total, 8 up, 8 in 2024-03-26T03:56:31.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:31 smithi175 bash[21527]: audit 2024-03-26T03:56:31.012961+0000 mon.a (mon.0) 2650 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-26T03:56:31.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:31 smithi175 bash[29284]: audit 2024-03-26T03:56:30.131284+0000 mgr.y (mgr.24431) 475 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:56:31.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:31 smithi175 bash[29284]: audit 2024-03-26T03:56:31.002979+0000 mon.a (mon.0) 2648 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_tier","val": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:56:31.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:31 smithi175 bash[29284]: cluster 2024-03-26T03:56:31.011015+0000 mon.a (mon.0) 2649 : cluster [DBG] osdmap e439: 8 total, 8 up, 8 in 2024-03-26T03:56:31.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:31 smithi175 bash[29284]: audit 2024-03-26T03:56:31.012961+0000 mon.a (mon.0) 2650 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-26T03:56:32.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:32 smithi190 bash[23392]: cluster 2024-03-26T03:56:30.853920+0000 mgr.y (mgr.24431) 476 : cluster [DBG] pgmap v702: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-03-26T03:56:32.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:32 smithi190 bash[23392]: audit 2024-03-26T03:56:32.005630+0000 mon.a (mon.0) 2651 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-26T03:56:32.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:32 smithi190 bash[23392]: cluster 2024-03-26T03:56:32.012554+0000 mon.a (mon.0) 2652 : cluster [DBG] osdmap e440: 8 total, 8 up, 8 in 2024-03-26T03:56:32.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:32 smithi190 bash[23392]: audit 2024-03-26T03:56:32.016149+0000 mon.a (mon.0) 2653 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-26T03:56:32.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:32 smithi175 bash[29284]: cluster 2024-03-26T03:56:30.853920+0000 mgr.y (mgr.24431) 476 : cluster [DBG] pgmap v702: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-03-26T03:56:32.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:32 smithi175 bash[29284]: audit 2024-03-26T03:56:32.005630+0000 mon.a (mon.0) 2651 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-26T03:56:32.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:32 smithi175 bash[29284]: cluster 2024-03-26T03:56:32.012554+0000 mon.a (mon.0) 2652 : cluster [DBG] osdmap e440: 8 total, 8 up, 8 in 2024-03-26T03:56:32.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:32 smithi175 bash[29284]: audit 2024-03-26T03:56:32.016149+0000 mon.a (mon.0) 2653 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-26T03:56:32.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:32 smithi175 bash[21527]: cluster 2024-03-26T03:56:30.853920+0000 mgr.y (mgr.24431) 476 : cluster [DBG] pgmap v702: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-03-26T03:56:32.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:32 smithi175 bash[21527]: audit 2024-03-26T03:56:32.005630+0000 mon.a (mon.0) 2651 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-26T03:56:32.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:32 smithi175 bash[21527]: cluster 2024-03-26T03:56:32.012554+0000 mon.a (mon.0) 2652 : cluster [DBG] osdmap e440: 8 total, 8 up, 8 in 2024-03-26T03:56:32.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:32 smithi175 bash[21527]: audit 2024-03-26T03:56:32.016149+0000 mon.a (mon.0) 2653 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-26T03:56:33.992 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 03:56:33 smithi190 bash[50860]: logger=cleanup t=2024-03-26T03:56:33.586339255Z level=info msg="Completed cleanup jobs" duration=81.31968ms 2024-03-26T03:56:34.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:34 smithi190 bash[23392]: cluster 2024-03-26T03:56:32.854764+0000 mgr.y (mgr.24431) 477 : cluster [DBG] pgmap v705: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-26T03:56:34.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:34 smithi190 bash[23392]: audit 2024-03-26T03:56:33.008376+0000 mon.a (mon.0) 2654 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-26T03:56:34.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:34 smithi190 bash[23392]: cluster 2024-03-26T03:56:33.015252+0000 mon.a (mon.0) 2655 : cluster [DBG] osdmap e441: 8 total, 8 up, 8 in 2024-03-26T03:56:34.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:34 smithi190 bash[23392]: audit 2024-03-26T03:56:33.068084+0000 mon.a (mon.0) 2656 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-03-26T03:56:34.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:34 smithi175 bash[29284]: cluster 2024-03-26T03:56:32.854764+0000 mgr.y (mgr.24431) 477 : cluster [DBG] pgmap v705: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-26T03:56:34.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:34 smithi175 bash[29284]: audit 2024-03-26T03:56:33.008376+0000 mon.a (mon.0) 2654 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-26T03:56:34.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:34 smithi175 bash[29284]: cluster 2024-03-26T03:56:33.015252+0000 mon.a (mon.0) 2655 : cluster [DBG] osdmap e441: 8 total, 8 up, 8 in 2024-03-26T03:56:34.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:34 smithi175 bash[29284]: audit 2024-03-26T03:56:33.068084+0000 mon.a (mon.0) 2656 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-03-26T03:56:34.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:34 smithi175 bash[21527]: cluster 2024-03-26T03:56:32.854764+0000 mgr.y (mgr.24431) 477 : cluster [DBG] pgmap v705: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-26T03:56:34.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:34 smithi175 bash[21527]: audit 2024-03-26T03:56:33.008376+0000 mon.a (mon.0) 2654 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-26T03:56:34.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:34 smithi175 bash[21527]: cluster 2024-03-26T03:56:33.015252+0000 mon.a (mon.0) 2655 : cluster [DBG] osdmap e441: 8 total, 8 up, 8 in 2024-03-26T03:56:34.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:34 smithi175 bash[21527]: audit 2024-03-26T03:56:33.068084+0000 mon.a (mon.0) 2656 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-03-26T03:56:35.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:35 smithi190 bash[23392]: audit 2024-03-26T03:56:34.042634+0000 mon.a (mon.0) 2657 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-03-26T03:56:35.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:35 smithi190 bash[23392]: cluster 2024-03-26T03:56:34.050522+0000 mon.a (mon.0) 2658 : cluster [DBG] osdmap e442: 8 total, 8 up, 8 in 2024-03-26T03:56:35.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:35 smithi190 bash[23392]: audit 2024-03-26T03:56:34.103357+0000 mon.a (mon.0) 2659 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-03-26T03:56:35.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:35 smithi175 bash[29284]: audit 2024-03-26T03:56:34.042634+0000 mon.a (mon.0) 2657 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-03-26T03:56:35.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:35 smithi175 bash[29284]: cluster 2024-03-26T03:56:34.050522+0000 mon.a (mon.0) 2658 : cluster [DBG] osdmap e442: 8 total, 8 up, 8 in 2024-03-26T03:56:35.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:35 smithi175 bash[29284]: audit 2024-03-26T03:56:34.103357+0000 mon.a (mon.0) 2659 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-03-26T03:56:35.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:35 smithi175 bash[21527]: audit 2024-03-26T03:56:34.042634+0000 mon.a (mon.0) 2657 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-03-26T03:56:35.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:35 smithi175 bash[21527]: cluster 2024-03-26T03:56:34.050522+0000 mon.a (mon.0) 2658 : cluster [DBG] osdmap e442: 8 total, 8 up, 8 in 2024-03-26T03:56:35.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:35 smithi175 bash[21527]: audit 2024-03-26T03:56:34.103357+0000 mon.a (mon.0) 2659 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-03-26T03:56:36.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:36 smithi190 bash[23392]: cluster 2024-03-26T03:56:34.857169+0000 mgr.y (mgr.24431) 478 : cluster [DBG] pgmap v708: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 8.7 KiB/s wr, 18 op/s 2024-03-26T03:56:36.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:36 smithi190 bash[23392]: audit 2024-03-26T03:56:35.054696+0000 mon.a (mon.0) 2660 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-03-26T03:56:36.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:36 smithi190 bash[23392]: cluster 2024-03-26T03:56:35.064800+0000 mon.a (mon.0) 2661 : cluster [DBG] osdmap e443: 8 total, 8 up, 8 in 2024-03-26T03:56:36.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:36 smithi190 bash[23392]: audit 2024-03-26T03:56:35.101465+0000 mon.a (mon.0) 2662 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-26T03:56:36.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:36 smithi190 bash[23392]: audit 2024-03-26T03:56:35.210276+0000 mon.c (mon.2) 530 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:56:36.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:36 smithi175 bash[21527]: cluster 2024-03-26T03:56:34.857169+0000 mgr.y (mgr.24431) 478 : cluster [DBG] pgmap v708: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 8.7 KiB/s wr, 18 op/s 2024-03-26T03:56:36.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:36 smithi175 bash[21527]: audit 2024-03-26T03:56:35.054696+0000 mon.a (mon.0) 2660 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-03-26T03:56:36.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:36 smithi175 bash[21527]: cluster 2024-03-26T03:56:35.064800+0000 mon.a (mon.0) 2661 : cluster [DBG] osdmap e443: 8 total, 8 up, 8 in 2024-03-26T03:56:36.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:36 smithi175 bash[21527]: audit 2024-03-26T03:56:35.101465+0000 mon.a (mon.0) 2662 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-26T03:56:36.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:36 smithi175 bash[21527]: audit 2024-03-26T03:56:35.210276+0000 mon.c (mon.2) 530 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:56:36.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:36 smithi175 bash[29284]: cluster 2024-03-26T03:56:34.857169+0000 mgr.y (mgr.24431) 478 : cluster [DBG] pgmap v708: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 8.7 KiB/s wr, 18 op/s 2024-03-26T03:56:36.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:36 smithi175 bash[29284]: audit 2024-03-26T03:56:35.054696+0000 mon.a (mon.0) 2660 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-03-26T03:56:36.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:36 smithi175 bash[29284]: cluster 2024-03-26T03:56:35.064800+0000 mon.a (mon.0) 2661 : cluster [DBG] osdmap e443: 8 total, 8 up, 8 in 2024-03-26T03:56:36.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:36 smithi175 bash[29284]: audit 2024-03-26T03:56:35.101465+0000 mon.a (mon.0) 2662 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-26T03:56:36.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:36 smithi175 bash[29284]: audit 2024-03-26T03:56:35.210276+0000 mon.c (mon.2) 530 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:56:37.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:37 smithi190 bash[23392]: audit 2024-03-26T03:56:36.066715+0000 mon.a (mon.0) 2663 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-26T03:56:37.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:37 smithi190 bash[23392]: cluster 2024-03-26T03:56:36.078150+0000 mon.a (mon.0) 2664 : cluster [DBG] osdmap e444: 8 total, 8 up, 8 in 2024-03-26T03:56:37.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:37 smithi190 bash[23392]: audit 2024-03-26T03:56:36.157939+0000 mon.a (mon.0) 2665 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:56:37.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:37 smithi190 bash[23392]: audit 2024-03-26T03:56:36.158988+0000 mon.a (mon.0) 2666 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-81"}]: dispatch 2024-03-26T03:56:37.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:37 smithi190 bash[23392]: cluster 2024-03-26T03:56:36.560695+0000 mon.a (mon.0) 2667 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:56:37.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:37 smithi175 bash[29284]: audit 2024-03-26T03:56:36.066715+0000 mon.a (mon.0) 2663 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-26T03:56:37.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:37 smithi175 bash[29284]: cluster 2024-03-26T03:56:36.078150+0000 mon.a (mon.0) 2664 : cluster [DBG] osdmap e444: 8 total, 8 up, 8 in 2024-03-26T03:56:37.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:37 smithi175 bash[29284]: audit 2024-03-26T03:56:36.157939+0000 mon.a (mon.0) 2665 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:56:37.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:37 smithi175 bash[29284]: audit 2024-03-26T03:56:36.158988+0000 mon.a (mon.0) 2666 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-81"}]: dispatch 2024-03-26T03:56:37.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:37 smithi175 bash[29284]: cluster 2024-03-26T03:56:36.560695+0000 mon.a (mon.0) 2667 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:56:37.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:37 smithi175 bash[21527]: audit 2024-03-26T03:56:36.066715+0000 mon.a (mon.0) 2663 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-26T03:56:37.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:37 smithi175 bash[21527]: cluster 2024-03-26T03:56:36.078150+0000 mon.a (mon.0) 2664 : cluster [DBG] osdmap e444: 8 total, 8 up, 8 in 2024-03-26T03:56:37.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:37 smithi175 bash[21527]: audit 2024-03-26T03:56:36.157939+0000 mon.a (mon.0) 2665 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:56:37.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:37 smithi175 bash[21527]: audit 2024-03-26T03:56:36.158988+0000 mon.a (mon.0) 2666 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-81"}]: dispatch 2024-03-26T03:56:37.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:37 smithi175 bash[21527]: cluster 2024-03-26T03:56:36.560695+0000 mon.a (mon.0) 2667 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:56:38.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:38 smithi190 bash[23392]: cluster 2024-03-26T03:56:36.858079+0000 mgr.y (mgr.24431) 479 : cluster [DBG] pgmap v711: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 9.2 KiB/s wr, 21 op/s 2024-03-26T03:56:38.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:38 smithi190 bash[23392]: cluster 2024-03-26T03:56:37.085508+0000 mon.a (mon.0) 2668 : cluster [DBG] osdmap e445: 8 total, 8 up, 8 in 2024-03-26T03:56:38.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:38 smithi175 bash[21527]: cluster 2024-03-26T03:56:36.858079+0000 mgr.y (mgr.24431) 479 : cluster [DBG] pgmap v711: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 9.2 KiB/s wr, 21 op/s 2024-03-26T03:56:38.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:38 smithi175 bash[21527]: cluster 2024-03-26T03:56:37.085508+0000 mon.a (mon.0) 2668 : cluster [DBG] osdmap e445: 8 total, 8 up, 8 in 2024-03-26T03:56:38.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:38 smithi175 bash[29284]: cluster 2024-03-26T03:56:36.858079+0000 mgr.y (mgr.24431) 479 : cluster [DBG] pgmap v711: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 9.2 KiB/s wr, 21 op/s 2024-03-26T03:56:38.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:38 smithi175 bash[29284]: cluster 2024-03-26T03:56:37.085508+0000 mon.a (mon.0) 2668 : cluster [DBG] osdmap e445: 8 total, 8 up, 8 in 2024-03-26T03:56:39.139 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:56:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:56:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:56:39.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:39 smithi190 bash[23392]: cluster 2024-03-26T03:56:38.092463+0000 mon.a (mon.0) 2669 : cluster [DBG] osdmap e446: 8 total, 8 up, 8 in 2024-03-26T03:56:39.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:39 smithi190 bash[23392]: audit 2024-03-26T03:56:38.110487+0000 mon.a (mon.0) 2670 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:56:39.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:39 smithi190 bash[23392]: audit 2024-03-26T03:56:39.083068+0000 mon.a (mon.0) 2671 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:56:39.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:39 smithi190 bash[23392]: cluster 2024-03-26T03:56:39.090816+0000 mon.a (mon.0) 2672 : cluster [DBG] osdmap e447: 8 total, 8 up, 8 in 2024-03-26T03:56:39.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:39 smithi175 bash[21527]: cluster 2024-03-26T03:56:38.092463+0000 mon.a (mon.0) 2669 : cluster [DBG] osdmap e446: 8 total, 8 up, 8 in 2024-03-26T03:56:39.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:39 smithi175 bash[21527]: audit 2024-03-26T03:56:38.110487+0000 mon.a (mon.0) 2670 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:56:39.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:39 smithi175 bash[21527]: audit 2024-03-26T03:56:39.083068+0000 mon.a (mon.0) 2671 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:56:39.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:39 smithi175 bash[21527]: cluster 2024-03-26T03:56:39.090816+0000 mon.a (mon.0) 2672 : cluster [DBG] osdmap e447: 8 total, 8 up, 8 in 2024-03-26T03:56:39.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:39 smithi175 bash[29284]: cluster 2024-03-26T03:56:38.092463+0000 mon.a (mon.0) 2669 : cluster [DBG] osdmap e446: 8 total, 8 up, 8 in 2024-03-26T03:56:39.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:39 smithi175 bash[29284]: audit 2024-03-26T03:56:38.110487+0000 mon.a (mon.0) 2670 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:56:39.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:39 smithi175 bash[29284]: audit 2024-03-26T03:56:39.083068+0000 mon.a (mon.0) 2671 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:56:39.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:39 smithi175 bash[29284]: cluster 2024-03-26T03:56:39.090816+0000 mon.a (mon.0) 2672 : cluster [DBG] osdmap e447: 8 total, 8 up, 8 in 2024-03-26T03:56:40.491 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:56:40 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:56:40.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:40 smithi190 bash[23392]: cluster 2024-03-26T03:56:38.858997+0000 mgr.y (mgr.24431) 480 : cluster [DBG] pgmap v714: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-26T03:56:40.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:40 smithi190 bash[23392]: audit 2024-03-26T03:56:39.104528+0000 mon.a (mon.0) 2673 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:56:40.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:40 smithi190 bash[23392]: audit 2024-03-26T03:56:39.139657+0000 mon.a (mon.0) 2674 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-26T03:56:40.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:40 smithi190 bash[23392]: audit 2024-03-26T03:56:40.133995+0000 mon.a (mon.0) 2675 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-26T03:56:40.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:40 smithi175 bash[21527]: cluster 2024-03-26T03:56:38.858997+0000 mgr.y (mgr.24431) 480 : cluster [DBG] pgmap v714: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-26T03:56:40.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:40 smithi175 bash[21527]: audit 2024-03-26T03:56:39.104528+0000 mon.a (mon.0) 2673 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:56:40.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:40 smithi175 bash[21527]: audit 2024-03-26T03:56:39.139657+0000 mon.a (mon.0) 2674 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-26T03:56:40.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:40 smithi175 bash[21527]: audit 2024-03-26T03:56:40.133995+0000 mon.a (mon.0) 2675 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-26T03:56:40.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:40 smithi175 bash[29284]: cluster 2024-03-26T03:56:38.858997+0000 mgr.y (mgr.24431) 480 : cluster [DBG] pgmap v714: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-26T03:56:40.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:40 smithi175 bash[29284]: audit 2024-03-26T03:56:39.104528+0000 mon.a (mon.0) 2673 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:56:40.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:40 smithi175 bash[29284]: audit 2024-03-26T03:56:39.139657+0000 mon.a (mon.0) 2674 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-26T03:56:40.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:40 smithi175 bash[29284]: audit 2024-03-26T03:56:40.133995+0000 mon.a (mon.0) 2675 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-26T03:56:41.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:41 smithi190 bash[23392]: audit 2024-03-26T03:56:40.143861+0000 mgr.y (mgr.24431) 481 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:56:41.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:41 smithi190 bash[23392]: cluster 2024-03-26T03:56:40.144339+0000 mon.a (mon.0) 2676 : cluster [DBG] osdmap e448: 8 total, 8 up, 8 in 2024-03-26T03:56:41.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:41 smithi190 bash[23392]: audit 2024-03-26T03:56:40.148420+0000 mon.a (mon.0) 2677 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-83","var": "dedup_tier","val": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:56:41.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:41 smithi190 bash[23392]: audit 2024-03-26T03:56:41.136897+0000 mon.a (mon.0) 2678 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-83","var": "dedup_tier","val": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:56:41.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:41 smithi190 bash[23392]: cluster 2024-03-26T03:56:41.143907+0000 mon.a (mon.0) 2679 : cluster [DBG] osdmap e449: 8 total, 8 up, 8 in 2024-03-26T03:56:41.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:41 smithi190 bash[23392]: audit 2024-03-26T03:56:41.145991+0000 mon.a (mon.0) 2680 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-26T03:56:41.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:41 smithi175 bash[29284]: audit 2024-03-26T03:56:40.143861+0000 mgr.y (mgr.24431) 481 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:56:41.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:41 smithi175 bash[29284]: cluster 2024-03-26T03:56:40.144339+0000 mon.a (mon.0) 2676 : cluster [DBG] osdmap e448: 8 total, 8 up, 8 in 2024-03-26T03:56:41.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:41 smithi175 bash[29284]: audit 2024-03-26T03:56:40.148420+0000 mon.a (mon.0) 2677 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-83","var": "dedup_tier","val": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:56:41.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:41 smithi175 bash[29284]: audit 2024-03-26T03:56:41.136897+0000 mon.a (mon.0) 2678 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-83","var": "dedup_tier","val": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:56:41.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:41 smithi175 bash[29284]: cluster 2024-03-26T03:56:41.143907+0000 mon.a (mon.0) 2679 : cluster [DBG] osdmap e449: 8 total, 8 up, 8 in 2024-03-26T03:56:41.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:41 smithi175 bash[29284]: audit 2024-03-26T03:56:41.145991+0000 mon.a (mon.0) 2680 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-26T03:56:41.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:41 smithi175 bash[21527]: audit 2024-03-26T03:56:40.143861+0000 mgr.y (mgr.24431) 481 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:56:41.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:41 smithi175 bash[21527]: cluster 2024-03-26T03:56:40.144339+0000 mon.a (mon.0) 2676 : cluster [DBG] osdmap e448: 8 total, 8 up, 8 in 2024-03-26T03:56:41.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:41 smithi175 bash[21527]: audit 2024-03-26T03:56:40.148420+0000 mon.a (mon.0) 2677 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-83","var": "dedup_tier","val": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:56:41.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:41 smithi175 bash[21527]: audit 2024-03-26T03:56:41.136897+0000 mon.a (mon.0) 2678 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-83","var": "dedup_tier","val": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:56:41.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:41 smithi175 bash[21527]: cluster 2024-03-26T03:56:41.143907+0000 mon.a (mon.0) 2679 : cluster [DBG] osdmap e449: 8 total, 8 up, 8 in 2024-03-26T03:56:41.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:41 smithi175 bash[21527]: audit 2024-03-26T03:56:41.145991+0000 mon.a (mon.0) 2680 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-26T03:56:42.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:42 smithi190 bash[23392]: cluster 2024-03-26T03:56:40.859916+0000 mgr.y (mgr.24431) 482 : cluster [DBG] pgmap v717: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 3.0 KiB/s wr, 18 op/s 2024-03-26T03:56:42.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:42 smithi190 bash[23392]: audit 2024-03-26T03:56:42.139568+0000 mon.a (mon.0) 2681 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-26T03:56:42.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:42 smithi190 bash[23392]: cluster 2024-03-26T03:56:42.146624+0000 mon.a (mon.0) 2682 : cluster [DBG] osdmap e450: 8 total, 8 up, 8 in 2024-03-26T03:56:42.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:42 smithi190 bash[23392]: audit 2024-03-26T03:56:42.148024+0000 mon.a (mon.0) 2683 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-26T03:56:42.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:42 smithi175 bash[29284]: cluster 2024-03-26T03:56:40.859916+0000 mgr.y (mgr.24431) 482 : cluster [DBG] pgmap v717: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 3.0 KiB/s wr, 18 op/s 2024-03-26T03:56:42.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:42 smithi175 bash[29284]: audit 2024-03-26T03:56:42.139568+0000 mon.a (mon.0) 2681 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-26T03:56:42.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:42 smithi175 bash[29284]: cluster 2024-03-26T03:56:42.146624+0000 mon.a (mon.0) 2682 : cluster [DBG] osdmap e450: 8 total, 8 up, 8 in 2024-03-26T03:56:42.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:42 smithi175 bash[29284]: audit 2024-03-26T03:56:42.148024+0000 mon.a (mon.0) 2683 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-26T03:56:42.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:42 smithi175 bash[21527]: cluster 2024-03-26T03:56:40.859916+0000 mgr.y (mgr.24431) 482 : cluster [DBG] pgmap v717: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 3.0 KiB/s wr, 18 op/s 2024-03-26T03:56:42.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:42 smithi175 bash[21527]: audit 2024-03-26T03:56:42.139568+0000 mon.a (mon.0) 2681 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-26T03:56:42.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:42 smithi175 bash[21527]: cluster 2024-03-26T03:56:42.146624+0000 mon.a (mon.0) 2682 : cluster [DBG] osdmap e450: 8 total, 8 up, 8 in 2024-03-26T03:56:42.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:42 smithi175 bash[21527]: audit 2024-03-26T03:56:42.148024+0000 mon.a (mon.0) 2683 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-26T03:56:44.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:44 smithi190 bash[23392]: cluster 2024-03-26T03:56:42.860815+0000 mgr.y (mgr.24431) 483 : cluster [DBG] pgmap v720: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 19 op/s 2024-03-26T03:56:44.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:44 smithi190 bash[23392]: audit 2024-03-26T03:56:43.142241+0000 mon.a (mon.0) 2684 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-26T03:56:44.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:44 smithi190 bash[23392]: cluster 2024-03-26T03:56:43.149645+0000 mon.a (mon.0) 2685 : cluster [DBG] osdmap e451: 8 total, 8 up, 8 in 2024-03-26T03:56:44.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:44 smithi175 bash[29284]: cluster 2024-03-26T03:56:42.860815+0000 mgr.y (mgr.24431) 483 : cluster [DBG] pgmap v720: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 19 op/s 2024-03-26T03:56:44.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:44 smithi175 bash[29284]: audit 2024-03-26T03:56:43.142241+0000 mon.a (mon.0) 2684 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-26T03:56:44.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:44 smithi175 bash[29284]: cluster 2024-03-26T03:56:43.149645+0000 mon.a (mon.0) 2685 : cluster [DBG] osdmap e451: 8 total, 8 up, 8 in 2024-03-26T03:56:44.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:44 smithi175 bash[21527]: cluster 2024-03-26T03:56:42.860815+0000 mgr.y (mgr.24431) 483 : cluster [DBG] pgmap v720: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 19 op/s 2024-03-26T03:56:44.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:44 smithi175 bash[21527]: audit 2024-03-26T03:56:43.142241+0000 mon.a (mon.0) 2684 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-26T03:56:44.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:44 smithi175 bash[21527]: cluster 2024-03-26T03:56:43.149645+0000 mon.a (mon.0) 2685 : cluster [DBG] osdmap e451: 8 total, 8 up, 8 in 2024-03-26T03:56:45.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:45 smithi190 bash[23392]: cluster 2024-03-26T03:56:44.178018+0000 mon.a (mon.0) 2686 : cluster [DBG] osdmap e452: 8 total, 8 up, 8 in 2024-03-26T03:56:45.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:45 smithi175 bash[29284]: cluster 2024-03-26T03:56:44.178018+0000 mon.a (mon.0) 2686 : cluster [DBG] osdmap e452: 8 total, 8 up, 8 in 2024-03-26T03:56:45.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:45 smithi175 bash[21527]: cluster 2024-03-26T03:56:44.178018+0000 mon.a (mon.0) 2686 : cluster [DBG] osdmap e452: 8 total, 8 up, 8 in 2024-03-26T03:56:46.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:46 smithi190 bash[23392]: cluster 2024-03-26T03:56:44.863173+0000 mgr.y (mgr.24431) 484 : cluster [DBG] pgmap v723: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 2.5 KiB/s wr, 1 op/s 2024-03-26T03:56:46.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:46 smithi190 bash[23392]: cluster 2024-03-26T03:56:45.188586+0000 mon.a (mon.0) 2687 : cluster [DBG] osdmap e453: 8 total, 8 up, 8 in 2024-03-26T03:56:46.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:46 smithi190 bash[23392]: audit 2024-03-26T03:56:45.276871+0000 mon.a (mon.0) 2688 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:56:46.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:46 smithi190 bash[23392]: audit 2024-03-26T03:56:45.277757+0000 mon.a (mon.0) 2689 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-83"}]: dispatch 2024-03-26T03:56:46.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:46 smithi175 bash[21527]: cluster 2024-03-26T03:56:44.863173+0000 mgr.y (mgr.24431) 484 : cluster [DBG] pgmap v723: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 2.5 KiB/s wr, 1 op/s 2024-03-26T03:56:46.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:46 smithi175 bash[21527]: cluster 2024-03-26T03:56:45.188586+0000 mon.a (mon.0) 2687 : cluster [DBG] osdmap e453: 8 total, 8 up, 8 in 2024-03-26T03:56:46.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:46 smithi175 bash[21527]: audit 2024-03-26T03:56:45.276871+0000 mon.a (mon.0) 2688 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:56:46.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:46 smithi175 bash[21527]: audit 2024-03-26T03:56:45.277757+0000 mon.a (mon.0) 2689 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-83"}]: dispatch 2024-03-26T03:56:46.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:46 smithi175 bash[29284]: cluster 2024-03-26T03:56:44.863173+0000 mgr.y (mgr.24431) 484 : cluster [DBG] pgmap v723: 292 pgs: 292 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 2.5 KiB/s wr, 1 op/s 2024-03-26T03:56:46.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:46 smithi175 bash[29284]: cluster 2024-03-26T03:56:45.188586+0000 mon.a (mon.0) 2687 : cluster [DBG] osdmap e453: 8 total, 8 up, 8 in 2024-03-26T03:56:46.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:46 smithi175 bash[29284]: audit 2024-03-26T03:56:45.276871+0000 mon.a (mon.0) 2688 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:56:46.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:46 smithi175 bash[29284]: audit 2024-03-26T03:56:45.277757+0000 mon.a (mon.0) 2689 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-83"}]: dispatch 2024-03-26T03:56:47.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:47 smithi190 bash[23392]: cluster 2024-03-26T03:56:46.190961+0000 mon.a (mon.0) 2690 : cluster [DBG] osdmap e454: 8 total, 8 up, 8 in 2024-03-26T03:56:47.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:47 smithi190 bash[23392]: cluster 2024-03-26T03:56:47.199325+0000 mon.a (mon.0) 2691 : cluster [DBG] osdmap e455: 8 total, 8 up, 8 in 2024-03-26T03:56:47.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:47 smithi190 bash[23392]: audit 2024-03-26T03:56:47.204812+0000 mon.a (mon.0) 2692 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:56:47.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:47 smithi175 bash[21527]: cluster 2024-03-26T03:56:46.190961+0000 mon.a (mon.0) 2690 : cluster [DBG] osdmap e454: 8 total, 8 up, 8 in 2024-03-26T03:56:47.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:47 smithi175 bash[21527]: cluster 2024-03-26T03:56:47.199325+0000 mon.a (mon.0) 2691 : cluster [DBG] osdmap e455: 8 total, 8 up, 8 in 2024-03-26T03:56:47.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:47 smithi175 bash[21527]: audit 2024-03-26T03:56:47.204812+0000 mon.a (mon.0) 2692 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:56:47.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:47 smithi175 bash[29284]: cluster 2024-03-26T03:56:46.190961+0000 mon.a (mon.0) 2690 : cluster [DBG] osdmap e454: 8 total, 8 up, 8 in 2024-03-26T03:56:47.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:47 smithi175 bash[29284]: cluster 2024-03-26T03:56:47.199325+0000 mon.a (mon.0) 2691 : cluster [DBG] osdmap e455: 8 total, 8 up, 8 in 2024-03-26T03:56:47.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:47 smithi175 bash[29284]: audit 2024-03-26T03:56:47.204812+0000 mon.a (mon.0) 2692 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:56:48.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:48 smithi190 bash[23392]: cluster 2024-03-26T03:56:46.864059+0000 mgr.y (mgr.24431) 485 : cluster [DBG] pgmap v726: 260 pgs: 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T03:56:48.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:48 smithi190 bash[23392]: audit 2024-03-26T03:56:48.190444+0000 mon.a (mon.0) 2693 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:56:48.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:48 smithi190 bash[23392]: cluster 2024-03-26T03:56:48.198244+0000 mon.a (mon.0) 2694 : cluster [DBG] osdmap e456: 8 total, 8 up, 8 in 2024-03-26T03:56:48.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:48 smithi190 bash[23392]: audit 2024-03-26T03:56:48.203615+0000 mon.a (mon.0) 2695 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:56:48.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:48 smithi190 bash[23392]: audit 2024-03-26T03:56:48.221447+0000 mon.a (mon.0) 2696 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-26T03:56:48.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:48 smithi175 bash[29284]: cluster 2024-03-26T03:56:46.864059+0000 mgr.y (mgr.24431) 485 : cluster [DBG] pgmap v726: 260 pgs: 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T03:56:48.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:48 smithi175 bash[29284]: audit 2024-03-26T03:56:48.190444+0000 mon.a (mon.0) 2693 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:56:48.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:48 smithi175 bash[29284]: cluster 2024-03-26T03:56:48.198244+0000 mon.a (mon.0) 2694 : cluster [DBG] osdmap e456: 8 total, 8 up, 8 in 2024-03-26T03:56:48.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:48 smithi175 bash[29284]: audit 2024-03-26T03:56:48.203615+0000 mon.a (mon.0) 2695 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:56:48.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:48 smithi175 bash[29284]: audit 2024-03-26T03:56:48.221447+0000 mon.a (mon.0) 2696 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-26T03:56:48.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:48 smithi175 bash[21527]: cluster 2024-03-26T03:56:46.864059+0000 mgr.y (mgr.24431) 485 : cluster [DBG] pgmap v726: 260 pgs: 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T03:56:48.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:48 smithi175 bash[21527]: audit 2024-03-26T03:56:48.190444+0000 mon.a (mon.0) 2693 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:56:48.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:48 smithi175 bash[21527]: cluster 2024-03-26T03:56:48.198244+0000 mon.a (mon.0) 2694 : cluster [DBG] osdmap e456: 8 total, 8 up, 8 in 2024-03-26T03:56:48.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:48 smithi175 bash[21527]: audit 2024-03-26T03:56:48.203615+0000 mon.a (mon.0) 2695 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:56:48.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:48 smithi175 bash[21527]: audit 2024-03-26T03:56:48.221447+0000 mon.a (mon.0) 2696 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-26T03:56:49.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:56:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:56:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:56:50.492 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:56:50 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:56:50.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:50 smithi190 bash[23392]: cluster 2024-03-26T03:56:48.864888+0000 mgr.y (mgr.24431) 486 : cluster [DBG] pgmap v729: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:56:50.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:50 smithi190 bash[23392]: audit 2024-03-26T03:56:49.193355+0000 mon.a (mon.0) 2697 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-26T03:56:50.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:50 smithi190 bash[23392]: cluster 2024-03-26T03:56:49.201300+0000 mon.a (mon.0) 2698 : cluster [DBG] osdmap e457: 8 total, 8 up, 8 in 2024-03-26T03:56:50.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:50 smithi190 bash[23392]: audit 2024-03-26T03:56:49.202302+0000 mon.a (mon.0) 2699 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-85","var": "dedup_tier","val": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:56:50.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:50 smithi175 bash[21527]: cluster 2024-03-26T03:56:48.864888+0000 mgr.y (mgr.24431) 486 : cluster [DBG] pgmap v729: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:56:50.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:50 smithi175 bash[21527]: audit 2024-03-26T03:56:49.193355+0000 mon.a (mon.0) 2697 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-26T03:56:50.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:50 smithi175 bash[21527]: cluster 2024-03-26T03:56:49.201300+0000 mon.a (mon.0) 2698 : cluster [DBG] osdmap e457: 8 total, 8 up, 8 in 2024-03-26T03:56:50.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:50 smithi175 bash[21527]: audit 2024-03-26T03:56:49.202302+0000 mon.a (mon.0) 2699 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-85","var": "dedup_tier","val": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:56:50.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:50 smithi175 bash[29284]: cluster 2024-03-26T03:56:48.864888+0000 mgr.y (mgr.24431) 486 : cluster [DBG] pgmap v729: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:56:50.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:50 smithi175 bash[29284]: audit 2024-03-26T03:56:49.193355+0000 mon.a (mon.0) 2697 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-26T03:56:50.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:50 smithi175 bash[29284]: cluster 2024-03-26T03:56:49.201300+0000 mon.a (mon.0) 2698 : cluster [DBG] osdmap e457: 8 total, 8 up, 8 in 2024-03-26T03:56:50.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:50 smithi175 bash[29284]: audit 2024-03-26T03:56:49.202302+0000 mon.a (mon.0) 2699 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-85","var": "dedup_tier","val": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:56:51.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:51 smithi190 bash[23392]: audit 2024-03-26T03:56:50.155418+0000 mgr.y (mgr.24431) 487 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:56:51.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:51 smithi190 bash[23392]: audit 2024-03-26T03:56:50.195999+0000 mon.a (mon.0) 2700 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-85","var": "dedup_tier","val": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:56:51.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:51 smithi190 bash[23392]: cluster 2024-03-26T03:56:50.203310+0000 mon.a (mon.0) 2701 : cluster [DBG] osdmap e458: 8 total, 8 up, 8 in 2024-03-26T03:56:51.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:51 smithi190 bash[23392]: audit 2024-03-26T03:56:50.205078+0000 mon.a (mon.0) 2702 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-26T03:56:51.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:51 smithi190 bash[23392]: audit 2024-03-26T03:56:50.244381+0000 mon.c (mon.2) 531 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:56:51.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:51 smithi190 bash[23392]: audit 2024-03-26T03:56:51.198746+0000 mon.a (mon.0) 2703 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-26T03:56:51.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:51 smithi190 bash[23392]: cluster 2024-03-26T03:56:51.205585+0000 mon.a (mon.0) 2704 : cluster [DBG] osdmap e459: 8 total, 8 up, 8 in 2024-03-26T03:56:51.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:51 smithi190 bash[23392]: audit 2024-03-26T03:56:51.207432+0000 mon.a (mon.0) 2705 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-26T03:56:51.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:51 smithi175 bash[21527]: audit 2024-03-26T03:56:50.155418+0000 mgr.y (mgr.24431) 487 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:56:51.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:51 smithi175 bash[21527]: audit 2024-03-26T03:56:50.195999+0000 mon.a (mon.0) 2700 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-85","var": "dedup_tier","val": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:56:51.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:51 smithi175 bash[21527]: cluster 2024-03-26T03:56:50.203310+0000 mon.a (mon.0) 2701 : cluster [DBG] osdmap e458: 8 total, 8 up, 8 in 2024-03-26T03:56:51.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:51 smithi175 bash[21527]: audit 2024-03-26T03:56:50.205078+0000 mon.a (mon.0) 2702 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-26T03:56:51.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:51 smithi175 bash[21527]: audit 2024-03-26T03:56:50.244381+0000 mon.c (mon.2) 531 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:56:51.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:51 smithi175 bash[21527]: audit 2024-03-26T03:56:51.198746+0000 mon.a (mon.0) 2703 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-26T03:56:51.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:51 smithi175 bash[21527]: cluster 2024-03-26T03:56:51.205585+0000 mon.a (mon.0) 2704 : cluster [DBG] osdmap e459: 8 total, 8 up, 8 in 2024-03-26T03:56:51.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:51 smithi175 bash[21527]: audit 2024-03-26T03:56:51.207432+0000 mon.a (mon.0) 2705 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-26T03:56:51.495 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:51 smithi175 bash[29284]: audit 2024-03-26T03:56:50.155418+0000 mgr.y (mgr.24431) 487 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:56:51.495 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:51 smithi175 bash[29284]: audit 2024-03-26T03:56:50.195999+0000 mon.a (mon.0) 2700 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-85","var": "dedup_tier","val": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:56:51.495 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:51 smithi175 bash[29284]: cluster 2024-03-26T03:56:50.203310+0000 mon.a (mon.0) 2701 : cluster [DBG] osdmap e458: 8 total, 8 up, 8 in 2024-03-26T03:56:51.495 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:51 smithi175 bash[29284]: audit 2024-03-26T03:56:50.205078+0000 mon.a (mon.0) 2702 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-26T03:56:51.495 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:51 smithi175 bash[29284]: audit 2024-03-26T03:56:50.244381+0000 mon.c (mon.2) 531 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:56:51.496 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:51 smithi175 bash[29284]: audit 2024-03-26T03:56:51.198746+0000 mon.a (mon.0) 2703 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-26T03:56:51.496 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:51 smithi175 bash[29284]: cluster 2024-03-26T03:56:51.205585+0000 mon.a (mon.0) 2704 : cluster [DBG] osdmap e459: 8 total, 8 up, 8 in 2024-03-26T03:56:51.496 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:51 smithi175 bash[29284]: audit 2024-03-26T03:56:51.207432+0000 mon.a (mon.0) 2705 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-26T03:56:52.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:52 smithi190 bash[23392]: cluster 2024-03-26T03:56:50.865776+0000 mgr.y (mgr.24431) 488 : cluster [DBG] pgmap v732: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-03-26T03:56:52.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:52 smithi190 bash[23392]: audit 2024-03-26T03:56:52.201549+0000 mon.a (mon.0) 2706 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-26T03:56:52.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:52 smithi190 bash[23392]: cluster 2024-03-26T03:56:52.210200+0000 mon.a (mon.0) 2707 : cluster [DBG] osdmap e460: 8 total, 8 up, 8 in 2024-03-26T03:56:52.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:52 smithi175 bash[21527]: cluster 2024-03-26T03:56:50.865776+0000 mgr.y (mgr.24431) 488 : cluster [DBG] pgmap v732: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-03-26T03:56:52.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:52 smithi175 bash[21527]: audit 2024-03-26T03:56:52.201549+0000 mon.a (mon.0) 2706 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-26T03:56:52.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:52 smithi175 bash[21527]: cluster 2024-03-26T03:56:52.210200+0000 mon.a (mon.0) 2707 : cluster [DBG] osdmap e460: 8 total, 8 up, 8 in 2024-03-26T03:56:52.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:52 smithi175 bash[29284]: cluster 2024-03-26T03:56:50.865776+0000 mgr.y (mgr.24431) 488 : cluster [DBG] pgmap v732: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-03-26T03:56:52.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:52 smithi175 bash[29284]: audit 2024-03-26T03:56:52.201549+0000 mon.a (mon.0) 2706 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-26T03:56:52.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:52 smithi175 bash[29284]: cluster 2024-03-26T03:56:52.210200+0000 mon.a (mon.0) 2707 : cluster [DBG] osdmap e460: 8 total, 8 up, 8 in 2024-03-26T03:56:54.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:54 smithi190 bash[23392]: cluster 2024-03-26T03:56:52.866662+0000 mgr.y (mgr.24431) 489 : cluster [DBG] pgmap v735: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-03-26T03:56:54.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:54 smithi190 bash[23392]: cluster 2024-03-26T03:56:53.245806+0000 mon.a (mon.0) 2708 : cluster [DBG] osdmap e461: 8 total, 8 up, 8 in 2024-03-26T03:56:54.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:54 smithi175 bash[21527]: cluster 2024-03-26T03:56:52.866662+0000 mgr.y (mgr.24431) 489 : cluster [DBG] pgmap v735: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-03-26T03:56:54.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:54 smithi175 bash[21527]: cluster 2024-03-26T03:56:53.245806+0000 mon.a (mon.0) 2708 : cluster [DBG] osdmap e461: 8 total, 8 up, 8 in 2024-03-26T03:56:54.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:54 smithi175 bash[29284]: cluster 2024-03-26T03:56:52.866662+0000 mgr.y (mgr.24431) 489 : cluster [DBG] pgmap v735: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-03-26T03:56:54.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:54 smithi175 bash[29284]: cluster 2024-03-26T03:56:53.245806+0000 mon.a (mon.0) 2708 : cluster [DBG] osdmap e461: 8 total, 8 up, 8 in 2024-03-26T03:56:55.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:55 smithi190 bash[23392]: cluster 2024-03-26T03:56:54.255230+0000 mon.a (mon.0) 2709 : cluster [DBG] osdmap e462: 8 total, 8 up, 8 in 2024-03-26T03:56:55.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:55 smithi190 bash[23392]: audit 2024-03-26T03:56:54.356491+0000 mon.a (mon.0) 2710 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:56:55.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:55 smithi190 bash[23392]: audit 2024-03-26T03:56:54.357429+0000 mon.a (mon.0) 2711 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-85"}]: dispatch 2024-03-26T03:56:55.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:55 smithi175 bash[21527]: cluster 2024-03-26T03:56:54.255230+0000 mon.a (mon.0) 2709 : cluster [DBG] osdmap e462: 8 total, 8 up, 8 in 2024-03-26T03:56:55.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:55 smithi175 bash[21527]: audit 2024-03-26T03:56:54.356491+0000 mon.a (mon.0) 2710 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:56:55.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:55 smithi175 bash[21527]: audit 2024-03-26T03:56:54.357429+0000 mon.a (mon.0) 2711 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-85"}]: dispatch 2024-03-26T03:56:55.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:55 smithi175 bash[29284]: cluster 2024-03-26T03:56:54.255230+0000 mon.a (mon.0) 2709 : cluster [DBG] osdmap e462: 8 total, 8 up, 8 in 2024-03-26T03:56:55.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:55 smithi175 bash[29284]: audit 2024-03-26T03:56:54.356491+0000 mon.a (mon.0) 2710 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:56:55.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:55 smithi175 bash[29284]: audit 2024-03-26T03:56:54.357429+0000 mon.a (mon.0) 2711 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-85"}]: dispatch 2024-03-26T03:56:56.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:56 smithi190 bash[23392]: cluster 2024-03-26T03:56:54.869105+0000 mgr.y (mgr.24431) 490 : cluster [DBG] pgmap v738: 292 pgs: 292 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.2 KiB/s wr, 9 op/s 2024-03-26T03:56:56.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:56 smithi190 bash[23392]: cluster 2024-03-26T03:56:55.282839+0000 mon.a (mon.0) 2712 : cluster [DBG] osdmap e463: 8 total, 8 up, 8 in 2024-03-26T03:56:56.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:56 smithi175 bash[21527]: cluster 2024-03-26T03:56:54.869105+0000 mgr.y (mgr.24431) 490 : cluster [DBG] pgmap v738: 292 pgs: 292 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.2 KiB/s wr, 9 op/s 2024-03-26T03:56:56.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:56 smithi175 bash[21527]: cluster 2024-03-26T03:56:55.282839+0000 mon.a (mon.0) 2712 : cluster [DBG] osdmap e463: 8 total, 8 up, 8 in 2024-03-26T03:56:56.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:56 smithi175 bash[29284]: cluster 2024-03-26T03:56:54.869105+0000 mgr.y (mgr.24431) 490 : cluster [DBG] pgmap v738: 292 pgs: 292 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.2 KiB/s wr, 9 op/s 2024-03-26T03:56:56.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:56 smithi175 bash[29284]: cluster 2024-03-26T03:56:55.282839+0000 mon.a (mon.0) 2712 : cluster [DBG] osdmap e463: 8 total, 8 up, 8 in 2024-03-26T03:56:57.324 INFO:tasks.workunit.client.0.smithi175.stdout: OK ] LibRadosTwoPoolsPP.ProxyRead (18210 ms) 2024-03-26T03:56:57.324 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2024-03-26T03:56:57.324 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (22440 ms) 2024-03-26T03:56:57.324 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-03-26T03:56:57.324 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (3037 ms) 2024-03-26T03:56:57.324 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-03-26T03:56:57.325 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: require_osd_release = squid 2024-03-26T03:56:57.325 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (3034 ms) 2024-03-26T03:56:57.325 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-03-26T03:56:57.325 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (2997 ms) 2024-03-26T03:56:57.325 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-03-26T03:56:57.325 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: require_osd_release = squid 2024-03-26T03:56:57.325 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (3022 ms) 2024-03-26T03:56:57.325 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-03-26T03:56:57.325 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: require_osd_release = squid 2024-03-26T03:56:57.325 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (4049 ms) 2024-03-26T03:56:57.325 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-03-26T03:56:57.325 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: require_osd_release = squid 2024-03-26T03:56:57.325 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (39856 ms) 2024-03-26T03:56:57.325 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-03-26T03:56:57.325 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: require_osd_release = squid 2024-03-26T03:56:57.326 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (16324 ms) 2024-03-26T03:56:57.326 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-03-26T03:56:57.326 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: require_osd_release = squid 2024-03-26T03:56:57.326 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (4059 ms) 2024-03-26T03:56:57.326 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-03-26T03:56:57.326 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: require_osd_release = squid 2024-03-26T03:56:57.326 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (3054 ms) 2024-03-26T03:56:57.326 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-03-26T03:56:57.326 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: require_osd_release = squid 2024-03-26T03:56:57.326 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (25166 ms) 2024-03-26T03:56:57.326 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-03-26T03:56:57.326 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: require_osd_release = squid 2024-03-26T03:56:57.326 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (14427 ms) 2024-03-26T03:56:57.326 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-03-26T03:56:57.326 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: require_osd_release = squid 2024-03-26T03:56:57.327 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (5084 ms) 2024-03-26T03:56:57.327 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-03-26T03:56:57.327 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: require_osd_release = squid 2024-03-26T03:56:57.327 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictPromote (4083 ms) 2024-03-26T03:56:57.327 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-03-26T03:56:57.327 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: require_osd_release = squid 2024-03-26T03:56:57.327 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: waiting for scrubs... 2024-03-26T03:56:57.327 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: done waiting 2024-03-26T03:56:57.327 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (25199 ms) 2024-03-26T03:56:57.327 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-03-26T03:56:57.327 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: require_osd_release = squid 2024-03-26T03:56:57.327 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (10128 ms) 2024-03-26T03:56:57.327 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-03-26T03:56:57.327 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: require_osd_release = squid 2024-03-26T03:56:57.327 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (9105 ms) 2024-03-26T03:56:57.327 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-03-26T03:56:57.328 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: require_osd_release = squid 2024-03-26T03:56:57.328 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (9098 ms) 2024-03-26T03:56:57.328 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-03-26T03:56:57.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:57 smithi190 bash[23392]: cluster 2024-03-26T03:56:56.291731+0000 mon.a (mon.0) 2713 : cluster [DBG] osdmap e464: 8 total, 8 up, 8 in 2024-03-26T03:56:57.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:57 smithi190 bash[23392]: audit 2024-03-26T03:56:56.295862+0000 mon.a (mon.0) 2714 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:56:57.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:57 smithi190 bash[23392]: audit 2024-03-26T03:56:57.067299+0000 mon.c (mon.2) 532 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:56:57.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:57 smithi175 bash[21527]: cluster 2024-03-26T03:56:56.291731+0000 mon.a (mon.0) 2713 : cluster [DBG] osdmap e464: 8 total, 8 up, 8 in 2024-03-26T03:56:57.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:57 smithi175 bash[21527]: audit 2024-03-26T03:56:56.295862+0000 mon.a (mon.0) 2714 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:56:57.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:57 smithi175 bash[21527]: audit 2024-03-26T03:56:57.067299+0000 mon.c (mon.2) 532 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:56:57.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:57 smithi175 bash[29284]: cluster 2024-03-26T03:56:56.291731+0000 mon.a (mon.0) 2713 : cluster [DBG] osdmap e464: 8 total, 8 up, 8 in 2024-03-26T03:56:57.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:57 smithi175 bash[29284]: audit 2024-03-26T03:56:56.295862+0000 mon.a (mon.0) 2714 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:56:57.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:57 smithi175 bash[29284]: audit 2024-03-26T03:56:57.067299+0000 mon.c (mon.2) 532 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:56:58.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:58 smithi190 bash[23392]: cluster 2024-03-26T03:56:56.870032+0000 mgr.y (mgr.24431) 491 : cluster [DBG] pgmap v741: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-03-26T03:56:58.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:58 smithi190 bash[23392]: cluster 2024-03-26T03:56:57.284302+0000 mon.a (mon.0) 2715 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:56:58.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:58 smithi190 bash[23392]: audit 2024-03-26T03:56:57.295173+0000 mon.a (mon.0) 2716 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:56:58.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:58 smithi190 bash[23392]: cluster 2024-03-26T03:56:57.307805+0000 mon.a (mon.0) 2717 : cluster [DBG] osdmap e465: 8 total, 8 up, 8 in 2024-03-26T03:56:58.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:58 smithi190 bash[23392]: audit 2024-03-26T03:56:57.309948+0000 mon.a (mon.0) 2718 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:56:58.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:58 smithi175 bash[21527]: cluster 2024-03-26T03:56:56.870032+0000 mgr.y (mgr.24431) 491 : cluster [DBG] pgmap v741: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-03-26T03:56:58.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:58 smithi175 bash[21527]: cluster 2024-03-26T03:56:57.284302+0000 mon.a (mon.0) 2715 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:56:58.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:58 smithi175 bash[21527]: audit 2024-03-26T03:56:57.295173+0000 mon.a (mon.0) 2716 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:56:58.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:58 smithi175 bash[21527]: cluster 2024-03-26T03:56:57.307805+0000 mon.a (mon.0) 2717 : cluster [DBG] osdmap e465: 8 total, 8 up, 8 in 2024-03-26T03:56:58.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:58 smithi175 bash[21527]: audit 2024-03-26T03:56:57.309948+0000 mon.a (mon.0) 2718 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:56:58.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:58 smithi175 bash[29284]: cluster 2024-03-26T03:56:56.870032+0000 mgr.y (mgr.24431) 491 : cluster [DBG] pgmap v741: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-03-26T03:56:58.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:58 smithi175 bash[29284]: cluster 2024-03-26T03:56:57.284302+0000 mon.a (mon.0) 2715 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:56:58.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:58 smithi175 bash[29284]: audit 2024-03-26T03:56:57.295173+0000 mon.a (mon.0) 2716 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:56:58.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:58 smithi175 bash[29284]: cluster 2024-03-26T03:56:57.307805+0000 mon.a (mon.0) 2717 : cluster [DBG] osdmap e465: 8 total, 8 up, 8 in 2024-03-26T03:56:58.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:58 smithi175 bash[29284]: audit 2024-03-26T03:56:57.309948+0000 mon.a (mon.0) 2718 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:56:59.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:56:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:56:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:56:59.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:59 smithi190 bash[23392]: cluster 2024-03-26T03:56:58.309970+0000 mon.a (mon.0) 2719 : cluster [DBG] osdmap e466: 8 total, 8 up, 8 in 2024-03-26T03:56:59.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:59 smithi190 bash[23392]: audit 2024-03-26T03:56:58.316074+0000 mon.a (mon.0) 2720 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-26T03:56:59.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:56:59 smithi190 bash[23392]: audit 2024-03-26T03:56:59.302541+0000 mon.a (mon.0) 2721 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-26T03:56:59.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:59 smithi175 bash[21527]: cluster 2024-03-26T03:56:58.309970+0000 mon.a (mon.0) 2719 : cluster [DBG] osdmap e466: 8 total, 8 up, 8 in 2024-03-26T03:56:59.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:59 smithi175 bash[21527]: audit 2024-03-26T03:56:58.316074+0000 mon.a (mon.0) 2720 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-26T03:56:59.747 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:56:59 smithi175 bash[21527]: audit 2024-03-26T03:56:59.302541+0000 mon.a (mon.0) 2721 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-26T03:56:59.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:59 smithi175 bash[29284]: cluster 2024-03-26T03:56:58.309970+0000 mon.a (mon.0) 2719 : cluster [DBG] osdmap e466: 8 total, 8 up, 8 in 2024-03-26T03:56:59.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:59 smithi175 bash[29284]: audit 2024-03-26T03:56:58.316074+0000 mon.a (mon.0) 2720 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-26T03:56:59.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:56:59 smithi175 bash[29284]: audit 2024-03-26T03:56:59.302541+0000 mon.a (mon.0) 2721 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-26T03:57:00.492 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:57:00 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:57:00.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:00 smithi190 bash[23392]: cluster 2024-03-26T03:56:58.871102+0000 mgr.y (mgr.24431) 492 : cluster [DBG] pgmap v744: 324 pgs: 64 unknown, 260 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:57:00.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:00 smithi190 bash[23392]: cluster 2024-03-26T03:56:59.310614+0000 mon.a (mon.0) 2722 : cluster [DBG] osdmap e467: 8 total, 8 up, 8 in 2024-03-26T03:57:00.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:00 smithi190 bash[23392]: audit 2024-03-26T03:56:59.311286+0000 mon.a (mon.0) 2723 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-87","var": "dedup_tier","val": "test-rados-api-smithi175-64594-89-test-flush"}]: dispatch 2024-03-26T03:57:00.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:00 smithi190 bash[23392]: audit 2024-03-26T03:57:00.305679+0000 mon.a (mon.0) 2724 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-87","var": "dedup_tier","val": "test-rados-api-smithi175-64594-89-test-flush"}]': finished 2024-03-26T03:57:00.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:00 smithi190 bash[23392]: cluster 2024-03-26T03:57:00.313593+0000 mon.a (mon.0) 2725 : cluster [DBG] osdmap e468: 8 total, 8 up, 8 in 2024-03-26T03:57:00.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:00 smithi190 bash[23392]: audit 2024-03-26T03:57:00.316323+0000 mon.a (mon.0) 2726 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-26T03:57:00.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:00 smithi175 bash[21527]: cluster 2024-03-26T03:56:58.871102+0000 mgr.y (mgr.24431) 492 : cluster [DBG] pgmap v744: 324 pgs: 64 unknown, 260 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:57:00.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:00 smithi175 bash[21527]: cluster 2024-03-26T03:56:59.310614+0000 mon.a (mon.0) 2722 : cluster [DBG] osdmap e467: 8 total, 8 up, 8 in 2024-03-26T03:57:00.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:00 smithi175 bash[21527]: audit 2024-03-26T03:56:59.311286+0000 mon.a (mon.0) 2723 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-87","var": "dedup_tier","val": "test-rados-api-smithi175-64594-89-test-flush"}]: dispatch 2024-03-26T03:57:00.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:00 smithi175 bash[21527]: audit 2024-03-26T03:57:00.305679+0000 mon.a (mon.0) 2724 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-87","var": "dedup_tier","val": "test-rados-api-smithi175-64594-89-test-flush"}]': finished 2024-03-26T03:57:00.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:00 smithi175 bash[21527]: cluster 2024-03-26T03:57:00.313593+0000 mon.a (mon.0) 2725 : cluster [DBG] osdmap e468: 8 total, 8 up, 8 in 2024-03-26T03:57:00.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:00 smithi175 bash[21527]: audit 2024-03-26T03:57:00.316323+0000 mon.a (mon.0) 2726 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-26T03:57:00.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:00 smithi175 bash[29284]: cluster 2024-03-26T03:56:58.871102+0000 mgr.y (mgr.24431) 492 : cluster [DBG] pgmap v744: 324 pgs: 64 unknown, 260 active+clean; 8.4 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:57:00.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:00 smithi175 bash[29284]: cluster 2024-03-26T03:56:59.310614+0000 mon.a (mon.0) 2722 : cluster [DBG] osdmap e467: 8 total, 8 up, 8 in 2024-03-26T03:57:00.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:00 smithi175 bash[29284]: audit 2024-03-26T03:56:59.311286+0000 mon.a (mon.0) 2723 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-87","var": "dedup_tier","val": "test-rados-api-smithi175-64594-89-test-flush"}]: dispatch 2024-03-26T03:57:00.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:00 smithi175 bash[29284]: audit 2024-03-26T03:57:00.305679+0000 mon.a (mon.0) 2724 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-87","var": "dedup_tier","val": "test-rados-api-smithi175-64594-89-test-flush"}]': finished 2024-03-26T03:57:00.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:00 smithi175 bash[29284]: cluster 2024-03-26T03:57:00.313593+0000 mon.a (mon.0) 2725 : cluster [DBG] osdmap e468: 8 total, 8 up, 8 in 2024-03-26T03:57:00.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:00 smithi175 bash[29284]: audit 2024-03-26T03:57:00.316323+0000 mon.a (mon.0) 2726 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-26T03:57:01.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:01 smithi190 bash[23392]: audit 2024-03-26T03:57:00.165826+0000 mgr.y (mgr.24431) 493 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:57:01.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:01 smithi190 bash[23392]: audit 2024-03-26T03:57:01.308577+0000 mon.a (mon.0) 2727 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-26T03:57:01.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:01 smithi190 bash[23392]: cluster 2024-03-26T03:57:01.315931+0000 mon.a (mon.0) 2728 : cluster [DBG] osdmap e469: 8 total, 8 up, 8 in 2024-03-26T03:57:01.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:01 smithi190 bash[23392]: audit 2024-03-26T03:57:01.320764+0000 mon.a (mon.0) 2729 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-26T03:57:01.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:01 smithi175 bash[21527]: audit 2024-03-26T03:57:00.165826+0000 mgr.y (mgr.24431) 493 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:57:01.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:01 smithi175 bash[21527]: audit 2024-03-26T03:57:01.308577+0000 mon.a (mon.0) 2727 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-26T03:57:01.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:01 smithi175 bash[21527]: cluster 2024-03-26T03:57:01.315931+0000 mon.a (mon.0) 2728 : cluster [DBG] osdmap e469: 8 total, 8 up, 8 in 2024-03-26T03:57:01.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:01 smithi175 bash[21527]: audit 2024-03-26T03:57:01.320764+0000 mon.a (mon.0) 2729 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-26T03:57:01.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:01 smithi175 bash[29284]: audit 2024-03-26T03:57:00.165826+0000 mgr.y (mgr.24431) 493 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:57:01.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:01 smithi175 bash[29284]: audit 2024-03-26T03:57:01.308577+0000 mon.a (mon.0) 2727 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-26T03:57:01.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:01 smithi175 bash[29284]: cluster 2024-03-26T03:57:01.315931+0000 mon.a (mon.0) 2728 : cluster [DBG] osdmap e469: 8 total, 8 up, 8 in 2024-03-26T03:57:01.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:01 smithi175 bash[29284]: audit 2024-03-26T03:57:01.320764+0000 mon.a (mon.0) 2729 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-26T03:57:02.741 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:02 smithi190 bash[23392]: cluster 2024-03-26T03:57:00.872120+0000 mgr.y (mgr.24431) 494 : cluster [DBG] pgmap v747: 324 pgs: 324 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:57:02.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:02 smithi190 bash[23392]: audit 2024-03-26T03:57:02.310291+0000 mon.a (mon.0) 2730 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-26T03:57:02.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:02 smithi190 bash[23392]: cluster 2024-03-26T03:57:02.315718+0000 mon.a (mon.0) 2731 : cluster [DBG] osdmap e470: 8 total, 8 up, 8 in 2024-03-26T03:57:02.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:02 smithi175 bash[21527]: cluster 2024-03-26T03:57:00.872120+0000 mgr.y (mgr.24431) 494 : cluster [DBG] pgmap v747: 324 pgs: 324 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:57:02.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:02 smithi175 bash[21527]: audit 2024-03-26T03:57:02.310291+0000 mon.a (mon.0) 2730 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-26T03:57:02.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:02 smithi175 bash[21527]: cluster 2024-03-26T03:57:02.315718+0000 mon.a (mon.0) 2731 : cluster [DBG] osdmap e470: 8 total, 8 up, 8 in 2024-03-26T03:57:02.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:02 smithi175 bash[29284]: cluster 2024-03-26T03:57:00.872120+0000 mgr.y (mgr.24431) 494 : cluster [DBG] pgmap v747: 324 pgs: 324 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:57:02.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:02 smithi175 bash[29284]: audit 2024-03-26T03:57:02.310291+0000 mon.a (mon.0) 2730 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-26T03:57:02.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:02 smithi175 bash[29284]: cluster 2024-03-26T03:57:02.315718+0000 mon.a (mon.0) 2731 : cluster [DBG] osdmap e470: 8 total, 8 up, 8 in 2024-03-26T03:57:04.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:03 smithi190 bash[23392]: audit 2024-03-26T03:57:02.757338+0000 mon.a (mon.0) 2732 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:57:04.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:03 smithi190 bash[23392]: audit 2024-03-26T03:57:02.764128+0000 mon.a (mon.0) 2733 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:57:04.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:03 smithi190 bash[23392]: cluster 2024-03-26T03:57:02.872988+0000 mgr.y (mgr.24431) 495 : cluster [DBG] pgmap v750: 324 pgs: 324 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:57:04.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:03 smithi190 bash[23392]: audit 2024-03-26T03:57:03.211946+0000 mon.a (mon.0) 2734 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:57:04.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:03 smithi190 bash[23392]: audit 2024-03-26T03:57:03.220160+0000 mon.a (mon.0) 2735 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:57:04.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:03 smithi190 bash[23392]: cluster 2024-03-26T03:57:03.320961+0000 mon.a (mon.0) 2736 : cluster [DBG] osdmap e471: 8 total, 8 up, 8 in 2024-03-26T03:57:04.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:03 smithi190 bash[23392]: audit 2024-03-26T03:57:03.407827+0000 mon.a (mon.0) 2737 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:57:04.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:03 smithi190 bash[23392]: audit 2024-03-26T03:57:03.408405+0000 mon.a (mon.0) 2738 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-87"}]: dispatch 2024-03-26T03:57:04.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:03 smithi175 bash[21527]: audit 2024-03-26T03:57:02.757338+0000 mon.a (mon.0) 2732 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:57:04.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:03 smithi175 bash[21527]: audit 2024-03-26T03:57:02.764128+0000 mon.a (mon.0) 2733 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:57:04.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:03 smithi175 bash[21527]: cluster 2024-03-26T03:57:02.872988+0000 mgr.y (mgr.24431) 495 : cluster [DBG] pgmap v750: 324 pgs: 324 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:57:04.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:03 smithi175 bash[21527]: audit 2024-03-26T03:57:03.211946+0000 mon.a (mon.0) 2734 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:57:04.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:03 smithi175 bash[21527]: audit 2024-03-26T03:57:03.220160+0000 mon.a (mon.0) 2735 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:57:04.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:03 smithi175 bash[21527]: cluster 2024-03-26T03:57:03.320961+0000 mon.a (mon.0) 2736 : cluster [DBG] osdmap e471: 8 total, 8 up, 8 in 2024-03-26T03:57:04.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:03 smithi175 bash[21527]: audit 2024-03-26T03:57:03.407827+0000 mon.a (mon.0) 2737 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:57:04.246 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:03 smithi175 bash[21527]: audit 2024-03-26T03:57:03.408405+0000 mon.a (mon.0) 2738 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-87"}]: dispatch 2024-03-26T03:57:04.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:03 smithi175 bash[29284]: audit 2024-03-26T03:57:02.757338+0000 mon.a (mon.0) 2732 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:57:04.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:03 smithi175 bash[29284]: audit 2024-03-26T03:57:02.764128+0000 mon.a (mon.0) 2733 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:57:04.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:03 smithi175 bash[29284]: cluster 2024-03-26T03:57:02.872988+0000 mgr.y (mgr.24431) 495 : cluster [DBG] pgmap v750: 324 pgs: 324 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:57:04.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:03 smithi175 bash[29284]: audit 2024-03-26T03:57:03.211946+0000 mon.a (mon.0) 2734 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:57:04.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:03 smithi175 bash[29284]: audit 2024-03-26T03:57:03.220160+0000 mon.a (mon.0) 2735 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:57:04.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:03 smithi175 bash[29284]: cluster 2024-03-26T03:57:03.320961+0000 mon.a (mon.0) 2736 : cluster [DBG] osdmap e471: 8 total, 8 up, 8 in 2024-03-26T03:57:04.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:03 smithi175 bash[29284]: audit 2024-03-26T03:57:03.407827+0000 mon.a (mon.0) 2737 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:57:04.246 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:03 smithi175 bash[29284]: audit 2024-03-26T03:57:03.408405+0000 mon.a (mon.0) 2738 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-87"}]: dispatch 2024-03-26T03:57:05.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:04 smithi190 bash[23392]: audit 2024-03-26T03:57:03.960797+0000 mon.c (mon.2) 533 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:57:05.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:04 smithi190 bash[23392]: audit 2024-03-26T03:57:03.962530+0000 mon.c (mon.2) 534 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:57:05.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:04 smithi190 bash[23392]: audit 2024-03-26T03:57:03.973847+0000 mon.a (mon.0) 2739 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:57:05.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:04 smithi190 bash[23392]: cluster 2024-03-26T03:57:04.323317+0000 mon.a (mon.0) 2740 : cluster [DBG] osdmap e472: 8 total, 8 up, 8 in 2024-03-26T03:57:05.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:04 smithi175 bash[29284]: audit 2024-03-26T03:57:03.960797+0000 mon.c (mon.2) 533 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:57:05.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:04 smithi175 bash[29284]: audit 2024-03-26T03:57:03.962530+0000 mon.c (mon.2) 534 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:57:05.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:04 smithi175 bash[29284]: audit 2024-03-26T03:57:03.973847+0000 mon.a (mon.0) 2739 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:57:05.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:04 smithi175 bash[29284]: cluster 2024-03-26T03:57:04.323317+0000 mon.a (mon.0) 2740 : cluster [DBG] osdmap e472: 8 total, 8 up, 8 in 2024-03-26T03:57:05.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:04 smithi175 bash[21527]: audit 2024-03-26T03:57:03.960797+0000 mon.c (mon.2) 533 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:57:05.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:04 smithi175 bash[21527]: audit 2024-03-26T03:57:03.962530+0000 mon.c (mon.2) 534 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:57:05.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:04 smithi175 bash[21527]: audit 2024-03-26T03:57:03.973847+0000 mon.a (mon.0) 2739 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:57:05.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:04 smithi175 bash[21527]: cluster 2024-03-26T03:57:04.323317+0000 mon.a (mon.0) 2740 : cluster [DBG] osdmap e472: 8 total, 8 up, 8 in 2024-03-26T03:57:06.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:05 smithi190 bash[23392]: cluster 2024-03-26T03:57:04.875155+0000 mgr.y (mgr.24431) 496 : cluster [DBG] pgmap v753: 260 pgs: 260 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:57:06.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:05 smithi190 bash[23392]: audit 2024-03-26T03:57:05.278753+0000 mon.c (mon.2) 535 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:57:06.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:05 smithi190 bash[23392]: cluster 2024-03-26T03:57:05.331608+0000 mon.a (mon.0) 2741 : cluster [DBG] osdmap e473: 8 total, 8 up, 8 in 2024-03-26T03:57:06.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:05 smithi190 bash[23392]: audit 2024-03-26T03:57:05.339505+0000 mon.a (mon.0) 2742 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:57:06.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:05 smithi175 bash[21527]: cluster 2024-03-26T03:57:04.875155+0000 mgr.y (mgr.24431) 496 : cluster [DBG] pgmap v753: 260 pgs: 260 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:57:06.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:05 smithi175 bash[21527]: audit 2024-03-26T03:57:05.278753+0000 mon.c (mon.2) 535 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:57:06.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:05 smithi175 bash[21527]: cluster 2024-03-26T03:57:05.331608+0000 mon.a (mon.0) 2741 : cluster [DBG] osdmap e473: 8 total, 8 up, 8 in 2024-03-26T03:57:06.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:05 smithi175 bash[21527]: audit 2024-03-26T03:57:05.339505+0000 mon.a (mon.0) 2742 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:57:06.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:05 smithi175 bash[29284]: cluster 2024-03-26T03:57:04.875155+0000 mgr.y (mgr.24431) 496 : cluster [DBG] pgmap v753: 260 pgs: 260 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:57:06.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:05 smithi175 bash[29284]: audit 2024-03-26T03:57:05.278753+0000 mon.c (mon.2) 535 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:57:06.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:05 smithi175 bash[29284]: cluster 2024-03-26T03:57:05.331608+0000 mon.a (mon.0) 2741 : cluster [DBG] osdmap e473: 8 total, 8 up, 8 in 2024-03-26T03:57:06.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:05 smithi175 bash[29284]: audit 2024-03-26T03:57:05.339505+0000 mon.a (mon.0) 2742 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:57:07.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:06 smithi190 bash[23392]: cluster 2024-03-26T03:57:05.768568+0000 mon.a (mon.0) 2743 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:57:07.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:06 smithi190 bash[23392]: audit 2024-03-26T03:57:06.322114+0000 mon.a (mon.0) 2744 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:57:07.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:06 smithi190 bash[23392]: cluster 2024-03-26T03:57:06.331979+0000 mon.a (mon.0) 2745 : cluster [DBG] osdmap e474: 8 total, 8 up, 8 in 2024-03-26T03:57:07.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:06 smithi190 bash[23392]: audit 2024-03-26T03:57:06.337288+0000 mon.a (mon.0) 2746 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:57:07.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:06 smithi190 bash[23392]: audit 2024-03-26T03:57:06.356782+0000 mon.a (mon.0) 2747 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-26T03:57:07.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:06 smithi175 bash[21527]: cluster 2024-03-26T03:57:05.768568+0000 mon.a (mon.0) 2743 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:57:07.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:06 smithi175 bash[21527]: audit 2024-03-26T03:57:06.322114+0000 mon.a (mon.0) 2744 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:57:07.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:06 smithi175 bash[21527]: cluster 2024-03-26T03:57:06.331979+0000 mon.a (mon.0) 2745 : cluster [DBG] osdmap e474: 8 total, 8 up, 8 in 2024-03-26T03:57:07.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:06 smithi175 bash[21527]: audit 2024-03-26T03:57:06.337288+0000 mon.a (mon.0) 2746 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:57:07.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:06 smithi175 bash[21527]: audit 2024-03-26T03:57:06.356782+0000 mon.a (mon.0) 2747 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-26T03:57:07.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:06 smithi175 bash[29284]: cluster 2024-03-26T03:57:05.768568+0000 mon.a (mon.0) 2743 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:57:07.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:06 smithi175 bash[29284]: audit 2024-03-26T03:57:06.322114+0000 mon.a (mon.0) 2744 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:57:07.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:06 smithi175 bash[29284]: cluster 2024-03-26T03:57:06.331979+0000 mon.a (mon.0) 2745 : cluster [DBG] osdmap e474: 8 total, 8 up, 8 in 2024-03-26T03:57:07.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:06 smithi175 bash[29284]: audit 2024-03-26T03:57:06.337288+0000 mon.a (mon.0) 2746 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:57:07.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:06 smithi175 bash[29284]: audit 2024-03-26T03:57:06.356782+0000 mon.a (mon.0) 2747 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-26T03:57:08.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:08 smithi190 bash[23392]: cluster 2024-03-26T03:57:06.876093+0000 mgr.y (mgr.24431) 497 : cluster [DBG] pgmap v756: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T03:57:08.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:08 smithi190 bash[23392]: audit 2024-03-26T03:57:07.324908+0000 mon.a (mon.0) 2748 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-26T03:57:08.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:08 smithi190 bash[23392]: cluster 2024-03-26T03:57:07.333685+0000 mon.a (mon.0) 2749 : cluster [DBG] osdmap e475: 8 total, 8 up, 8 in 2024-03-26T03:57:08.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:08 smithi175 bash[29284]: cluster 2024-03-26T03:57:06.876093+0000 mgr.y (mgr.24431) 497 : cluster [DBG] pgmap v756: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T03:57:08.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:08 smithi175 bash[29284]: audit 2024-03-26T03:57:07.324908+0000 mon.a (mon.0) 2748 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-26T03:57:08.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:08 smithi175 bash[29284]: cluster 2024-03-26T03:57:07.333685+0000 mon.a (mon.0) 2749 : cluster [DBG] osdmap e475: 8 total, 8 up, 8 in 2024-03-26T03:57:08.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:08 smithi175 bash[21527]: cluster 2024-03-26T03:57:06.876093+0000 mgr.y (mgr.24431) 497 : cluster [DBG] pgmap v756: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T03:57:08.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:08 smithi175 bash[21527]: audit 2024-03-26T03:57:07.324908+0000 mon.a (mon.0) 2748 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-26T03:57:08.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:08 smithi175 bash[21527]: cluster 2024-03-26T03:57:07.333685+0000 mon.a (mon.0) 2749 : cluster [DBG] osdmap e475: 8 total, 8 up, 8 in 2024-03-26T03:57:09.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:57:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:57:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:57:09.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:09 smithi190 bash[23392]: cluster 2024-03-26T03:57:08.335209+0000 mon.a (mon.0) 2750 : cluster [DBG] osdmap e476: 8 total, 8 up, 8 in 2024-03-26T03:57:09.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:09 smithi175 bash[29284]: cluster 2024-03-26T03:57:08.335209+0000 mon.a (mon.0) 2750 : cluster [DBG] osdmap e476: 8 total, 8 up, 8 in 2024-03-26T03:57:09.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:09 smithi175 bash[21527]: cluster 2024-03-26T03:57:08.335209+0000 mon.a (mon.0) 2750 : cluster [DBG] osdmap e476: 8 total, 8 up, 8 in 2024-03-26T03:57:10.492 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:57:10 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:57:10.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:10 smithi190 bash[23392]: cluster 2024-03-26T03:57:08.877023+0000 mgr.y (mgr.24431) 498 : cluster [DBG] pgmap v759: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:57:10.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:10 smithi190 bash[23392]: cluster 2024-03-26T03:57:09.362308+0000 mon.a (mon.0) 2751 : cluster [DBG] osdmap e477: 8 total, 8 up, 8 in 2024-03-26T03:57:10.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:10 smithi190 bash[23392]: audit 2024-03-26T03:57:09.461466+0000 mon.a (mon.0) 2752 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:57:10.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:10 smithi190 bash[23392]: audit 2024-03-26T03:57:09.462384+0000 mon.a (mon.0) 2753 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-90"}]: dispatch 2024-03-26T03:57:10.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:10 smithi175 bash[29284]: cluster 2024-03-26T03:57:08.877023+0000 mgr.y (mgr.24431) 498 : cluster [DBG] pgmap v759: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:57:10.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:10 smithi175 bash[29284]: cluster 2024-03-26T03:57:09.362308+0000 mon.a (mon.0) 2751 : cluster [DBG] osdmap e477: 8 total, 8 up, 8 in 2024-03-26T03:57:10.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:10 smithi175 bash[29284]: audit 2024-03-26T03:57:09.461466+0000 mon.a (mon.0) 2752 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:57:10.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:10 smithi175 bash[29284]: audit 2024-03-26T03:57:09.462384+0000 mon.a (mon.0) 2753 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-90"}]: dispatch 2024-03-26T03:57:10.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:10 smithi175 bash[21527]: cluster 2024-03-26T03:57:08.877023+0000 mgr.y (mgr.24431) 498 : cluster [DBG] pgmap v759: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:57:10.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:10 smithi175 bash[21527]: cluster 2024-03-26T03:57:09.362308+0000 mon.a (mon.0) 2751 : cluster [DBG] osdmap e477: 8 total, 8 up, 8 in 2024-03-26T03:57:10.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:10 smithi175 bash[21527]: audit 2024-03-26T03:57:09.461466+0000 mon.a (mon.0) 2752 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:57:10.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:10 smithi175 bash[21527]: audit 2024-03-26T03:57:09.462384+0000 mon.a (mon.0) 2753 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-90"}]: dispatch 2024-03-26T03:57:11.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:11 smithi190 bash[23392]: audit 2024-03-26T03:57:10.173963+0000 mgr.y (mgr.24431) 499 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:57:11.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:11 smithi190 bash[23392]: cluster 2024-03-26T03:57:10.363103+0000 mon.a (mon.0) 2754 : cluster [DBG] osdmap e478: 8 total, 8 up, 8 in 2024-03-26T03:57:11.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:11 smithi190 bash[23392]: cluster 2024-03-26T03:57:11.368956+0000 mon.a (mon.0) 2755 : cluster [DBG] osdmap e479: 8 total, 8 up, 8 in 2024-03-26T03:57:11.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:11 smithi190 bash[23392]: audit 2024-03-26T03:57:11.372138+0000 mon.a (mon.0) 2756 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:57:11.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:11 smithi175 bash[21527]: audit 2024-03-26T03:57:10.173963+0000 mgr.y (mgr.24431) 499 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:57:11.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:11 smithi175 bash[21527]: cluster 2024-03-26T03:57:10.363103+0000 mon.a (mon.0) 2754 : cluster [DBG] osdmap e478: 8 total, 8 up, 8 in 2024-03-26T03:57:11.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:11 smithi175 bash[21527]: cluster 2024-03-26T03:57:11.368956+0000 mon.a (mon.0) 2755 : cluster [DBG] osdmap e479: 8 total, 8 up, 8 in 2024-03-26T03:57:11.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:11 smithi175 bash[21527]: audit 2024-03-26T03:57:11.372138+0000 mon.a (mon.0) 2756 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:57:11.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:11 smithi175 bash[29284]: audit 2024-03-26T03:57:10.173963+0000 mgr.y (mgr.24431) 499 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:57:11.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:11 smithi175 bash[29284]: cluster 2024-03-26T03:57:10.363103+0000 mon.a (mon.0) 2754 : cluster [DBG] osdmap e478: 8 total, 8 up, 8 in 2024-03-26T03:57:11.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:11 smithi175 bash[29284]: cluster 2024-03-26T03:57:11.368956+0000 mon.a (mon.0) 2755 : cluster [DBG] osdmap e479: 8 total, 8 up, 8 in 2024-03-26T03:57:11.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:11 smithi175 bash[29284]: audit 2024-03-26T03:57:11.372138+0000 mon.a (mon.0) 2756 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:57:12.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:12 smithi190 bash[23392]: cluster 2024-03-26T03:57:10.878039+0000 mgr.y (mgr.24431) 500 : cluster [DBG] pgmap v762: 260 pgs: 260 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-26T03:57:12.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:12 smithi190 bash[23392]: audit 2024-03-26T03:57:12.363377+0000 mon.a (mon.0) 2757 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:57:12.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:12 smithi190 bash[23392]: cluster 2024-03-26T03:57:12.373954+0000 mon.a (mon.0) 2758 : cluster [DBG] osdmap e480: 8 total, 8 up, 8 in 2024-03-26T03:57:12.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:12 smithi190 bash[23392]: audit 2024-03-26T03:57:12.379880+0000 mon.a (mon.0) 2759 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:57:12.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:12 smithi175 bash[29284]: cluster 2024-03-26T03:57:10.878039+0000 mgr.y (mgr.24431) 500 : cluster [DBG] pgmap v762: 260 pgs: 260 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-26T03:57:12.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:12 smithi175 bash[29284]: audit 2024-03-26T03:57:12.363377+0000 mon.a (mon.0) 2757 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:57:12.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:12 smithi175 bash[29284]: cluster 2024-03-26T03:57:12.373954+0000 mon.a (mon.0) 2758 : cluster [DBG] osdmap e480: 8 total, 8 up, 8 in 2024-03-26T03:57:12.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:12 smithi175 bash[29284]: audit 2024-03-26T03:57:12.379880+0000 mon.a (mon.0) 2759 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:57:12.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:12 smithi175 bash[21527]: cluster 2024-03-26T03:57:10.878039+0000 mgr.y (mgr.24431) 500 : cluster [DBG] pgmap v762: 260 pgs: 260 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-26T03:57:12.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:12 smithi175 bash[21527]: audit 2024-03-26T03:57:12.363377+0000 mon.a (mon.0) 2757 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:57:12.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:12 smithi175 bash[21527]: cluster 2024-03-26T03:57:12.373954+0000 mon.a (mon.0) 2758 : cluster [DBG] osdmap e480: 8 total, 8 up, 8 in 2024-03-26T03:57:12.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:12 smithi175 bash[21527]: audit 2024-03-26T03:57:12.379880+0000 mon.a (mon.0) 2759 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:57:14.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:14 smithi190 bash[23392]: cluster 2024-03-26T03:57:12.878951+0000 mgr.y (mgr.24431) 501 : cluster [DBG] pgmap v765: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-26T03:57:14.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:14 smithi190 bash[23392]: cluster 2024-03-26T03:57:13.372728+0000 mon.a (mon.0) 2760 : cluster [DBG] osdmap e481: 8 total, 8 up, 8 in 2024-03-26T03:57:14.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:14 smithi175 bash[21527]: cluster 2024-03-26T03:57:12.878951+0000 mgr.y (mgr.24431) 501 : cluster [DBG] pgmap v765: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-26T03:57:14.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:14 smithi175 bash[21527]: cluster 2024-03-26T03:57:13.372728+0000 mon.a (mon.0) 2760 : cluster [DBG] osdmap e481: 8 total, 8 up, 8 in 2024-03-26T03:57:14.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:14 smithi175 bash[29284]: cluster 2024-03-26T03:57:12.878951+0000 mgr.y (mgr.24431) 501 : cluster [DBG] pgmap v765: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-26T03:57:14.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:14 smithi175 bash[29284]: cluster 2024-03-26T03:57:13.372728+0000 mon.a (mon.0) 2760 : cluster [DBG] osdmap e481: 8 total, 8 up, 8 in 2024-03-26T03:57:15.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:15 smithi190 bash[23392]: cluster 2024-03-26T03:57:14.378373+0000 mon.a (mon.0) 2761 : cluster [DBG] osdmap e482: 8 total, 8 up, 8 in 2024-03-26T03:57:15.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:15 smithi190 bash[23392]: audit 2024-03-26T03:57:14.469020+0000 mon.a (mon.0) 2762 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:57:15.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:15 smithi190 bash[23392]: audit 2024-03-26T03:57:14.470139+0000 mon.a (mon.0) 2763 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-92"}]: dispatch 2024-03-26T03:57:15.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:15 smithi175 bash[21527]: cluster 2024-03-26T03:57:14.378373+0000 mon.a (mon.0) 2761 : cluster [DBG] osdmap e482: 8 total, 8 up, 8 in 2024-03-26T03:57:15.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:15 smithi175 bash[21527]: audit 2024-03-26T03:57:14.469020+0000 mon.a (mon.0) 2762 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:57:15.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:15 smithi175 bash[21527]: audit 2024-03-26T03:57:14.470139+0000 mon.a (mon.0) 2763 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-92"}]: dispatch 2024-03-26T03:57:15.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:15 smithi175 bash[29284]: cluster 2024-03-26T03:57:14.378373+0000 mon.a (mon.0) 2761 : cluster [DBG] osdmap e482: 8 total, 8 up, 8 in 2024-03-26T03:57:15.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:15 smithi175 bash[29284]: audit 2024-03-26T03:57:14.469020+0000 mon.a (mon.0) 2762 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:57:15.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:15 smithi175 bash[29284]: audit 2024-03-26T03:57:14.470139+0000 mon.a (mon.0) 2763 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-92"}]: dispatch 2024-03-26T03:57:16.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:16 smithi190 bash[23392]: cluster 2024-03-26T03:57:14.881268+0000 mgr.y (mgr.24431) 502 : cluster [DBG] pgmap v768: 292 pgs: 4 unknown, 288 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 511 B/s wr, 2 op/s 2024-03-26T03:57:16.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:16 smithi190 bash[23392]: cluster 2024-03-26T03:57:15.405801+0000 mon.a (mon.0) 2764 : cluster [DBG] osdmap e483: 8 total, 8 up, 8 in 2024-03-26T03:57:16.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:16 smithi175 bash[21527]: cluster 2024-03-26T03:57:14.881268+0000 mgr.y (mgr.24431) 502 : cluster [DBG] pgmap v768: 292 pgs: 4 unknown, 288 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 511 B/s wr, 2 op/s 2024-03-26T03:57:16.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:16 smithi175 bash[21527]: cluster 2024-03-26T03:57:15.405801+0000 mon.a (mon.0) 2764 : cluster [DBG] osdmap e483: 8 total, 8 up, 8 in 2024-03-26T03:57:16.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:16 smithi175 bash[29284]: cluster 2024-03-26T03:57:14.881268+0000 mgr.y (mgr.24431) 502 : cluster [DBG] pgmap v768: 292 pgs: 4 unknown, 288 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 511 B/s wr, 2 op/s 2024-03-26T03:57:16.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:16 smithi175 bash[29284]: cluster 2024-03-26T03:57:15.405801+0000 mon.a (mon.0) 2764 : cluster [DBG] osdmap e483: 8 total, 8 up, 8 in 2024-03-26T03:57:17.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:17 smithi190 bash[23392]: cluster 2024-03-26T03:57:16.417484+0000 mon.a (mon.0) 2765 : cluster [DBG] osdmap e484: 8 total, 8 up, 8 in 2024-03-26T03:57:17.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:17 smithi190 bash[23392]: audit 2024-03-26T03:57:16.432875+0000 mon.a (mon.0) 2766 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:57:17.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:17 smithi190 bash[23392]: audit 2024-03-26T03:57:17.403976+0000 mon.a (mon.0) 2767 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:57:17.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:17 smithi190 bash[23392]: cluster 2024-03-26T03:57:17.412582+0000 mon.a (mon.0) 2768 : cluster [DBG] osdmap e485: 8 total, 8 up, 8 in 2024-03-26T03:57:17.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:17 smithi175 bash[29284]: cluster 2024-03-26T03:57:16.417484+0000 mon.a (mon.0) 2765 : cluster [DBG] osdmap e484: 8 total, 8 up, 8 in 2024-03-26T03:57:17.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:17 smithi175 bash[29284]: audit 2024-03-26T03:57:16.432875+0000 mon.a (mon.0) 2766 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:57:17.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:17 smithi175 bash[29284]: audit 2024-03-26T03:57:17.403976+0000 mon.a (mon.0) 2767 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:57:17.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:17 smithi175 bash[29284]: cluster 2024-03-26T03:57:17.412582+0000 mon.a (mon.0) 2768 : cluster [DBG] osdmap e485: 8 total, 8 up, 8 in 2024-03-26T03:57:17.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:17 smithi175 bash[21527]: cluster 2024-03-26T03:57:16.417484+0000 mon.a (mon.0) 2765 : cluster [DBG] osdmap e484: 8 total, 8 up, 8 in 2024-03-26T03:57:17.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:17 smithi175 bash[21527]: audit 2024-03-26T03:57:16.432875+0000 mon.a (mon.0) 2766 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:57:17.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:17 smithi175 bash[21527]: audit 2024-03-26T03:57:17.403976+0000 mon.a (mon.0) 2767 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:57:17.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:17 smithi175 bash[21527]: cluster 2024-03-26T03:57:17.412582+0000 mon.a (mon.0) 2768 : cluster [DBG] osdmap e485: 8 total, 8 up, 8 in 2024-03-26T03:57:18.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:18 smithi190 bash[23392]: cluster 2024-03-26T03:57:16.882258+0000 mgr.y (mgr.24431) 503 : cluster [DBG] pgmap v771: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-26T03:57:18.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:18 smithi190 bash[23392]: audit 2024-03-26T03:57:17.416587+0000 mon.a (mon.0) 2769 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:57:18.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:18 smithi190 bash[23392]: cluster 2024-03-26T03:57:17.441394+0000 mon.a (mon.0) 2770 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:57:18.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:18 smithi190 bash[23392]: cluster 2024-03-26T03:57:18.416012+0000 mon.a (mon.0) 2771 : cluster [DBG] osdmap e486: 8 total, 8 up, 8 in 2024-03-26T03:57:18.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:18 smithi175 bash[21527]: cluster 2024-03-26T03:57:16.882258+0000 mgr.y (mgr.24431) 503 : cluster [DBG] pgmap v771: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-26T03:57:18.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:18 smithi175 bash[21527]: audit 2024-03-26T03:57:17.416587+0000 mon.a (mon.0) 2769 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:57:18.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:18 smithi175 bash[21527]: cluster 2024-03-26T03:57:17.441394+0000 mon.a (mon.0) 2770 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:57:18.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:18 smithi175 bash[21527]: cluster 2024-03-26T03:57:18.416012+0000 mon.a (mon.0) 2771 : cluster [DBG] osdmap e486: 8 total, 8 up, 8 in 2024-03-26T03:57:18.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:18 smithi175 bash[29284]: cluster 2024-03-26T03:57:16.882258+0000 mgr.y (mgr.24431) 503 : cluster [DBG] pgmap v771: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-26T03:57:18.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:18 smithi175 bash[29284]: audit 2024-03-26T03:57:17.416587+0000 mon.a (mon.0) 2769 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:57:18.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:18 smithi175 bash[29284]: cluster 2024-03-26T03:57:17.441394+0000 mon.a (mon.0) 2770 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:57:18.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:18 smithi175 bash[29284]: cluster 2024-03-26T03:57:18.416012+0000 mon.a (mon.0) 2771 : cluster [DBG] osdmap e486: 8 total, 8 up, 8 in 2024-03-26T03:57:19.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:57:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:57:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:57:20.492 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:57:20 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:57:20.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:20 smithi190 bash[23392]: cluster 2024-03-26T03:57:18.883208+0000 mgr.y (mgr.24431) 504 : cluster [DBG] pgmap v774: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-26T03:57:20.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:20 smithi190 bash[23392]: cluster 2024-03-26T03:57:19.417148+0000 mon.a (mon.0) 2772 : cluster [DBG] osdmap e487: 8 total, 8 up, 8 in 2024-03-26T03:57:20.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:20 smithi190 bash[23392]: audit 2024-03-26T03:57:20.313867+0000 mon.c (mon.2) 536 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:57:20.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:20 smithi175 bash[21527]: cluster 2024-03-26T03:57:18.883208+0000 mgr.y (mgr.24431) 504 : cluster [DBG] pgmap v774: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-26T03:57:20.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:20 smithi175 bash[21527]: cluster 2024-03-26T03:57:19.417148+0000 mon.a (mon.0) 2772 : cluster [DBG] osdmap e487: 8 total, 8 up, 8 in 2024-03-26T03:57:20.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:20 smithi175 bash[21527]: audit 2024-03-26T03:57:20.313867+0000 mon.c (mon.2) 536 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:57:20.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:20 smithi175 bash[29284]: cluster 2024-03-26T03:57:18.883208+0000 mgr.y (mgr.24431) 504 : cluster [DBG] pgmap v774: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-26T03:57:20.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:20 smithi175 bash[29284]: cluster 2024-03-26T03:57:19.417148+0000 mon.a (mon.0) 2772 : cluster [DBG] osdmap e487: 8 total, 8 up, 8 in 2024-03-26T03:57:20.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:20 smithi175 bash[29284]: audit 2024-03-26T03:57:20.313867+0000 mon.c (mon.2) 536 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:57:21.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:21 smithi190 bash[23392]: audit 2024-03-26T03:57:20.185488+0000 mgr.y (mgr.24431) 505 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:57:21.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:21 smithi175 bash[21527]: audit 2024-03-26T03:57:20.185488+0000 mgr.y (mgr.24431) 505 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:57:21.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:21 smithi175 bash[29284]: audit 2024-03-26T03:57:20.185488+0000 mgr.y (mgr.24431) 505 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:57:22.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:22 smithi190 bash[23392]: cluster 2024-03-26T03:57:20.885851+0000 mgr.y (mgr.24431) 506 : cluster [DBG] pgmap v776: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-03-26T03:57:22.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:22 smithi175 bash[21527]: cluster 2024-03-26T03:57:20.885851+0000 mgr.y (mgr.24431) 506 : cluster [DBG] pgmap v776: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-03-26T03:57:22.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:22 smithi175 bash[29284]: cluster 2024-03-26T03:57:20.885851+0000 mgr.y (mgr.24431) 506 : cluster [DBG] pgmap v776: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-03-26T03:57:24.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:24 smithi190 bash[23392]: cluster 2024-03-26T03:57:22.886723+0000 mgr.y (mgr.24431) 507 : cluster [DBG] pgmap v777: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-03-26T03:57:24.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:24 smithi175 bash[21527]: cluster 2024-03-26T03:57:22.886723+0000 mgr.y (mgr.24431) 507 : cluster [DBG] pgmap v777: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-03-26T03:57:24.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:24 smithi175 bash[29284]: cluster 2024-03-26T03:57:22.886723+0000 mgr.y (mgr.24431) 507 : cluster [DBG] pgmap v777: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-03-26T03:57:26.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:26 smithi190 bash[23392]: cluster 2024-03-26T03:57:24.889114+0000 mgr.y (mgr.24431) 508 : cluster [DBG] pgmap v778: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-03-26T03:57:26.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:26 smithi175 bash[21527]: cluster 2024-03-26T03:57:24.889114+0000 mgr.y (mgr.24431) 508 : cluster [DBG] pgmap v778: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-03-26T03:57:26.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:26 smithi175 bash[29284]: cluster 2024-03-26T03:57:24.889114+0000 mgr.y (mgr.24431) 508 : cluster [DBG] pgmap v778: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-03-26T03:57:27.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:27 smithi190 bash[23392]: cluster 2024-03-26T03:57:26.574005+0000 mon.a (mon.0) 2773 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:57:27.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:27 smithi175 bash[21527]: cluster 2024-03-26T03:57:26.574005+0000 mon.a (mon.0) 2773 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:57:27.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:27 smithi175 bash[29284]: cluster 2024-03-26T03:57:26.574005+0000 mon.a (mon.0) 2773 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:57:28.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:28 smithi190 bash[23392]: cluster 2024-03-26T03:57:26.890312+0000 mgr.y (mgr.24431) 509 : cluster [DBG] pgmap v779: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 966 B/s wr, 4 op/s 2024-03-26T03:57:28.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:28 smithi175 bash[21527]: cluster 2024-03-26T03:57:26.890312+0000 mgr.y (mgr.24431) 509 : cluster [DBG] pgmap v779: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 966 B/s wr, 4 op/s 2024-03-26T03:57:28.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:28 smithi175 bash[29284]: cluster 2024-03-26T03:57:26.890312+0000 mgr.y (mgr.24431) 509 : cluster [DBG] pgmap v779: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 966 B/s wr, 4 op/s 2024-03-26T03:57:29.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:57:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:57:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:57:30.492 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:57:30 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:57:30.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:30 smithi190 bash[23392]: cluster 2024-03-26T03:57:28.891323+0000 mgr.y (mgr.24431) 510 : cluster [DBG] pgmap v780: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-03-26T03:57:30.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:30 smithi175 bash[21527]: cluster 2024-03-26T03:57:28.891323+0000 mgr.y (mgr.24431) 510 : cluster [DBG] pgmap v780: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-03-26T03:57:30.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:30 smithi175 bash[29284]: cluster 2024-03-26T03:57:28.891323+0000 mgr.y (mgr.24431) 510 : cluster [DBG] pgmap v780: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-03-26T03:57:31.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:31 smithi190 bash[23392]: audit 2024-03-26T03:57:30.196912+0000 mgr.y (mgr.24431) 511 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:57:31.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:31 smithi190 bash[23392]: cluster 2024-03-26T03:57:30.455152+0000 mon.a (mon.0) 2774 : cluster [DBG] osdmap e488: 8 total, 8 up, 8 in 2024-03-26T03:57:31.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:31 smithi175 bash[29284]: audit 2024-03-26T03:57:30.196912+0000 mgr.y (mgr.24431) 511 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:57:31.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:31 smithi175 bash[29284]: cluster 2024-03-26T03:57:30.455152+0000 mon.a (mon.0) 2774 : cluster [DBG] osdmap e488: 8 total, 8 up, 8 in 2024-03-26T03:57:31.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:31 smithi175 bash[21527]: audit 2024-03-26T03:57:30.196912+0000 mgr.y (mgr.24431) 511 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:57:31.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:31 smithi175 bash[21527]: cluster 2024-03-26T03:57:30.455152+0000 mon.a (mon.0) 2774 : cluster [DBG] osdmap e488: 8 total, 8 up, 8 in 2024-03-26T03:57:32.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:32 smithi190 bash[23392]: cluster 2024-03-26T03:57:30.892213+0000 mgr.y (mgr.24431) 512 : cluster [DBG] pgmap v782: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T03:57:32.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:32 smithi175 bash[29284]: cluster 2024-03-26T03:57:30.892213+0000 mgr.y (mgr.24431) 512 : cluster [DBG] pgmap v782: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T03:57:32.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:32 smithi175 bash[21527]: cluster 2024-03-26T03:57:30.892213+0000 mgr.y (mgr.24431) 512 : cluster [DBG] pgmap v782: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T03:57:34.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:34 smithi190 bash[23392]: cluster 2024-03-26T03:57:32.893093+0000 mgr.y (mgr.24431) 513 : cluster [DBG] pgmap v783: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T03:57:34.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:34 smithi175 bash[21527]: cluster 2024-03-26T03:57:32.893093+0000 mgr.y (mgr.24431) 513 : cluster [DBG] pgmap v783: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T03:57:34.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:34 smithi175 bash[29284]: cluster 2024-03-26T03:57:32.893093+0000 mgr.y (mgr.24431) 513 : cluster [DBG] pgmap v783: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T03:57:35.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:35 smithi190 bash[23392]: audit 2024-03-26T03:57:35.350625+0000 mon.c (mon.2) 537 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:57:35.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:35 smithi175 bash[21527]: audit 2024-03-26T03:57:35.350625+0000 mon.c (mon.2) 537 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:57:35.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:35 smithi175 bash[29284]: audit 2024-03-26T03:57:35.350625+0000 mon.c (mon.2) 537 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:57:36.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:36 smithi190 bash[23392]: cluster 2024-03-26T03:57:34.895670+0000 mgr.y (mgr.24431) 514 : cluster [DBG] pgmap v784: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T03:57:36.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:36 smithi175 bash[21527]: cluster 2024-03-26T03:57:34.895670+0000 mgr.y (mgr.24431) 514 : cluster [DBG] pgmap v784: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T03:57:36.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:36 smithi175 bash[29284]: cluster 2024-03-26T03:57:34.895670+0000 mgr.y (mgr.24431) 514 : cluster [DBG] pgmap v784: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T03:57:38.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:38 smithi190 bash[23392]: cluster 2024-03-26T03:57:36.896839+0000 mgr.y (mgr.24431) 515 : cluster [DBG] pgmap v785: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T03:57:38.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:38 smithi175 bash[21527]: cluster 2024-03-26T03:57:36.896839+0000 mgr.y (mgr.24431) 515 : cluster [DBG] pgmap v785: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T03:57:38.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:38 smithi175 bash[29284]: cluster 2024-03-26T03:57:36.896839+0000 mgr.y (mgr.24431) 515 : cluster [DBG] pgmap v785: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T03:57:39.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:57:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:57:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:57:40.489 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:57:40 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:57:40.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:40 smithi190 bash[23392]: cluster 2024-03-26T03:57:38.897723+0000 mgr.y (mgr.24431) 516 : cluster [DBG] pgmap v786: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T03:57:40.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:40 smithi175 bash[29284]: cluster 2024-03-26T03:57:38.897723+0000 mgr.y (mgr.24431) 516 : cluster [DBG] pgmap v786: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T03:57:40.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:40 smithi175 bash[21527]: cluster 2024-03-26T03:57:38.897723+0000 mgr.y (mgr.24431) 516 : cluster [DBG] pgmap v786: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T03:57:41.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:41 smithi190 bash[23392]: audit 2024-03-26T03:57:40.208321+0000 mgr.y (mgr.24431) 517 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:57:41.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:41 smithi190 bash[23392]: audit 2024-03-26T03:57:40.505012+0000 mon.a (mon.0) 2775 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:57:41.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:41 smithi190 bash[23392]: audit 2024-03-26T03:57:40.505838+0000 mon.a (mon.0) 2776 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-94"}]: dispatch 2024-03-26T03:57:41.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:41 smithi175 bash[21527]: audit 2024-03-26T03:57:40.208321+0000 mgr.y (mgr.24431) 517 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:57:41.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:41 smithi175 bash[21527]: audit 2024-03-26T03:57:40.505012+0000 mon.a (mon.0) 2775 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:57:41.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:41 smithi175 bash[21527]: audit 2024-03-26T03:57:40.505838+0000 mon.a (mon.0) 2776 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-94"}]: dispatch 2024-03-26T03:57:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:41 smithi175 bash[29284]: audit 2024-03-26T03:57:40.208321+0000 mgr.y (mgr.24431) 517 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:57:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:41 smithi175 bash[29284]: audit 2024-03-26T03:57:40.505012+0000 mon.a (mon.0) 2775 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:57:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:41 smithi175 bash[29284]: audit 2024-03-26T03:57:40.505838+0000 mon.a (mon.0) 2776 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-94"}]: dispatch 2024-03-26T03:57:42.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:42 smithi190 bash[23392]: cluster 2024-03-26T03:57:40.900399+0000 mgr.y (mgr.24431) 518 : cluster [DBG] pgmap v787: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T03:57:42.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:42 smithi190 bash[23392]: cluster 2024-03-26T03:57:41.516119+0000 mon.a (mon.0) 2777 : cluster [DBG] osdmap e489: 8 total, 8 up, 8 in 2024-03-26T03:57:42.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:42 smithi190 bash[23392]: cluster 2024-03-26T03:57:41.602297+0000 mon.a (mon.0) 2778 : cluster [DBG] osdmap e490: 8 total, 8 up, 8 in 2024-03-26T03:57:42.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:42 smithi190 bash[23392]: audit 2024-03-26T03:57:41.604860+0000 mon.a (mon.0) 2779 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:57:42.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:42 smithi175 bash[21527]: cluster 2024-03-26T03:57:40.900399+0000 mgr.y (mgr.24431) 518 : cluster [DBG] pgmap v787: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T03:57:42.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:42 smithi175 bash[21527]: cluster 2024-03-26T03:57:41.516119+0000 mon.a (mon.0) 2777 : cluster [DBG] osdmap e489: 8 total, 8 up, 8 in 2024-03-26T03:57:42.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:42 smithi175 bash[21527]: cluster 2024-03-26T03:57:41.602297+0000 mon.a (mon.0) 2778 : cluster [DBG] osdmap e490: 8 total, 8 up, 8 in 2024-03-26T03:57:42.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:42 smithi175 bash[21527]: audit 2024-03-26T03:57:41.604860+0000 mon.a (mon.0) 2779 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:57:42.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:42 smithi175 bash[29284]: cluster 2024-03-26T03:57:40.900399+0000 mgr.y (mgr.24431) 518 : cluster [DBG] pgmap v787: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T03:57:42.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:42 smithi175 bash[29284]: cluster 2024-03-26T03:57:41.516119+0000 mon.a (mon.0) 2777 : cluster [DBG] osdmap e489: 8 total, 8 up, 8 in 2024-03-26T03:57:42.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:42 smithi175 bash[29284]: cluster 2024-03-26T03:57:41.602297+0000 mon.a (mon.0) 2778 : cluster [DBG] osdmap e490: 8 total, 8 up, 8 in 2024-03-26T03:57:42.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:42 smithi175 bash[29284]: audit 2024-03-26T03:57:41.604860+0000 mon.a (mon.0) 2779 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:57:43.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:43 smithi190 bash[23392]: audit 2024-03-26T03:57:42.598574+0000 mon.a (mon.0) 2780 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:57:43.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:43 smithi190 bash[23392]: cluster 2024-03-26T03:57:42.606531+0000 mon.a (mon.0) 2781 : cluster [DBG] osdmap e491: 8 total, 8 up, 8 in 2024-03-26T03:57:43.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:43 smithi190 bash[23392]: audit 2024-03-26T03:57:42.609236+0000 mon.a (mon.0) 2782 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:57:43.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:43 smithi190 bash[23392]: cluster 2024-03-26T03:57:42.901261+0000 mgr.y (mgr.24431) 519 : cluster [DBG] pgmap v791: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:57:43.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:43 smithi175 bash[21527]: audit 2024-03-26T03:57:42.598574+0000 mon.a (mon.0) 2780 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:57:43.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:43 smithi175 bash[21527]: cluster 2024-03-26T03:57:42.606531+0000 mon.a (mon.0) 2781 : cluster [DBG] osdmap e491: 8 total, 8 up, 8 in 2024-03-26T03:57:43.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:43 smithi175 bash[21527]: audit 2024-03-26T03:57:42.609236+0000 mon.a (mon.0) 2782 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:57:43.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:43 smithi175 bash[21527]: cluster 2024-03-26T03:57:42.901261+0000 mgr.y (mgr.24431) 519 : cluster [DBG] pgmap v791: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:57:43.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:43 smithi175 bash[29284]: audit 2024-03-26T03:57:42.598574+0000 mon.a (mon.0) 2780 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:57:43.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:43 smithi175 bash[29284]: cluster 2024-03-26T03:57:42.606531+0000 mon.a (mon.0) 2781 : cluster [DBG] osdmap e491: 8 total, 8 up, 8 in 2024-03-26T03:57:43.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:43 smithi175 bash[29284]: audit 2024-03-26T03:57:42.609236+0000 mon.a (mon.0) 2782 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:57:43.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:43 smithi175 bash[29284]: cluster 2024-03-26T03:57:42.901261+0000 mgr.y (mgr.24431) 519 : cluster [DBG] pgmap v791: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:57:44.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:44 smithi190 bash[23392]: cluster 2024-03-26T03:57:43.618357+0000 mon.a (mon.0) 2783 : cluster [DBG] osdmap e492: 8 total, 8 up, 8 in 2024-03-26T03:57:44.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:44 smithi175 bash[29284]: cluster 2024-03-26T03:57:43.618357+0000 mon.a (mon.0) 2783 : cluster [DBG] osdmap e492: 8 total, 8 up, 8 in 2024-03-26T03:57:44.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:44 smithi175 bash[21527]: cluster 2024-03-26T03:57:43.618357+0000 mon.a (mon.0) 2783 : cluster [DBG] osdmap e492: 8 total, 8 up, 8 in 2024-03-26T03:57:45.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:45 smithi190 bash[23392]: cluster 2024-03-26T03:57:44.903541+0000 mgr.y (mgr.24431) 520 : cluster [DBG] pgmap v793: 292 pgs: 4 unknown, 288 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:57:45.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:45 smithi175 bash[21527]: cluster 2024-03-26T03:57:44.903541+0000 mgr.y (mgr.24431) 520 : cluster [DBG] pgmap v793: 292 pgs: 4 unknown, 288 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:57:45.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:45 smithi175 bash[29284]: cluster 2024-03-26T03:57:44.903541+0000 mgr.y (mgr.24431) 520 : cluster [DBG] pgmap v793: 292 pgs: 4 unknown, 288 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:57:47.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:47 smithi190 bash[23392]: cluster 2024-03-26T03:57:46.904723+0000 mgr.y (mgr.24431) 521 : cluster [DBG] pgmap v794: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 948 B/s rd, 1.1 KiB/s wr, 3 op/s 2024-03-26T03:57:47.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:47 smithi175 bash[29284]: cluster 2024-03-26T03:57:46.904723+0000 mgr.y (mgr.24431) 521 : cluster [DBG] pgmap v794: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 948 B/s rd, 1.1 KiB/s wr, 3 op/s 2024-03-26T03:57:47.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:47 smithi175 bash[21527]: cluster 2024-03-26T03:57:46.904723+0000 mgr.y (mgr.24431) 521 : cluster [DBG] pgmap v794: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 948 B/s rd, 1.1 KiB/s wr, 3 op/s 2024-03-26T03:57:49.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:57:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:57:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:57:50.218 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:49 smithi190 bash[23392]: cluster 2024-03-26T03:57:48.905614+0000 mgr.y (mgr.24431) 522 : cluster [DBG] pgmap v795: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 700 B/s rd, 841 B/s wr, 2 op/s 2024-03-26T03:57:50.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:49 smithi175 bash[29284]: cluster 2024-03-26T03:57:48.905614+0000 mgr.y (mgr.24431) 522 : cluster [DBG] pgmap v795: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 700 B/s rd, 841 B/s wr, 2 op/s 2024-03-26T03:57:50.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:49 smithi175 bash[21527]: cluster 2024-03-26T03:57:48.905614+0000 mgr.y (mgr.24431) 522 : cluster [DBG] pgmap v795: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 700 B/s rd, 841 B/s wr, 2 op/s 2024-03-26T03:57:50.492 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:57:50 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:57:51.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:50 smithi190 bash[23392]: audit 2024-03-26T03:57:50.219784+0000 mgr.y (mgr.24431) 523 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:57:51.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:50 smithi190 bash[23392]: audit 2024-03-26T03:57:50.386104+0000 mon.c (mon.2) 538 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:57:51.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:50 smithi175 bash[29284]: audit 2024-03-26T03:57:50.219784+0000 mgr.y (mgr.24431) 523 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:57:51.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:50 smithi175 bash[29284]: audit 2024-03-26T03:57:50.386104+0000 mon.c (mon.2) 538 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:57:51.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:50 smithi175 bash[21527]: audit 2024-03-26T03:57:50.219784+0000 mgr.y (mgr.24431) 523 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:57:51.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:50 smithi175 bash[21527]: audit 2024-03-26T03:57:50.386104+0000 mon.c (mon.2) 538 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:57:52.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:51 smithi190 bash[23392]: cluster 2024-03-26T03:57:50.908323+0000 mgr.y (mgr.24431) 524 : cluster [DBG] pgmap v796: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 739 B/s wr, 2 op/s 2024-03-26T03:57:52.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:51 smithi175 bash[29284]: cluster 2024-03-26T03:57:50.908323+0000 mgr.y (mgr.24431) 524 : cluster [DBG] pgmap v796: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 739 B/s wr, 2 op/s 2024-03-26T03:57:52.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:51 smithi175 bash[21527]: cluster 2024-03-26T03:57:50.908323+0000 mgr.y (mgr.24431) 524 : cluster [DBG] pgmap v796: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 739 B/s wr, 2 op/s 2024-03-26T03:57:53.979 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 26 03:57:53 smithi175 bash[43199]: debug 2024-03-26T03:57:53.673+0000 7f96ad223700 -1 snap_mapper.add_oid found existing snaps mapped on 105:c164d9e0:test-rados-api-smithi175-64594-97::foo:21, removing 2024-03-26T03:57:53.979 INFO:journalctl@ceph.osd.0.smithi175.stdout:Mar 26 03:57:53 smithi175 bash[32304]: debug 2024-03-26T03:57:53.673+0000 7ff3b2928700 -1 snap_mapper.add_oid found existing snaps mapped on 105:c164d9e0:test-rados-api-smithi175-64594-97::foo:21, removing 2024-03-26T03:57:53.979 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 03:57:53 smithi190 bash[43655]: debug 2024-03-26T03:57:53.682+0000 7fef9837c700 -1 snap_mapper.add_oid found existing snaps mapped on 105:c164d9e0:test-rados-api-smithi175-64594-97::foo:21, removing 2024-03-26T03:57:54.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:53 smithi190 bash[23392]: cluster 2024-03-26T03:57:52.909207+0000 mgr.y (mgr.24431) 525 : cluster [DBG] pgmap v797: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 613 B/s wr, 2 op/s 2024-03-26T03:57:54.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:53 smithi190 bash[23392]: audit 2024-03-26T03:57:53.714838+0000 mon.a (mon.0) 2784 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:57:54.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:53 smithi190 bash[23392]: audit 2024-03-26T03:57:53.715834+0000 mon.a (mon.0) 2785 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-96"}]: dispatch 2024-03-26T03:57:54.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:53 smithi175 bash[29284]: cluster 2024-03-26T03:57:52.909207+0000 mgr.y (mgr.24431) 525 : cluster [DBG] pgmap v797: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 613 B/s wr, 2 op/s 2024-03-26T03:57:54.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:53 smithi175 bash[29284]: audit 2024-03-26T03:57:53.714838+0000 mon.a (mon.0) 2784 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:57:54.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:53 smithi175 bash[29284]: audit 2024-03-26T03:57:53.715834+0000 mon.a (mon.0) 2785 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-96"}]: dispatch 2024-03-26T03:57:54.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:53 smithi175 bash[21527]: cluster 2024-03-26T03:57:52.909207+0000 mgr.y (mgr.24431) 525 : cluster [DBG] pgmap v797: 292 pgs: 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 613 B/s wr, 2 op/s 2024-03-26T03:57:54.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:53 smithi175 bash[21527]: audit 2024-03-26T03:57:53.714838+0000 mon.a (mon.0) 2784 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:57:54.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:53 smithi175 bash[21527]: audit 2024-03-26T03:57:53.715834+0000 mon.a (mon.0) 2785 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-96"}]: dispatch 2024-03-26T03:57:55.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:55 smithi190 bash[23392]: cluster 2024-03-26T03:57:53.995860+0000 mon.a (mon.0) 2786 : cluster [DBG] osdmap e493: 8 total, 8 up, 8 in 2024-03-26T03:57:55.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:55 smithi175 bash[29284]: cluster 2024-03-26T03:57:53.995860+0000 mon.a (mon.0) 2786 : cluster [DBG] osdmap e493: 8 total, 8 up, 8 in 2024-03-26T03:57:55.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:55 smithi175 bash[21527]: cluster 2024-03-26T03:57:53.995860+0000 mon.a (mon.0) 2786 : cluster [DBG] osdmap e493: 8 total, 8 up, 8 in 2024-03-26T03:57:56.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:56 smithi190 bash[23392]: cluster 2024-03-26T03:57:54.911391+0000 mgr.y (mgr.24431) 526 : cluster [DBG] pgmap v799: 260 pgs: 260 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 op/s 2024-03-26T03:57:56.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:56 smithi190 bash[23392]: cluster 2024-03-26T03:57:55.011373+0000 mon.a (mon.0) 2787 : cluster [DBG] osdmap e494: 8 total, 8 up, 8 in 2024-03-26T03:57:56.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:56 smithi190 bash[23392]: audit 2024-03-26T03:57:55.017358+0000 mon.a (mon.0) 2788 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:57:56.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:56 smithi175 bash[21527]: cluster 2024-03-26T03:57:54.911391+0000 mgr.y (mgr.24431) 526 : cluster [DBG] pgmap v799: 260 pgs: 260 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 op/s 2024-03-26T03:57:56.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:56 smithi175 bash[21527]: cluster 2024-03-26T03:57:55.011373+0000 mon.a (mon.0) 2787 : cluster [DBG] osdmap e494: 8 total, 8 up, 8 in 2024-03-26T03:57:56.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:56 smithi175 bash[21527]: audit 2024-03-26T03:57:55.017358+0000 mon.a (mon.0) 2788 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:57:56.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:56 smithi175 bash[29284]: cluster 2024-03-26T03:57:54.911391+0000 mgr.y (mgr.24431) 526 : cluster [DBG] pgmap v799: 260 pgs: 260 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 op/s 2024-03-26T03:57:56.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:56 smithi175 bash[29284]: cluster 2024-03-26T03:57:55.011373+0000 mon.a (mon.0) 2787 : cluster [DBG] osdmap e494: 8 total, 8 up, 8 in 2024-03-26T03:57:56.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:56 smithi175 bash[29284]: audit 2024-03-26T03:57:55.017358+0000 mon.a (mon.0) 2788 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:57:57.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:57 smithi190 bash[23392]: audit 2024-03-26T03:57:56.004204+0000 mon.a (mon.0) 2789 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:57:57.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:57 smithi190 bash[23392]: cluster 2024-03-26T03:57:56.012117+0000 mon.a (mon.0) 2790 : cluster [DBG] osdmap e495: 8 total, 8 up, 8 in 2024-03-26T03:57:57.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:57 smithi190 bash[23392]: audit 2024-03-26T03:57:56.069765+0000 mon.a (mon.0) 2791 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:57:57.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:57 smithi175 bash[29284]: audit 2024-03-26T03:57:56.004204+0000 mon.a (mon.0) 2789 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:57:57.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:57 smithi175 bash[29284]: cluster 2024-03-26T03:57:56.012117+0000 mon.a (mon.0) 2790 : cluster [DBG] osdmap e495: 8 total, 8 up, 8 in 2024-03-26T03:57:57.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:57 smithi175 bash[29284]: audit 2024-03-26T03:57:56.069765+0000 mon.a (mon.0) 2791 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:57:57.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:57 smithi175 bash[21527]: audit 2024-03-26T03:57:56.004204+0000 mon.a (mon.0) 2789 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:57:57.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:57 smithi175 bash[21527]: cluster 2024-03-26T03:57:56.012117+0000 mon.a (mon.0) 2790 : cluster [DBG] osdmap e495: 8 total, 8 up, 8 in 2024-03-26T03:57:57.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:57 smithi175 bash[21527]: audit 2024-03-26T03:57:56.069765+0000 mon.a (mon.0) 2791 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:57:58.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:58 smithi190 bash[23392]: cluster 2024-03-26T03:57:56.912267+0000 mgr.y (mgr.24431) 527 : cluster [DBG] pgmap v802: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T03:57:58.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:58 smithi190 bash[23392]: audit 2024-03-26T03:57:57.045670+0000 mon.a (mon.0) 2792 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-98", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:57:58.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:58 smithi190 bash[23392]: cluster 2024-03-26T03:57:57.052675+0000 mon.a (mon.0) 2793 : cluster [DBG] osdmap e496: 8 total, 8 up, 8 in 2024-03-26T03:57:58.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:58 smithi190 bash[23392]: audit 2024-03-26T03:57:57.053501+0000 mon.a (mon.0) 2794 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-98", "mode": "writeback"}]: dispatch 2024-03-26T03:57:58.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:58 smithi175 bash[21527]: cluster 2024-03-26T03:57:56.912267+0000 mgr.y (mgr.24431) 527 : cluster [DBG] pgmap v802: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T03:57:58.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:58 smithi175 bash[21527]: audit 2024-03-26T03:57:57.045670+0000 mon.a (mon.0) 2792 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-98", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:57:58.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:58 smithi175 bash[21527]: cluster 2024-03-26T03:57:57.052675+0000 mon.a (mon.0) 2793 : cluster [DBG] osdmap e496: 8 total, 8 up, 8 in 2024-03-26T03:57:58.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:58 smithi175 bash[21527]: audit 2024-03-26T03:57:57.053501+0000 mon.a (mon.0) 2794 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-98", "mode": "writeback"}]: dispatch 2024-03-26T03:57:58.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:58 smithi175 bash[29284]: cluster 2024-03-26T03:57:56.912267+0000 mgr.y (mgr.24431) 527 : cluster [DBG] pgmap v802: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T03:57:58.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:58 smithi175 bash[29284]: audit 2024-03-26T03:57:57.045670+0000 mon.a (mon.0) 2792 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-98", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:57:58.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:58 smithi175 bash[29284]: cluster 2024-03-26T03:57:57.052675+0000 mon.a (mon.0) 2793 : cluster [DBG] osdmap e496: 8 total, 8 up, 8 in 2024-03-26T03:57:58.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:58 smithi175 bash[29284]: audit 2024-03-26T03:57:57.053501+0000 mon.a (mon.0) 2794 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-98", "mode": "writeback"}]: dispatch 2024-03-26T03:57:59.071 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:57:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:57:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:57:59.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:59 smithi190 bash[23392]: cluster 2024-03-26T03:57:58.046119+0000 mon.a (mon.0) 2795 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:57:59.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:59 smithi190 bash[23392]: audit 2024-03-26T03:57:58.048411+0000 mon.a (mon.0) 2796 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-98", "mode": "writeback"}]': finished 2024-03-26T03:57:59.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:59 smithi190 bash[23392]: cluster 2024-03-26T03:57:58.055520+0000 mon.a (mon.0) 2797 : cluster [DBG] osdmap e497: 8 total, 8 up, 8 in 2024-03-26T03:57:59.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:59 smithi190 bash[23392]: audit 2024-03-26T03:57:58.057221+0000 mon.a (mon.0) 2798 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-98"}]: dispatch 2024-03-26T03:57:59.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:57:59 smithi190 bash[23392]: audit 2024-03-26T03:57:59.051159+0000 mon.a (mon.0) 2799 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-98"}]': finished 2024-03-26T03:57:59.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:59 smithi175 bash[21527]: cluster 2024-03-26T03:57:58.046119+0000 mon.a (mon.0) 2795 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:57:59.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:59 smithi175 bash[21527]: audit 2024-03-26T03:57:58.048411+0000 mon.a (mon.0) 2796 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-98", "mode": "writeback"}]': finished 2024-03-26T03:57:59.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:59 smithi175 bash[21527]: cluster 2024-03-26T03:57:58.055520+0000 mon.a (mon.0) 2797 : cluster [DBG] osdmap e497: 8 total, 8 up, 8 in 2024-03-26T03:57:59.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:59 smithi175 bash[21527]: audit 2024-03-26T03:57:58.057221+0000 mon.a (mon.0) 2798 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-98"}]: dispatch 2024-03-26T03:57:59.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:57:59 smithi175 bash[21527]: audit 2024-03-26T03:57:59.051159+0000 mon.a (mon.0) 2799 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-98"}]': finished 2024-03-26T03:57:59.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:59 smithi175 bash[29284]: cluster 2024-03-26T03:57:58.046119+0000 mon.a (mon.0) 2795 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:57:59.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:59 smithi175 bash[29284]: audit 2024-03-26T03:57:58.048411+0000 mon.a (mon.0) 2796 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-98", "mode": "writeback"}]': finished 2024-03-26T03:57:59.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:59 smithi175 bash[29284]: cluster 2024-03-26T03:57:58.055520+0000 mon.a (mon.0) 2797 : cluster [DBG] osdmap e497: 8 total, 8 up, 8 in 2024-03-26T03:57:59.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:59 smithi175 bash[29284]: audit 2024-03-26T03:57:58.057221+0000 mon.a (mon.0) 2798 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-98"}]: dispatch 2024-03-26T03:57:59.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:57:59 smithi175 bash[29284]: audit 2024-03-26T03:57:59.051159+0000 mon.a (mon.0) 2799 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-98"}]': finished 2024-03-26T03:58:00.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:00 smithi190 bash[23392]: cluster 2024-03-26T03:57:58.913167+0000 mgr.y (mgr.24431) 528 : cluster [DBG] pgmap v805: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:00.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:00 smithi190 bash[23392]: cluster 2024-03-26T03:57:59.059012+0000 mon.a (mon.0) 2800 : cluster [DBG] osdmap e498: 8 total, 8 up, 8 in 2024-03-26T03:58:00.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:00 smithi190 bash[23392]: audit 2024-03-26T03:57:59.062035+0000 mon.a (mon.0) 2801 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-26T03:58:00.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:00 smithi190 bash[23392]: cluster 2024-03-26T03:58:00.051675+0000 mon.a (mon.0) 2802 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:58:00.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:00 smithi190 bash[23392]: audit 2024-03-26T03:58:00.054227+0000 mon.a (mon.0) 2803 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-98","var": "hit_set_type","val": "bloom"}]': finished 2024-03-26T03:58:00.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:00 smithi190 bash[23392]: cluster 2024-03-26T03:58:00.062155+0000 mon.a (mon.0) 2804 : cluster [DBG] osdmap e499: 8 total, 8 up, 8 in 2024-03-26T03:58:00.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:00 smithi190 bash[23392]: audit 2024-03-26T03:58:00.064803+0000 mon.a (mon.0) 2805 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-98","var": "hit_set_count","val": "1"}]: dispatch 2024-03-26T03:58:00.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:58:00 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:58:00.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:00 smithi175 bash[21527]: cluster 2024-03-26T03:57:58.913167+0000 mgr.y (mgr.24431) 528 : cluster [DBG] pgmap v805: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:00.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:00 smithi175 bash[21527]: cluster 2024-03-26T03:57:59.059012+0000 mon.a (mon.0) 2800 : cluster [DBG] osdmap e498: 8 total, 8 up, 8 in 2024-03-26T03:58:00.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:00 smithi175 bash[21527]: audit 2024-03-26T03:57:59.062035+0000 mon.a (mon.0) 2801 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-26T03:58:00.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:00 smithi175 bash[21527]: cluster 2024-03-26T03:58:00.051675+0000 mon.a (mon.0) 2802 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:58:00.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:00 smithi175 bash[21527]: audit 2024-03-26T03:58:00.054227+0000 mon.a (mon.0) 2803 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-98","var": "hit_set_type","val": "bloom"}]': finished 2024-03-26T03:58:00.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:00 smithi175 bash[21527]: cluster 2024-03-26T03:58:00.062155+0000 mon.a (mon.0) 2804 : cluster [DBG] osdmap e499: 8 total, 8 up, 8 in 2024-03-26T03:58:00.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:00 smithi175 bash[21527]: audit 2024-03-26T03:58:00.064803+0000 mon.a (mon.0) 2805 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-98","var": "hit_set_count","val": "1"}]: dispatch 2024-03-26T03:58:00.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:00 smithi175 bash[29284]: cluster 2024-03-26T03:57:58.913167+0000 mgr.y (mgr.24431) 528 : cluster [DBG] pgmap v805: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:00.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:00 smithi175 bash[29284]: cluster 2024-03-26T03:57:59.059012+0000 mon.a (mon.0) 2800 : cluster [DBG] osdmap e498: 8 total, 8 up, 8 in 2024-03-26T03:58:00.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:00 smithi175 bash[29284]: audit 2024-03-26T03:57:59.062035+0000 mon.a (mon.0) 2801 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-26T03:58:00.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:00 smithi175 bash[29284]: cluster 2024-03-26T03:58:00.051675+0000 mon.a (mon.0) 2802 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:58:00.495 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:00 smithi175 bash[29284]: audit 2024-03-26T03:58:00.054227+0000 mon.a (mon.0) 2803 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-98","var": "hit_set_type","val": "bloom"}]': finished 2024-03-26T03:58:00.495 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:00 smithi175 bash[29284]: cluster 2024-03-26T03:58:00.062155+0000 mon.a (mon.0) 2804 : cluster [DBG] osdmap e499: 8 total, 8 up, 8 in 2024-03-26T03:58:00.495 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:00 smithi175 bash[29284]: audit 2024-03-26T03:58:00.064803+0000 mon.a (mon.0) 2805 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-98","var": "hit_set_count","val": "1"}]: dispatch 2024-03-26T03:58:01.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:01 smithi190 bash[23392]: audit 2024-03-26T03:58:00.230225+0000 mgr.y (mgr.24431) 529 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:58:01.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:01 smithi190 bash[23392]: audit 2024-03-26T03:58:01.056980+0000 mon.a (mon.0) 2806 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-98","var": "hit_set_count","val": "1"}]': finished 2024-03-26T03:58:01.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:01 smithi190 bash[23392]: cluster 2024-03-26T03:58:01.063850+0000 mon.a (mon.0) 2807 : cluster [DBG] osdmap e500: 8 total, 8 up, 8 in 2024-03-26T03:58:01.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:01 smithi190 bash[23392]: audit 2024-03-26T03:58:01.064795+0000 mon.a (mon.0) 2808 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-98","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T03:58:01.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:01 smithi175 bash[21527]: audit 2024-03-26T03:58:00.230225+0000 mgr.y (mgr.24431) 529 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:58:01.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:01 smithi175 bash[21527]: audit 2024-03-26T03:58:01.056980+0000 mon.a (mon.0) 2806 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-98","var": "hit_set_count","val": "1"}]': finished 2024-03-26T03:58:01.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:01 smithi175 bash[21527]: cluster 2024-03-26T03:58:01.063850+0000 mon.a (mon.0) 2807 : cluster [DBG] osdmap e500: 8 total, 8 up, 8 in 2024-03-26T03:58:01.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:01 smithi175 bash[21527]: audit 2024-03-26T03:58:01.064795+0000 mon.a (mon.0) 2808 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-98","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T03:58:01.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:01 smithi175 bash[29284]: audit 2024-03-26T03:58:00.230225+0000 mgr.y (mgr.24431) 529 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:58:01.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:01 smithi175 bash[29284]: audit 2024-03-26T03:58:01.056980+0000 mon.a (mon.0) 2806 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-98","var": "hit_set_count","val": "1"}]': finished 2024-03-26T03:58:01.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:01 smithi175 bash[29284]: cluster 2024-03-26T03:58:01.063850+0000 mon.a (mon.0) 2807 : cluster [DBG] osdmap e500: 8 total, 8 up, 8 in 2024-03-26T03:58:01.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:01 smithi175 bash[29284]: audit 2024-03-26T03:58:01.064795+0000 mon.a (mon.0) 2808 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-98","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T03:58:02.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:02 smithi190 bash[23392]: cluster 2024-03-26T03:58:00.914077+0000 mgr.y (mgr.24431) 530 : cluster [DBG] pgmap v808: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:58:02.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:02 smithi190 bash[23392]: audit 2024-03-26T03:58:02.061845+0000 mon.a (mon.0) 2809 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-98","var": "hit_set_period","val": "600"}]': finished 2024-03-26T03:58:02.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:02 smithi190 bash[23392]: cluster 2024-03-26T03:58:02.068756+0000 mon.a (mon.0) 2810 : cluster [DBG] osdmap e501: 8 total, 8 up, 8 in 2024-03-26T03:58:02.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:02 smithi190 bash[23392]: audit 2024-03-26T03:58:02.069584+0000 mon.a (mon.0) 2811 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-98","var": "target_max_objects","val": "250"}]: dispatch 2024-03-26T03:58:02.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:02 smithi175 bash[21527]: cluster 2024-03-26T03:58:00.914077+0000 mgr.y (mgr.24431) 530 : cluster [DBG] pgmap v808: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:58:02.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:02 smithi175 bash[21527]: audit 2024-03-26T03:58:02.061845+0000 mon.a (mon.0) 2809 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-98","var": "hit_set_period","val": "600"}]': finished 2024-03-26T03:58:02.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:02 smithi175 bash[21527]: cluster 2024-03-26T03:58:02.068756+0000 mon.a (mon.0) 2810 : cluster [DBG] osdmap e501: 8 total, 8 up, 8 in 2024-03-26T03:58:02.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:02 smithi175 bash[21527]: audit 2024-03-26T03:58:02.069584+0000 mon.a (mon.0) 2811 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-98","var": "target_max_objects","val": "250"}]: dispatch 2024-03-26T03:58:02.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:02 smithi175 bash[29284]: cluster 2024-03-26T03:58:00.914077+0000 mgr.y (mgr.24431) 530 : cluster [DBG] pgmap v808: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:58:02.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:02 smithi175 bash[29284]: audit 2024-03-26T03:58:02.061845+0000 mon.a (mon.0) 2809 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-98","var": "hit_set_period","val": "600"}]': finished 2024-03-26T03:58:02.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:02 smithi175 bash[29284]: cluster 2024-03-26T03:58:02.068756+0000 mon.a (mon.0) 2810 : cluster [DBG] osdmap e501: 8 total, 8 up, 8 in 2024-03-26T03:58:02.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:02 smithi175 bash[29284]: audit 2024-03-26T03:58:02.069584+0000 mon.a (mon.0) 2811 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-98","var": "target_max_objects","val": "250"}]: dispatch 2024-03-26T03:58:04.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:04 smithi190 bash[23392]: cluster 2024-03-26T03:58:02.914987+0000 mgr.y (mgr.24431) 531 : cluster [DBG] pgmap v811: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:58:04.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:04 smithi190 bash[23392]: audit 2024-03-26T03:58:03.064460+0000 mon.a (mon.0) 2812 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-98","var": "target_max_objects","val": "250"}]': finished 2024-03-26T03:58:04.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:04 smithi190 bash[23392]: cluster 2024-03-26T03:58:03.073660+0000 mon.a (mon.0) 2813 : cluster [DBG] osdmap e502: 8 total, 8 up, 8 in 2024-03-26T03:58:04.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:04 smithi190 bash[23392]: audit 2024-03-26T03:58:03.147601+0000 mon.a (mon.0) 2814 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:58:04.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:04 smithi175 bash[29284]: cluster 2024-03-26T03:58:02.914987+0000 mgr.y (mgr.24431) 531 : cluster [DBG] pgmap v811: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:58:04.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:04 smithi175 bash[29284]: audit 2024-03-26T03:58:03.064460+0000 mon.a (mon.0) 2812 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-98","var": "target_max_objects","val": "250"}]': finished 2024-03-26T03:58:04.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:04 smithi175 bash[29284]: cluster 2024-03-26T03:58:03.073660+0000 mon.a (mon.0) 2813 : cluster [DBG] osdmap e502: 8 total, 8 up, 8 in 2024-03-26T03:58:04.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:04 smithi175 bash[29284]: audit 2024-03-26T03:58:03.147601+0000 mon.a (mon.0) 2814 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:58:04.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:04 smithi175 bash[21527]: cluster 2024-03-26T03:58:02.914987+0000 mgr.y (mgr.24431) 531 : cluster [DBG] pgmap v811: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:58:04.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:04 smithi175 bash[21527]: audit 2024-03-26T03:58:03.064460+0000 mon.a (mon.0) 2812 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-98","var": "target_max_objects","val": "250"}]': finished 2024-03-26T03:58:04.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:04 smithi175 bash[21527]: cluster 2024-03-26T03:58:03.073660+0000 mon.a (mon.0) 2813 : cluster [DBG] osdmap e502: 8 total, 8 up, 8 in 2024-03-26T03:58:04.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:04 smithi175 bash[21527]: audit 2024-03-26T03:58:03.147601+0000 mon.a (mon.0) 2814 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:58:05.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:05 smithi190 bash[23392]: audit 2024-03-26T03:58:04.073890+0000 mon.c (mon.2) 539 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:58:05.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:05 smithi190 bash[23392]: audit 2024-03-26T03:58:04.100495+0000 mon.a (mon.0) 2815 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:58:05.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:05 smithi190 bash[23392]: cluster 2024-03-26T03:58:04.108093+0000 mon.a (mon.0) 2816 : cluster [DBG] osdmap e503: 8 total, 8 up, 8 in 2024-03-26T03:58:05.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:05 smithi190 bash[23392]: audit 2024-03-26T03:58:04.114404+0000 mon.a (mon.0) 2817 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-98"}]: dispatch 2024-03-26T03:58:05.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:05 smithi190 bash[23392]: audit 2024-03-26T03:58:04.858725+0000 mon.c (mon.2) 540 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:58:05.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:05 smithi190 bash[23392]: audit 2024-03-26T03:58:04.860301+0000 mon.c (mon.2) 541 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:58:05.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:05 smithi190 bash[23392]: audit 2024-03-26T03:58:04.867735+0000 mon.a (mon.0) 2818 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:58:05.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:05 smithi175 bash[21527]: audit 2024-03-26T03:58:04.073890+0000 mon.c (mon.2) 539 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:58:05.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:05 smithi175 bash[21527]: audit 2024-03-26T03:58:04.100495+0000 mon.a (mon.0) 2815 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:58:05.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:05 smithi175 bash[21527]: cluster 2024-03-26T03:58:04.108093+0000 mon.a (mon.0) 2816 : cluster [DBG] osdmap e503: 8 total, 8 up, 8 in 2024-03-26T03:58:05.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:05 smithi175 bash[21527]: audit 2024-03-26T03:58:04.114404+0000 mon.a (mon.0) 2817 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-98"}]: dispatch 2024-03-26T03:58:05.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:05 smithi175 bash[21527]: audit 2024-03-26T03:58:04.858725+0000 mon.c (mon.2) 540 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:58:05.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:05 smithi175 bash[21527]: audit 2024-03-26T03:58:04.860301+0000 mon.c (mon.2) 541 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:58:05.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:05 smithi175 bash[21527]: audit 2024-03-26T03:58:04.867735+0000 mon.a (mon.0) 2818 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:58:05.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:05 smithi175 bash[29284]: audit 2024-03-26T03:58:04.073890+0000 mon.c (mon.2) 539 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:58:05.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:05 smithi175 bash[29284]: audit 2024-03-26T03:58:04.100495+0000 mon.a (mon.0) 2815 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:58:05.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:05 smithi175 bash[29284]: cluster 2024-03-26T03:58:04.108093+0000 mon.a (mon.0) 2816 : cluster [DBG] osdmap e503: 8 total, 8 up, 8 in 2024-03-26T03:58:05.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:05 smithi175 bash[29284]: audit 2024-03-26T03:58:04.114404+0000 mon.a (mon.0) 2817 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-98"}]: dispatch 2024-03-26T03:58:05.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:05 smithi175 bash[29284]: audit 2024-03-26T03:58:04.858725+0000 mon.c (mon.2) 540 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:58:05.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:05 smithi175 bash[29284]: audit 2024-03-26T03:58:04.860301+0000 mon.c (mon.2) 541 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:58:05.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:05 smithi175 bash[29284]: audit 2024-03-26T03:58:04.867735+0000 mon.a (mon.0) 2818 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:58:06.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:06 smithi190 bash[23392]: cluster 2024-03-26T03:58:04.917395+0000 mgr.y (mgr.24431) 532 : cluster [DBG] pgmap v814: 292 pgs: 292 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 0 op/s 2024-03-26T03:58:06.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:06 smithi190 bash[23392]: audit 2024-03-26T03:58:05.103258+0000 mon.a (mon.0) 2819 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-98"}]': finished 2024-03-26T03:58:06.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:06 smithi190 bash[23392]: cluster 2024-03-26T03:58:05.110881+0000 mon.a (mon.0) 2820 : cluster [DBG] osdmap e504: 8 total, 8 up, 8 in 2024-03-26T03:58:06.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:06 smithi190 bash[23392]: audit 2024-03-26T03:58:05.421570+0000 mon.c (mon.2) 542 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:58:06.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:06 smithi175 bash[29284]: cluster 2024-03-26T03:58:04.917395+0000 mgr.y (mgr.24431) 532 : cluster [DBG] pgmap v814: 292 pgs: 292 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 0 op/s 2024-03-26T03:58:06.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:06 smithi175 bash[29284]: audit 2024-03-26T03:58:05.103258+0000 mon.a (mon.0) 2819 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-98"}]': finished 2024-03-26T03:58:06.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:06 smithi175 bash[29284]: cluster 2024-03-26T03:58:05.110881+0000 mon.a (mon.0) 2820 : cluster [DBG] osdmap e504: 8 total, 8 up, 8 in 2024-03-26T03:58:06.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:06 smithi175 bash[29284]: audit 2024-03-26T03:58:05.421570+0000 mon.c (mon.2) 542 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:58:06.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:06 smithi175 bash[21527]: cluster 2024-03-26T03:58:04.917395+0000 mgr.y (mgr.24431) 532 : cluster [DBG] pgmap v814: 292 pgs: 292 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 0 op/s 2024-03-26T03:58:06.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:06 smithi175 bash[21527]: audit 2024-03-26T03:58:05.103258+0000 mon.a (mon.0) 2819 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-98"}]': finished 2024-03-26T03:58:06.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:06 smithi175 bash[21527]: cluster 2024-03-26T03:58:05.110881+0000 mon.a (mon.0) 2820 : cluster [DBG] osdmap e504: 8 total, 8 up, 8 in 2024-03-26T03:58:06.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:06 smithi175 bash[21527]: audit 2024-03-26T03:58:05.421570+0000 mon.c (mon.2) 542 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:58:07.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:07 smithi190 bash[23392]: cluster 2024-03-26T03:58:06.113081+0000 mon.a (mon.0) 2821 : cluster [DBG] osdmap e505: 8 total, 8 up, 8 in 2024-03-26T03:58:07.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:07 smithi190 bash[23392]: cluster 2024-03-26T03:58:07.119962+0000 mon.a (mon.0) 2822 : cluster [DBG] osdmap e506: 8 total, 8 up, 8 in 2024-03-26T03:58:07.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:07 smithi190 bash[23392]: audit 2024-03-26T03:58:07.126473+0000 mon.a (mon.0) 2823 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:58:07.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:07 smithi175 bash[29284]: cluster 2024-03-26T03:58:06.113081+0000 mon.a (mon.0) 2821 : cluster [DBG] osdmap e505: 8 total, 8 up, 8 in 2024-03-26T03:58:07.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:07 smithi175 bash[29284]: cluster 2024-03-26T03:58:07.119962+0000 mon.a (mon.0) 2822 : cluster [DBG] osdmap e506: 8 total, 8 up, 8 in 2024-03-26T03:58:07.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:07 smithi175 bash[29284]: audit 2024-03-26T03:58:07.126473+0000 mon.a (mon.0) 2823 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:58:07.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:07 smithi175 bash[21527]: cluster 2024-03-26T03:58:06.113081+0000 mon.a (mon.0) 2821 : cluster [DBG] osdmap e505: 8 total, 8 up, 8 in 2024-03-26T03:58:07.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:07 smithi175 bash[21527]: cluster 2024-03-26T03:58:07.119962+0000 mon.a (mon.0) 2822 : cluster [DBG] osdmap e506: 8 total, 8 up, 8 in 2024-03-26T03:58:07.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:07 smithi175 bash[21527]: audit 2024-03-26T03:58:07.126473+0000 mon.a (mon.0) 2823 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:58:08.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:08 smithi190 bash[23392]: cluster 2024-03-26T03:58:06.918273+0000 mgr.y (mgr.24431) 533 : cluster [DBG] pgmap v817: 260 pgs: 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:08.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:08 smithi190 bash[23392]: audit 2024-03-26T03:58:08.112572+0000 mon.a (mon.0) 2824 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:58:08.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:08 smithi190 bash[23392]: cluster 2024-03-26T03:58:08.120357+0000 mon.a (mon.0) 2825 : cluster [DBG] osdmap e507: 8 total, 8 up, 8 in 2024-03-26T03:58:08.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:08 smithi190 bash[23392]: audit 2024-03-26T03:58:08.125850+0000 mon.a (mon.0) 2826 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:58:08.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:08 smithi175 bash[21527]: cluster 2024-03-26T03:58:06.918273+0000 mgr.y (mgr.24431) 533 : cluster [DBG] pgmap v817: 260 pgs: 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:08.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:08 smithi175 bash[21527]: audit 2024-03-26T03:58:08.112572+0000 mon.a (mon.0) 2824 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:58:08.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:08 smithi175 bash[21527]: cluster 2024-03-26T03:58:08.120357+0000 mon.a (mon.0) 2825 : cluster [DBG] osdmap e507: 8 total, 8 up, 8 in 2024-03-26T03:58:08.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:08 smithi175 bash[21527]: audit 2024-03-26T03:58:08.125850+0000 mon.a (mon.0) 2826 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:58:08.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:08 smithi175 bash[29284]: cluster 2024-03-26T03:58:06.918273+0000 mgr.y (mgr.24431) 533 : cluster [DBG] pgmap v817: 260 pgs: 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:08.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:08 smithi175 bash[29284]: audit 2024-03-26T03:58:08.112572+0000 mon.a (mon.0) 2824 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:58:08.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:08 smithi175 bash[29284]: cluster 2024-03-26T03:58:08.120357+0000 mon.a (mon.0) 2825 : cluster [DBG] osdmap e507: 8 total, 8 up, 8 in 2024-03-26T03:58:08.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:08 smithi175 bash[29284]: audit 2024-03-26T03:58:08.125850+0000 mon.a (mon.0) 2826 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:58:09.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:58:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:58:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:58:10.492 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:58:10 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:58:10.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:10 smithi190 bash[23392]: cluster 2024-03-26T03:58:08.919106+0000 mgr.y (mgr.24431) 534 : cluster [DBG] pgmap v820: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:10.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:10 smithi190 bash[23392]: audit 2024-03-26T03:58:09.115516+0000 mon.a (mon.0) 2827 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-100", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:58:10.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:10 smithi190 bash[23392]: cluster 2024-03-26T03:58:09.122860+0000 mon.a (mon.0) 2828 : cluster [DBG] osdmap e508: 8 total, 8 up, 8 in 2024-03-26T03:58:10.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:10 smithi190 bash[23392]: audit 2024-03-26T03:58:09.125239+0000 mon.a (mon.0) 2829 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-100"}]: dispatch 2024-03-26T03:58:10.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:10 smithi175 bash[29284]: cluster 2024-03-26T03:58:08.919106+0000 mgr.y (mgr.24431) 534 : cluster [DBG] pgmap v820: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:10.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:10 smithi175 bash[29284]: audit 2024-03-26T03:58:09.115516+0000 mon.a (mon.0) 2827 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-100", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:58:10.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:10 smithi175 bash[29284]: cluster 2024-03-26T03:58:09.122860+0000 mon.a (mon.0) 2828 : cluster [DBG] osdmap e508: 8 total, 8 up, 8 in 2024-03-26T03:58:10.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:10 smithi175 bash[29284]: audit 2024-03-26T03:58:09.125239+0000 mon.a (mon.0) 2829 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-100"}]: dispatch 2024-03-26T03:58:10.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:10 smithi175 bash[21527]: cluster 2024-03-26T03:58:08.919106+0000 mgr.y (mgr.24431) 534 : cluster [DBG] pgmap v820: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:10.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:10 smithi175 bash[21527]: audit 2024-03-26T03:58:09.115516+0000 mon.a (mon.0) 2827 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-100", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:58:10.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:10 smithi175 bash[21527]: cluster 2024-03-26T03:58:09.122860+0000 mon.a (mon.0) 2828 : cluster [DBG] osdmap e508: 8 total, 8 up, 8 in 2024-03-26T03:58:10.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:10 smithi175 bash[21527]: audit 2024-03-26T03:58:09.125239+0000 mon.a (mon.0) 2829 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-100"}]: dispatch 2024-03-26T03:58:11.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:11 smithi190 bash[23392]: audit 2024-03-26T03:58:10.118176+0000 mon.a (mon.0) 2830 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-100"}]': finished 2024-03-26T03:58:11.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:11 smithi190 bash[23392]: cluster 2024-03-26T03:58:10.126598+0000 mon.a (mon.0) 2831 : cluster [DBG] osdmap e509: 8 total, 8 up, 8 in 2024-03-26T03:58:11.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:11 smithi190 bash[23392]: audit 2024-03-26T03:58:10.130371+0000 mon.a (mon.0) 2832 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-100", "mode": "writeback"}]: dispatch 2024-03-26T03:58:11.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:11 smithi190 bash[23392]: audit 2024-03-26T03:58:10.241811+0000 mgr.y (mgr.24431) 535 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:58:11.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:11 smithi190 bash[23392]: cluster 2024-03-26T03:58:11.118601+0000 mon.a (mon.0) 2833 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:58:11.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:11 smithi190 bash[23392]: audit 2024-03-26T03:58:11.120974+0000 mon.a (mon.0) 2834 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-100", "mode": "writeback"}]': finished 2024-03-26T03:58:11.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:11 smithi190 bash[23392]: cluster 2024-03-26T03:58:11.129500+0000 mon.a (mon.0) 2835 : cluster [DBG] osdmap e510: 8 total, 8 up, 8 in 2024-03-26T03:58:11.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:11 smithi190 bash[23392]: audit 2024-03-26T03:58:11.133836+0000 mon.a (mon.0) 2836 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-100","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T03:58:11.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:11 smithi175 bash[29284]: audit 2024-03-26T03:58:10.118176+0000 mon.a (mon.0) 2830 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-100"}]': finished 2024-03-26T03:58:11.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:11 smithi175 bash[29284]: cluster 2024-03-26T03:58:10.126598+0000 mon.a (mon.0) 2831 : cluster [DBG] osdmap e509: 8 total, 8 up, 8 in 2024-03-26T03:58:11.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:11 smithi175 bash[29284]: audit 2024-03-26T03:58:10.130371+0000 mon.a (mon.0) 2832 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-100", "mode": "writeback"}]: dispatch 2024-03-26T03:58:11.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:11 smithi175 bash[29284]: audit 2024-03-26T03:58:10.241811+0000 mgr.y (mgr.24431) 535 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:58:11.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:11 smithi175 bash[29284]: cluster 2024-03-26T03:58:11.118601+0000 mon.a (mon.0) 2833 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:58:11.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:11 smithi175 bash[29284]: audit 2024-03-26T03:58:11.120974+0000 mon.a (mon.0) 2834 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-100", "mode": "writeback"}]': finished 2024-03-26T03:58:11.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:11 smithi175 bash[29284]: cluster 2024-03-26T03:58:11.129500+0000 mon.a (mon.0) 2835 : cluster [DBG] osdmap e510: 8 total, 8 up, 8 in 2024-03-26T03:58:11.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:11 smithi175 bash[29284]: audit 2024-03-26T03:58:11.133836+0000 mon.a (mon.0) 2836 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-100","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T03:58:11.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:11 smithi175 bash[21527]: audit 2024-03-26T03:58:10.118176+0000 mon.a (mon.0) 2830 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-6", "overlaypool": "test-rados-api-smithi175-64594-100"}]': finished 2024-03-26T03:58:11.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:11 smithi175 bash[21527]: cluster 2024-03-26T03:58:10.126598+0000 mon.a (mon.0) 2831 : cluster [DBG] osdmap e509: 8 total, 8 up, 8 in 2024-03-26T03:58:11.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:11 smithi175 bash[21527]: audit 2024-03-26T03:58:10.130371+0000 mon.a (mon.0) 2832 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-100", "mode": "writeback"}]: dispatch 2024-03-26T03:58:11.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:11 smithi175 bash[21527]: audit 2024-03-26T03:58:10.241811+0000 mgr.y (mgr.24431) 535 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:58:11.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:11 smithi175 bash[21527]: cluster 2024-03-26T03:58:11.118601+0000 mon.a (mon.0) 2833 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:58:11.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:11 smithi175 bash[21527]: audit 2024-03-26T03:58:11.120974+0000 mon.a (mon.0) 2834 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-100", "mode": "writeback"}]': finished 2024-03-26T03:58:11.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:11 smithi175 bash[21527]: cluster 2024-03-26T03:58:11.129500+0000 mon.a (mon.0) 2835 : cluster [DBG] osdmap e510: 8 total, 8 up, 8 in 2024-03-26T03:58:11.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:11 smithi175 bash[21527]: audit 2024-03-26T03:58:11.133836+0000 mon.a (mon.0) 2836 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-100","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T03:58:12.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:12 smithi190 bash[23392]: cluster 2024-03-26T03:58:10.920047+0000 mgr.y (mgr.24431) 536 : cluster [DBG] pgmap v823: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:12.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:12 smithi190 bash[23392]: audit 2024-03-26T03:58:12.123847+0000 mon.a (mon.0) 2837 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-100","var": "hit_set_count","val": "2"}]': finished 2024-03-26T03:58:12.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:12 smithi190 bash[23392]: cluster 2024-03-26T03:58:12.130686+0000 mon.a (mon.0) 2838 : cluster [DBG] osdmap e511: 8 total, 8 up, 8 in 2024-03-26T03:58:12.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:12 smithi190 bash[23392]: audit 2024-03-26T03:58:12.133361+0000 mon.a (mon.0) 2839 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-100","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T03:58:12.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:12 smithi175 bash[29284]: cluster 2024-03-26T03:58:10.920047+0000 mgr.y (mgr.24431) 536 : cluster [DBG] pgmap v823: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:12.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:12 smithi175 bash[29284]: audit 2024-03-26T03:58:12.123847+0000 mon.a (mon.0) 2837 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-100","var": "hit_set_count","val": "2"}]': finished 2024-03-26T03:58:12.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:12 smithi175 bash[29284]: cluster 2024-03-26T03:58:12.130686+0000 mon.a (mon.0) 2838 : cluster [DBG] osdmap e511: 8 total, 8 up, 8 in 2024-03-26T03:58:12.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:12 smithi175 bash[29284]: audit 2024-03-26T03:58:12.133361+0000 mon.a (mon.0) 2839 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-100","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T03:58:12.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:12 smithi175 bash[21527]: cluster 2024-03-26T03:58:10.920047+0000 mgr.y (mgr.24431) 536 : cluster [DBG] pgmap v823: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:12.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:12 smithi175 bash[21527]: audit 2024-03-26T03:58:12.123847+0000 mon.a (mon.0) 2837 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-100","var": "hit_set_count","val": "2"}]': finished 2024-03-26T03:58:12.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:12 smithi175 bash[21527]: cluster 2024-03-26T03:58:12.130686+0000 mon.a (mon.0) 2838 : cluster [DBG] osdmap e511: 8 total, 8 up, 8 in 2024-03-26T03:58:12.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:12 smithi175 bash[21527]: audit 2024-03-26T03:58:12.133361+0000 mon.a (mon.0) 2839 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-100","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T03:58:14.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:14 smithi190 bash[23392]: cluster 2024-03-26T03:58:12.920965+0000 mgr.y (mgr.24431) 537 : cluster [DBG] pgmap v826: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:14.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:14 smithi190 bash[23392]: audit 2024-03-26T03:58:13.126413+0000 mon.a (mon.0) 2840 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-100","var": "hit_set_period","val": "600"}]': finished 2024-03-26T03:58:14.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:14 smithi190 bash[23392]: cluster 2024-03-26T03:58:13.133720+0000 mon.a (mon.0) 2841 : cluster [DBG] osdmap e512: 8 total, 8 up, 8 in 2024-03-26T03:58:14.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:14 smithi190 bash[23392]: audit 2024-03-26T03:58:13.134746+0000 mon.a (mon.0) 2842 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-26T03:58:14.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:14 smithi175 bash[21527]: cluster 2024-03-26T03:58:12.920965+0000 mgr.y (mgr.24431) 537 : cluster [DBG] pgmap v826: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:14.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:14 smithi175 bash[21527]: audit 2024-03-26T03:58:13.126413+0000 mon.a (mon.0) 2840 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-100","var": "hit_set_period","val": "600"}]': finished 2024-03-26T03:58:14.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:14 smithi175 bash[21527]: cluster 2024-03-26T03:58:13.133720+0000 mon.a (mon.0) 2841 : cluster [DBG] osdmap e512: 8 total, 8 up, 8 in 2024-03-26T03:58:14.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:14 smithi175 bash[21527]: audit 2024-03-26T03:58:13.134746+0000 mon.a (mon.0) 2842 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-26T03:58:14.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:14 smithi175 bash[29284]: cluster 2024-03-26T03:58:12.920965+0000 mgr.y (mgr.24431) 537 : cluster [DBG] pgmap v826: 292 pgs: 292 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:14.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:14 smithi175 bash[29284]: audit 2024-03-26T03:58:13.126413+0000 mon.a (mon.0) 2840 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-100","var": "hit_set_period","val": "600"}]': finished 2024-03-26T03:58:14.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:14 smithi175 bash[29284]: cluster 2024-03-26T03:58:13.133720+0000 mon.a (mon.0) 2841 : cluster [DBG] osdmap e512: 8 total, 8 up, 8 in 2024-03-26T03:58:14.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:14 smithi175 bash[29284]: audit 2024-03-26T03:58:13.134746+0000 mon.a (mon.0) 2842 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-26T03:58:15.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:15 smithi190 bash[23392]: cluster 2024-03-26T03:58:14.126847+0000 mon.a (mon.0) 2843 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:58:15.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:15 smithi190 bash[23392]: audit 2024-03-26T03:58:14.129176+0000 mon.a (mon.0) 2844 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-26T03:58:15.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:15 smithi190 bash[23392]: cluster 2024-03-26T03:58:14.138523+0000 mon.a (mon.0) 2845 : cluster [DBG] osdmap e513: 8 total, 8 up, 8 in 2024-03-26T03:58:15.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:15 smithi190 bash[23392]: audit 2024-03-26T03:58:14.140203+0000 mon.a (mon.0) 2846 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-03-26T03:58:15.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:15 smithi190 bash[23392]: audit 2024-03-26T03:58:15.132237+0000 mon.a (mon.0) 2847 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-03-26T03:58:15.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:15 smithi175 bash[21527]: cluster 2024-03-26T03:58:14.126847+0000 mon.a (mon.0) 2843 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:58:15.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:15 smithi175 bash[21527]: audit 2024-03-26T03:58:14.129176+0000 mon.a (mon.0) 2844 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-26T03:58:15.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:15 smithi175 bash[21527]: cluster 2024-03-26T03:58:14.138523+0000 mon.a (mon.0) 2845 : cluster [DBG] osdmap e513: 8 total, 8 up, 8 in 2024-03-26T03:58:15.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:15 smithi175 bash[21527]: audit 2024-03-26T03:58:14.140203+0000 mon.a (mon.0) 2846 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-03-26T03:58:15.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:15 smithi175 bash[21527]: audit 2024-03-26T03:58:15.132237+0000 mon.a (mon.0) 2847 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-03-26T03:58:15.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:15 smithi175 bash[29284]: cluster 2024-03-26T03:58:14.126847+0000 mon.a (mon.0) 2843 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:58:15.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:15 smithi175 bash[29284]: audit 2024-03-26T03:58:14.129176+0000 mon.a (mon.0) 2844 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-26T03:58:15.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:15 smithi175 bash[29284]: cluster 2024-03-26T03:58:14.138523+0000 mon.a (mon.0) 2845 : cluster [DBG] osdmap e513: 8 total, 8 up, 8 in 2024-03-26T03:58:15.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:15 smithi175 bash[29284]: audit 2024-03-26T03:58:14.140203+0000 mon.a (mon.0) 2846 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-03-26T03:58:15.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:15 smithi175 bash[29284]: audit 2024-03-26T03:58:15.132237+0000 mon.a (mon.0) 2847 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-03-26T03:58:16.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:16 smithi190 bash[23392]: cluster 2024-03-26T03:58:14.923297+0000 mgr.y (mgr.24431) 538 : cluster [DBG] pgmap v829: 292 pgs: 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:58:16.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:16 smithi190 bash[23392]: cluster 2024-03-26T03:58:15.142463+0000 mon.a (mon.0) 2848 : cluster [DBG] osdmap e514: 8 total, 8 up, 8 in 2024-03-26T03:58:16.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:16 smithi190 bash[23392]: audit 2024-03-26T03:58:15.232415+0000 mon.a (mon.0) 2849 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:58:16.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:16 smithi175 bash[21527]: cluster 2024-03-26T03:58:14.923297+0000 mgr.y (mgr.24431) 538 : cluster [DBG] pgmap v829: 292 pgs: 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:58:16.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:16 smithi175 bash[21527]: cluster 2024-03-26T03:58:15.142463+0000 mon.a (mon.0) 2848 : cluster [DBG] osdmap e514: 8 total, 8 up, 8 in 2024-03-26T03:58:16.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:16 smithi175 bash[21527]: audit 2024-03-26T03:58:15.232415+0000 mon.a (mon.0) 2849 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:58:16.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:16 smithi175 bash[29284]: cluster 2024-03-26T03:58:14.923297+0000 mgr.y (mgr.24431) 538 : cluster [DBG] pgmap v829: 292 pgs: 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:58:16.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:16 smithi175 bash[29284]: cluster 2024-03-26T03:58:15.142463+0000 mon.a (mon.0) 2848 : cluster [DBG] osdmap e514: 8 total, 8 up, 8 in 2024-03-26T03:58:16.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:16 smithi175 bash[29284]: audit 2024-03-26T03:58:15.232415+0000 mon.a (mon.0) 2849 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:58:17.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:17 smithi190 bash[23392]: audit 2024-03-26T03:58:16.167461+0000 mon.a (mon.0) 2850 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:58:17.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:17 smithi190 bash[23392]: cluster 2024-03-26T03:58:16.177262+0000 mon.a (mon.0) 2851 : cluster [DBG] osdmap e515: 8 total, 8 up, 8 in 2024-03-26T03:58:17.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:17 smithi190 bash[23392]: audit 2024-03-26T03:58:16.181389+0000 mon.a (mon.0) 2852 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-100"}]: dispatch 2024-03-26T03:58:17.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:17 smithi175 bash[21527]: audit 2024-03-26T03:58:16.167461+0000 mon.a (mon.0) 2850 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:58:17.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:17 smithi175 bash[21527]: cluster 2024-03-26T03:58:16.177262+0000 mon.a (mon.0) 2851 : cluster [DBG] osdmap e515: 8 total, 8 up, 8 in 2024-03-26T03:58:17.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:17 smithi175 bash[21527]: audit 2024-03-26T03:58:16.181389+0000 mon.a (mon.0) 2852 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-100"}]: dispatch 2024-03-26T03:58:17.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:17 smithi175 bash[29284]: audit 2024-03-26T03:58:16.167461+0000 mon.a (mon.0) 2850 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]': finished 2024-03-26T03:58:17.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:17 smithi175 bash[29284]: cluster 2024-03-26T03:58:16.177262+0000 mon.a (mon.0) 2851 : cluster [DBG] osdmap e515: 8 total, 8 up, 8 in 2024-03-26T03:58:17.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:17 smithi175 bash[29284]: audit 2024-03-26T03:58:16.181389+0000 mon.a (mon.0) 2852 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-100"}]: dispatch 2024-03-26T03:58:18.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:18 smithi190 bash[23392]: cluster 2024-03-26T03:58:16.924193+0000 mgr.y (mgr.24431) 539 : cluster [DBG] pgmap v832: 292 pgs: 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:18.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:18 smithi190 bash[23392]: audit 2024-03-26T03:58:17.170296+0000 mon.a (mon.0) 2853 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-100"}]': finished 2024-03-26T03:58:18.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:18 smithi190 bash[23392]: cluster 2024-03-26T03:58:17.178612+0000 mon.a (mon.0) 2854 : cluster [DBG] osdmap e516: 8 total, 8 up, 8 in 2024-03-26T03:58:18.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:18 smithi175 bash[21527]: cluster 2024-03-26T03:58:16.924193+0000 mgr.y (mgr.24431) 539 : cluster [DBG] pgmap v832: 292 pgs: 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:18.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:18 smithi175 bash[21527]: audit 2024-03-26T03:58:17.170296+0000 mon.a (mon.0) 2853 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-100"}]': finished 2024-03-26T03:58:18.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:18 smithi175 bash[21527]: cluster 2024-03-26T03:58:17.178612+0000 mon.a (mon.0) 2854 : cluster [DBG] osdmap e516: 8 total, 8 up, 8 in 2024-03-26T03:58:18.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:18 smithi175 bash[29284]: cluster 2024-03-26T03:58:16.924193+0000 mgr.y (mgr.24431) 539 : cluster [DBG] pgmap v832: 292 pgs: 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:18.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:18 smithi175 bash[29284]: audit 2024-03-26T03:58:17.170296+0000 mon.a (mon.0) 2853 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-100"}]': finished 2024-03-26T03:58:18.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:18 smithi175 bash[29284]: cluster 2024-03-26T03:58:17.178612+0000 mon.a (mon.0) 2854 : cluster [DBG] osdmap e516: 8 total, 8 up, 8 in 2024-03-26T03:58:19.222 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:58:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:58:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:58:19.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:19 smithi190 bash[23392]: cluster 2024-03-26T03:58:18.210557+0000 mon.a (mon.0) 2855 : cluster [DBG] osdmap e517: 8 total, 8 up, 8 in 2024-03-26T03:58:19.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:19 smithi175 bash[21527]: cluster 2024-03-26T03:58:18.210557+0000 mon.a (mon.0) 2855 : cluster [DBG] osdmap e517: 8 total, 8 up, 8 in 2024-03-26T03:58:19.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:19 smithi175 bash[29284]: cluster 2024-03-26T03:58:18.210557+0000 mon.a (mon.0) 2855 : cluster [DBG] osdmap e517: 8 total, 8 up, 8 in 2024-03-26T03:58:20.492 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:58:20 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:58:20.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:20 smithi190 bash[23392]: cluster 2024-03-26T03:58:18.925015+0000 mgr.y (mgr.24431) 540 : cluster [DBG] pgmap v835: 260 pgs: 260 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:20.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:20 smithi190 bash[23392]: cluster 2024-03-26T03:58:19.233573+0000 mon.a (mon.0) 2856 : cluster [DBG] osdmap e518: 8 total, 8 up, 8 in 2024-03-26T03:58:20.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:20 smithi190 bash[23392]: audit 2024-03-26T03:58:19.240156+0000 mon.a (mon.0) 2857 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:58:20.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:20 smithi175 bash[21527]: cluster 2024-03-26T03:58:18.925015+0000 mgr.y (mgr.24431) 540 : cluster [DBG] pgmap v835: 260 pgs: 260 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:20.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:20 smithi175 bash[21527]: cluster 2024-03-26T03:58:19.233573+0000 mon.a (mon.0) 2856 : cluster [DBG] osdmap e518: 8 total, 8 up, 8 in 2024-03-26T03:58:20.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:20 smithi175 bash[21527]: audit 2024-03-26T03:58:19.240156+0000 mon.a (mon.0) 2857 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:58:20.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:20 smithi175 bash[29284]: cluster 2024-03-26T03:58:18.925015+0000 mgr.y (mgr.24431) 540 : cluster [DBG] pgmap v835: 260 pgs: 260 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:20.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:20 smithi175 bash[29284]: cluster 2024-03-26T03:58:19.233573+0000 mon.a (mon.0) 2856 : cluster [DBG] osdmap e518: 8 total, 8 up, 8 in 2024-03-26T03:58:20.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:20 smithi175 bash[29284]: audit 2024-03-26T03:58:19.240156+0000 mon.a (mon.0) 2857 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:58:21.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:21 smithi190 bash[23392]: audit 2024-03-26T03:58:20.227822+0000 mon.a (mon.0) 2858 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:58:21.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:21 smithi190 bash[23392]: cluster 2024-03-26T03:58:20.235766+0000 mon.a (mon.0) 2859 : cluster [DBG] osdmap e519: 8 total, 8 up, 8 in 2024-03-26T03:58:21.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:21 smithi190 bash[23392]: audit 2024-03-26T03:58:20.241719+0000 mon.a (mon.0) 2860 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:58:21.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:21 smithi190 bash[23392]: audit 2024-03-26T03:58:20.256414+0000 mgr.y (mgr.24431) 541 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:58:21.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:21 smithi190 bash[23392]: audit 2024-03-26T03:58:20.259638+0000 mon.a (mon.0) 2861 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-26T03:58:21.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:21 smithi190 bash[23392]: audit 2024-03-26T03:58:20.456215+0000 mon.c (mon.2) 543 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:58:21.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:21 smithi175 bash[21527]: audit 2024-03-26T03:58:20.227822+0000 mon.a (mon.0) 2858 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:58:21.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:21 smithi175 bash[21527]: cluster 2024-03-26T03:58:20.235766+0000 mon.a (mon.0) 2859 : cluster [DBG] osdmap e519: 8 total, 8 up, 8 in 2024-03-26T03:58:21.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:21 smithi175 bash[21527]: audit 2024-03-26T03:58:20.241719+0000 mon.a (mon.0) 2860 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:58:21.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:21 smithi175 bash[21527]: audit 2024-03-26T03:58:20.256414+0000 mgr.y (mgr.24431) 541 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:58:21.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:21 smithi175 bash[21527]: audit 2024-03-26T03:58:20.259638+0000 mon.a (mon.0) 2861 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-26T03:58:21.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:21 smithi175 bash[21527]: audit 2024-03-26T03:58:20.456215+0000 mon.c (mon.2) 543 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:58:21.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:21 smithi175 bash[29284]: audit 2024-03-26T03:58:20.227822+0000 mon.a (mon.0) 2858 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:58:21.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:21 smithi175 bash[29284]: cluster 2024-03-26T03:58:20.235766+0000 mon.a (mon.0) 2859 : cluster [DBG] osdmap e519: 8 total, 8 up, 8 in 2024-03-26T03:58:21.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:21 smithi175 bash[29284]: audit 2024-03-26T03:58:20.241719+0000 mon.a (mon.0) 2860 : audit [DBG] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-26T03:58:21.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:21 smithi175 bash[29284]: audit 2024-03-26T03:58:20.256414+0000 mgr.y (mgr.24431) 541 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:58:21.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:21 smithi175 bash[29284]: audit 2024-03-26T03:58:20.259638+0000 mon.a (mon.0) 2861 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-26T03:58:21.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:21 smithi175 bash[29284]: audit 2024-03-26T03:58:20.456215+0000 mon.c (mon.2) 543 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:58:22.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:22 smithi190 bash[23392]: cluster 2024-03-26T03:58:20.925952+0000 mgr.y (mgr.24431) 542 : cluster [DBG] pgmap v838: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T03:58:22.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:22 smithi190 bash[23392]: audit 2024-03-26T03:58:21.236802+0000 mon.a (mon.0) 2862 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-26T03:58:22.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:22 smithi190 bash[23392]: cluster 2024-03-26T03:58:21.246484+0000 mon.a (mon.0) 2863 : cluster [DBG] osdmap e520: 8 total, 8 up, 8 in 2024-03-26T03:58:22.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:22 smithi190 bash[23392]: audit 2024-03-26T03:58:21.249884+0000 mon.a (mon.0) 2864 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-26T03:58:22.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:22 smithi175 bash[21527]: cluster 2024-03-26T03:58:20.925952+0000 mgr.y (mgr.24431) 542 : cluster [DBG] pgmap v838: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T03:58:22.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:22 smithi175 bash[21527]: audit 2024-03-26T03:58:21.236802+0000 mon.a (mon.0) 2862 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-26T03:58:22.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:22 smithi175 bash[21527]: cluster 2024-03-26T03:58:21.246484+0000 mon.a (mon.0) 2863 : cluster [DBG] osdmap e520: 8 total, 8 up, 8 in 2024-03-26T03:58:22.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:22 smithi175 bash[21527]: audit 2024-03-26T03:58:21.249884+0000 mon.a (mon.0) 2864 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-26T03:58:22.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:22 smithi175 bash[29284]: cluster 2024-03-26T03:58:20.925952+0000 mgr.y (mgr.24431) 542 : cluster [DBG] pgmap v838: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T03:58:22.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:22 smithi175 bash[29284]: audit 2024-03-26T03:58:21.236802+0000 mon.a (mon.0) 2862 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-26T03:58:22.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:22 smithi175 bash[29284]: cluster 2024-03-26T03:58:21.246484+0000 mon.a (mon.0) 2863 : cluster [DBG] osdmap e520: 8 total, 8 up, 8 in 2024-03-26T03:58:22.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:22 smithi175 bash[29284]: audit 2024-03-26T03:58:21.249884+0000 mon.a (mon.0) 2864 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-26T03:58:23.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:23 smithi190 bash[23392]: audit 2024-03-26T03:58:22.239429+0000 mon.a (mon.0) 2865 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-26T03:58:23.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:23 smithi190 bash[23392]: cluster 2024-03-26T03:58:22.245694+0000 mon.a (mon.0) 2866 : cluster [DBG] osdmap e521: 8 total, 8 up, 8 in 2024-03-26T03:58:23.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:23 smithi190 bash[23392]: audit 2024-03-26T03:58:22.247067+0000 mon.a (mon.0) 2867 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-26T03:58:23.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:23 smithi190 bash[23392]: audit 2024-03-26T03:58:23.242435+0000 mon.a (mon.0) 2868 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-26T03:58:23.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:23 smithi190 bash[23392]: cluster 2024-03-26T03:58:23.252526+0000 mon.a (mon.0) 2869 : cluster [DBG] osdmap e522: 8 total, 8 up, 8 in 2024-03-26T03:58:23.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:23 smithi175 bash[29284]: audit 2024-03-26T03:58:22.239429+0000 mon.a (mon.0) 2865 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-26T03:58:23.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:23 smithi175 bash[29284]: cluster 2024-03-26T03:58:22.245694+0000 mon.a (mon.0) 2866 : cluster [DBG] osdmap e521: 8 total, 8 up, 8 in 2024-03-26T03:58:23.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:23 smithi175 bash[29284]: audit 2024-03-26T03:58:22.247067+0000 mon.a (mon.0) 2867 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-26T03:58:23.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:23 smithi175 bash[29284]: audit 2024-03-26T03:58:23.242435+0000 mon.a (mon.0) 2868 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-26T03:58:23.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:23 smithi175 bash[29284]: cluster 2024-03-26T03:58:23.252526+0000 mon.a (mon.0) 2869 : cluster [DBG] osdmap e522: 8 total, 8 up, 8 in 2024-03-26T03:58:23.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:23 smithi175 bash[21527]: audit 2024-03-26T03:58:22.239429+0000 mon.a (mon.0) 2865 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-26T03:58:23.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:23 smithi175 bash[21527]: cluster 2024-03-26T03:58:22.245694+0000 mon.a (mon.0) 2866 : cluster [DBG] osdmap e521: 8 total, 8 up, 8 in 2024-03-26T03:58:23.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:23 smithi175 bash[21527]: audit 2024-03-26T03:58:22.247067+0000 mon.a (mon.0) 2867 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-26T03:58:23.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:23 smithi175 bash[21527]: audit 2024-03-26T03:58:23.242435+0000 mon.a (mon.0) 2868 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-26T03:58:23.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:23 smithi175 bash[21527]: cluster 2024-03-26T03:58:23.252526+0000 mon.a (mon.0) 2869 : cluster [DBG] osdmap e522: 8 total, 8 up, 8 in 2024-03-26T03:58:24.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:24 smithi190 bash[23392]: cluster 2024-03-26T03:58:22.926845+0000 mgr.y (mgr.24431) 543 : cluster [DBG] pgmap v841: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T03:58:24.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:24 smithi190 bash[23392]: audit 2024-03-26T03:58:23.333807+0000 mon.a (mon.0) 2870 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:58:24.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:24 smithi190 bash[23392]: audit 2024-03-26T03:58:23.334698+0000 mon.a (mon.0) 2871 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-102"}]: dispatch 2024-03-26T03:58:24.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:24 smithi175 bash[21527]: cluster 2024-03-26T03:58:22.926845+0000 mgr.y (mgr.24431) 543 : cluster [DBG] pgmap v841: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T03:58:24.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:24 smithi175 bash[21527]: audit 2024-03-26T03:58:23.333807+0000 mon.a (mon.0) 2870 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:58:24.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:24 smithi175 bash[21527]: audit 2024-03-26T03:58:23.334698+0000 mon.a (mon.0) 2871 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-102"}]: dispatch 2024-03-26T03:58:24.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:24 smithi175 bash[29284]: cluster 2024-03-26T03:58:22.926845+0000 mgr.y (mgr.24431) 543 : cluster [DBG] pgmap v841: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T03:58:24.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:24 smithi175 bash[29284]: audit 2024-03-26T03:58:23.333807+0000 mon.a (mon.0) 2870 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-6"}]: dispatch 2024-03-26T03:58:24.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:24 smithi175 bash[29284]: audit 2024-03-26T03:58:23.334698+0000 mon.a (mon.0) 2871 : audit [INF] from='client.? 172.21.15.175:0/2317141914' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-6", "tierpool": "test-rados-api-smithi175-64594-102"}]: dispatch 2024-03-26T03:58:25.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:25 smithi190 bash[23392]: cluster 2024-03-26T03:58:24.278415+0000 mon.a (mon.0) 2872 : cluster [DBG] osdmap e523: 8 total, 8 up, 8 in 2024-03-26T03:58:25.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:25 smithi190 bash[23392]: cluster 2024-03-26T03:58:25.281633+0000 mon.a (mon.0) 2873 : cluster [DBG] osdmap e524: 8 total, 8 up, 8 in 2024-03-26T03:58:25.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:25 smithi175 bash[21527]: cluster 2024-03-26T03:58:24.278415+0000 mon.a (mon.0) 2872 : cluster [DBG] osdmap e523: 8 total, 8 up, 8 in 2024-03-26T03:58:25.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:25 smithi175 bash[21527]: cluster 2024-03-26T03:58:25.281633+0000 mon.a (mon.0) 2873 : cluster [DBG] osdmap e524: 8 total, 8 up, 8 in 2024-03-26T03:58:25.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:25 smithi175 bash[29284]: cluster 2024-03-26T03:58:24.278415+0000 mon.a (mon.0) 2872 : cluster [DBG] osdmap e523: 8 total, 8 up, 8 in 2024-03-26T03:58:25.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:25 smithi175 bash[29284]: cluster 2024-03-26T03:58:25.281633+0000 mon.a (mon.0) 2873 : cluster [DBG] osdmap e524: 8 total, 8 up, 8 in 2024-03-26T03:58:26.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:26 smithi190 bash[23392]: cluster 2024-03-26T03:58:24.928990+0000 mgr.y (mgr.24431) 544 : cluster [DBG] pgmap v844: 260 pgs: 260 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:58:26.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:26 smithi190 bash[23392]: audit 2024-03-26T03:58:25.341207+0000 mon.a (mon.0) 2874 : audit [INF] from='client.? 172.21.15.175:0/2594920259' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi175-64594-104"}]: dispatch 2024-03-26T03:58:26.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:26 smithi190 bash[23392]: audit 2024-03-26T03:58:25.342819+0000 mon.a (mon.0) 2875 : audit [INF] from='client.? 172.21.15.175:0/2594920259' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi175-64594-104"}]: dispatch 2024-03-26T03:58:26.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:26 smithi190 bash[23392]: audit 2024-03-26T03:58:25.343251+0000 mon.a (mon.0) 2876 : audit [INF] from='client.? 172.21.15.175:0/2594920259' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi175-64594-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:58:26.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:26 smithi175 bash[29284]: cluster 2024-03-26T03:58:24.928990+0000 mgr.y (mgr.24431) 544 : cluster [DBG] pgmap v844: 260 pgs: 260 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:58:26.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:26 smithi175 bash[29284]: audit 2024-03-26T03:58:25.341207+0000 mon.a (mon.0) 2874 : audit [INF] from='client.? 172.21.15.175:0/2594920259' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi175-64594-104"}]: dispatch 2024-03-26T03:58:26.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:26 smithi175 bash[29284]: audit 2024-03-26T03:58:25.342819+0000 mon.a (mon.0) 2875 : audit [INF] from='client.? 172.21.15.175:0/2594920259' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi175-64594-104"}]: dispatch 2024-03-26T03:58:26.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:26 smithi175 bash[29284]: audit 2024-03-26T03:58:25.343251+0000 mon.a (mon.0) 2876 : audit [INF] from='client.? 172.21.15.175:0/2594920259' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi175-64594-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:58:26.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:26 smithi175 bash[21527]: cluster 2024-03-26T03:58:24.928990+0000 mgr.y (mgr.24431) 544 : cluster [DBG] pgmap v844: 260 pgs: 260 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:58:26.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:26 smithi175 bash[21527]: audit 2024-03-26T03:58:25.341207+0000 mon.a (mon.0) 2874 : audit [INF] from='client.? 172.21.15.175:0/2594920259' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi175-64594-104"}]: dispatch 2024-03-26T03:58:26.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:26 smithi175 bash[21527]: audit 2024-03-26T03:58:25.342819+0000 mon.a (mon.0) 2875 : audit [INF] from='client.? 172.21.15.175:0/2594920259' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi175-64594-104"}]: dispatch 2024-03-26T03:58:26.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:26 smithi175 bash[21527]: audit 2024-03-26T03:58:25.343251+0000 mon.a (mon.0) 2876 : audit [INF] from='client.? 172.21.15.175:0/2594920259' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi175-64594-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:58:27.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:27 smithi190 bash[23392]: audit 2024-03-26T03:58:26.307976+0000 mon.a (mon.0) 2877 : audit [INF] from='client.? 172.21.15.175:0/2594920259' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi175-64594-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:58:27.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:27 smithi190 bash[23392]: cluster 2024-03-26T03:58:26.314139+0000 mon.a (mon.0) 2878 : cluster [DBG] osdmap e525: 8 total, 8 up, 8 in 2024-03-26T03:58:27.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:27 smithi190 bash[23392]: audit 2024-03-26T03:58:26.317522+0000 mon.a (mon.0) 2879 : audit [INF] from='client.? 172.21.15.175:0/2594920259' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi175-64594-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi175-64594-104"}]: dispatch 2024-03-26T03:58:27.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:27 smithi175 bash[21527]: audit 2024-03-26T03:58:26.307976+0000 mon.a (mon.0) 2877 : audit [INF] from='client.? 172.21.15.175:0/2594920259' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi175-64594-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:58:27.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:27 smithi175 bash[21527]: cluster 2024-03-26T03:58:26.314139+0000 mon.a (mon.0) 2878 : cluster [DBG] osdmap e525: 8 total, 8 up, 8 in 2024-03-26T03:58:27.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:27 smithi175 bash[21527]: audit 2024-03-26T03:58:26.317522+0000 mon.a (mon.0) 2879 : audit [INF] from='client.? 172.21.15.175:0/2594920259' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi175-64594-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi175-64594-104"}]: dispatch 2024-03-26T03:58:27.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:27 smithi175 bash[29284]: audit 2024-03-26T03:58:26.307976+0000 mon.a (mon.0) 2877 : audit [INF] from='client.? 172.21.15.175:0/2594920259' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi175-64594-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:58:27.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:27 smithi175 bash[29284]: cluster 2024-03-26T03:58:26.314139+0000 mon.a (mon.0) 2878 : cluster [DBG] osdmap e525: 8 total, 8 up, 8 in 2024-03-26T03:58:27.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:27 smithi175 bash[29284]: audit 2024-03-26T03:58:26.317522+0000 mon.a (mon.0) 2879 : audit [INF] from='client.? 172.21.15.175:0/2594920259' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi175-64594-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi175-64594-104"}]: dispatch 2024-03-26T03:58:28.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:28 smithi190 bash[23392]: cluster 2024-03-26T03:58:26.929815+0000 mgr.y (mgr.24431) 545 : cluster [DBG] pgmap v847: 228 pgs: 228 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:58:28.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:28 smithi190 bash[23392]: cluster 2024-03-26T03:58:27.337802+0000 mon.a (mon.0) 2880 : cluster [DBG] osdmap e526: 8 total, 8 up, 8 in 2024-03-26T03:58:28.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:28 smithi190 bash[23392]: audit 2024-03-26T03:58:28.315995+0000 mon.a (mon.0) 2881 : audit [INF] from='client.? 172.21.15.175:0/2594920259' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi175-64594-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi175-64594-104"}]': finished 2024-03-26T03:58:28.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:28 smithi190 bash[23392]: cluster 2024-03-26T03:58:28.324643+0000 mon.a (mon.0) 2882 : cluster [DBG] osdmap e527: 8 total, 8 up, 8 in 2024-03-26T03:58:28.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:28 smithi175 bash[29284]: cluster 2024-03-26T03:58:26.929815+0000 mgr.y (mgr.24431) 545 : cluster [DBG] pgmap v847: 228 pgs: 228 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:58:28.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:28 smithi175 bash[29284]: cluster 2024-03-26T03:58:27.337802+0000 mon.a (mon.0) 2880 : cluster [DBG] osdmap e526: 8 total, 8 up, 8 in 2024-03-26T03:58:28.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:28 smithi175 bash[29284]: audit 2024-03-26T03:58:28.315995+0000 mon.a (mon.0) 2881 : audit [INF] from='client.? 172.21.15.175:0/2594920259' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi175-64594-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi175-64594-104"}]': finished 2024-03-26T03:58:28.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:28 smithi175 bash[29284]: cluster 2024-03-26T03:58:28.324643+0000 mon.a (mon.0) 2882 : cluster [DBG] osdmap e527: 8 total, 8 up, 8 in 2024-03-26T03:58:28.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:28 smithi175 bash[21527]: cluster 2024-03-26T03:58:26.929815+0000 mgr.y (mgr.24431) 545 : cluster [DBG] pgmap v847: 228 pgs: 228 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:58:28.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:28 smithi175 bash[21527]: cluster 2024-03-26T03:58:27.337802+0000 mon.a (mon.0) 2880 : cluster [DBG] osdmap e526: 8 total, 8 up, 8 in 2024-03-26T03:58:28.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:28 smithi175 bash[21527]: audit 2024-03-26T03:58:28.315995+0000 mon.a (mon.0) 2881 : audit [INF] from='client.? 172.21.15.175:0/2594920259' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi175-64594-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi175-64594-104"}]': finished 2024-03-26T03:58:28.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:28 smithi175 bash[21527]: cluster 2024-03-26T03:58:28.324643+0000 mon.a (mon.0) 2882 : cluster [DBG] osdmap e527: 8 total, 8 up, 8 in 2024-03-26T03:58:29.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:58:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:58:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:58:29.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:29 smithi190 bash[23392]: cluster 2024-03-26T03:58:29.359673+0000 mon.a (mon.0) 2883 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:58:29.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:29 smithi190 bash[23392]: cluster 2024-03-26T03:58:29.379918+0000 mon.a (mon.0) 2884 : cluster [DBG] osdmap e528: 8 total, 8 up, 8 in 2024-03-26T03:58:29.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:29 smithi175 bash[29284]: cluster 2024-03-26T03:58:29.359673+0000 mon.a (mon.0) 2883 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:58:29.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:29 smithi175 bash[29284]: cluster 2024-03-26T03:58:29.379918+0000 mon.a (mon.0) 2884 : cluster [DBG] osdmap e528: 8 total, 8 up, 8 in 2024-03-26T03:58:29.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:29 smithi175 bash[21527]: cluster 2024-03-26T03:58:29.359673+0000 mon.a (mon.0) 2883 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:58:29.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:29 smithi175 bash[21527]: cluster 2024-03-26T03:58:29.379918+0000 mon.a (mon.0) 2884 : cluster [DBG] osdmap e528: 8 total, 8 up, 8 in 2024-03-26T03:58:30.742 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:58:30 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:58:30.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:30 smithi190 bash[23392]: cluster 2024-03-26T03:58:28.930696+0000 mgr.y (mgr.24431) 546 : cluster [DBG] pgmap v850: 236 pgs: 8 unknown, 228 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:58:30.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:30 smithi190 bash[23392]: cluster 2024-03-26T03:58:30.383489+0000 mon.a (mon.0) 2885 : cluster [DBG] osdmap e529: 8 total, 8 up, 8 in 2024-03-26T03:58:30.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:30 smithi190 bash[23392]: audit 2024-03-26T03:58:30.387705+0000 mon.c (mon.2) 544 : audit [INF] from='client.? 172.21.15.175:0/1901179583' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:58:30.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:30 smithi190 bash[23392]: audit 2024-03-26T03:58:30.389416+0000 mon.a (mon.0) 2886 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:58:30.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:30 smithi175 bash[29284]: cluster 2024-03-26T03:58:28.930696+0000 mgr.y (mgr.24431) 546 : cluster [DBG] pgmap v850: 236 pgs: 8 unknown, 228 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:58:30.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:30 smithi175 bash[29284]: cluster 2024-03-26T03:58:30.383489+0000 mon.a (mon.0) 2885 : cluster [DBG] osdmap e529: 8 total, 8 up, 8 in 2024-03-26T03:58:30.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:30 smithi175 bash[29284]: audit 2024-03-26T03:58:30.387705+0000 mon.c (mon.2) 544 : audit [INF] from='client.? 172.21.15.175:0/1901179583' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:58:30.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:30 smithi175 bash[29284]: audit 2024-03-26T03:58:30.389416+0000 mon.a (mon.0) 2886 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:58:30.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:30 smithi175 bash[21527]: cluster 2024-03-26T03:58:28.930696+0000 mgr.y (mgr.24431) 546 : cluster [DBG] pgmap v850: 236 pgs: 8 unknown, 228 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:58:30.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:30 smithi175 bash[21527]: cluster 2024-03-26T03:58:30.383489+0000 mon.a (mon.0) 2885 : cluster [DBG] osdmap e529: 8 total, 8 up, 8 in 2024-03-26T03:58:30.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:30 smithi175 bash[21527]: audit 2024-03-26T03:58:30.387705+0000 mon.c (mon.2) 544 : audit [INF] from='client.? 172.21.15.175:0/1901179583' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:58:30.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:30 smithi175 bash[21527]: audit 2024-03-26T03:58:30.389416+0000 mon.a (mon.0) 2886 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:58:31.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:31 smithi190 bash[23392]: audit 2024-03-26T03:58:30.267997+0000 mgr.y (mgr.24431) 547 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:58:31.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:31 smithi190 bash[23392]: audit 2024-03-26T03:58:31.376782+0000 mon.a (mon.0) 2887 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:58:31.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:31 smithi190 bash[23392]: cluster 2024-03-26T03:58:31.385730+0000 mon.a (mon.0) 2888 : cluster [DBG] osdmap e530: 8 total, 8 up, 8 in 2024-03-26T03:58:31.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:31 smithi175 bash[21527]: audit 2024-03-26T03:58:30.267997+0000 mgr.y (mgr.24431) 547 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:58:31.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:31 smithi175 bash[21527]: audit 2024-03-26T03:58:31.376782+0000 mon.a (mon.0) 2887 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:58:31.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:31 smithi175 bash[21527]: cluster 2024-03-26T03:58:31.385730+0000 mon.a (mon.0) 2888 : cluster [DBG] osdmap e530: 8 total, 8 up, 8 in 2024-03-26T03:58:31.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:31 smithi175 bash[29284]: audit 2024-03-26T03:58:30.267997+0000 mgr.y (mgr.24431) 547 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:58:31.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:31 smithi175 bash[29284]: audit 2024-03-26T03:58:31.376782+0000 mon.a (mon.0) 2887 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:58:31.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:31 smithi175 bash[29284]: cluster 2024-03-26T03:58:31.385730+0000 mon.a (mon.0) 2888 : cluster [DBG] osdmap e530: 8 total, 8 up, 8 in 2024-03-26T03:58:32.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:32 smithi190 bash[23392]: cluster 2024-03-26T03:58:30.931669+0000 mgr.y (mgr.24431) 548 : cluster [DBG] pgmap v853: 268 pgs: 32 unknown, 1 creating+activating, 235 active+clean; 582 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T03:58:32.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:32 smithi190 bash[23392]: cluster 2024-03-26T03:58:32.387885+0000 mon.a (mon.0) 2889 : cluster [DBG] osdmap e531: 8 total, 8 up, 8 in 2024-03-26T03:58:32.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:32 smithi190 bash[23392]: audit 2024-03-26T03:58:32.391807+0000 mon.c (mon.2) 545 : audit [INF] from='client.? 172.21.15.175:0/1901179583' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:58:32.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:32 smithi190 bash[23392]: audit 2024-03-26T03:58:32.393305+0000 mon.a (mon.0) 2890 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:58:32.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:32 smithi175 bash[21527]: cluster 2024-03-26T03:58:30.931669+0000 mgr.y (mgr.24431) 548 : cluster [DBG] pgmap v853: 268 pgs: 32 unknown, 1 creating+activating, 235 active+clean; 582 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T03:58:32.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:32 smithi175 bash[21527]: cluster 2024-03-26T03:58:32.387885+0000 mon.a (mon.0) 2889 : cluster [DBG] osdmap e531: 8 total, 8 up, 8 in 2024-03-26T03:58:32.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:32 smithi175 bash[21527]: audit 2024-03-26T03:58:32.391807+0000 mon.c (mon.2) 545 : audit [INF] from='client.? 172.21.15.175:0/1901179583' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:58:32.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:32 smithi175 bash[21527]: audit 2024-03-26T03:58:32.393305+0000 mon.a (mon.0) 2890 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:58:32.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:32 smithi175 bash[29284]: cluster 2024-03-26T03:58:30.931669+0000 mgr.y (mgr.24431) 548 : cluster [DBG] pgmap v853: 268 pgs: 32 unknown, 1 creating+activating, 235 active+clean; 582 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T03:58:32.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:32 smithi175 bash[29284]: cluster 2024-03-26T03:58:32.387885+0000 mon.a (mon.0) 2889 : cluster [DBG] osdmap e531: 8 total, 8 up, 8 in 2024-03-26T03:58:32.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:32 smithi175 bash[29284]: audit 2024-03-26T03:58:32.391807+0000 mon.c (mon.2) 545 : audit [INF] from='client.? 172.21.15.175:0/1901179583' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:58:32.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:32 smithi175 bash[29284]: audit 2024-03-26T03:58:32.393305+0000 mon.a (mon.0) 2890 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:58:34.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:34 smithi190 bash[23392]: cluster 2024-03-26T03:58:32.932604+0000 mgr.y (mgr.24431) 549 : cluster [DBG] pgmap v856: 300 pgs: 64 unknown, 1 creating+activating, 235 active+clean; 582 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T03:58:34.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:34 smithi190 bash[23392]: audit 2024-03-26T03:58:33.383314+0000 mon.a (mon.0) 2891 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:58:34.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:34 smithi190 bash[23392]: cluster 2024-03-26T03:58:33.392684+0000 mon.a (mon.0) 2892 : cluster [DBG] osdmap e532: 8 total, 8 up, 8 in 2024-03-26T03:58:34.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:34 smithi190 bash[23392]: audit 2024-03-26T03:58:33.401111+0000 mon.c (mon.2) 546 : audit [INF] from='client.? 172.21.15.175:0/1901179583' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-107", "tierpool": "test-rados-api-smithi175-64594-107-cache"}]: dispatch 2024-03-26T03:58:34.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:34 smithi190 bash[23392]: audit 2024-03-26T03:58:33.402058+0000 mon.a (mon.0) 2893 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-107", "tierpool": "test-rados-api-smithi175-64594-107-cache"}]: dispatch 2024-03-26T03:58:34.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:34 smithi175 bash[29284]: cluster 2024-03-26T03:58:32.932604+0000 mgr.y (mgr.24431) 549 : cluster [DBG] pgmap v856: 300 pgs: 64 unknown, 1 creating+activating, 235 active+clean; 582 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T03:58:34.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:34 smithi175 bash[29284]: audit 2024-03-26T03:58:33.383314+0000 mon.a (mon.0) 2891 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:58:34.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:34 smithi175 bash[29284]: cluster 2024-03-26T03:58:33.392684+0000 mon.a (mon.0) 2892 : cluster [DBG] osdmap e532: 8 total, 8 up, 8 in 2024-03-26T03:58:34.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:34 smithi175 bash[29284]: audit 2024-03-26T03:58:33.401111+0000 mon.c (mon.2) 546 : audit [INF] from='client.? 172.21.15.175:0/1901179583' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-107", "tierpool": "test-rados-api-smithi175-64594-107-cache"}]: dispatch 2024-03-26T03:58:34.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:34 smithi175 bash[29284]: audit 2024-03-26T03:58:33.402058+0000 mon.a (mon.0) 2893 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-107", "tierpool": "test-rados-api-smithi175-64594-107-cache"}]: dispatch 2024-03-26T03:58:34.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:34 smithi175 bash[21527]: cluster 2024-03-26T03:58:32.932604+0000 mgr.y (mgr.24431) 549 : cluster [DBG] pgmap v856: 300 pgs: 64 unknown, 1 creating+activating, 235 active+clean; 582 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T03:58:34.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:34 smithi175 bash[21527]: audit 2024-03-26T03:58:33.383314+0000 mon.a (mon.0) 2891 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:58:34.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:34 smithi175 bash[21527]: cluster 2024-03-26T03:58:33.392684+0000 mon.a (mon.0) 2892 : cluster [DBG] osdmap e532: 8 total, 8 up, 8 in 2024-03-26T03:58:34.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:34 smithi175 bash[21527]: audit 2024-03-26T03:58:33.401111+0000 mon.c (mon.2) 546 : audit [INF] from='client.? 172.21.15.175:0/1901179583' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-107", "tierpool": "test-rados-api-smithi175-64594-107-cache"}]: dispatch 2024-03-26T03:58:34.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:34 smithi175 bash[21527]: audit 2024-03-26T03:58:33.402058+0000 mon.a (mon.0) 2893 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-107", "tierpool": "test-rados-api-smithi175-64594-107-cache"}]: dispatch 2024-03-26T03:58:35.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:35 smithi190 bash[23392]: audit 2024-03-26T03:58:34.386298+0000 mon.a (mon.0) 2894 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-107", "tierpool": "test-rados-api-smithi175-64594-107-cache"}]': finished 2024-03-26T03:58:35.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:35 smithi190 bash[23392]: cluster 2024-03-26T03:58:34.396724+0000 mon.a (mon.0) 2895 : cluster [DBG] osdmap e533: 8 total, 8 up, 8 in 2024-03-26T03:58:35.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:35 smithi190 bash[23392]: audit 2024-03-26T03:58:34.399525+0000 mon.c (mon.2) 547 : audit [INF] from='client.? 172.21.15.175:0/1901179583' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-107", "overlaypool": "test-rados-api-smithi175-64594-107-cache"}]: dispatch 2024-03-26T03:58:35.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:35 smithi190 bash[23392]: audit 2024-03-26T03:58:34.400815+0000 mon.a (mon.0) 2896 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-107", "overlaypool": "test-rados-api-smithi175-64594-107-cache"}]: dispatch 2024-03-26T03:58:35.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:35 smithi190 bash[23392]: audit 2024-03-26T03:58:35.389276+0000 mon.a (mon.0) 2897 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-107", "overlaypool": "test-rados-api-smithi175-64594-107-cache"}]': finished 2024-03-26T03:58:35.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:35 smithi190 bash[23392]: cluster 2024-03-26T03:58:35.396631+0000 mon.a (mon.0) 2898 : cluster [DBG] osdmap e534: 8 total, 8 up, 8 in 2024-03-26T03:58:35.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:35 smithi190 bash[23392]: audit 2024-03-26T03:58:35.401614+0000 mon.c (mon.2) 548 : audit [INF] from='client.? 172.21.15.175:0/1901179583' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-107-cache", "mode": "writeback"}]: dispatch 2024-03-26T03:58:35.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:35 smithi190 bash[23392]: audit 2024-03-26T03:58:35.402703+0000 mon.a (mon.0) 2899 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-107-cache", "mode": "writeback"}]: dispatch 2024-03-26T03:58:35.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:35 smithi175 bash[21527]: audit 2024-03-26T03:58:34.386298+0000 mon.a (mon.0) 2894 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-107", "tierpool": "test-rados-api-smithi175-64594-107-cache"}]': finished 2024-03-26T03:58:35.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:35 smithi175 bash[21527]: cluster 2024-03-26T03:58:34.396724+0000 mon.a (mon.0) 2895 : cluster [DBG] osdmap e533: 8 total, 8 up, 8 in 2024-03-26T03:58:35.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:35 smithi175 bash[21527]: audit 2024-03-26T03:58:34.399525+0000 mon.c (mon.2) 547 : audit [INF] from='client.? 172.21.15.175:0/1901179583' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-107", "overlaypool": "test-rados-api-smithi175-64594-107-cache"}]: dispatch 2024-03-26T03:58:35.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:35 smithi175 bash[21527]: audit 2024-03-26T03:58:34.400815+0000 mon.a (mon.0) 2896 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-107", "overlaypool": "test-rados-api-smithi175-64594-107-cache"}]: dispatch 2024-03-26T03:58:35.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:35 smithi175 bash[21527]: audit 2024-03-26T03:58:35.389276+0000 mon.a (mon.0) 2897 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-107", "overlaypool": "test-rados-api-smithi175-64594-107-cache"}]': finished 2024-03-26T03:58:35.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:35 smithi175 bash[21527]: cluster 2024-03-26T03:58:35.396631+0000 mon.a (mon.0) 2898 : cluster [DBG] osdmap e534: 8 total, 8 up, 8 in 2024-03-26T03:58:35.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:35 smithi175 bash[21527]: audit 2024-03-26T03:58:35.401614+0000 mon.c (mon.2) 548 : audit [INF] from='client.? 172.21.15.175:0/1901179583' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-107-cache", "mode": "writeback"}]: dispatch 2024-03-26T03:58:35.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:35 smithi175 bash[21527]: audit 2024-03-26T03:58:35.402703+0000 mon.a (mon.0) 2899 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-107-cache", "mode": "writeback"}]: dispatch 2024-03-26T03:58:35.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:35 smithi175 bash[29284]: audit 2024-03-26T03:58:34.386298+0000 mon.a (mon.0) 2894 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-107", "tierpool": "test-rados-api-smithi175-64594-107-cache"}]': finished 2024-03-26T03:58:35.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:35 smithi175 bash[29284]: cluster 2024-03-26T03:58:34.396724+0000 mon.a (mon.0) 2895 : cluster [DBG] osdmap e533: 8 total, 8 up, 8 in 2024-03-26T03:58:35.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:35 smithi175 bash[29284]: audit 2024-03-26T03:58:34.399525+0000 mon.c (mon.2) 547 : audit [INF] from='client.? 172.21.15.175:0/1901179583' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-107", "overlaypool": "test-rados-api-smithi175-64594-107-cache"}]: dispatch 2024-03-26T03:58:35.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:35 smithi175 bash[29284]: audit 2024-03-26T03:58:34.400815+0000 mon.a (mon.0) 2896 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-107", "overlaypool": "test-rados-api-smithi175-64594-107-cache"}]: dispatch 2024-03-26T03:58:35.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:35 smithi175 bash[29284]: audit 2024-03-26T03:58:35.389276+0000 mon.a (mon.0) 2897 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-107", "overlaypool": "test-rados-api-smithi175-64594-107-cache"}]': finished 2024-03-26T03:58:35.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:35 smithi175 bash[29284]: cluster 2024-03-26T03:58:35.396631+0000 mon.a (mon.0) 2898 : cluster [DBG] osdmap e534: 8 total, 8 up, 8 in 2024-03-26T03:58:35.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:35 smithi175 bash[29284]: audit 2024-03-26T03:58:35.401614+0000 mon.c (mon.2) 548 : audit [INF] from='client.? 172.21.15.175:0/1901179583' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-107-cache", "mode": "writeback"}]: dispatch 2024-03-26T03:58:35.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:35 smithi175 bash[29284]: audit 2024-03-26T03:58:35.402703+0000 mon.a (mon.0) 2899 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-107-cache", "mode": "writeback"}]: dispatch 2024-03-26T03:58:36.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:36 smithi190 bash[23392]: cluster 2024-03-26T03:58:34.935014+0000 mgr.y (mgr.24431) 550 : cluster [DBG] pgmap v859: 300 pgs: 5 unknown, 295 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:58:36.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:36 smithi190 bash[23392]: cluster 2024-03-26T03:58:35.403862+0000 mon.a (mon.0) 2900 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:58:36.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:36 smithi190 bash[23392]: audit 2024-03-26T03:58:35.491293+0000 mon.c (mon.2) 549 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:58:36.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:36 smithi190 bash[23392]: cluster 2024-03-26T03:58:36.389704+0000 mon.a (mon.0) 2901 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:58:36.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:36 smithi190 bash[23392]: audit 2024-03-26T03:58:36.392333+0000 mon.a (mon.0) 2902 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-107-cache", "mode": "writeback"}]': finished 2024-03-26T03:58:36.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:36 smithi190 bash[23392]: cluster 2024-03-26T03:58:36.400092+0000 mon.a (mon.0) 2903 : cluster [DBG] osdmap e535: 8 total, 8 up, 8 in 2024-03-26T03:58:36.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:36 smithi175 bash[21527]: cluster 2024-03-26T03:58:34.935014+0000 mgr.y (mgr.24431) 550 : cluster [DBG] pgmap v859: 300 pgs: 5 unknown, 295 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:58:36.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:36 smithi175 bash[21527]: cluster 2024-03-26T03:58:35.403862+0000 mon.a (mon.0) 2900 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:58:36.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:36 smithi175 bash[21527]: audit 2024-03-26T03:58:35.491293+0000 mon.c (mon.2) 549 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:58:36.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:36 smithi175 bash[21527]: cluster 2024-03-26T03:58:36.389704+0000 mon.a (mon.0) 2901 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:58:36.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:36 smithi175 bash[21527]: audit 2024-03-26T03:58:36.392333+0000 mon.a (mon.0) 2902 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-107-cache", "mode": "writeback"}]': finished 2024-03-26T03:58:36.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:36 smithi175 bash[21527]: cluster 2024-03-26T03:58:36.400092+0000 mon.a (mon.0) 2903 : cluster [DBG] osdmap e535: 8 total, 8 up, 8 in 2024-03-26T03:58:36.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:36 smithi175 bash[29284]: cluster 2024-03-26T03:58:34.935014+0000 mgr.y (mgr.24431) 550 : cluster [DBG] pgmap v859: 300 pgs: 5 unknown, 295 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:58:36.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:36 smithi175 bash[29284]: cluster 2024-03-26T03:58:35.403862+0000 mon.a (mon.0) 2900 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:58:36.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:36 smithi175 bash[29284]: audit 2024-03-26T03:58:35.491293+0000 mon.c (mon.2) 549 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:58:36.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:36 smithi175 bash[29284]: cluster 2024-03-26T03:58:36.389704+0000 mon.a (mon.0) 2901 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:58:36.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:36 smithi175 bash[29284]: audit 2024-03-26T03:58:36.392333+0000 mon.a (mon.0) 2902 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-107-cache", "mode": "writeback"}]': finished 2024-03-26T03:58:36.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:36 smithi175 bash[29284]: cluster 2024-03-26T03:58:36.400092+0000 mon.a (mon.0) 2903 : cluster [DBG] osdmap e535: 8 total, 8 up, 8 in 2024-03-26T03:58:37.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:37 smithi190 bash[23392]: audit 2024-03-26T03:58:36.446572+0000 mon.c (mon.2) 550 : audit [INF] from='client.? 172.21.15.175:0/1901179583' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-107"}]: dispatch 2024-03-26T03:58:37.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:37 smithi190 bash[23392]: audit 2024-03-26T03:58:36.447115+0000 mon.a (mon.0) 2904 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-107"}]: dispatch 2024-03-26T03:58:37.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:37 smithi175 bash[29284]: audit 2024-03-26T03:58:36.446572+0000 mon.c (mon.2) 550 : audit [INF] from='client.? 172.21.15.175:0/1901179583' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-107"}]: dispatch 2024-03-26T03:58:37.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:37 smithi175 bash[29284]: audit 2024-03-26T03:58:36.447115+0000 mon.a (mon.0) 2904 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-107"}]: dispatch 2024-03-26T03:58:37.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:37 smithi175 bash[21527]: audit 2024-03-26T03:58:36.446572+0000 mon.c (mon.2) 550 : audit [INF] from='client.? 172.21.15.175:0/1901179583' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-107"}]: dispatch 2024-03-26T03:58:37.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:37 smithi175 bash[21527]: audit 2024-03-26T03:58:36.447115+0000 mon.a (mon.0) 2904 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-107"}]: dispatch 2024-03-26T03:58:38.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:38 smithi190 bash[23392]: cluster 2024-03-26T03:58:36.936041+0000 mgr.y (mgr.24431) 551 : cluster [DBG] pgmap v862: 300 pgs: 300 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:38.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:38 smithi190 bash[23392]: audit 2024-03-26T03:58:37.429781+0000 mon.a (mon.0) 2905 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-107"}]': finished 2024-03-26T03:58:38.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:38 smithi190 bash[23392]: cluster 2024-03-26T03:58:37.437948+0000 mon.a (mon.0) 2906 : cluster [DBG] osdmap e536: 8 total, 8 up, 8 in 2024-03-26T03:58:38.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:38 smithi190 bash[23392]: audit 2024-03-26T03:58:37.439975+0000 mon.c (mon.2) 551 : audit [INF] from='client.? 172.21.15.175:0/1901179583' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-107", "tierpool": "test-rados-api-smithi175-64594-107-cache"}]: dispatch 2024-03-26T03:58:38.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:38 smithi190 bash[23392]: audit 2024-03-26T03:58:37.441102+0000 mon.a (mon.0) 2907 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-107", "tierpool": "test-rados-api-smithi175-64594-107-cache"}]: dispatch 2024-03-26T03:58:38.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:38 smithi175 bash[29284]: cluster 2024-03-26T03:58:36.936041+0000 mgr.y (mgr.24431) 551 : cluster [DBG] pgmap v862: 300 pgs: 300 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:38.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:38 smithi175 bash[29284]: audit 2024-03-26T03:58:37.429781+0000 mon.a (mon.0) 2905 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-107"}]': finished 2024-03-26T03:58:38.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:38 smithi175 bash[29284]: cluster 2024-03-26T03:58:37.437948+0000 mon.a (mon.0) 2906 : cluster [DBG] osdmap e536: 8 total, 8 up, 8 in 2024-03-26T03:58:38.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:38 smithi175 bash[29284]: audit 2024-03-26T03:58:37.439975+0000 mon.c (mon.2) 551 : audit [INF] from='client.? 172.21.15.175:0/1901179583' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-107", "tierpool": "test-rados-api-smithi175-64594-107-cache"}]: dispatch 2024-03-26T03:58:38.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:38 smithi175 bash[29284]: audit 2024-03-26T03:58:37.441102+0000 mon.a (mon.0) 2907 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-107", "tierpool": "test-rados-api-smithi175-64594-107-cache"}]: dispatch 2024-03-26T03:58:38.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:38 smithi175 bash[21527]: cluster 2024-03-26T03:58:36.936041+0000 mgr.y (mgr.24431) 551 : cluster [DBG] pgmap v862: 300 pgs: 300 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:38.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:38 smithi175 bash[21527]: audit 2024-03-26T03:58:37.429781+0000 mon.a (mon.0) 2905 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-107"}]': finished 2024-03-26T03:58:38.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:38 smithi175 bash[21527]: cluster 2024-03-26T03:58:37.437948+0000 mon.a (mon.0) 2906 : cluster [DBG] osdmap e536: 8 total, 8 up, 8 in 2024-03-26T03:58:38.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:38 smithi175 bash[21527]: audit 2024-03-26T03:58:37.439975+0000 mon.c (mon.2) 551 : audit [INF] from='client.? 172.21.15.175:0/1901179583' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-107", "tierpool": "test-rados-api-smithi175-64594-107-cache"}]: dispatch 2024-03-26T03:58:38.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:38 smithi175 bash[21527]: audit 2024-03-26T03:58:37.441102+0000 mon.a (mon.0) 2907 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-107", "tierpool": "test-rados-api-smithi175-64594-107-cache"}]: dispatch 2024-03-26T03:58:39.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:58:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:58:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:58:39.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:39 smithi190 bash[23392]: cluster 2024-03-26T03:58:38.430936+0000 mon.a (mon.0) 2908 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:58:39.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:39 smithi190 bash[23392]: audit 2024-03-26T03:58:38.440784+0000 mon.a (mon.0) 2909 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-107", "tierpool": "test-rados-api-smithi175-64594-107-cache"}]': finished 2024-03-26T03:58:39.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:39 smithi190 bash[23392]: cluster 2024-03-26T03:58:38.447725+0000 mon.a (mon.0) 2910 : cluster [DBG] osdmap e537: 8 total, 8 up, 8 in 2024-03-26T03:58:39.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:39 smithi175 bash[29284]: cluster 2024-03-26T03:58:38.430936+0000 mon.a (mon.0) 2908 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:58:39.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:39 smithi175 bash[29284]: audit 2024-03-26T03:58:38.440784+0000 mon.a (mon.0) 2909 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-107", "tierpool": "test-rados-api-smithi175-64594-107-cache"}]': finished 2024-03-26T03:58:39.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:39 smithi175 bash[29284]: cluster 2024-03-26T03:58:38.447725+0000 mon.a (mon.0) 2910 : cluster [DBG] osdmap e537: 8 total, 8 up, 8 in 2024-03-26T03:58:39.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:39 smithi175 bash[21527]: cluster 2024-03-26T03:58:38.430936+0000 mon.a (mon.0) 2908 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:58:39.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:39 smithi175 bash[21527]: audit 2024-03-26T03:58:38.440784+0000 mon.a (mon.0) 2909 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-107", "tierpool": "test-rados-api-smithi175-64594-107-cache"}]': finished 2024-03-26T03:58:39.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:39 smithi175 bash[21527]: cluster 2024-03-26T03:58:38.447725+0000 mon.a (mon.0) 2910 : cluster [DBG] osdmap e537: 8 total, 8 up, 8 in 2024-03-26T03:58:40.742 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:58:40 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:58:40.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:40 smithi190 bash[23392]: cluster 2024-03-26T03:58:38.936943+0000 mgr.y (mgr.24431) 552 : cluster [DBG] pgmap v865: 300 pgs: 300 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:40.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:40 smithi190 bash[23392]: cluster 2024-03-26T03:58:39.456473+0000 mon.a (mon.0) 2911 : cluster [DBG] osdmap e538: 8 total, 8 up, 8 in 2024-03-26T03:58:40.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:40 smithi175 bash[21527]: cluster 2024-03-26T03:58:38.936943+0000 mgr.y (mgr.24431) 552 : cluster [DBG] pgmap v865: 300 pgs: 300 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:40.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:40 smithi175 bash[21527]: cluster 2024-03-26T03:58:39.456473+0000 mon.a (mon.0) 2911 : cluster [DBG] osdmap e538: 8 total, 8 up, 8 in 2024-03-26T03:58:40.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:40 smithi175 bash[29284]: cluster 2024-03-26T03:58:38.936943+0000 mgr.y (mgr.24431) 552 : cluster [DBG] pgmap v865: 300 pgs: 300 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:40.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:40 smithi175 bash[29284]: cluster 2024-03-26T03:58:39.456473+0000 mon.a (mon.0) 2911 : cluster [DBG] osdmap e538: 8 total, 8 up, 8 in 2024-03-26T03:58:41.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:41 smithi190 bash[23392]: audit 2024-03-26T03:58:40.279549+0000 mgr.y (mgr.24431) 553 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:58:41.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:41 smithi190 bash[23392]: cluster 2024-03-26T03:58:40.459147+0000 mon.a (mon.0) 2912 : cluster [DBG] osdmap e539: 8 total, 8 up, 8 in 2024-03-26T03:58:41.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:41 smithi190 bash[23392]: audit 2024-03-26T03:58:40.516863+0000 mon.c (mon.2) 552 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:41.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:41 smithi190 bash[23392]: audit 2024-03-26T03:58:40.517263+0000 mon.a (mon.0) 2913 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:41.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:41 smithi190 bash[23392]: audit 2024-03-26T03:58:40.518597+0000 mon.c (mon.2) 553 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:41.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:41 smithi190 bash[23392]: audit 2024-03-26T03:58:40.518919+0000 mon.a (mon.0) 2914 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:41.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:41 smithi190 bash[23392]: audit 2024-03-26T03:58:40.519479+0000 mon.c (mon.2) 554 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi175-64594-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:58:41.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:41 smithi190 bash[23392]: audit 2024-03-26T03:58:40.519784+0000 mon.a (mon.0) 2915 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi175-64594-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:58:41.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:41 smithi175 bash[21527]: audit 2024-03-26T03:58:40.279549+0000 mgr.y (mgr.24431) 553 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:58:41.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:41 smithi175 bash[21527]: cluster 2024-03-26T03:58:40.459147+0000 mon.a (mon.0) 2912 : cluster [DBG] osdmap e539: 8 total, 8 up, 8 in 2024-03-26T03:58:41.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:41 smithi175 bash[21527]: audit 2024-03-26T03:58:40.516863+0000 mon.c (mon.2) 552 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:41.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:41 smithi175 bash[21527]: audit 2024-03-26T03:58:40.517263+0000 mon.a (mon.0) 2913 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:41.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:41 smithi175 bash[21527]: audit 2024-03-26T03:58:40.518597+0000 mon.c (mon.2) 553 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:41.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:41 smithi175 bash[21527]: audit 2024-03-26T03:58:40.518919+0000 mon.a (mon.0) 2914 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:41.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:41 smithi175 bash[21527]: audit 2024-03-26T03:58:40.519479+0000 mon.c (mon.2) 554 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi175-64594-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:58:41.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:41 smithi175 bash[21527]: audit 2024-03-26T03:58:40.519784+0000 mon.a (mon.0) 2915 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi175-64594-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:58:41.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:41 smithi175 bash[29284]: audit 2024-03-26T03:58:40.279549+0000 mgr.y (mgr.24431) 553 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:58:41.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:41 smithi175 bash[29284]: cluster 2024-03-26T03:58:40.459147+0000 mon.a (mon.0) 2912 : cluster [DBG] osdmap e539: 8 total, 8 up, 8 in 2024-03-26T03:58:41.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:41 smithi175 bash[29284]: audit 2024-03-26T03:58:40.516863+0000 mon.c (mon.2) 552 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:41.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:41 smithi175 bash[29284]: audit 2024-03-26T03:58:40.517263+0000 mon.a (mon.0) 2913 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:41.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:41 smithi175 bash[29284]: audit 2024-03-26T03:58:40.518597+0000 mon.c (mon.2) 553 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:41.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:41 smithi175 bash[29284]: audit 2024-03-26T03:58:40.518919+0000 mon.a (mon.0) 2914 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:41.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:41 smithi175 bash[29284]: audit 2024-03-26T03:58:40.519479+0000 mon.c (mon.2) 554 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi175-64594-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:58:41.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:41 smithi175 bash[29284]: audit 2024-03-26T03:58:40.519784+0000 mon.a (mon.0) 2915 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi175-64594-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:58:42.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:42 smithi190 bash[23392]: cluster 2024-03-26T03:58:40.937825+0000 mgr.y (mgr.24431) 554 : cluster [DBG] pgmap v868: 236 pgs: 236 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:42.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:42 smithi190 bash[23392]: audit 2024-03-26T03:58:41.480756+0000 mon.a (mon.0) 2916 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi175-64594-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:58:42.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:42 smithi190 bash[23392]: cluster 2024-03-26T03:58:41.490369+0000 mon.a (mon.0) 2917 : cluster [DBG] osdmap e540: 8 total, 8 up, 8 in 2024-03-26T03:58:42.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:42 smithi190 bash[23392]: audit 2024-03-26T03:58:41.492426+0000 mon.c (mon.2) 555 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64594-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:42.742 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:42 smithi190 bash[23392]: audit 2024-03-26T03:58:41.493533+0000 mon.a (mon.0) 2918 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64594-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:42.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:42 smithi175 bash[29284]: cluster 2024-03-26T03:58:40.937825+0000 mgr.y (mgr.24431) 554 : cluster [DBG] pgmap v868: 236 pgs: 236 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:42.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:42 smithi175 bash[29284]: audit 2024-03-26T03:58:41.480756+0000 mon.a (mon.0) 2916 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi175-64594-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:58:42.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:42 smithi175 bash[29284]: cluster 2024-03-26T03:58:41.490369+0000 mon.a (mon.0) 2917 : cluster [DBG] osdmap e540: 8 total, 8 up, 8 in 2024-03-26T03:58:42.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:42 smithi175 bash[29284]: audit 2024-03-26T03:58:41.492426+0000 mon.c (mon.2) 555 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64594-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:42.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:42 smithi175 bash[29284]: audit 2024-03-26T03:58:41.493533+0000 mon.a (mon.0) 2918 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64594-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:42.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:42 smithi175 bash[21527]: cluster 2024-03-26T03:58:40.937825+0000 mgr.y (mgr.24431) 554 : cluster [DBG] pgmap v868: 236 pgs: 236 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:42.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:42 smithi175 bash[21527]: audit 2024-03-26T03:58:41.480756+0000 mon.a (mon.0) 2916 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi175-64594-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:58:42.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:42 smithi175 bash[21527]: cluster 2024-03-26T03:58:41.490369+0000 mon.a (mon.0) 2917 : cluster [DBG] osdmap e540: 8 total, 8 up, 8 in 2024-03-26T03:58:42.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:42 smithi175 bash[21527]: audit 2024-03-26T03:58:41.492426+0000 mon.c (mon.2) 555 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64594-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:42.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:42 smithi175 bash[21527]: audit 2024-03-26T03:58:41.493533+0000 mon.a (mon.0) 2918 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64594-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:43.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:43 smithi190 bash[23392]: cluster 2024-03-26T03:58:42.513674+0000 mon.a (mon.0) 2919 : cluster [DBG] osdmap e541: 8 total, 8 up, 8 in 2024-03-26T03:58:43.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:43 smithi175 bash[21527]: cluster 2024-03-26T03:58:42.513674+0000 mon.a (mon.0) 2919 : cluster [DBG] osdmap e541: 8 total, 8 up, 8 in 2024-03-26T03:58:43.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:43 smithi175 bash[29284]: cluster 2024-03-26T03:58:42.513674+0000 mon.a (mon.0) 2919 : cluster [DBG] osdmap e541: 8 total, 8 up, 8 in 2024-03-26T03:58:44.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:44 smithi190 bash[23392]: cluster 2024-03-26T03:58:42.938748+0000 mgr.y (mgr.24431) 555 : cluster [DBG] pgmap v871: 236 pgs: 236 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:44.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:44 smithi190 bash[23392]: audit 2024-03-26T03:58:43.504609+0000 mon.a (mon.0) 2920 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64594-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi175-64594-109"}]': finished 2024-03-26T03:58:44.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:44 smithi190 bash[23392]: cluster 2024-03-26T03:58:43.513163+0000 mon.a (mon.0) 2921 : cluster [DBG] osdmap e542: 8 total, 8 up, 8 in 2024-03-26T03:58:44.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:44 smithi175 bash[21527]: cluster 2024-03-26T03:58:42.938748+0000 mgr.y (mgr.24431) 555 : cluster [DBG] pgmap v871: 236 pgs: 236 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:44.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:44 smithi175 bash[21527]: audit 2024-03-26T03:58:43.504609+0000 mon.a (mon.0) 2920 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64594-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi175-64594-109"}]': finished 2024-03-26T03:58:44.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:44 smithi175 bash[21527]: cluster 2024-03-26T03:58:43.513163+0000 mon.a (mon.0) 2921 : cluster [DBG] osdmap e542: 8 total, 8 up, 8 in 2024-03-26T03:58:44.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:44 smithi175 bash[29284]: cluster 2024-03-26T03:58:42.938748+0000 mgr.y (mgr.24431) 555 : cluster [DBG] pgmap v871: 236 pgs: 236 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:44.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:44 smithi175 bash[29284]: audit 2024-03-26T03:58:43.504609+0000 mon.a (mon.0) 2920 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64594-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi175-64594-109"}]': finished 2024-03-26T03:58:44.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:44 smithi175 bash[29284]: cluster 2024-03-26T03:58:43.513163+0000 mon.a (mon.0) 2921 : cluster [DBG] osdmap e542: 8 total, 8 up, 8 in 2024-03-26T03:58:45.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:45 smithi190 bash[23392]: cluster 2024-03-26T03:58:44.520425+0000 mon.a (mon.0) 2922 : cluster [DBG] osdmap e543: 8 total, 8 up, 8 in 2024-03-26T03:58:45.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:45 smithi190 bash[23392]: audit 2024-03-26T03:58:44.528556+0000 mon.c (mon.2) 556 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:58:45.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:45 smithi190 bash[23392]: audit 2024-03-26T03:58:44.546312+0000 mon.a (mon.0) 2923 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:58:45.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:45 smithi190 bash[23392]: audit 2024-03-26T03:58:45.512412+0000 mon.a (mon.0) 2924 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:58:45.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:45 smithi190 bash[23392]: cluster 2024-03-26T03:58:45.520167+0000 mon.a (mon.0) 2925 : cluster [DBG] osdmap e544: 8 total, 8 up, 8 in 2024-03-26T03:58:45.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:45 smithi175 bash[21527]: cluster 2024-03-26T03:58:44.520425+0000 mon.a (mon.0) 2922 : cluster [DBG] osdmap e543: 8 total, 8 up, 8 in 2024-03-26T03:58:45.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:45 smithi175 bash[21527]: audit 2024-03-26T03:58:44.528556+0000 mon.c (mon.2) 556 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:58:45.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:45 smithi175 bash[21527]: audit 2024-03-26T03:58:44.546312+0000 mon.a (mon.0) 2923 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:58:45.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:45 smithi175 bash[21527]: audit 2024-03-26T03:58:45.512412+0000 mon.a (mon.0) 2924 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:58:45.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:45 smithi175 bash[21527]: cluster 2024-03-26T03:58:45.520167+0000 mon.a (mon.0) 2925 : cluster [DBG] osdmap e544: 8 total, 8 up, 8 in 2024-03-26T03:58:45.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:45 smithi175 bash[29284]: cluster 2024-03-26T03:58:44.520425+0000 mon.a (mon.0) 2922 : cluster [DBG] osdmap e543: 8 total, 8 up, 8 in 2024-03-26T03:58:45.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:45 smithi175 bash[29284]: audit 2024-03-26T03:58:44.528556+0000 mon.c (mon.2) 556 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:58:45.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:45 smithi175 bash[29284]: audit 2024-03-26T03:58:44.546312+0000 mon.a (mon.0) 2923 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:58:45.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:45 smithi175 bash[29284]: audit 2024-03-26T03:58:45.512412+0000 mon.a (mon.0) 2924 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:58:45.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:45 smithi175 bash[29284]: cluster 2024-03-26T03:58:45.520167+0000 mon.a (mon.0) 2925 : cluster [DBG] osdmap e544: 8 total, 8 up, 8 in 2024-03-26T03:58:46.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:46 smithi190 bash[23392]: cluster 2024-03-26T03:58:44.940984+0000 mgr.y (mgr.24431) 556 : cluster [DBG] pgmap v874: 276 pgs: 28 creating+peering, 5 unknown, 243 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:58:46.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:46 smithi190 bash[23392]: cluster 2024-03-26T03:58:45.529039+0000 mon.a (mon.0) 2926 : cluster [WRN] Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:58:46.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:46 smithi190 bash[23392]: audit 2024-03-26T03:58:45.540568+0000 mon.c (mon.2) 557 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-109", "tierpool": "test-rados-api-smithi175-64594-109-cache"}]: dispatch 2024-03-26T03:58:46.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:46 smithi190 bash[23392]: audit 2024-03-26T03:58:45.553394+0000 mon.a (mon.0) 2927 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-109", "tierpool": "test-rados-api-smithi175-64594-109-cache"}]: dispatch 2024-03-26T03:58:46.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:46 smithi190 bash[23392]: audit 2024-03-26T03:58:46.544112+0000 mon.a (mon.0) 2928 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-109", "tierpool": "test-rados-api-smithi175-64594-109-cache"}]': finished 2024-03-26T03:58:46.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:46 smithi190 bash[23392]: cluster 2024-03-26T03:58:46.553827+0000 mon.a (mon.0) 2929 : cluster [DBG] osdmap e545: 8 total, 8 up, 8 in 2024-03-26T03:58:46.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:46 smithi190 bash[23392]: audit 2024-03-26T03:58:46.556702+0000 mon.c (mon.2) 558 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-109", "overlaypool": "test-rados-api-smithi175-64594-109-cache"}]: dispatch 2024-03-26T03:58:46.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:46 smithi190 bash[23392]: audit 2024-03-26T03:58:46.559859+0000 mon.a (mon.0) 2930 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-109", "overlaypool": "test-rados-api-smithi175-64594-109-cache"}]: dispatch 2024-03-26T03:58:46.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:46 smithi175 bash[21527]: cluster 2024-03-26T03:58:44.940984+0000 mgr.y (mgr.24431) 556 : cluster [DBG] pgmap v874: 276 pgs: 28 creating+peering, 5 unknown, 243 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:58:46.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:46 smithi175 bash[21527]: cluster 2024-03-26T03:58:45.529039+0000 mon.a (mon.0) 2926 : cluster [WRN] Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:58:46.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:46 smithi175 bash[21527]: audit 2024-03-26T03:58:45.540568+0000 mon.c (mon.2) 557 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-109", "tierpool": "test-rados-api-smithi175-64594-109-cache"}]: dispatch 2024-03-26T03:58:46.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:46 smithi175 bash[21527]: audit 2024-03-26T03:58:45.553394+0000 mon.a (mon.0) 2927 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-109", "tierpool": "test-rados-api-smithi175-64594-109-cache"}]: dispatch 2024-03-26T03:58:46.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:46 smithi175 bash[21527]: audit 2024-03-26T03:58:46.544112+0000 mon.a (mon.0) 2928 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-109", "tierpool": "test-rados-api-smithi175-64594-109-cache"}]': finished 2024-03-26T03:58:46.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:46 smithi175 bash[21527]: cluster 2024-03-26T03:58:46.553827+0000 mon.a (mon.0) 2929 : cluster [DBG] osdmap e545: 8 total, 8 up, 8 in 2024-03-26T03:58:46.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:46 smithi175 bash[21527]: audit 2024-03-26T03:58:46.556702+0000 mon.c (mon.2) 558 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-109", "overlaypool": "test-rados-api-smithi175-64594-109-cache"}]: dispatch 2024-03-26T03:58:46.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:46 smithi175 bash[21527]: audit 2024-03-26T03:58:46.559859+0000 mon.a (mon.0) 2930 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-109", "overlaypool": "test-rados-api-smithi175-64594-109-cache"}]: dispatch 2024-03-26T03:58:46.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:46 smithi175 bash[29284]: cluster 2024-03-26T03:58:44.940984+0000 mgr.y (mgr.24431) 556 : cluster [DBG] pgmap v874: 276 pgs: 28 creating+peering, 5 unknown, 243 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:58:46.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:46 smithi175 bash[29284]: cluster 2024-03-26T03:58:45.529039+0000 mon.a (mon.0) 2926 : cluster [WRN] Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:58:46.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:46 smithi175 bash[29284]: audit 2024-03-26T03:58:45.540568+0000 mon.c (mon.2) 557 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-109", "tierpool": "test-rados-api-smithi175-64594-109-cache"}]: dispatch 2024-03-26T03:58:46.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:46 smithi175 bash[29284]: audit 2024-03-26T03:58:45.553394+0000 mon.a (mon.0) 2927 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-109", "tierpool": "test-rados-api-smithi175-64594-109-cache"}]: dispatch 2024-03-26T03:58:46.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:46 smithi175 bash[29284]: audit 2024-03-26T03:58:46.544112+0000 mon.a (mon.0) 2928 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-109", "tierpool": "test-rados-api-smithi175-64594-109-cache"}]': finished 2024-03-26T03:58:46.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:46 smithi175 bash[29284]: cluster 2024-03-26T03:58:46.553827+0000 mon.a (mon.0) 2929 : cluster [DBG] osdmap e545: 8 total, 8 up, 8 in 2024-03-26T03:58:46.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:46 smithi175 bash[29284]: audit 2024-03-26T03:58:46.556702+0000 mon.c (mon.2) 558 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-109", "overlaypool": "test-rados-api-smithi175-64594-109-cache"}]: dispatch 2024-03-26T03:58:46.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:46 smithi175 bash[29284]: audit 2024-03-26T03:58:46.559859+0000 mon.a (mon.0) 2930 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-109", "overlaypool": "test-rados-api-smithi175-64594-109-cache"}]: dispatch 2024-03-26T03:58:48.992 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:58:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:58:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:58:48.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:48 smithi190 bash[23392]: cluster 2024-03-26T03:58:46.941905+0000 mgr.y (mgr.24431) 557 : cluster [DBG] pgmap v877: 276 pgs: 32 creating+peering, 244 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:48.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:48 smithi190 bash[23392]: audit 2024-03-26T03:58:47.546921+0000 mon.a (mon.0) 2931 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-109", "overlaypool": "test-rados-api-smithi175-64594-109-cache"}]': finished 2024-03-26T03:58:48.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:48 smithi190 bash[23392]: cluster 2024-03-26T03:58:47.554180+0000 mon.a (mon.0) 2932 : cluster [DBG] osdmap e546: 8 total, 8 up, 8 in 2024-03-26T03:58:48.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:48 smithi190 bash[23392]: audit 2024-03-26T03:58:47.556349+0000 mon.c (mon.2) 559 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-109-cache", "mode": "writeback"}]: dispatch 2024-03-26T03:58:48.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:48 smithi190 bash[23392]: audit 2024-03-26T03:58:47.558188+0000 mon.a (mon.0) 2933 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-109-cache", "mode": "writeback"}]: dispatch 2024-03-26T03:58:48.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:48 smithi175 bash[21527]: cluster 2024-03-26T03:58:46.941905+0000 mgr.y (mgr.24431) 557 : cluster [DBG] pgmap v877: 276 pgs: 32 creating+peering, 244 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:48.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:48 smithi175 bash[21527]: audit 2024-03-26T03:58:47.546921+0000 mon.a (mon.0) 2931 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-109", "overlaypool": "test-rados-api-smithi175-64594-109-cache"}]': finished 2024-03-26T03:58:48.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:48 smithi175 bash[21527]: cluster 2024-03-26T03:58:47.554180+0000 mon.a (mon.0) 2932 : cluster [DBG] osdmap e546: 8 total, 8 up, 8 in 2024-03-26T03:58:48.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:48 smithi175 bash[21527]: audit 2024-03-26T03:58:47.556349+0000 mon.c (mon.2) 559 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-109-cache", "mode": "writeback"}]: dispatch 2024-03-26T03:58:48.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:48 smithi175 bash[21527]: audit 2024-03-26T03:58:47.558188+0000 mon.a (mon.0) 2933 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-109-cache", "mode": "writeback"}]: dispatch 2024-03-26T03:58:48.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:48 smithi175 bash[29284]: cluster 2024-03-26T03:58:46.941905+0000 mgr.y (mgr.24431) 557 : cluster [DBG] pgmap v877: 276 pgs: 32 creating+peering, 244 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:48.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:48 smithi175 bash[29284]: audit 2024-03-26T03:58:47.546921+0000 mon.a (mon.0) 2931 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-109", "overlaypool": "test-rados-api-smithi175-64594-109-cache"}]': finished 2024-03-26T03:58:48.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:48 smithi175 bash[29284]: cluster 2024-03-26T03:58:47.554180+0000 mon.a (mon.0) 2932 : cluster [DBG] osdmap e546: 8 total, 8 up, 8 in 2024-03-26T03:58:48.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:48 smithi175 bash[29284]: audit 2024-03-26T03:58:47.556349+0000 mon.c (mon.2) 559 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-109-cache", "mode": "writeback"}]: dispatch 2024-03-26T03:58:48.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:48 smithi175 bash[29284]: audit 2024-03-26T03:58:47.558188+0000 mon.a (mon.0) 2933 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-109-cache", "mode": "writeback"}]: dispatch 2024-03-26T03:58:49.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:49 smithi190 bash[23392]: cluster 2024-03-26T03:58:48.547311+0000 mon.a (mon.0) 2934 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:58:49.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:49 smithi190 bash[23392]: audit 2024-03-26T03:58:48.549662+0000 mon.a (mon.0) 2935 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-109-cache", "mode": "writeback"}]': finished 2024-03-26T03:58:49.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:49 smithi190 bash[23392]: cluster 2024-03-26T03:58:48.558655+0000 mon.a (mon.0) 2936 : cluster [DBG] osdmap e547: 8 total, 8 up, 8 in 2024-03-26T03:58:49.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:49 smithi190 bash[23392]: audit 2024-03-26T03:58:48.561521+0000 mon.c (mon.2) 560 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T03:58:49.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:49 smithi190 bash[23392]: audit 2024-03-26T03:58:48.573918+0000 mon.a (mon.0) 2937 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T03:58:49.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:49 smithi175 bash[21527]: cluster 2024-03-26T03:58:48.547311+0000 mon.a (mon.0) 2934 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:58:49.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:49 smithi175 bash[21527]: audit 2024-03-26T03:58:48.549662+0000 mon.a (mon.0) 2935 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-109-cache", "mode": "writeback"}]': finished 2024-03-26T03:58:49.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:49 smithi175 bash[21527]: cluster 2024-03-26T03:58:48.558655+0000 mon.a (mon.0) 2936 : cluster [DBG] osdmap e547: 8 total, 8 up, 8 in 2024-03-26T03:58:49.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:49 smithi175 bash[21527]: audit 2024-03-26T03:58:48.561521+0000 mon.c (mon.2) 560 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T03:58:49.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:49 smithi175 bash[21527]: audit 2024-03-26T03:58:48.573918+0000 mon.a (mon.0) 2937 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T03:58:49.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:49 smithi175 bash[29284]: cluster 2024-03-26T03:58:48.547311+0000 mon.a (mon.0) 2934 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:58:49.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:49 smithi175 bash[29284]: audit 2024-03-26T03:58:48.549662+0000 mon.a (mon.0) 2935 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-109-cache", "mode": "writeback"}]': finished 2024-03-26T03:58:49.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:49 smithi175 bash[29284]: cluster 2024-03-26T03:58:48.558655+0000 mon.a (mon.0) 2936 : cluster [DBG] osdmap e547: 8 total, 8 up, 8 in 2024-03-26T03:58:49.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:49 smithi175 bash[29284]: audit 2024-03-26T03:58:48.561521+0000 mon.c (mon.2) 560 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T03:58:49.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:49 smithi175 bash[29284]: audit 2024-03-26T03:58:48.573918+0000 mon.a (mon.0) 2937 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T03:58:50.594 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:58:50 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:58:50.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:50 smithi190 bash[23392]: cluster 2024-03-26T03:58:48.942848+0000 mgr.y (mgr.24431) 558 : cluster [DBG] pgmap v880: 276 pgs: 32 creating+peering, 244 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:50.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:50 smithi190 bash[23392]: audit 2024-03-26T03:58:49.573133+0000 mon.a (mon.0) 2938 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-03-26T03:58:50.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:50 smithi190 bash[23392]: cluster 2024-03-26T03:58:49.583930+0000 mon.a (mon.0) 2939 : cluster [DBG] osdmap e548: 8 total, 8 up, 8 in 2024-03-26T03:58:50.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:50 smithi190 bash[23392]: audit 2024-03-26T03:58:49.586551+0000 mon.c (mon.2) 561 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T03:58:50.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:50 smithi190 bash[23392]: audit 2024-03-26T03:58:49.587574+0000 mon.a (mon.0) 2940 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T03:58:50.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:50 smithi190 bash[23392]: audit 2024-03-26T03:58:50.532864+0000 mon.a (mon.0) 2941 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:58:50.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:50 smithi190 bash[23392]: audit 2024-03-26T03:58:50.535923+0000 mon.c (mon.2) 562 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:58:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:50 smithi175 bash[21527]: cluster 2024-03-26T03:58:48.942848+0000 mgr.y (mgr.24431) 558 : cluster [DBG] pgmap v880: 276 pgs: 32 creating+peering, 244 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:50 smithi175 bash[21527]: audit 2024-03-26T03:58:49.573133+0000 mon.a (mon.0) 2938 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-03-26T03:58:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:50 smithi175 bash[21527]: cluster 2024-03-26T03:58:49.583930+0000 mon.a (mon.0) 2939 : cluster [DBG] osdmap e548: 8 total, 8 up, 8 in 2024-03-26T03:58:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:50 smithi175 bash[21527]: audit 2024-03-26T03:58:49.586551+0000 mon.c (mon.2) 561 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T03:58:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:50 smithi175 bash[21527]: audit 2024-03-26T03:58:49.587574+0000 mon.a (mon.0) 2940 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T03:58:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:50 smithi175 bash[21527]: audit 2024-03-26T03:58:50.532864+0000 mon.a (mon.0) 2941 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:58:50.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:50 smithi175 bash[21527]: audit 2024-03-26T03:58:50.535923+0000 mon.c (mon.2) 562 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:58:50.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:50 smithi175 bash[29284]: cluster 2024-03-26T03:58:48.942848+0000 mgr.y (mgr.24431) 558 : cluster [DBG] pgmap v880: 276 pgs: 32 creating+peering, 244 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:50.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:50 smithi175 bash[29284]: audit 2024-03-26T03:58:49.573133+0000 mon.a (mon.0) 2938 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-03-26T03:58:50.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:50 smithi175 bash[29284]: cluster 2024-03-26T03:58:49.583930+0000 mon.a (mon.0) 2939 : cluster [DBG] osdmap e548: 8 total, 8 up, 8 in 2024-03-26T03:58:50.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:50 smithi175 bash[29284]: audit 2024-03-26T03:58:49.586551+0000 mon.c (mon.2) 561 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T03:58:50.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:50 smithi175 bash[29284]: audit 2024-03-26T03:58:49.587574+0000 mon.a (mon.0) 2940 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T03:58:50.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:50 smithi175 bash[29284]: audit 2024-03-26T03:58:50.532864+0000 mon.a (mon.0) 2941 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:58:50.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:50 smithi175 bash[29284]: audit 2024-03-26T03:58:50.535923+0000 mon.c (mon.2) 562 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:58:51.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:51 smithi190 bash[23392]: audit 2024-03-26T03:58:50.291051+0000 mgr.y (mgr.24431) 559 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:58:51.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:51 smithi190 bash[23392]: audit 2024-03-26T03:58:50.576020+0000 mon.a (mon.0) 2942 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-03-26T03:58:51.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:51 smithi190 bash[23392]: cluster 2024-03-26T03:58:50.583374+0000 mon.a (mon.0) 2943 : cluster [DBG] osdmap e549: 8 total, 8 up, 8 in 2024-03-26T03:58:51.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:51 smithi190 bash[23392]: audit 2024-03-26T03:58:50.590376+0000 mon.c (mon.2) 563 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-26T03:58:51.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:51 smithi190 bash[23392]: audit 2024-03-26T03:58:50.605639+0000 mon.a (mon.0) 2944 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-26T03:58:51.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:51 smithi190 bash[23392]: cluster 2024-03-26T03:58:50.943789+0000 mgr.y (mgr.24431) 560 : cluster [DBG] pgmap v883: 276 pgs: 276 active+clean; 582 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:51.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:51 smithi175 bash[21527]: audit 2024-03-26T03:58:50.291051+0000 mgr.y (mgr.24431) 559 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:58:51.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:51 smithi175 bash[21527]: audit 2024-03-26T03:58:50.576020+0000 mon.a (mon.0) 2942 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-03-26T03:58:51.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:51 smithi175 bash[21527]: cluster 2024-03-26T03:58:50.583374+0000 mon.a (mon.0) 2943 : cluster [DBG] osdmap e549: 8 total, 8 up, 8 in 2024-03-26T03:58:51.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:51 smithi175 bash[21527]: audit 2024-03-26T03:58:50.590376+0000 mon.c (mon.2) 563 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-26T03:58:51.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:51 smithi175 bash[21527]: audit 2024-03-26T03:58:50.605639+0000 mon.a (mon.0) 2944 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-26T03:58:51.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:51 smithi175 bash[21527]: cluster 2024-03-26T03:58:50.943789+0000 mgr.y (mgr.24431) 560 : cluster [DBG] pgmap v883: 276 pgs: 276 active+clean; 582 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:51.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:51 smithi175 bash[29284]: audit 2024-03-26T03:58:50.291051+0000 mgr.y (mgr.24431) 559 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:58:51.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:51 smithi175 bash[29284]: audit 2024-03-26T03:58:50.576020+0000 mon.a (mon.0) 2942 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-03-26T03:58:51.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:51 smithi175 bash[29284]: cluster 2024-03-26T03:58:50.583374+0000 mon.a (mon.0) 2943 : cluster [DBG] osdmap e549: 8 total, 8 up, 8 in 2024-03-26T03:58:51.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:51 smithi175 bash[29284]: audit 2024-03-26T03:58:50.590376+0000 mon.c (mon.2) 563 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-26T03:58:51.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:51 smithi175 bash[29284]: audit 2024-03-26T03:58:50.605639+0000 mon.a (mon.0) 2944 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-26T03:58:51.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:51 smithi175 bash[29284]: cluster 2024-03-26T03:58:50.943789+0000 mgr.y (mgr.24431) 560 : cluster [DBG] pgmap v883: 276 pgs: 276 active+clean; 582 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:52.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:52 smithi190 bash[23392]: cluster 2024-03-26T03:58:51.596052+0000 mon.a (mon.0) 2945 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:58:52.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:52 smithi190 bash[23392]: audit 2024-03-26T03:58:51.598513+0000 mon.a (mon.0) 2946 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-26T03:58:52.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:52 smithi190 bash[23392]: cluster 2024-03-26T03:58:51.605854+0000 mon.a (mon.0) 2947 : cluster [DBG] osdmap e550: 8 total, 8 up, 8 in 2024-03-26T03:58:52.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:52 smithi190 bash[23392]: audit 2024-03-26T03:58:51.607304+0000 mon.c (mon.2) 564 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-03-26T03:58:52.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:52 smithi190 bash[23392]: cluster 2024-03-26T03:58:51.611314+0000 mon.a (mon.0) 2948 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:58:52.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:52 smithi190 bash[23392]: audit 2024-03-26T03:58:51.613876+0000 mon.a (mon.0) 2949 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-03-26T03:58:52.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:52 smithi190 bash[23392]: audit 2024-03-26T03:58:52.601384+0000 mon.a (mon.0) 2950 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-03-26T03:58:52.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:52 smithi190 bash[23392]: cluster 2024-03-26T03:58:52.609918+0000 mon.a (mon.0) 2951 : cluster [DBG] osdmap e551: 8 total, 8 up, 8 in 2024-03-26T03:58:52.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:52 smithi175 bash[21527]: cluster 2024-03-26T03:58:51.596052+0000 mon.a (mon.0) 2945 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:58:52.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:52 smithi175 bash[21527]: audit 2024-03-26T03:58:51.598513+0000 mon.a (mon.0) 2946 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-26T03:58:52.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:52 smithi175 bash[21527]: cluster 2024-03-26T03:58:51.605854+0000 mon.a (mon.0) 2947 : cluster [DBG] osdmap e550: 8 total, 8 up, 8 in 2024-03-26T03:58:52.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:52 smithi175 bash[21527]: audit 2024-03-26T03:58:51.607304+0000 mon.c (mon.2) 564 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-03-26T03:58:52.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:52 smithi175 bash[21527]: cluster 2024-03-26T03:58:51.611314+0000 mon.a (mon.0) 2948 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:58:52.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:52 smithi175 bash[21527]: audit 2024-03-26T03:58:51.613876+0000 mon.a (mon.0) 2949 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-03-26T03:58:52.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:52 smithi175 bash[21527]: audit 2024-03-26T03:58:52.601384+0000 mon.a (mon.0) 2950 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-03-26T03:58:52.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:52 smithi175 bash[21527]: cluster 2024-03-26T03:58:52.609918+0000 mon.a (mon.0) 2951 : cluster [DBG] osdmap e551: 8 total, 8 up, 8 in 2024-03-26T03:58:52.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:52 smithi175 bash[29284]: cluster 2024-03-26T03:58:51.596052+0000 mon.a (mon.0) 2945 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:58:52.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:52 smithi175 bash[29284]: audit 2024-03-26T03:58:51.598513+0000 mon.a (mon.0) 2946 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-26T03:58:52.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:52 smithi175 bash[29284]: cluster 2024-03-26T03:58:51.605854+0000 mon.a (mon.0) 2947 : cluster [DBG] osdmap e550: 8 total, 8 up, 8 in 2024-03-26T03:58:52.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:52 smithi175 bash[29284]: audit 2024-03-26T03:58:51.607304+0000 mon.c (mon.2) 564 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-03-26T03:58:52.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:52 smithi175 bash[29284]: cluster 2024-03-26T03:58:51.611314+0000 mon.a (mon.0) 2948 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:58:52.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:52 smithi175 bash[29284]: audit 2024-03-26T03:58:51.613876+0000 mon.a (mon.0) 2949 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-03-26T03:58:52.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:52 smithi175 bash[29284]: audit 2024-03-26T03:58:52.601384+0000 mon.a (mon.0) 2950 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-03-26T03:58:52.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:52 smithi175 bash[29284]: cluster 2024-03-26T03:58:52.609918+0000 mon.a (mon.0) 2951 : cluster [DBG] osdmap e551: 8 total, 8 up, 8 in 2024-03-26T03:58:53.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:53 smithi190 bash[23392]: audit 2024-03-26T03:58:52.699751+0000 mon.c (mon.2) 565 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:53.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:53 smithi190 bash[23392]: audit 2024-03-26T03:58:52.700420+0000 mon.a (mon.0) 2952 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:53.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:53 smithi190 bash[23392]: cluster 2024-03-26T03:58:52.944646+0000 mgr.y (mgr.24431) 561 : cluster [DBG] pgmap v886: 276 pgs: 276 active+clean; 582 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:53.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:53 smithi175 bash[29284]: audit 2024-03-26T03:58:52.699751+0000 mon.c (mon.2) 565 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:53.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:53 smithi175 bash[29284]: audit 2024-03-26T03:58:52.700420+0000 mon.a (mon.0) 2952 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:53.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:53 smithi175 bash[29284]: cluster 2024-03-26T03:58:52.944646+0000 mgr.y (mgr.24431) 561 : cluster [DBG] pgmap v886: 276 pgs: 276 active+clean; 582 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:53.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:53 smithi175 bash[21527]: audit 2024-03-26T03:58:52.699751+0000 mon.c (mon.2) 565 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:53.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:53 smithi175 bash[21527]: audit 2024-03-26T03:58:52.700420+0000 mon.a (mon.0) 2952 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:53.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:53 smithi175 bash[21527]: cluster 2024-03-26T03:58:52.944646+0000 mgr.y (mgr.24431) 561 : cluster [DBG] pgmap v886: 276 pgs: 276 active+clean; 582 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:54.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:54 smithi190 bash[23392]: audit 2024-03-26T03:58:53.629031+0000 mon.a (mon.0) 2953 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-109"}]': finished 2024-03-26T03:58:54.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:54 smithi190 bash[23392]: cluster 2024-03-26T03:58:53.636270+0000 mon.a (mon.0) 2954 : cluster [DBG] osdmap e552: 8 total, 8 up, 8 in 2024-03-26T03:58:54.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:54 smithi190 bash[23392]: audit 2024-03-26T03:58:53.638326+0000 mon.c (mon.2) 566 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-109", "tierpool": "test-rados-api-smithi175-64594-109-cache"}]: dispatch 2024-03-26T03:58:54.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:54 smithi190 bash[23392]: audit 2024-03-26T03:58:53.639316+0000 mon.a (mon.0) 2955 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-109", "tierpool": "test-rados-api-smithi175-64594-109-cache"}]: dispatch 2024-03-26T03:58:54.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:54 smithi175 bash[21527]: audit 2024-03-26T03:58:53.629031+0000 mon.a (mon.0) 2953 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-109"}]': finished 2024-03-26T03:58:54.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:54 smithi175 bash[21527]: cluster 2024-03-26T03:58:53.636270+0000 mon.a (mon.0) 2954 : cluster [DBG] osdmap e552: 8 total, 8 up, 8 in 2024-03-26T03:58:54.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:54 smithi175 bash[21527]: audit 2024-03-26T03:58:53.638326+0000 mon.c (mon.2) 566 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-109", "tierpool": "test-rados-api-smithi175-64594-109-cache"}]: dispatch 2024-03-26T03:58:54.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:54 smithi175 bash[21527]: audit 2024-03-26T03:58:53.639316+0000 mon.a (mon.0) 2955 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-109", "tierpool": "test-rados-api-smithi175-64594-109-cache"}]: dispatch 2024-03-26T03:58:54.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:54 smithi175 bash[29284]: audit 2024-03-26T03:58:53.629031+0000 mon.a (mon.0) 2953 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-109"}]': finished 2024-03-26T03:58:54.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:54 smithi175 bash[29284]: cluster 2024-03-26T03:58:53.636270+0000 mon.a (mon.0) 2954 : cluster [DBG] osdmap e552: 8 total, 8 up, 8 in 2024-03-26T03:58:54.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:54 smithi175 bash[29284]: audit 2024-03-26T03:58:53.638326+0000 mon.c (mon.2) 566 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-109", "tierpool": "test-rados-api-smithi175-64594-109-cache"}]: dispatch 2024-03-26T03:58:54.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:54 smithi175 bash[29284]: audit 2024-03-26T03:58:53.639316+0000 mon.a (mon.0) 2955 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-109", "tierpool": "test-rados-api-smithi175-64594-109-cache"}]: dispatch 2024-03-26T03:58:55.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:55 smithi190 bash[23392]: audit 2024-03-26T03:58:54.631833+0000 mon.a (mon.0) 2956 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-109", "tierpool": "test-rados-api-smithi175-64594-109-cache"}]': finished 2024-03-26T03:58:55.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:55 smithi190 bash[23392]: cluster 2024-03-26T03:58:54.639934+0000 mon.a (mon.0) 2957 : cluster [DBG] osdmap e553: 8 total, 8 up, 8 in 2024-03-26T03:58:55.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:55 smithi190 bash[23392]: cluster 2024-03-26T03:58:54.947263+0000 mgr.y (mgr.24431) 562 : cluster [DBG] pgmap v889: 276 pgs: 276 active+clean; 582 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T03:58:55.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:55 smithi190 bash[23392]: cluster 2024-03-26T03:58:55.642462+0000 mon.a (mon.0) 2958 : cluster [DBG] osdmap e554: 8 total, 8 up, 8 in 2024-03-26T03:58:55.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:55 smithi175 bash[29284]: audit 2024-03-26T03:58:54.631833+0000 mon.a (mon.0) 2956 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-109", "tierpool": "test-rados-api-smithi175-64594-109-cache"}]': finished 2024-03-26T03:58:55.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:55 smithi175 bash[29284]: cluster 2024-03-26T03:58:54.639934+0000 mon.a (mon.0) 2957 : cluster [DBG] osdmap e553: 8 total, 8 up, 8 in 2024-03-26T03:58:55.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:55 smithi175 bash[29284]: cluster 2024-03-26T03:58:54.947263+0000 mgr.y (mgr.24431) 562 : cluster [DBG] pgmap v889: 276 pgs: 276 active+clean; 582 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T03:58:55.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:55 smithi175 bash[29284]: cluster 2024-03-26T03:58:55.642462+0000 mon.a (mon.0) 2958 : cluster [DBG] osdmap e554: 8 total, 8 up, 8 in 2024-03-26T03:58:55.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:55 smithi175 bash[21527]: audit 2024-03-26T03:58:54.631833+0000 mon.a (mon.0) 2956 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-109", "tierpool": "test-rados-api-smithi175-64594-109-cache"}]': finished 2024-03-26T03:58:55.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:55 smithi175 bash[21527]: cluster 2024-03-26T03:58:54.639934+0000 mon.a (mon.0) 2957 : cluster [DBG] osdmap e553: 8 total, 8 up, 8 in 2024-03-26T03:58:55.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:55 smithi175 bash[21527]: cluster 2024-03-26T03:58:54.947263+0000 mgr.y (mgr.24431) 562 : cluster [DBG] pgmap v889: 276 pgs: 276 active+clean; 582 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T03:58:55.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:55 smithi175 bash[21527]: cluster 2024-03-26T03:58:55.642462+0000 mon.a (mon.0) 2958 : cluster [DBG] osdmap e554: 8 total, 8 up, 8 in 2024-03-26T03:58:56.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:56 smithi190 bash[23392]: cluster 2024-03-26T03:58:56.612879+0000 mon.a (mon.0) 2959 : cluster [WRN] Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:58:56.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:56 smithi190 bash[23392]: cluster 2024-03-26T03:58:56.659485+0000 mon.a (mon.0) 2960 : cluster [DBG] osdmap e555: 8 total, 8 up, 8 in 2024-03-26T03:58:56.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:56 smithi190 bash[23392]: audit 2024-03-26T03:58:56.662568+0000 mon.c (mon.2) 567 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:56.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:56 smithi190 bash[23392]: audit 2024-03-26T03:58:56.663700+0000 mon.a (mon.0) 2961 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:56.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:56 smithi175 bash[29284]: cluster 2024-03-26T03:58:56.612879+0000 mon.a (mon.0) 2959 : cluster [WRN] Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:58:56.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:56 smithi175 bash[29284]: cluster 2024-03-26T03:58:56.659485+0000 mon.a (mon.0) 2960 : cluster [DBG] osdmap e555: 8 total, 8 up, 8 in 2024-03-26T03:58:56.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:56 smithi175 bash[29284]: audit 2024-03-26T03:58:56.662568+0000 mon.c (mon.2) 567 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:56.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:56 smithi175 bash[29284]: audit 2024-03-26T03:58:56.663700+0000 mon.a (mon.0) 2961 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:56.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:56 smithi175 bash[21527]: cluster 2024-03-26T03:58:56.612879+0000 mon.a (mon.0) 2959 : cluster [WRN] Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:58:56.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:56 smithi175 bash[21527]: cluster 2024-03-26T03:58:56.659485+0000 mon.a (mon.0) 2960 : cluster [DBG] osdmap e555: 8 total, 8 up, 8 in 2024-03-26T03:58:56.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:56 smithi175 bash[21527]: audit 2024-03-26T03:58:56.662568+0000 mon.c (mon.2) 567 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:56.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:56 smithi175 bash[21527]: audit 2024-03-26T03:58:56.663700+0000 mon.a (mon.0) 2961 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:57.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:57 smithi190 bash[23392]: cluster 2024-03-26T03:58:56.948415+0000 mgr.y (mgr.24431) 563 : cluster [DBG] pgmap v892: 236 pgs: 236 active+clean; 582 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:57.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:57 smithi190 bash[23392]: audit 2024-03-26T03:58:57.655527+0000 mon.a (mon.0) 2962 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64594-109"}]': finished 2024-03-26T03:58:57.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:57 smithi190 bash[23392]: cluster 2024-03-26T03:58:57.662539+0000 mon.a (mon.0) 2963 : cluster [DBG] osdmap e556: 8 total, 8 up, 8 in 2024-03-26T03:58:57.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:57 smithi190 bash[23392]: audit 2024-03-26T03:58:57.666445+0000 mon.c (mon.2) 568 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:57.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:57 smithi190 bash[23392]: audit 2024-03-26T03:58:57.668053+0000 mon.a (mon.0) 2964 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:57.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:57 smithi175 bash[21527]: cluster 2024-03-26T03:58:56.948415+0000 mgr.y (mgr.24431) 563 : cluster [DBG] pgmap v892: 236 pgs: 236 active+clean; 582 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:57.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:57 smithi175 bash[21527]: audit 2024-03-26T03:58:57.655527+0000 mon.a (mon.0) 2962 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64594-109"}]': finished 2024-03-26T03:58:57.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:57 smithi175 bash[21527]: cluster 2024-03-26T03:58:57.662539+0000 mon.a (mon.0) 2963 : cluster [DBG] osdmap e556: 8 total, 8 up, 8 in 2024-03-26T03:58:57.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:57 smithi175 bash[21527]: audit 2024-03-26T03:58:57.666445+0000 mon.c (mon.2) 568 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:57.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:57 smithi175 bash[21527]: audit 2024-03-26T03:58:57.668053+0000 mon.a (mon.0) 2964 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:57.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:57 smithi175 bash[29284]: cluster 2024-03-26T03:58:56.948415+0000 mgr.y (mgr.24431) 563 : cluster [DBG] pgmap v892: 236 pgs: 236 active+clean; 582 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:58:57.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:57 smithi175 bash[29284]: audit 2024-03-26T03:58:57.655527+0000 mon.a (mon.0) 2962 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64594-109"}]': finished 2024-03-26T03:58:57.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:57 smithi175 bash[29284]: cluster 2024-03-26T03:58:57.662539+0000 mon.a (mon.0) 2963 : cluster [DBG] osdmap e556: 8 total, 8 up, 8 in 2024-03-26T03:58:57.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:57 smithi175 bash[29284]: audit 2024-03-26T03:58:57.666445+0000 mon.c (mon.2) 568 : audit [INF] from='client.? 172.21.15.175:0/4081036163' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:57.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:57 smithi175 bash[29284]: audit 2024-03-26T03:58:57.668053+0000 mon.a (mon.0) 2964 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64594-109"}]: dispatch 2024-03-26T03:58:59.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:58:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:58:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:58:59.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:59 smithi190 bash[23392]: audit 2024-03-26T03:58:58.659078+0000 mon.a (mon.0) 2965 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64594-109"}]': finished 2024-03-26T03:58:59.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:59 smithi190 bash[23392]: cluster 2024-03-26T03:58:58.666243+0000 mon.a (mon.0) 2966 : cluster [DBG] osdmap e557: 8 total, 8 up, 8 in 2024-03-26T03:58:59.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:58:59 smithi190 bash[23392]: cluster 2024-03-26T03:58:58.949253+0000 mgr.y (mgr.24431) 564 : cluster [DBG] pgmap v895: 236 pgs: 236 active+clean; 582 KiB data, 611 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:58:59.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:59 smithi175 bash[21527]: audit 2024-03-26T03:58:58.659078+0000 mon.a (mon.0) 2965 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64594-109"}]': finished 2024-03-26T03:58:59.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:59 smithi175 bash[21527]: cluster 2024-03-26T03:58:58.666243+0000 mon.a (mon.0) 2966 : cluster [DBG] osdmap e557: 8 total, 8 up, 8 in 2024-03-26T03:58:59.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:58:59 smithi175 bash[21527]: cluster 2024-03-26T03:58:58.949253+0000 mgr.y (mgr.24431) 564 : cluster [DBG] pgmap v895: 236 pgs: 236 active+clean; 582 KiB data, 611 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:58:59.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:59 smithi175 bash[29284]: audit 2024-03-26T03:58:58.659078+0000 mon.a (mon.0) 2965 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64594-109"}]': finished 2024-03-26T03:58:59.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:59 smithi175 bash[29284]: cluster 2024-03-26T03:58:58.666243+0000 mon.a (mon.0) 2966 : cluster [DBG] osdmap e557: 8 total, 8 up, 8 in 2024-03-26T03:58:59.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:58:59 smithi175 bash[29284]: cluster 2024-03-26T03:58:58.949253+0000 mgr.y (mgr.24431) 564 : cluster [DBG] pgmap v895: 236 pgs: 236 active+clean; 582 KiB data, 611 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:59:00.684 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:59:00 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:59:00.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:00 smithi190 bash[23392]: cluster 2024-03-26T03:58:59.674178+0000 mon.a (mon.0) 2967 : cluster [DBG] osdmap e558: 8 total, 8 up, 8 in 2024-03-26T03:59:00.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:00 smithi190 bash[23392]: audit 2024-03-26T03:58:59.679745+0000 mon.a (mon.0) 2968 : audit [INF] from='client.? 172.21.15.175:0/2594920259' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi175-64594-104"}]: dispatch 2024-03-26T03:59:00.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:00 smithi190 bash[23392]: audit 2024-03-26T03:59:00.302501+0000 mgr.y (mgr.24431) 565 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:59:00.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:00 smithi190 bash[23392]: audit 2024-03-26T03:59:00.665141+0000 mon.a (mon.0) 2969 : audit [INF] from='client.? 172.21.15.175:0/2594920259' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi175-64594-104"}]': finished 2024-03-26T03:59:00.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:00 smithi190 bash[23392]: cluster 2024-03-26T03:59:00.672135+0000 mon.a (mon.0) 2970 : cluster [DBG] osdmap e559: 8 total, 8 up, 8 in 2024-03-26T03:59:00.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:00 smithi190 bash[23392]: audit 2024-03-26T03:59:00.674188+0000 mon.a (mon.0) 2971 : audit [INF] from='client.? 172.21.15.175:0/2594920259' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi175-64594-104"}]: dispatch 2024-03-26T03:59:00.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:00 smithi175 bash[21527]: cluster 2024-03-26T03:58:59.674178+0000 mon.a (mon.0) 2967 : cluster [DBG] osdmap e558: 8 total, 8 up, 8 in 2024-03-26T03:59:00.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:00 smithi175 bash[21527]: audit 2024-03-26T03:58:59.679745+0000 mon.a (mon.0) 2968 : audit [INF] from='client.? 172.21.15.175:0/2594920259' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi175-64594-104"}]: dispatch 2024-03-26T03:59:00.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:00 smithi175 bash[21527]: audit 2024-03-26T03:59:00.302501+0000 mgr.y (mgr.24431) 565 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:59:00.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:00 smithi175 bash[21527]: audit 2024-03-26T03:59:00.665141+0000 mon.a (mon.0) 2969 : audit [INF] from='client.? 172.21.15.175:0/2594920259' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi175-64594-104"}]': finished 2024-03-26T03:59:00.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:00 smithi175 bash[21527]: cluster 2024-03-26T03:59:00.672135+0000 mon.a (mon.0) 2970 : cluster [DBG] osdmap e559: 8 total, 8 up, 8 in 2024-03-26T03:59:00.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:00 smithi175 bash[21527]: audit 2024-03-26T03:59:00.674188+0000 mon.a (mon.0) 2971 : audit [INF] from='client.? 172.21.15.175:0/2594920259' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi175-64594-104"}]: dispatch 2024-03-26T03:59:00.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:00 smithi175 bash[29284]: cluster 2024-03-26T03:58:59.674178+0000 mon.a (mon.0) 2967 : cluster [DBG] osdmap e558: 8 total, 8 up, 8 in 2024-03-26T03:59:00.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:00 smithi175 bash[29284]: audit 2024-03-26T03:58:59.679745+0000 mon.a (mon.0) 2968 : audit [INF] from='client.? 172.21.15.175:0/2594920259' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi175-64594-104"}]: dispatch 2024-03-26T03:59:00.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:00 smithi175 bash[29284]: audit 2024-03-26T03:59:00.302501+0000 mgr.y (mgr.24431) 565 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:59:00.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:00 smithi175 bash[29284]: audit 2024-03-26T03:59:00.665141+0000 mon.a (mon.0) 2969 : audit [INF] from='client.? 172.21.15.175:0/2594920259' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi175-64594-104"}]': finished 2024-03-26T03:59:00.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:00 smithi175 bash[29284]: cluster 2024-03-26T03:59:00.672135+0000 mon.a (mon.0) 2970 : cluster [DBG] osdmap e559: 8 total, 8 up, 8 in 2024-03-26T03:59:00.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:00 smithi175 bash[29284]: audit 2024-03-26T03:59:00.674188+0000 mon.a (mon.0) 2971 : audit [INF] from='client.? 172.21.15.175:0/2594920259' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi175-64594-104"}]: dispatch 2024-03-26T03:59:01.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:01 smithi190 bash[23392]: cluster 2024-03-26T03:59:00.950088+0000 mgr.y (mgr.24431) 566 : cluster [DBG] pgmap v898: 228 pgs: 228 active+clean; 582 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-26T03:59:01.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:01 smithi190 bash[23392]: cluster 2024-03-26T03:59:01.614057+0000 mon.a (mon.0) 2972 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:59:01.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:01 smithi190 bash[23392]: audit 2024-03-26T03:59:01.668619+0000 mon.a (mon.0) 2973 : audit [INF] from='client.? 172.21.15.175:0/2594920259' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi175-64594-104"}]': finished 2024-03-26T03:59:01.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:01 smithi190 bash[23392]: cluster 2024-03-26T03:59:01.675764+0000 mon.a (mon.0) 2974 : cluster [DBG] osdmap e560: 8 total, 8 up, 8 in 2024-03-26T03:59:01.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:01 smithi175 bash[21527]: cluster 2024-03-26T03:59:00.950088+0000 mgr.y (mgr.24431) 566 : cluster [DBG] pgmap v898: 228 pgs: 228 active+clean; 582 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-26T03:59:01.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:01 smithi175 bash[21527]: cluster 2024-03-26T03:59:01.614057+0000 mon.a (mon.0) 2972 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:59:01.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:01 smithi175 bash[21527]: audit 2024-03-26T03:59:01.668619+0000 mon.a (mon.0) 2973 : audit [INF] from='client.? 172.21.15.175:0/2594920259' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi175-64594-104"}]': finished 2024-03-26T03:59:01.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:01 smithi175 bash[21527]: cluster 2024-03-26T03:59:01.675764+0000 mon.a (mon.0) 2974 : cluster [DBG] osdmap e560: 8 total, 8 up, 8 in 2024-03-26T03:59:01.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:01 smithi175 bash[29284]: cluster 2024-03-26T03:59:00.950088+0000 mgr.y (mgr.24431) 566 : cluster [DBG] pgmap v898: 228 pgs: 228 active+clean; 582 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-26T03:59:01.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:01 smithi175 bash[29284]: cluster 2024-03-26T03:59:01.614057+0000 mon.a (mon.0) 2972 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:59:01.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:01 smithi175 bash[29284]: audit 2024-03-26T03:59:01.668619+0000 mon.a (mon.0) 2973 : audit [INF] from='client.? 172.21.15.175:0/2594920259' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi175-64594-104"}]': finished 2024-03-26T03:59:01.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:01 smithi175 bash[29284]: cluster 2024-03-26T03:59:01.675764+0000 mon.a (mon.0) 2974 : cluster [DBG] osdmap e560: 8 total, 8 up, 8 in 2024-03-26T03:59:02.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:02 smithi190 bash[23392]: audit 2024-03-26T03:59:01.725841+0000 mon.c (mon.2) 569 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:02.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:02 smithi190 bash[23392]: audit 2024-03-26T03:59:01.726204+0000 mon.a (mon.0) 2975 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:02.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:02 smithi190 bash[23392]: audit 2024-03-26T03:59:01.727299+0000 mon.c (mon.2) 570 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:02.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:02 smithi190 bash[23392]: audit 2024-03-26T03:59:01.727615+0000 mon.a (mon.0) 2976 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:02.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:02 smithi190 bash[23392]: audit 2024-03-26T03:59:01.728191+0000 mon.c (mon.2) 571 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi175-64594-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:59:02.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:02 smithi190 bash[23392]: audit 2024-03-26T03:59:01.728505+0000 mon.a (mon.0) 2977 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi175-64594-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:59:02.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:02 smithi175 bash[21527]: audit 2024-03-26T03:59:01.725841+0000 mon.c (mon.2) 569 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:02.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:02 smithi175 bash[21527]: audit 2024-03-26T03:59:01.726204+0000 mon.a (mon.0) 2975 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:02.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:02 smithi175 bash[21527]: audit 2024-03-26T03:59:01.727299+0000 mon.c (mon.2) 570 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:02.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:02 smithi175 bash[21527]: audit 2024-03-26T03:59:01.727615+0000 mon.a (mon.0) 2976 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:02.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:02 smithi175 bash[21527]: audit 2024-03-26T03:59:01.728191+0000 mon.c (mon.2) 571 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi175-64594-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:59:02.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:02 smithi175 bash[21527]: audit 2024-03-26T03:59:01.728505+0000 mon.a (mon.0) 2977 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi175-64594-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:59:02.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:02 smithi175 bash[29284]: audit 2024-03-26T03:59:01.725841+0000 mon.c (mon.2) 569 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:02.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:02 smithi175 bash[29284]: audit 2024-03-26T03:59:01.726204+0000 mon.a (mon.0) 2975 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:02.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:02 smithi175 bash[29284]: audit 2024-03-26T03:59:01.727299+0000 mon.c (mon.2) 570 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:02.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:02 smithi175 bash[29284]: audit 2024-03-26T03:59:01.727615+0000 mon.a (mon.0) 2976 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:02.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:02 smithi175 bash[29284]: audit 2024-03-26T03:59:01.728191+0000 mon.c (mon.2) 571 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi175-64594-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:59:02.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:02 smithi175 bash[29284]: audit 2024-03-26T03:59:01.728505+0000 mon.a (mon.0) 2977 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi175-64594-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T03:59:03.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:03 smithi190 bash[23392]: audit 2024-03-26T03:59:02.697057+0000 mon.a (mon.0) 2978 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi175-64594-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:59:03.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:03 smithi190 bash[23392]: cluster 2024-03-26T03:59:02.706139+0000 mon.a (mon.0) 2979 : cluster [DBG] osdmap e561: 8 total, 8 up, 8 in 2024-03-26T03:59:03.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:03 smithi190 bash[23392]: audit 2024-03-26T03:59:02.707257+0000 mon.c (mon.2) 572 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64594-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:03.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:03 smithi190 bash[23392]: audit 2024-03-26T03:59:02.709875+0000 mon.a (mon.0) 2980 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64594-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:03.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:03 smithi190 bash[23392]: cluster 2024-03-26T03:59:02.950952+0000 mgr.y (mgr.24431) 567 : cluster [DBG] pgmap v901: 228 pgs: 228 active+clean; 582 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-26T03:59:03.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:03 smithi175 bash[21527]: audit 2024-03-26T03:59:02.697057+0000 mon.a (mon.0) 2978 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi175-64594-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:59:03.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:03 smithi175 bash[21527]: cluster 2024-03-26T03:59:02.706139+0000 mon.a (mon.0) 2979 : cluster [DBG] osdmap e561: 8 total, 8 up, 8 in 2024-03-26T03:59:03.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:03 smithi175 bash[21527]: audit 2024-03-26T03:59:02.707257+0000 mon.c (mon.2) 572 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64594-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:03.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:03 smithi175 bash[21527]: audit 2024-03-26T03:59:02.709875+0000 mon.a (mon.0) 2980 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64594-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:03.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:03 smithi175 bash[21527]: cluster 2024-03-26T03:59:02.950952+0000 mgr.y (mgr.24431) 567 : cluster [DBG] pgmap v901: 228 pgs: 228 active+clean; 582 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-26T03:59:03.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:03 smithi175 bash[29284]: audit 2024-03-26T03:59:02.697057+0000 mon.a (mon.0) 2978 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi175-64594-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-26T03:59:03.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:03 smithi175 bash[29284]: cluster 2024-03-26T03:59:02.706139+0000 mon.a (mon.0) 2979 : cluster [DBG] osdmap e561: 8 total, 8 up, 8 in 2024-03-26T03:59:03.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:03 smithi175 bash[29284]: audit 2024-03-26T03:59:02.707257+0000 mon.c (mon.2) 572 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64594-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:03.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:03 smithi175 bash[29284]: audit 2024-03-26T03:59:02.709875+0000 mon.a (mon.0) 2980 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64594-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:03.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:03 smithi175 bash[29284]: cluster 2024-03-26T03:59:02.950952+0000 mgr.y (mgr.24431) 567 : cluster [DBG] pgmap v901: 228 pgs: 228 active+clean; 582 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-26T03:59:04.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:04 smithi190 bash[23392]: cluster 2024-03-26T03:59:03.721969+0000 mon.a (mon.0) 2981 : cluster [DBG] osdmap e562: 8 total, 8 up, 8 in 2024-03-26T03:59:04.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:04 smithi190 bash[23392]: audit 2024-03-26T03:59:04.703636+0000 mon.a (mon.0) 2982 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64594-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T03:59:04.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:04 smithi190 bash[23392]: cluster 2024-03-26T03:59:04.711948+0000 mon.a (mon.0) 2983 : cluster [DBG] osdmap e563: 8 total, 8 up, 8 in 2024-03-26T03:59:04.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:04 smithi175 bash[29284]: cluster 2024-03-26T03:59:03.721969+0000 mon.a (mon.0) 2981 : cluster [DBG] osdmap e562: 8 total, 8 up, 8 in 2024-03-26T03:59:04.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:04 smithi175 bash[29284]: audit 2024-03-26T03:59:04.703636+0000 mon.a (mon.0) 2982 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64594-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T03:59:04.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:04 smithi175 bash[29284]: cluster 2024-03-26T03:59:04.711948+0000 mon.a (mon.0) 2983 : cluster [DBG] osdmap e563: 8 total, 8 up, 8 in 2024-03-26T03:59:04.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:04 smithi175 bash[21527]: cluster 2024-03-26T03:59:03.721969+0000 mon.a (mon.0) 2981 : cluster [DBG] osdmap e562: 8 total, 8 up, 8 in 2024-03-26T03:59:04.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:04 smithi175 bash[21527]: audit 2024-03-26T03:59:04.703636+0000 mon.a (mon.0) 2982 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi175-64594-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T03:59:04.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:04 smithi175 bash[21527]: cluster 2024-03-26T03:59:04.711948+0000 mon.a (mon.0) 2983 : cluster [DBG] osdmap e563: 8 total, 8 up, 8 in 2024-03-26T03:59:06.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:05 smithi190 bash[23392]: cluster 2024-03-26T03:59:04.953262+0000 mgr.y (mgr.24431) 568 : cluster [DBG] pgmap v904: 236 pgs: 8 creating+peering, 228 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:59:06.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:05 smithi190 bash[23392]: audit 2024-03-26T03:59:04.957741+0000 mon.c (mon.2) 573 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:59:06.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:05 smithi190 bash[23392]: audit 2024-03-26T03:59:05.579539+0000 mon.a (mon.0) 2984 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:59:06.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:05 smithi190 bash[23392]: audit 2024-03-26T03:59:05.588219+0000 mon.c (mon.2) 574 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:59:06.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:05 smithi190 bash[23392]: audit 2024-03-26T03:59:05.699505+0000 mon.c (mon.2) 575 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:59:06.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:05 smithi190 bash[23392]: audit 2024-03-26T03:59:05.700898+0000 mon.c (mon.2) 576 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:59:06.243 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:05 smithi190 bash[23392]: cluster 2024-03-26T03:59:05.711129+0000 mon.a (mon.0) 2985 : cluster [DBG] osdmap e564: 8 total, 8 up, 8 in 2024-03-26T03:59:06.243 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:05 smithi190 bash[23392]: audit 2024-03-26T03:59:05.717359+0000 mon.c (mon.2) 577 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:59:06.243 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:05 smithi190 bash[23392]: audit 2024-03-26T03:59:05.730968+0000 mon.a (mon.0) 2986 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:59:06.243 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:05 smithi190 bash[23392]: audit 2024-03-26T03:59:05.736515+0000 mon.a (mon.0) 2987 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:59:06.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:05 smithi175 bash[21527]: cluster 2024-03-26T03:59:04.953262+0000 mgr.y (mgr.24431) 568 : cluster [DBG] pgmap v904: 236 pgs: 8 creating+peering, 228 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:59:06.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:05 smithi175 bash[21527]: audit 2024-03-26T03:59:04.957741+0000 mon.c (mon.2) 573 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:59:06.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:05 smithi175 bash[21527]: audit 2024-03-26T03:59:05.579539+0000 mon.a (mon.0) 2984 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:59:06.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:05 smithi175 bash[21527]: audit 2024-03-26T03:59:05.588219+0000 mon.c (mon.2) 574 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:59:06.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:05 smithi175 bash[21527]: audit 2024-03-26T03:59:05.699505+0000 mon.c (mon.2) 575 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:59:06.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:05 smithi175 bash[21527]: audit 2024-03-26T03:59:05.700898+0000 mon.c (mon.2) 576 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:59:06.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:05 smithi175 bash[21527]: cluster 2024-03-26T03:59:05.711129+0000 mon.a (mon.0) 2985 : cluster [DBG] osdmap e564: 8 total, 8 up, 8 in 2024-03-26T03:59:06.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:05 smithi175 bash[21527]: audit 2024-03-26T03:59:05.717359+0000 mon.c (mon.2) 577 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:59:06.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:05 smithi175 bash[21527]: audit 2024-03-26T03:59:05.730968+0000 mon.a (mon.0) 2986 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:59:06.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:05 smithi175 bash[21527]: audit 2024-03-26T03:59:05.736515+0000 mon.a (mon.0) 2987 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:59:06.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:05 smithi175 bash[29284]: cluster 2024-03-26T03:59:04.953262+0000 mgr.y (mgr.24431) 568 : cluster [DBG] pgmap v904: 236 pgs: 8 creating+peering, 228 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:59:06.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:05 smithi175 bash[29284]: audit 2024-03-26T03:59:04.957741+0000 mon.c (mon.2) 573 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T03:59:06.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:05 smithi175 bash[29284]: audit 2024-03-26T03:59:05.579539+0000 mon.a (mon.0) 2984 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:59:06.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:05 smithi175 bash[29284]: audit 2024-03-26T03:59:05.588219+0000 mon.c (mon.2) 574 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:59:06.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:05 smithi175 bash[29284]: audit 2024-03-26T03:59:05.699505+0000 mon.c (mon.2) 575 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T03:59:06.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:05 smithi175 bash[29284]: audit 2024-03-26T03:59:05.700898+0000 mon.c (mon.2) 576 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T03:59:06.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:05 smithi175 bash[29284]: cluster 2024-03-26T03:59:05.711129+0000 mon.a (mon.0) 2985 : cluster [DBG] osdmap e564: 8 total, 8 up, 8 in 2024-03-26T03:59:06.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:05 smithi175 bash[29284]: audit 2024-03-26T03:59:05.717359+0000 mon.c (mon.2) 577 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:59:06.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:05 smithi175 bash[29284]: audit 2024-03-26T03:59:05.730968+0000 mon.a (mon.0) 2986 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:59:06.245 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:05 smithi175 bash[29284]: audit 2024-03-26T03:59:05.736515+0000 mon.a (mon.0) 2987 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:59:07.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:07 smithi190 bash[23392]: audit 2024-03-26T03:59:06.711032+0000 mon.a (mon.0) 2988 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:59:07.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:07 smithi190 bash[23392]: cluster 2024-03-26T03:59:06.721630+0000 mon.a (mon.0) 2989 : cluster [DBG] osdmap e565: 8 total, 8 up, 8 in 2024-03-26T03:59:07.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:07 smithi190 bash[23392]: audit 2024-03-26T03:59:06.857327+0000 mon.c (mon.2) 578 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:59:07.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:07 smithi190 bash[23392]: audit 2024-03-26T03:59:06.857969+0000 mon.a (mon.0) 2990 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:59:07.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:07 smithi190 bash[23392]: cluster 2024-03-26T03:59:06.954212+0000 mgr.y (mgr.24431) 569 : cluster [DBG] pgmap v907: 268 pgs: 32 unknown, 8 creating+peering, 228 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:59:07.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:07 smithi175 bash[21527]: audit 2024-03-26T03:59:06.711032+0000 mon.a (mon.0) 2988 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:59:07.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:07 smithi175 bash[21527]: cluster 2024-03-26T03:59:06.721630+0000 mon.a (mon.0) 2989 : cluster [DBG] osdmap e565: 8 total, 8 up, 8 in 2024-03-26T03:59:07.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:07 smithi175 bash[21527]: audit 2024-03-26T03:59:06.857327+0000 mon.c (mon.2) 578 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:59:07.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:07 smithi175 bash[21527]: audit 2024-03-26T03:59:06.857969+0000 mon.a (mon.0) 2990 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:59:07.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:07 smithi175 bash[21527]: cluster 2024-03-26T03:59:06.954212+0000 mgr.y (mgr.24431) 569 : cluster [DBG] pgmap v907: 268 pgs: 32 unknown, 8 creating+peering, 228 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:59:07.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:07 smithi175 bash[29284]: audit 2024-03-26T03:59:06.711032+0000 mon.a (mon.0) 2988 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:59:07.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:07 smithi175 bash[29284]: cluster 2024-03-26T03:59:06.721630+0000 mon.a (mon.0) 2989 : cluster [DBG] osdmap e565: 8 total, 8 up, 8 in 2024-03-26T03:59:07.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:07 smithi175 bash[29284]: audit 2024-03-26T03:59:06.857327+0000 mon.c (mon.2) 578 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:59:07.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:07 smithi175 bash[29284]: audit 2024-03-26T03:59:06.857969+0000 mon.a (mon.0) 2990 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:59:07.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:07 smithi175 bash[29284]: cluster 2024-03-26T03:59:06.954212+0000 mgr.y (mgr.24431) 569 : cluster [DBG] pgmap v907: 268 pgs: 32 unknown, 8 creating+peering, 228 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:59:08.992 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:59:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:59:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:59:08.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:08 smithi190 bash[23392]: audit 2024-03-26T03:59:07.713753+0000 mon.a (mon.0) 2991 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-112", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:59:08.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:08 smithi190 bash[23392]: cluster 2024-03-26T03:59:07.721027+0000 mon.a (mon.0) 2992 : cluster [DBG] osdmap e566: 8 total, 8 up, 8 in 2024-03-26T03:59:08.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:08 smithi190 bash[23392]: audit 2024-03-26T03:59:07.725407+0000 mon.c (mon.2) 579 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-112"}]: dispatch 2024-03-26T03:59:08.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:08 smithi190 bash[23392]: audit 2024-03-26T03:59:07.727119+0000 mon.a (mon.0) 2993 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-112"}]: dispatch 2024-03-26T03:59:08.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:08 smithi190 bash[23392]: audit 2024-03-26T03:59:08.716693+0000 mon.a (mon.0) 2994 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-112"}]': finished 2024-03-26T03:59:08.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:08 smithi190 bash[23392]: cluster 2024-03-26T03:59:08.724769+0000 mon.a (mon.0) 2995 : cluster [DBG] osdmap e567: 8 total, 8 up, 8 in 2024-03-26T03:59:08.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:08 smithi175 bash[29284]: audit 2024-03-26T03:59:07.713753+0000 mon.a (mon.0) 2991 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-112", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:59:08.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:08 smithi175 bash[29284]: cluster 2024-03-26T03:59:07.721027+0000 mon.a (mon.0) 2992 : cluster [DBG] osdmap e566: 8 total, 8 up, 8 in 2024-03-26T03:59:08.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:08 smithi175 bash[29284]: audit 2024-03-26T03:59:07.725407+0000 mon.c (mon.2) 579 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-112"}]: dispatch 2024-03-26T03:59:08.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:08 smithi175 bash[29284]: audit 2024-03-26T03:59:07.727119+0000 mon.a (mon.0) 2993 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-112"}]: dispatch 2024-03-26T03:59:08.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:08 smithi175 bash[29284]: audit 2024-03-26T03:59:08.716693+0000 mon.a (mon.0) 2994 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-112"}]': finished 2024-03-26T03:59:08.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:08 smithi175 bash[29284]: cluster 2024-03-26T03:59:08.724769+0000 mon.a (mon.0) 2995 : cluster [DBG] osdmap e567: 8 total, 8 up, 8 in 2024-03-26T03:59:08.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:08 smithi175 bash[21527]: audit 2024-03-26T03:59:07.713753+0000 mon.a (mon.0) 2991 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-112", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:59:08.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:08 smithi175 bash[21527]: cluster 2024-03-26T03:59:07.721027+0000 mon.a (mon.0) 2992 : cluster [DBG] osdmap e566: 8 total, 8 up, 8 in 2024-03-26T03:59:08.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:08 smithi175 bash[21527]: audit 2024-03-26T03:59:07.725407+0000 mon.c (mon.2) 579 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-112"}]: dispatch 2024-03-26T03:59:08.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:08 smithi175 bash[21527]: audit 2024-03-26T03:59:07.727119+0000 mon.a (mon.0) 2993 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-112"}]: dispatch 2024-03-26T03:59:08.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:08 smithi175 bash[21527]: audit 2024-03-26T03:59:08.716693+0000 mon.a (mon.0) 2994 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-112"}]': finished 2024-03-26T03:59:08.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:08 smithi175 bash[21527]: cluster 2024-03-26T03:59:08.724769+0000 mon.a (mon.0) 2995 : cluster [DBG] osdmap e567: 8 total, 8 up, 8 in 2024-03-26T03:59:09.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:09 smithi190 bash[23392]: audit 2024-03-26T03:59:08.808237+0000 mon.c (mon.2) 580 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:09.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:09 smithi190 bash[23392]: audit 2024-03-26T03:59:08.808896+0000 mon.a (mon.0) 2996 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:09.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:09 smithi190 bash[23392]: cluster 2024-03-26T03:59:08.955140+0000 mgr.y (mgr.24431) 570 : cluster [DBG] pgmap v910: 268 pgs: 32 unknown, 8 creating+peering, 228 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:59:09.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:09 smithi175 bash[29284]: audit 2024-03-26T03:59:08.808237+0000 mon.c (mon.2) 580 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:09.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:09 smithi175 bash[29284]: audit 2024-03-26T03:59:08.808896+0000 mon.a (mon.0) 2996 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:09.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:09 smithi175 bash[29284]: cluster 2024-03-26T03:59:08.955140+0000 mgr.y (mgr.24431) 570 : cluster [DBG] pgmap v910: 268 pgs: 32 unknown, 8 creating+peering, 228 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:59:09.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:09 smithi175 bash[21527]: audit 2024-03-26T03:59:08.808237+0000 mon.c (mon.2) 580 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:09.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:09 smithi175 bash[21527]: audit 2024-03-26T03:59:08.808896+0000 mon.a (mon.0) 2996 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:09.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:09 smithi175 bash[21527]: cluster 2024-03-26T03:59:08.955140+0000 mgr.y (mgr.24431) 570 : cluster [DBG] pgmap v910: 268 pgs: 32 unknown, 8 creating+peering, 228 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:59:10.742 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:59:10 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:59:11.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:10 smithi190 bash[23392]: audit 2024-03-26T03:59:09.746519+0000 mon.a (mon.0) 2997 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T03:59:11.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:10 smithi190 bash[23392]: cluster 2024-03-26T03:59:09.756898+0000 mon.a (mon.0) 2998 : cluster [DBG] osdmap e568: 8 total, 8 up, 8 in 2024-03-26T03:59:11.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:10 smithi190 bash[23392]: audit 2024-03-26T03:59:09.762453+0000 mon.c (mon.2) 581 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-112"}]: dispatch 2024-03-26T03:59:11.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:10 smithi190 bash[23392]: audit 2024-03-26T03:59:09.763390+0000 mon.a (mon.0) 2999 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-112"}]: dispatch 2024-03-26T03:59:11.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:10 smithi190 bash[23392]: audit 2024-03-26T03:59:10.313896+0000 mgr.y (mgr.24431) 571 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:59:11.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:10 smithi175 bash[29284]: audit 2024-03-26T03:59:09.746519+0000 mon.a (mon.0) 2997 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T03:59:11.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:10 smithi175 bash[29284]: cluster 2024-03-26T03:59:09.756898+0000 mon.a (mon.0) 2998 : cluster [DBG] osdmap e568: 8 total, 8 up, 8 in 2024-03-26T03:59:11.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:10 smithi175 bash[29284]: audit 2024-03-26T03:59:09.762453+0000 mon.c (mon.2) 581 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-112"}]: dispatch 2024-03-26T03:59:11.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:10 smithi175 bash[29284]: audit 2024-03-26T03:59:09.763390+0000 mon.a (mon.0) 2999 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-112"}]: dispatch 2024-03-26T03:59:11.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:10 smithi175 bash[29284]: audit 2024-03-26T03:59:10.313896+0000 mgr.y (mgr.24431) 571 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:59:11.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:10 smithi175 bash[21527]: audit 2024-03-26T03:59:09.746519+0000 mon.a (mon.0) 2997 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T03:59:11.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:10 smithi175 bash[21527]: cluster 2024-03-26T03:59:09.756898+0000 mon.a (mon.0) 2998 : cluster [DBG] osdmap e568: 8 total, 8 up, 8 in 2024-03-26T03:59:11.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:10 smithi175 bash[21527]: audit 2024-03-26T03:59:09.762453+0000 mon.c (mon.2) 581 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-112"}]: dispatch 2024-03-26T03:59:11.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:10 smithi175 bash[21527]: audit 2024-03-26T03:59:09.763390+0000 mon.a (mon.0) 2999 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-112"}]: dispatch 2024-03-26T03:59:11.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:10 smithi175 bash[21527]: audit 2024-03-26T03:59:10.313896+0000 mgr.y (mgr.24431) 571 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:59:12.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:11 smithi190 bash[23392]: audit 2024-03-26T03:59:10.749306+0000 mon.a (mon.0) 3000 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-112"}]': finished 2024-03-26T03:59:12.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:11 smithi190 bash[23392]: cluster 2024-03-26T03:59:10.758871+0000 mon.a (mon.0) 3001 : cluster [DBG] osdmap e569: 8 total, 8 up, 8 in 2024-03-26T03:59:12.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:11 smithi190 bash[23392]: cluster 2024-03-26T03:59:10.956092+0000 mgr.y (mgr.24431) 572 : cluster [DBG] pgmap v913: 268 pgs: 268 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-26T03:59:12.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:11 smithi175 bash[29284]: audit 2024-03-26T03:59:10.749306+0000 mon.a (mon.0) 3000 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-112"}]': finished 2024-03-26T03:59:12.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:11 smithi175 bash[29284]: cluster 2024-03-26T03:59:10.758871+0000 mon.a (mon.0) 3001 : cluster [DBG] osdmap e569: 8 total, 8 up, 8 in 2024-03-26T03:59:12.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:11 smithi175 bash[29284]: cluster 2024-03-26T03:59:10.956092+0000 mgr.y (mgr.24431) 572 : cluster [DBG] pgmap v913: 268 pgs: 268 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-26T03:59:12.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:11 smithi175 bash[21527]: audit 2024-03-26T03:59:10.749306+0000 mon.a (mon.0) 3000 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-112"}]': finished 2024-03-26T03:59:12.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:11 smithi175 bash[21527]: cluster 2024-03-26T03:59:10.758871+0000 mon.a (mon.0) 3001 : cluster [DBG] osdmap e569: 8 total, 8 up, 8 in 2024-03-26T03:59:12.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:11 smithi175 bash[21527]: cluster 2024-03-26T03:59:10.956092+0000 mgr.y (mgr.24431) 572 : cluster [DBG] pgmap v913: 268 pgs: 268 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-26T03:59:13.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:12 smithi190 bash[23392]: cluster 2024-03-26T03:59:11.775909+0000 mon.a (mon.0) 3002 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:59:13.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:12 smithi190 bash[23392]: cluster 2024-03-26T03:59:11.796442+0000 mon.a (mon.0) 3003 : cluster [DBG] osdmap e570: 8 total, 8 up, 8 in 2024-03-26T03:59:13.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:12 smithi175 bash[29284]: cluster 2024-03-26T03:59:11.775909+0000 mon.a (mon.0) 3002 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:59:13.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:12 smithi175 bash[29284]: cluster 2024-03-26T03:59:11.796442+0000 mon.a (mon.0) 3003 : cluster [DBG] osdmap e570: 8 total, 8 up, 8 in 2024-03-26T03:59:13.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:12 smithi175 bash[21527]: cluster 2024-03-26T03:59:11.775909+0000 mon.a (mon.0) 3002 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:59:13.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:12 smithi175 bash[21527]: cluster 2024-03-26T03:59:11.796442+0000 mon.a (mon.0) 3003 : cluster [DBG] osdmap e570: 8 total, 8 up, 8 in 2024-03-26T03:59:14.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:13 smithi190 bash[23392]: cluster 2024-03-26T03:59:12.809714+0000 mon.a (mon.0) 3004 : cluster [DBG] osdmap e571: 8 total, 8 up, 8 in 2024-03-26T03:59:14.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:13 smithi190 bash[23392]: audit 2024-03-26T03:59:12.813531+0000 mon.c (mon.2) 582 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:59:14.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:13 smithi190 bash[23392]: audit 2024-03-26T03:59:12.815176+0000 mon.a (mon.0) 3005 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:59:14.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:13 smithi190 bash[23392]: cluster 2024-03-26T03:59:12.956958+0000 mgr.y (mgr.24431) 573 : cluster [DBG] pgmap v916: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:59:14.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:13 smithi175 bash[21527]: cluster 2024-03-26T03:59:12.809714+0000 mon.a (mon.0) 3004 : cluster [DBG] osdmap e571: 8 total, 8 up, 8 in 2024-03-26T03:59:14.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:13 smithi175 bash[21527]: audit 2024-03-26T03:59:12.813531+0000 mon.c (mon.2) 582 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:59:14.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:13 smithi175 bash[21527]: audit 2024-03-26T03:59:12.815176+0000 mon.a (mon.0) 3005 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:59:14.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:13 smithi175 bash[21527]: cluster 2024-03-26T03:59:12.956958+0000 mgr.y (mgr.24431) 573 : cluster [DBG] pgmap v916: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:59:14.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:13 smithi175 bash[29284]: cluster 2024-03-26T03:59:12.809714+0000 mon.a (mon.0) 3004 : cluster [DBG] osdmap e571: 8 total, 8 up, 8 in 2024-03-26T03:59:14.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:13 smithi175 bash[29284]: audit 2024-03-26T03:59:12.813531+0000 mon.c (mon.2) 582 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:59:14.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:13 smithi175 bash[29284]: audit 2024-03-26T03:59:12.815176+0000 mon.a (mon.0) 3005 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:59:14.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:13 smithi175 bash[29284]: cluster 2024-03-26T03:59:12.956958+0000 mgr.y (mgr.24431) 573 : cluster [DBG] pgmap v916: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:59:15.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:14 smithi190 bash[23392]: audit 2024-03-26T03:59:13.810757+0000 mon.a (mon.0) 3006 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:59:15.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:14 smithi190 bash[23392]: cluster 2024-03-26T03:59:13.822807+0000 mon.a (mon.0) 3007 : cluster [DBG] osdmap e572: 8 total, 8 up, 8 in 2024-03-26T03:59:15.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:14 smithi190 bash[23392]: audit 2024-03-26T03:59:13.862543+0000 mon.c (mon.2) 583 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:59:15.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:14 smithi190 bash[23392]: audit 2024-03-26T03:59:13.864860+0000 mon.a (mon.0) 3008 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:59:15.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:14 smithi175 bash[21527]: audit 2024-03-26T03:59:13.810757+0000 mon.a (mon.0) 3006 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:59:15.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:14 smithi175 bash[21527]: cluster 2024-03-26T03:59:13.822807+0000 mon.a (mon.0) 3007 : cluster [DBG] osdmap e572: 8 total, 8 up, 8 in 2024-03-26T03:59:15.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:14 smithi175 bash[21527]: audit 2024-03-26T03:59:13.862543+0000 mon.c (mon.2) 583 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:59:15.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:14 smithi175 bash[21527]: audit 2024-03-26T03:59:13.864860+0000 mon.a (mon.0) 3008 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:59:15.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:14 smithi175 bash[29284]: audit 2024-03-26T03:59:13.810757+0000 mon.a (mon.0) 3006 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:59:15.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:14 smithi175 bash[29284]: cluster 2024-03-26T03:59:13.822807+0000 mon.a (mon.0) 3007 : cluster [DBG] osdmap e572: 8 total, 8 up, 8 in 2024-03-26T03:59:15.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:14 smithi175 bash[29284]: audit 2024-03-26T03:59:13.862543+0000 mon.c (mon.2) 583 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:59:15.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:14 smithi175 bash[29284]: audit 2024-03-26T03:59:13.864860+0000 mon.a (mon.0) 3008 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:59:16.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:15 smithi190 bash[23392]: audit 2024-03-26T03:59:14.824755+0000 mon.a (mon.0) 3009 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-114", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:59:16.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:15 smithi190 bash[23392]: cluster 2024-03-26T03:59:14.834180+0000 mon.a (mon.0) 3010 : cluster [DBG] osdmap e573: 8 total, 8 up, 8 in 2024-03-26T03:59:16.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:15 smithi190 bash[23392]: audit 2024-03-26T03:59:14.838035+0000 mon.c (mon.2) 584 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-114"}]: dispatch 2024-03-26T03:59:16.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:15 smithi190 bash[23392]: audit 2024-03-26T03:59:14.839497+0000 mon.a (mon.0) 3011 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-114"}]: dispatch 2024-03-26T03:59:16.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:15 smithi190 bash[23392]: cluster 2024-03-26T03:59:14.958389+0000 mgr.y (mgr.24431) 574 : cluster [DBG] pgmap v919: 268 pgs: 268 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T03:59:16.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:15 smithi175 bash[21527]: audit 2024-03-26T03:59:14.824755+0000 mon.a (mon.0) 3009 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-114", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:59:16.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:15 smithi175 bash[21527]: cluster 2024-03-26T03:59:14.834180+0000 mon.a (mon.0) 3010 : cluster [DBG] osdmap e573: 8 total, 8 up, 8 in 2024-03-26T03:59:16.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:15 smithi175 bash[21527]: audit 2024-03-26T03:59:14.838035+0000 mon.c (mon.2) 584 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-114"}]: dispatch 2024-03-26T03:59:16.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:15 smithi175 bash[21527]: audit 2024-03-26T03:59:14.839497+0000 mon.a (mon.0) 3011 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-114"}]: dispatch 2024-03-26T03:59:16.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:15 smithi175 bash[21527]: cluster 2024-03-26T03:59:14.958389+0000 mgr.y (mgr.24431) 574 : cluster [DBG] pgmap v919: 268 pgs: 268 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T03:59:16.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:15 smithi175 bash[29284]: audit 2024-03-26T03:59:14.824755+0000 mon.a (mon.0) 3009 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-114", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:59:16.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:15 smithi175 bash[29284]: cluster 2024-03-26T03:59:14.834180+0000 mon.a (mon.0) 3010 : cluster [DBG] osdmap e573: 8 total, 8 up, 8 in 2024-03-26T03:59:16.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:15 smithi175 bash[29284]: audit 2024-03-26T03:59:14.838035+0000 mon.c (mon.2) 584 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-114"}]: dispatch 2024-03-26T03:59:16.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:15 smithi175 bash[29284]: audit 2024-03-26T03:59:14.839497+0000 mon.a (mon.0) 3011 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-114"}]: dispatch 2024-03-26T03:59:16.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:15 smithi175 bash[29284]: cluster 2024-03-26T03:59:14.958389+0000 mgr.y (mgr.24431) 574 : cluster [DBG] pgmap v919: 268 pgs: 268 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T03:59:17.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:16 smithi190 bash[23392]: audit 2024-03-26T03:59:15.836083+0000 mon.a (mon.0) 3012 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-114"}]': finished 2024-03-26T03:59:17.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:16 smithi190 bash[23392]: cluster 2024-03-26T03:59:15.846435+0000 mon.a (mon.0) 3013 : cluster [DBG] osdmap e574: 8 total, 8 up, 8 in 2024-03-26T03:59:17.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:16 smithi190 bash[23392]: audit 2024-03-26T03:59:15.847622+0000 mon.c (mon.2) 585 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-114", "mode": "writeback"}]: dispatch 2024-03-26T03:59:17.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:16 smithi190 bash[23392]: audit 2024-03-26T03:59:15.848627+0000 mon.a (mon.0) 3014 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-114", "mode": "writeback"}]: dispatch 2024-03-26T03:59:17.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:16 smithi175 bash[29284]: audit 2024-03-26T03:59:15.836083+0000 mon.a (mon.0) 3012 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-114"}]': finished 2024-03-26T03:59:17.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:16 smithi175 bash[29284]: cluster 2024-03-26T03:59:15.846435+0000 mon.a (mon.0) 3013 : cluster [DBG] osdmap e574: 8 total, 8 up, 8 in 2024-03-26T03:59:17.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:16 smithi175 bash[29284]: audit 2024-03-26T03:59:15.847622+0000 mon.c (mon.2) 585 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-114", "mode": "writeback"}]: dispatch 2024-03-26T03:59:17.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:16 smithi175 bash[29284]: audit 2024-03-26T03:59:15.848627+0000 mon.a (mon.0) 3014 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-114", "mode": "writeback"}]: dispatch 2024-03-26T03:59:17.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:16 smithi175 bash[21527]: audit 2024-03-26T03:59:15.836083+0000 mon.a (mon.0) 3012 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-114"}]': finished 2024-03-26T03:59:17.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:16 smithi175 bash[21527]: cluster 2024-03-26T03:59:15.846435+0000 mon.a (mon.0) 3013 : cluster [DBG] osdmap e574: 8 total, 8 up, 8 in 2024-03-26T03:59:17.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:16 smithi175 bash[21527]: audit 2024-03-26T03:59:15.847622+0000 mon.c (mon.2) 585 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-114", "mode": "writeback"}]: dispatch 2024-03-26T03:59:17.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:16 smithi175 bash[21527]: audit 2024-03-26T03:59:15.848627+0000 mon.a (mon.0) 3014 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-114", "mode": "writeback"}]: dispatch 2024-03-26T03:59:18.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:17 smithi190 bash[23392]: cluster 2024-03-26T03:59:16.836456+0000 mon.a (mon.0) 3015 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:59:18.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:17 smithi190 bash[23392]: audit 2024-03-26T03:59:16.838952+0000 mon.a (mon.0) 3016 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-114", "mode": "writeback"}]': finished 2024-03-26T03:59:18.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:17 smithi190 bash[23392]: cluster 2024-03-26T03:59:16.846570+0000 mon.a (mon.0) 3017 : cluster [DBG] osdmap e575: 8 total, 8 up, 8 in 2024-03-26T03:59:18.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:17 smithi190 bash[23392]: audit 2024-03-26T03:59:16.956641+0000 mon.c (mon.2) 586 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:18.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:17 smithi190 bash[23392]: audit 2024-03-26T03:59:16.957351+0000 mon.a (mon.0) 3018 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:18.243 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:17 smithi190 bash[23392]: cluster 2024-03-26T03:59:16.959439+0000 mgr.y (mgr.24431) 575 : cluster [DBG] pgmap v922: 268 pgs: 268 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:59:18.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:17 smithi175 bash[29284]: cluster 2024-03-26T03:59:16.836456+0000 mon.a (mon.0) 3015 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:59:18.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:17 smithi175 bash[29284]: audit 2024-03-26T03:59:16.838952+0000 mon.a (mon.0) 3016 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-114", "mode": "writeback"}]': finished 2024-03-26T03:59:18.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:17 smithi175 bash[29284]: cluster 2024-03-26T03:59:16.846570+0000 mon.a (mon.0) 3017 : cluster [DBG] osdmap e575: 8 total, 8 up, 8 in 2024-03-26T03:59:18.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:17 smithi175 bash[29284]: audit 2024-03-26T03:59:16.956641+0000 mon.c (mon.2) 586 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:18.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:17 smithi175 bash[29284]: audit 2024-03-26T03:59:16.957351+0000 mon.a (mon.0) 3018 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:18.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:17 smithi175 bash[29284]: cluster 2024-03-26T03:59:16.959439+0000 mgr.y (mgr.24431) 575 : cluster [DBG] pgmap v922: 268 pgs: 268 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:59:18.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:17 smithi175 bash[21527]: cluster 2024-03-26T03:59:16.836456+0000 mon.a (mon.0) 3015 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:59:18.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:17 smithi175 bash[21527]: audit 2024-03-26T03:59:16.838952+0000 mon.a (mon.0) 3016 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-114", "mode": "writeback"}]': finished 2024-03-26T03:59:18.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:17 smithi175 bash[21527]: cluster 2024-03-26T03:59:16.846570+0000 mon.a (mon.0) 3017 : cluster [DBG] osdmap e575: 8 total, 8 up, 8 in 2024-03-26T03:59:18.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:17 smithi175 bash[21527]: audit 2024-03-26T03:59:16.956641+0000 mon.c (mon.2) 586 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:18.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:17 smithi175 bash[21527]: audit 2024-03-26T03:59:16.957351+0000 mon.a (mon.0) 3018 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:18.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:17 smithi175 bash[21527]: cluster 2024-03-26T03:59:16.959439+0000 mgr.y (mgr.24431) 575 : cluster [DBG] pgmap v922: 268 pgs: 268 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T03:59:19.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:18 smithi190 bash[23392]: audit 2024-03-26T03:59:17.865377+0000 mon.a (mon.0) 3019 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T03:59:19.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:18 smithi190 bash[23392]: cluster 2024-03-26T03:59:17.871446+0000 mon.a (mon.0) 3020 : cluster [DBG] osdmap e576: 8 total, 8 up, 8 in 2024-03-26T03:59:19.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:18 smithi190 bash[23392]: audit 2024-03-26T03:59:17.876988+0000 mon.c (mon.2) 587 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-114"}]: dispatch 2024-03-26T03:59:19.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:18 smithi190 bash[23392]: audit 2024-03-26T03:59:17.878189+0000 mon.a (mon.0) 3021 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-114"}]: dispatch 2024-03-26T03:59:19.243 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:59:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:59:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:59:19.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:18 smithi175 bash[29284]: audit 2024-03-26T03:59:17.865377+0000 mon.a (mon.0) 3019 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T03:59:19.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:18 smithi175 bash[29284]: cluster 2024-03-26T03:59:17.871446+0000 mon.a (mon.0) 3020 : cluster [DBG] osdmap e576: 8 total, 8 up, 8 in 2024-03-26T03:59:19.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:18 smithi175 bash[29284]: audit 2024-03-26T03:59:17.876988+0000 mon.c (mon.2) 587 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-114"}]: dispatch 2024-03-26T03:59:19.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:18 smithi175 bash[29284]: audit 2024-03-26T03:59:17.878189+0000 mon.a (mon.0) 3021 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-114"}]: dispatch 2024-03-26T03:59:19.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:18 smithi175 bash[21527]: audit 2024-03-26T03:59:17.865377+0000 mon.a (mon.0) 3019 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T03:59:19.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:18 smithi175 bash[21527]: cluster 2024-03-26T03:59:17.871446+0000 mon.a (mon.0) 3020 : cluster [DBG] osdmap e576: 8 total, 8 up, 8 in 2024-03-26T03:59:19.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:18 smithi175 bash[21527]: audit 2024-03-26T03:59:17.876988+0000 mon.c (mon.2) 587 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-114"}]: dispatch 2024-03-26T03:59:19.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:18 smithi175 bash[21527]: audit 2024-03-26T03:59:17.878189+0000 mon.a (mon.0) 3021 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-114"}]: dispatch 2024-03-26T03:59:20.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:19 smithi190 bash[23392]: cluster 2024-03-26T03:59:18.865766+0000 mon.a (mon.0) 3022 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:59:20.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:19 smithi190 bash[23392]: audit 2024-03-26T03:59:18.868271+0000 mon.a (mon.0) 3023 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-114"}]': finished 2024-03-26T03:59:20.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:19 smithi190 bash[23392]: cluster 2024-03-26T03:59:18.879709+0000 mon.a (mon.0) 3024 : cluster [DBG] osdmap e577: 8 total, 8 up, 8 in 2024-03-26T03:59:20.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:19 smithi190 bash[23392]: cluster 2024-03-26T03:59:18.960443+0000 mgr.y (mgr.24431) 576 : cluster [DBG] pgmap v925: 268 pgs: 268 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:59:20.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:19 smithi175 bash[21527]: cluster 2024-03-26T03:59:18.865766+0000 mon.a (mon.0) 3022 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:59:20.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:19 smithi175 bash[21527]: audit 2024-03-26T03:59:18.868271+0000 mon.a (mon.0) 3023 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-114"}]': finished 2024-03-26T03:59:20.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:19 smithi175 bash[21527]: cluster 2024-03-26T03:59:18.879709+0000 mon.a (mon.0) 3024 : cluster [DBG] osdmap e577: 8 total, 8 up, 8 in 2024-03-26T03:59:20.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:19 smithi175 bash[21527]: cluster 2024-03-26T03:59:18.960443+0000 mgr.y (mgr.24431) 576 : cluster [DBG] pgmap v925: 268 pgs: 268 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:59:20.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:19 smithi175 bash[29284]: cluster 2024-03-26T03:59:18.865766+0000 mon.a (mon.0) 3022 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:59:20.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:19 smithi175 bash[29284]: audit 2024-03-26T03:59:18.868271+0000 mon.a (mon.0) 3023 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-114"}]': finished 2024-03-26T03:59:20.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:19 smithi175 bash[29284]: cluster 2024-03-26T03:59:18.879709+0000 mon.a (mon.0) 3024 : cluster [DBG] osdmap e577: 8 total, 8 up, 8 in 2024-03-26T03:59:20.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:19 smithi175 bash[29284]: cluster 2024-03-26T03:59:18.960443+0000 mgr.y (mgr.24431) 576 : cluster [DBG] pgmap v925: 268 pgs: 268 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:59:20.742 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:59:20 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:59:21.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:20 smithi190 bash[23392]: cluster 2024-03-26T03:59:19.883819+0000 mon.a (mon.0) 3025 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:59:21.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:20 smithi190 bash[23392]: cluster 2024-03-26T03:59:19.903094+0000 mon.a (mon.0) 3026 : cluster [DBG] osdmap e578: 8 total, 8 up, 8 in 2024-03-26T03:59:21.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:20 smithi190 bash[23392]: audit 2024-03-26T03:59:20.322140+0000 mgr.y (mgr.24431) 577 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:59:21.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:20 smithi190 bash[23392]: audit 2024-03-26T03:59:20.612389+0000 mon.c (mon.2) 588 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:59:21.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:20 smithi175 bash[21527]: cluster 2024-03-26T03:59:19.883819+0000 mon.a (mon.0) 3025 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:59:21.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:20 smithi175 bash[21527]: cluster 2024-03-26T03:59:19.903094+0000 mon.a (mon.0) 3026 : cluster [DBG] osdmap e578: 8 total, 8 up, 8 in 2024-03-26T03:59:21.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:20 smithi175 bash[21527]: audit 2024-03-26T03:59:20.322140+0000 mgr.y (mgr.24431) 577 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:59:21.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:20 smithi175 bash[21527]: audit 2024-03-26T03:59:20.612389+0000 mon.c (mon.2) 588 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:59:21.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:20 smithi175 bash[29284]: cluster 2024-03-26T03:59:19.883819+0000 mon.a (mon.0) 3025 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:59:21.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:20 smithi175 bash[29284]: cluster 2024-03-26T03:59:19.903094+0000 mon.a (mon.0) 3026 : cluster [DBG] osdmap e578: 8 total, 8 up, 8 in 2024-03-26T03:59:21.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:20 smithi175 bash[29284]: audit 2024-03-26T03:59:20.322140+0000 mgr.y (mgr.24431) 577 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:59:21.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:20 smithi175 bash[29284]: audit 2024-03-26T03:59:20.612389+0000 mon.c (mon.2) 588 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:59:22.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:21 smithi190 bash[23392]: cluster 2024-03-26T03:59:20.916733+0000 mon.a (mon.0) 3027 : cluster [DBG] osdmap e579: 8 total, 8 up, 8 in 2024-03-26T03:59:22.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:21 smithi190 bash[23392]: audit 2024-03-26T03:59:20.922356+0000 mon.c (mon.2) 589 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:59:22.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:21 smithi190 bash[23392]: audit 2024-03-26T03:59:20.923436+0000 mon.a (mon.0) 3028 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:59:22.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:21 smithi190 bash[23392]: cluster 2024-03-26T03:59:20.961444+0000 mgr.y (mgr.24431) 578 : cluster [DBG] pgmap v928: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:59:22.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:21 smithi175 bash[29284]: cluster 2024-03-26T03:59:20.916733+0000 mon.a (mon.0) 3027 : cluster [DBG] osdmap e579: 8 total, 8 up, 8 in 2024-03-26T03:59:22.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:21 smithi175 bash[29284]: audit 2024-03-26T03:59:20.922356+0000 mon.c (mon.2) 589 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:59:22.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:21 smithi175 bash[29284]: audit 2024-03-26T03:59:20.923436+0000 mon.a (mon.0) 3028 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:59:22.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:21 smithi175 bash[29284]: cluster 2024-03-26T03:59:20.961444+0000 mgr.y (mgr.24431) 578 : cluster [DBG] pgmap v928: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:59:22.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:21 smithi175 bash[21527]: cluster 2024-03-26T03:59:20.916733+0000 mon.a (mon.0) 3027 : cluster [DBG] osdmap e579: 8 total, 8 up, 8 in 2024-03-26T03:59:22.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:21 smithi175 bash[21527]: audit 2024-03-26T03:59:20.922356+0000 mon.c (mon.2) 589 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:59:22.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:21 smithi175 bash[21527]: audit 2024-03-26T03:59:20.923436+0000 mon.a (mon.0) 3028 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:59:22.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:21 smithi175 bash[21527]: cluster 2024-03-26T03:59:20.961444+0000 mgr.y (mgr.24431) 578 : cluster [DBG] pgmap v928: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:59:23.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:22 smithi190 bash[23392]: audit 2024-03-26T03:59:21.920798+0000 mon.a (mon.0) 3029 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:59:23.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:22 smithi190 bash[23392]: cluster 2024-03-26T03:59:21.932013+0000 mon.a (mon.0) 3030 : cluster [DBG] osdmap e580: 8 total, 8 up, 8 in 2024-03-26T03:59:23.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:22 smithi175 bash[21527]: audit 2024-03-26T03:59:21.920798+0000 mon.a (mon.0) 3029 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:59:23.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:22 smithi175 bash[21527]: cluster 2024-03-26T03:59:21.932013+0000 mon.a (mon.0) 3030 : cluster [DBG] osdmap e580: 8 total, 8 up, 8 in 2024-03-26T03:59:23.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:22 smithi175 bash[29284]: audit 2024-03-26T03:59:21.920798+0000 mon.a (mon.0) 3029 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:59:23.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:22 smithi175 bash[29284]: cluster 2024-03-26T03:59:21.932013+0000 mon.a (mon.0) 3030 : cluster [DBG] osdmap e580: 8 total, 8 up, 8 in 2024-03-26T03:59:24.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:23 smithi190 bash[23392]: cluster 2024-03-26T03:59:22.932643+0000 mon.a (mon.0) 3031 : cluster [DBG] osdmap e581: 8 total, 8 up, 8 in 2024-03-26T03:59:24.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:23 smithi190 bash[23392]: cluster 2024-03-26T03:59:22.962418+0000 mgr.y (mgr.24431) 579 : cluster [DBG] pgmap v931: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:59:24.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:23 smithi190 bash[23392]: audit 2024-03-26T03:59:22.970744+0000 mon.c (mon.2) 590 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:59:24.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:23 smithi190 bash[23392]: audit 2024-03-26T03:59:22.971607+0000 mon.a (mon.0) 3032 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:59:24.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:23 smithi175 bash[29284]: cluster 2024-03-26T03:59:22.932643+0000 mon.a (mon.0) 3031 : cluster [DBG] osdmap e581: 8 total, 8 up, 8 in 2024-03-26T03:59:24.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:23 smithi175 bash[29284]: cluster 2024-03-26T03:59:22.962418+0000 mgr.y (mgr.24431) 579 : cluster [DBG] pgmap v931: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:59:24.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:23 smithi175 bash[29284]: audit 2024-03-26T03:59:22.970744+0000 mon.c (mon.2) 590 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:59:24.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:23 smithi175 bash[29284]: audit 2024-03-26T03:59:22.971607+0000 mon.a (mon.0) 3032 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:59:24.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:23 smithi175 bash[21527]: cluster 2024-03-26T03:59:22.932643+0000 mon.a (mon.0) 3031 : cluster [DBG] osdmap e581: 8 total, 8 up, 8 in 2024-03-26T03:59:24.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:23 smithi175 bash[21527]: cluster 2024-03-26T03:59:22.962418+0000 mgr.y (mgr.24431) 579 : cluster [DBG] pgmap v931: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:59:24.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:23 smithi175 bash[21527]: audit 2024-03-26T03:59:22.970744+0000 mon.c (mon.2) 590 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:59:24.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:23 smithi175 bash[21527]: audit 2024-03-26T03:59:22.971607+0000 mon.a (mon.0) 3032 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:59:25.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:24 smithi190 bash[23392]: audit 2024-03-26T03:59:23.939858+0000 mon.a (mon.0) 3033 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-116", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:59:25.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:24 smithi190 bash[23392]: cluster 2024-03-26T03:59:23.948788+0000 mon.a (mon.0) 3034 : cluster [DBG] osdmap e582: 8 total, 8 up, 8 in 2024-03-26T03:59:25.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:24 smithi190 bash[23392]: audit 2024-03-26T03:59:23.952075+0000 mon.c (mon.2) 591 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-116"}]: dispatch 2024-03-26T03:59:25.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:24 smithi190 bash[23392]: audit 2024-03-26T03:59:23.966561+0000 mon.a (mon.0) 3035 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-116"}]: dispatch 2024-03-26T03:59:25.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:24 smithi175 bash[21527]: audit 2024-03-26T03:59:23.939858+0000 mon.a (mon.0) 3033 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-116", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:59:25.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:24 smithi175 bash[21527]: cluster 2024-03-26T03:59:23.948788+0000 mon.a (mon.0) 3034 : cluster [DBG] osdmap e582: 8 total, 8 up, 8 in 2024-03-26T03:59:25.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:24 smithi175 bash[21527]: audit 2024-03-26T03:59:23.952075+0000 mon.c (mon.2) 591 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-116"}]: dispatch 2024-03-26T03:59:25.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:24 smithi175 bash[21527]: audit 2024-03-26T03:59:23.966561+0000 mon.a (mon.0) 3035 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-116"}]: dispatch 2024-03-26T03:59:25.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:24 smithi175 bash[29284]: audit 2024-03-26T03:59:23.939858+0000 mon.a (mon.0) 3033 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-116", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:59:25.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:24 smithi175 bash[29284]: cluster 2024-03-26T03:59:23.948788+0000 mon.a (mon.0) 3034 : cluster [DBG] osdmap e582: 8 total, 8 up, 8 in 2024-03-26T03:59:25.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:24 smithi175 bash[29284]: audit 2024-03-26T03:59:23.952075+0000 mon.c (mon.2) 591 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-116"}]: dispatch 2024-03-26T03:59:25.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:24 smithi175 bash[29284]: audit 2024-03-26T03:59:23.966561+0000 mon.a (mon.0) 3035 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-116"}]: dispatch 2024-03-26T03:59:26.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:25 smithi190 bash[23392]: cluster 2024-03-26T03:59:24.965062+0000 mgr.y (mgr.24431) 580 : cluster [DBG] pgmap v933: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-26T03:59:26.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:25 smithi190 bash[23392]: audit 2024-03-26T03:59:24.971717+0000 mon.a (mon.0) 3036 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-116"}]': finished 2024-03-26T03:59:26.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:25 smithi190 bash[23392]: cluster 2024-03-26T03:59:24.978790+0000 mon.a (mon.0) 3037 : cluster [DBG] osdmap e583: 8 total, 8 up, 8 in 2024-03-26T03:59:26.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:25 smithi190 bash[23392]: audit 2024-03-26T03:59:24.982672+0000 mon.c (mon.2) 592 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-116", "mode": "writeback"}]: dispatch 2024-03-26T03:59:26.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:25 smithi190 bash[23392]: audit 2024-03-26T03:59:24.984229+0000 mon.a (mon.0) 3038 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-116", "mode": "writeback"}]: dispatch 2024-03-26T03:59:26.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:25 smithi175 bash[21527]: cluster 2024-03-26T03:59:24.965062+0000 mgr.y (mgr.24431) 580 : cluster [DBG] pgmap v933: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-26T03:59:26.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:25 smithi175 bash[21527]: audit 2024-03-26T03:59:24.971717+0000 mon.a (mon.0) 3036 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-116"}]': finished 2024-03-26T03:59:26.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:25 smithi175 bash[21527]: cluster 2024-03-26T03:59:24.978790+0000 mon.a (mon.0) 3037 : cluster [DBG] osdmap e583: 8 total, 8 up, 8 in 2024-03-26T03:59:26.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:25 smithi175 bash[21527]: audit 2024-03-26T03:59:24.982672+0000 mon.c (mon.2) 592 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-116", "mode": "writeback"}]: dispatch 2024-03-26T03:59:26.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:25 smithi175 bash[21527]: audit 2024-03-26T03:59:24.984229+0000 mon.a (mon.0) 3038 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-116", "mode": "writeback"}]: dispatch 2024-03-26T03:59:26.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:25 smithi175 bash[29284]: cluster 2024-03-26T03:59:24.965062+0000 mgr.y (mgr.24431) 580 : cluster [DBG] pgmap v933: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-26T03:59:26.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:25 smithi175 bash[29284]: audit 2024-03-26T03:59:24.971717+0000 mon.a (mon.0) 3036 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-116"}]': finished 2024-03-26T03:59:26.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:25 smithi175 bash[29284]: cluster 2024-03-26T03:59:24.978790+0000 mon.a (mon.0) 3037 : cluster [DBG] osdmap e583: 8 total, 8 up, 8 in 2024-03-26T03:59:26.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:25 smithi175 bash[29284]: audit 2024-03-26T03:59:24.982672+0000 mon.c (mon.2) 592 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-116", "mode": "writeback"}]: dispatch 2024-03-26T03:59:26.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:25 smithi175 bash[29284]: audit 2024-03-26T03:59:24.984229+0000 mon.a (mon.0) 3038 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-116", "mode": "writeback"}]: dispatch 2024-03-26T03:59:27.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:27 smithi190 bash[23392]: cluster 2024-03-26T03:59:25.972027+0000 mon.a (mon.0) 3039 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:59:27.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:27 smithi190 bash[23392]: audit 2024-03-26T03:59:25.974322+0000 mon.a (mon.0) 3040 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-116", "mode": "writeback"}]': finished 2024-03-26T03:59:27.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:27 smithi190 bash[23392]: cluster 2024-03-26T03:59:25.980961+0000 mon.a (mon.0) 3041 : cluster [DBG] osdmap e584: 8 total, 8 up, 8 in 2024-03-26T03:59:27.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:27 smithi190 bash[23392]: audit 2024-03-26T03:59:26.024786+0000 mon.c (mon.2) 593 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "319.5"}]: dispatch 2024-03-26T03:59:27.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:27 smithi190 bash[23392]: audit 2024-03-26T03:59:26.025250+0000 mgr.y (mgr.24431) 581 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "319.5"}]: dispatch 2024-03-26T03:59:27.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:27 smithi190 bash[23392]: cluster 2024-03-26T03:59:26.620701+0000 mon.a (mon.0) 3042 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:59:27.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:26 smithi175 bash[21527]: cluster 2024-03-26T03:59:25.972027+0000 mon.a (mon.0) 3039 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:59:27.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:26 smithi175 bash[21527]: audit 2024-03-26T03:59:25.974322+0000 mon.a (mon.0) 3040 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-116", "mode": "writeback"}]': finished 2024-03-26T03:59:27.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:26 smithi175 bash[21527]: cluster 2024-03-26T03:59:25.980961+0000 mon.a (mon.0) 3041 : cluster [DBG] osdmap e584: 8 total, 8 up, 8 in 2024-03-26T03:59:27.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:26 smithi175 bash[21527]: audit 2024-03-26T03:59:26.024786+0000 mon.c (mon.2) 593 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "319.5"}]: dispatch 2024-03-26T03:59:27.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:26 smithi175 bash[21527]: audit 2024-03-26T03:59:26.025250+0000 mgr.y (mgr.24431) 581 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "319.5"}]: dispatch 2024-03-26T03:59:27.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:26 smithi175 bash[21527]: cluster 2024-03-26T03:59:26.620701+0000 mon.a (mon.0) 3042 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:59:27.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:26 smithi175 bash[29284]: cluster 2024-03-26T03:59:25.972027+0000 mon.a (mon.0) 3039 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:59:27.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:26 smithi175 bash[29284]: audit 2024-03-26T03:59:25.974322+0000 mon.a (mon.0) 3040 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-116", "mode": "writeback"}]': finished 2024-03-26T03:59:27.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:26 smithi175 bash[29284]: cluster 2024-03-26T03:59:25.980961+0000 mon.a (mon.0) 3041 : cluster [DBG] osdmap e584: 8 total, 8 up, 8 in 2024-03-26T03:59:27.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:26 smithi175 bash[29284]: audit 2024-03-26T03:59:26.024786+0000 mon.c (mon.2) 593 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "319.5"}]: dispatch 2024-03-26T03:59:27.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:26 smithi175 bash[29284]: audit 2024-03-26T03:59:26.025250+0000 mgr.y (mgr.24431) 581 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "319.5"}]: dispatch 2024-03-26T03:59:27.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:26 smithi175 bash[29284]: cluster 2024-03-26T03:59:26.620701+0000 mon.a (mon.0) 3042 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:59:28.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:28 smithi190 bash[23392]: cluster 2024-03-26T03:59:26.269581+0000 osd.0 (osd.0) 15 : cluster [DBG] 319.5 scrub starts 2024-03-26T03:59:28.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:28 smithi190 bash[23392]: cluster 2024-03-26T03:59:26.273873+0000 osd.0 (osd.0) 16 : cluster [DBG] 319.5 scrub ok 2024-03-26T03:59:28.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:28 smithi190 bash[23392]: cluster 2024-03-26T03:59:26.965952+0000 mgr.y (mgr.24431) 582 : cluster [DBG] pgmap v936: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-26T03:59:28.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:28 smithi175 bash[29284]: cluster 2024-03-26T03:59:26.269581+0000 osd.0 (osd.0) 15 : cluster [DBG] 319.5 scrub starts 2024-03-26T03:59:28.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:28 smithi175 bash[29284]: cluster 2024-03-26T03:59:26.273873+0000 osd.0 (osd.0) 16 : cluster [DBG] 319.5 scrub ok 2024-03-26T03:59:28.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:28 smithi175 bash[29284]: cluster 2024-03-26T03:59:26.965952+0000 mgr.y (mgr.24431) 582 : cluster [DBG] pgmap v936: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-26T03:59:28.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:27 smithi175 bash[21527]: cluster 2024-03-26T03:59:26.269581+0000 osd.0 (osd.0) 15 : cluster [DBG] 319.5 scrub starts 2024-03-26T03:59:28.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:27 smithi175 bash[21527]: cluster 2024-03-26T03:59:26.273873+0000 osd.0 (osd.0) 16 : cluster [DBG] 319.5 scrub ok 2024-03-26T03:59:28.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:27 smithi175 bash[21527]: cluster 2024-03-26T03:59:26.965952+0000 mgr.y (mgr.24431) 582 : cluster [DBG] pgmap v936: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-26T03:59:29.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:59:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:59:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:59:30.331 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:30 smithi190 bash[23392]: cluster 2024-03-26T03:59:28.966877+0000 mgr.y (mgr.24431) 583 : cluster [DBG] pgmap v937: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-26T03:59:30.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:30 smithi175 bash[29284]: cluster 2024-03-26T03:59:28.966877+0000 mgr.y (mgr.24431) 583 : cluster [DBG] pgmap v937: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-26T03:59:30.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:30 smithi175 bash[21527]: cluster 2024-03-26T03:59:28.966877+0000 mgr.y (mgr.24431) 583 : cluster [DBG] pgmap v937: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-26T03:59:30.741 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:59:30 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:59:31.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:31 smithi190 bash[23392]: audit 2024-03-26T03:59:30.333449+0000 mgr.y (mgr.24431) 584 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:59:31.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:31 smithi175 bash[21527]: audit 2024-03-26T03:59:30.333449+0000 mgr.y (mgr.24431) 584 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:59:31.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:31 smithi175 bash[29284]: audit 2024-03-26T03:59:30.333449+0000 mgr.y (mgr.24431) 584 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:59:32.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:32 smithi190 bash[23392]: cluster 2024-03-26T03:59:30.969450+0000 mgr.y (mgr.24431) 585 : cluster [DBG] pgmap v938: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1020 B/s wr, 3 op/s 2024-03-26T03:59:32.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:32 smithi175 bash[21527]: cluster 2024-03-26T03:59:30.969450+0000 mgr.y (mgr.24431) 585 : cluster [DBG] pgmap v938: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1020 B/s wr, 3 op/s 2024-03-26T03:59:32.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:32 smithi175 bash[29284]: cluster 2024-03-26T03:59:30.969450+0000 mgr.y (mgr.24431) 585 : cluster [DBG] pgmap v938: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1020 B/s wr, 3 op/s 2024-03-26T03:59:34.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:34 smithi190 bash[23392]: cluster 2024-03-26T03:59:32.970346+0000 mgr.y (mgr.24431) 586 : cluster [DBG] pgmap v939: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 1 op/s 2024-03-26T03:59:34.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:34 smithi175 bash[21527]: cluster 2024-03-26T03:59:32.970346+0000 mgr.y (mgr.24431) 586 : cluster [DBG] pgmap v939: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 1 op/s 2024-03-26T03:59:34.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:34 smithi175 bash[29284]: cluster 2024-03-26T03:59:32.970346+0000 mgr.y (mgr.24431) 586 : cluster [DBG] pgmap v939: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 1 op/s 2024-03-26T03:59:36.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:36 smithi190 bash[23392]: cluster 2024-03-26T03:59:34.972833+0000 mgr.y (mgr.24431) 587 : cluster [DBG] pgmap v940: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:59:36.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:36 smithi190 bash[23392]: audit 2024-03-26T03:59:35.655211+0000 mon.a (mon.0) 3043 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:59:36.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:36 smithi190 bash[23392]: audit 2024-03-26T03:59:35.658223+0000 mon.c (mon.2) 594 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:59:36.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:36 smithi175 bash[29284]: cluster 2024-03-26T03:59:34.972833+0000 mgr.y (mgr.24431) 587 : cluster [DBG] pgmap v940: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:59:36.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:36 smithi175 bash[29284]: audit 2024-03-26T03:59:35.655211+0000 mon.a (mon.0) 3043 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:59:36.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:36 smithi175 bash[29284]: audit 2024-03-26T03:59:35.658223+0000 mon.c (mon.2) 594 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:59:36.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:36 smithi175 bash[21527]: cluster 2024-03-26T03:59:34.972833+0000 mgr.y (mgr.24431) 587 : cluster [DBG] pgmap v940: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:59:36.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:36 smithi175 bash[21527]: audit 2024-03-26T03:59:35.655211+0000 mon.a (mon.0) 3043 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T03:59:36.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:36 smithi175 bash[21527]: audit 2024-03-26T03:59:35.658223+0000 mon.c (mon.2) 594 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:59:38.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:38 smithi190 bash[23392]: cluster 2024-03-26T03:59:36.973675+0000 mgr.y (mgr.24431) 588 : cluster [DBG] pgmap v941: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-26T03:59:38.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:38 smithi175 bash[21527]: cluster 2024-03-26T03:59:36.973675+0000 mgr.y (mgr.24431) 588 : cluster [DBG] pgmap v941: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-26T03:59:38.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:38 smithi175 bash[29284]: cluster 2024-03-26T03:59:36.973675+0000 mgr.y (mgr.24431) 588 : cluster [DBG] pgmap v941: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-26T03:59:39.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:59:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:59:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:59:40.340 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:40 smithi190 bash[23392]: cluster 2024-03-26T03:59:38.974546+0000 mgr.y (mgr.24431) 589 : cluster [DBG] pgmap v942: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-03-26T03:59:40.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:40 smithi175 bash[21527]: cluster 2024-03-26T03:59:38.974546+0000 mgr.y (mgr.24431) 589 : cluster [DBG] pgmap v942: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-03-26T03:59:40.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:40 smithi175 bash[29284]: cluster 2024-03-26T03:59:38.974546+0000 mgr.y (mgr.24431) 589 : cluster [DBG] pgmap v942: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-03-26T03:59:40.741 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:59:40 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:59:41.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:41 smithi190 bash[23392]: audit 2024-03-26T03:59:40.342145+0000 mgr.y (mgr.24431) 590 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:59:41.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:41 smithi175 bash[29284]: audit 2024-03-26T03:59:40.342145+0000 mgr.y (mgr.24431) 590 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:59:41.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:41 smithi175 bash[21527]: audit 2024-03-26T03:59:40.342145+0000 mgr.y (mgr.24431) 590 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:59:42.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:42 smithi190 bash[23392]: cluster 2024-03-26T03:59:40.977072+0000 mgr.y (mgr.24431) 591 : cluster [DBG] pgmap v943: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-26T03:59:42.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:42 smithi175 bash[21527]: cluster 2024-03-26T03:59:40.977072+0000 mgr.y (mgr.24431) 591 : cluster [DBG] pgmap v943: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-26T03:59:42.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:42 smithi175 bash[29284]: cluster 2024-03-26T03:59:40.977072+0000 mgr.y (mgr.24431) 591 : cluster [DBG] pgmap v943: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-26T03:59:43.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:43 smithi190 bash[23392]: cluster 2024-03-26T03:59:42.064317+0000 mon.a (mon.0) 3044 : cluster [DBG] osdmap e585: 8 total, 8 up, 8 in 2024-03-26T03:59:43.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:43 smithi190 bash[23392]: audit 2024-03-26T03:59:42.132853+0000 mon.c (mon.2) 595 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:43.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:43 smithi190 bash[23392]: audit 2024-03-26T03:59:42.133526+0000 mon.a (mon.0) 3045 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:43.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:43 smithi175 bash[21527]: cluster 2024-03-26T03:59:42.064317+0000 mon.a (mon.0) 3044 : cluster [DBG] osdmap e585: 8 total, 8 up, 8 in 2024-03-26T03:59:43.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:43 smithi175 bash[21527]: audit 2024-03-26T03:59:42.132853+0000 mon.c (mon.2) 595 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:43.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:43 smithi175 bash[21527]: audit 2024-03-26T03:59:42.133526+0000 mon.a (mon.0) 3045 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:43.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:43 smithi175 bash[29284]: cluster 2024-03-26T03:59:42.064317+0000 mon.a (mon.0) 3044 : cluster [DBG] osdmap e585: 8 total, 8 up, 8 in 2024-03-26T03:59:43.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:43 smithi175 bash[29284]: audit 2024-03-26T03:59:42.132853+0000 mon.c (mon.2) 595 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:43.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:43 smithi175 bash[29284]: audit 2024-03-26T03:59:42.133526+0000 mon.a (mon.0) 3045 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:44.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:44 smithi190 bash[23392]: cluster 2024-03-26T03:59:42.977973+0000 mgr.y (mgr.24431) 592 : cluster [DBG] pgmap v945: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T03:59:44.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:44 smithi190 bash[23392]: audit 2024-03-26T03:59:43.064462+0000 mon.a (mon.0) 3046 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T03:59:44.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:44 smithi190 bash[23392]: cluster 2024-03-26T03:59:43.077245+0000 mon.a (mon.0) 3047 : cluster [DBG] osdmap e586: 8 total, 8 up, 8 in 2024-03-26T03:59:44.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:44 smithi190 bash[23392]: audit 2024-03-26T03:59:43.079870+0000 mon.c (mon.2) 596 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-116"}]: dispatch 2024-03-26T03:59:44.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:44 smithi190 bash[23392]: audit 2024-03-26T03:59:43.080822+0000 mon.a (mon.0) 3048 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-116"}]: dispatch 2024-03-26T03:59:44.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:44 smithi175 bash[29284]: cluster 2024-03-26T03:59:42.977973+0000 mgr.y (mgr.24431) 592 : cluster [DBG] pgmap v945: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T03:59:44.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:44 smithi175 bash[29284]: audit 2024-03-26T03:59:43.064462+0000 mon.a (mon.0) 3046 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T03:59:44.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:44 smithi175 bash[29284]: cluster 2024-03-26T03:59:43.077245+0000 mon.a (mon.0) 3047 : cluster [DBG] osdmap e586: 8 total, 8 up, 8 in 2024-03-26T03:59:44.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:44 smithi175 bash[29284]: audit 2024-03-26T03:59:43.079870+0000 mon.c (mon.2) 596 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-116"}]: dispatch 2024-03-26T03:59:44.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:44 smithi175 bash[29284]: audit 2024-03-26T03:59:43.080822+0000 mon.a (mon.0) 3048 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-116"}]: dispatch 2024-03-26T03:59:44.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:44 smithi175 bash[21527]: cluster 2024-03-26T03:59:42.977973+0000 mgr.y (mgr.24431) 592 : cluster [DBG] pgmap v945: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T03:59:44.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:44 smithi175 bash[21527]: audit 2024-03-26T03:59:43.064462+0000 mon.a (mon.0) 3046 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T03:59:44.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:44 smithi175 bash[21527]: cluster 2024-03-26T03:59:43.077245+0000 mon.a (mon.0) 3047 : cluster [DBG] osdmap e586: 8 total, 8 up, 8 in 2024-03-26T03:59:44.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:44 smithi175 bash[21527]: audit 2024-03-26T03:59:43.079870+0000 mon.c (mon.2) 596 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-116"}]: dispatch 2024-03-26T03:59:44.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:44 smithi175 bash[21527]: audit 2024-03-26T03:59:43.080822+0000 mon.a (mon.0) 3048 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-116"}]: dispatch 2024-03-26T03:59:45.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:45 smithi190 bash[23392]: cluster 2024-03-26T03:59:44.064834+0000 mon.a (mon.0) 3049 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:59:45.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:45 smithi190 bash[23392]: audit 2024-03-26T03:59:44.067222+0000 mon.a (mon.0) 3050 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-116"}]': finished 2024-03-26T03:59:45.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:45 smithi190 bash[23392]: cluster 2024-03-26T03:59:44.074543+0000 mon.a (mon.0) 3051 : cluster [DBG] osdmap e587: 8 total, 8 up, 8 in 2024-03-26T03:59:45.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:45 smithi175 bash[21527]: cluster 2024-03-26T03:59:44.064834+0000 mon.a (mon.0) 3049 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:59:45.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:45 smithi175 bash[21527]: audit 2024-03-26T03:59:44.067222+0000 mon.a (mon.0) 3050 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-116"}]': finished 2024-03-26T03:59:45.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:45 smithi175 bash[21527]: cluster 2024-03-26T03:59:44.074543+0000 mon.a (mon.0) 3051 : cluster [DBG] osdmap e587: 8 total, 8 up, 8 in 2024-03-26T03:59:45.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:45 smithi175 bash[29284]: cluster 2024-03-26T03:59:44.064834+0000 mon.a (mon.0) 3049 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:59:45.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:45 smithi175 bash[29284]: audit 2024-03-26T03:59:44.067222+0000 mon.a (mon.0) 3050 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-116"}]': finished 2024-03-26T03:59:45.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:45 smithi175 bash[29284]: cluster 2024-03-26T03:59:44.074543+0000 mon.a (mon.0) 3051 : cluster [DBG] osdmap e587: 8 total, 8 up, 8 in 2024-03-26T03:59:46.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:46 smithi190 bash[23392]: cluster 2024-03-26T03:59:44.980466+0000 mgr.y (mgr.24431) 593 : cluster [DBG] pgmap v948: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-26T03:59:46.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:46 smithi190 bash[23392]: cluster 2024-03-26T03:59:45.079322+0000 mon.a (mon.0) 3052 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:59:46.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:46 smithi190 bash[23392]: cluster 2024-03-26T03:59:45.098364+0000 mon.a (mon.0) 3053 : cluster [DBG] osdmap e588: 8 total, 8 up, 8 in 2024-03-26T03:59:46.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:46 smithi175 bash[21527]: cluster 2024-03-26T03:59:44.980466+0000 mgr.y (mgr.24431) 593 : cluster [DBG] pgmap v948: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-26T03:59:46.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:46 smithi175 bash[21527]: cluster 2024-03-26T03:59:45.079322+0000 mon.a (mon.0) 3052 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:59:46.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:46 smithi175 bash[21527]: cluster 2024-03-26T03:59:45.098364+0000 mon.a (mon.0) 3053 : cluster [DBG] osdmap e588: 8 total, 8 up, 8 in 2024-03-26T03:59:46.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:46 smithi175 bash[29284]: cluster 2024-03-26T03:59:44.980466+0000 mgr.y (mgr.24431) 593 : cluster [DBG] pgmap v948: 268 pgs: 268 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-26T03:59:46.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:46 smithi175 bash[29284]: cluster 2024-03-26T03:59:45.079322+0000 mon.a (mon.0) 3052 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:59:46.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:46 smithi175 bash[29284]: cluster 2024-03-26T03:59:45.098364+0000 mon.a (mon.0) 3053 : cluster [DBG] osdmap e588: 8 total, 8 up, 8 in 2024-03-26T03:59:47.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:47 smithi190 bash[23392]: cluster 2024-03-26T03:59:46.105270+0000 mon.a (mon.0) 3054 : cluster [DBG] osdmap e589: 8 total, 8 up, 8 in 2024-03-26T03:59:47.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:47 smithi190 bash[23392]: audit 2024-03-26T03:59:46.112724+0000 mon.c (mon.2) 597 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:59:47.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:47 smithi190 bash[23392]: audit 2024-03-26T03:59:46.118841+0000 mon.a (mon.0) 3055 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:59:47.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:47 smithi190 bash[23392]: audit 2024-03-26T03:59:46.628945+0000 mon.a (mon.0) 3056 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:59:47.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:47 smithi190 bash[23392]: cluster 2024-03-26T03:59:46.636884+0000 mon.a (mon.0) 3057 : cluster [DBG] osdmap e590: 8 total, 8 up, 8 in 2024-03-26T03:59:47.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:47 smithi175 bash[21527]: cluster 2024-03-26T03:59:46.105270+0000 mon.a (mon.0) 3054 : cluster [DBG] osdmap e589: 8 total, 8 up, 8 in 2024-03-26T03:59:47.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:47 smithi175 bash[21527]: audit 2024-03-26T03:59:46.112724+0000 mon.c (mon.2) 597 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:59:47.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:47 smithi175 bash[21527]: audit 2024-03-26T03:59:46.118841+0000 mon.a (mon.0) 3055 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:59:47.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:47 smithi175 bash[21527]: audit 2024-03-26T03:59:46.628945+0000 mon.a (mon.0) 3056 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:59:47.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:47 smithi175 bash[21527]: cluster 2024-03-26T03:59:46.636884+0000 mon.a (mon.0) 3057 : cluster [DBG] osdmap e590: 8 total, 8 up, 8 in 2024-03-26T03:59:47.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:47 smithi175 bash[29284]: cluster 2024-03-26T03:59:46.105270+0000 mon.a (mon.0) 3054 : cluster [DBG] osdmap e589: 8 total, 8 up, 8 in 2024-03-26T03:59:47.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:47 smithi175 bash[29284]: audit 2024-03-26T03:59:46.112724+0000 mon.c (mon.2) 597 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:59:47.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:47 smithi175 bash[29284]: audit 2024-03-26T03:59:46.118841+0000 mon.a (mon.0) 3055 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:59:47.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:47 smithi175 bash[29284]: audit 2024-03-26T03:59:46.628945+0000 mon.a (mon.0) 3056 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:59:47.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:47 smithi175 bash[29284]: cluster 2024-03-26T03:59:46.636884+0000 mon.a (mon.0) 3057 : cluster [DBG] osdmap e590: 8 total, 8 up, 8 in 2024-03-26T03:59:48.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:48 smithi190 bash[23392]: cluster 2024-03-26T03:59:46.981379+0000 mgr.y (mgr.24431) 594 : cluster [DBG] pgmap v952: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 784 B/s rd, 1 op/s 2024-03-26T03:59:48.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:48 smithi190 bash[23392]: cluster 2024-03-26T03:59:47.644591+0000 mon.a (mon.0) 3058 : cluster [DBG] osdmap e591: 8 total, 8 up, 8 in 2024-03-26T03:59:48.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:48 smithi190 bash[23392]: audit 2024-03-26T03:59:47.659260+0000 mon.c (mon.2) 598 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:59:48.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:48 smithi190 bash[23392]: audit 2024-03-26T03:59:47.660226+0000 mon.a (mon.0) 3059 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:59:48.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:48 smithi175 bash[29284]: cluster 2024-03-26T03:59:46.981379+0000 mgr.y (mgr.24431) 594 : cluster [DBG] pgmap v952: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 784 B/s rd, 1 op/s 2024-03-26T03:59:48.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:48 smithi175 bash[29284]: cluster 2024-03-26T03:59:47.644591+0000 mon.a (mon.0) 3058 : cluster [DBG] osdmap e591: 8 total, 8 up, 8 in 2024-03-26T03:59:48.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:48 smithi175 bash[29284]: audit 2024-03-26T03:59:47.659260+0000 mon.c (mon.2) 598 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:59:48.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:48 smithi175 bash[29284]: audit 2024-03-26T03:59:47.660226+0000 mon.a (mon.0) 3059 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:59:48.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:48 smithi175 bash[21527]: cluster 2024-03-26T03:59:46.981379+0000 mgr.y (mgr.24431) 594 : cluster [DBG] pgmap v952: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 784 B/s rd, 1 op/s 2024-03-26T03:59:48.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:48 smithi175 bash[21527]: cluster 2024-03-26T03:59:47.644591+0000 mon.a (mon.0) 3058 : cluster [DBG] osdmap e591: 8 total, 8 up, 8 in 2024-03-26T03:59:48.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:48 smithi175 bash[21527]: audit 2024-03-26T03:59:47.659260+0000 mon.c (mon.2) 598 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:59:48.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:48 smithi175 bash[21527]: audit 2024-03-26T03:59:47.660226+0000 mon.a (mon.0) 3059 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:59:49.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:59:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:59:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:59:49.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:49 smithi190 bash[23392]: audit 2024-03-26T03:59:48.643036+0000 mon.a (mon.0) 3060 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-118", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:59:49.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:49 smithi190 bash[23392]: cluster 2024-03-26T03:59:48.650115+0000 mon.a (mon.0) 3061 : cluster [DBG] osdmap e592: 8 total, 8 up, 8 in 2024-03-26T03:59:49.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:49 smithi190 bash[23392]: audit 2024-03-26T03:59:48.654466+0000 mon.c (mon.2) 599 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-118"}]: dispatch 2024-03-26T03:59:49.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:49 smithi190 bash[23392]: audit 2024-03-26T03:59:48.657023+0000 mon.a (mon.0) 3062 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-118"}]: dispatch 2024-03-26T03:59:49.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:49 smithi190 bash[23392]: cluster 2024-03-26T03:59:48.982284+0000 mgr.y (mgr.24431) 595 : cluster [DBG] pgmap v955: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:59:49.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:49 smithi175 bash[21527]: audit 2024-03-26T03:59:48.643036+0000 mon.a (mon.0) 3060 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-118", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:59:49.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:49 smithi175 bash[21527]: cluster 2024-03-26T03:59:48.650115+0000 mon.a (mon.0) 3061 : cluster [DBG] osdmap e592: 8 total, 8 up, 8 in 2024-03-26T03:59:49.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:49 smithi175 bash[21527]: audit 2024-03-26T03:59:48.654466+0000 mon.c (mon.2) 599 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-118"}]: dispatch 2024-03-26T03:59:49.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:49 smithi175 bash[21527]: audit 2024-03-26T03:59:48.657023+0000 mon.a (mon.0) 3062 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-118"}]: dispatch 2024-03-26T03:59:49.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:49 smithi175 bash[21527]: cluster 2024-03-26T03:59:48.982284+0000 mgr.y (mgr.24431) 595 : cluster [DBG] pgmap v955: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:59:49.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:49 smithi175 bash[29284]: audit 2024-03-26T03:59:48.643036+0000 mon.a (mon.0) 3060 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-118", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:59:49.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:49 smithi175 bash[29284]: cluster 2024-03-26T03:59:48.650115+0000 mon.a (mon.0) 3061 : cluster [DBG] osdmap e592: 8 total, 8 up, 8 in 2024-03-26T03:59:49.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:49 smithi175 bash[29284]: audit 2024-03-26T03:59:48.654466+0000 mon.c (mon.2) 599 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-118"}]: dispatch 2024-03-26T03:59:49.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:49 smithi175 bash[29284]: audit 2024-03-26T03:59:48.657023+0000 mon.a (mon.0) 3062 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-118"}]: dispatch 2024-03-26T03:59:49.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:49 smithi175 bash[29284]: cluster 2024-03-26T03:59:48.982284+0000 mgr.y (mgr.24431) 595 : cluster [DBG] pgmap v955: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:59:50.670 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 03:59:50 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T03:59:50.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:50 smithi190 bash[23392]: audit 2024-03-26T03:59:49.655263+0000 mon.a (mon.0) 3063 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-118"}]': finished 2024-03-26T03:59:50.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:50 smithi190 bash[23392]: cluster 2024-03-26T03:59:49.666502+0000 mon.a (mon.0) 3064 : cluster [DBG] osdmap e593: 8 total, 8 up, 8 in 2024-03-26T03:59:50.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:50 smithi190 bash[23392]: audit 2024-03-26T03:59:49.670704+0000 mon.c (mon.2) 600 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-118", "mode": "writeback"}]: dispatch 2024-03-26T03:59:50.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:50 smithi190 bash[23392]: audit 2024-03-26T03:59:49.672322+0000 mon.a (mon.0) 3065 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-118", "mode": "writeback"}]: dispatch 2024-03-26T03:59:50.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:50 smithi190 bash[23392]: audit 2024-03-26T03:59:50.353569+0000 mgr.y (mgr.24431) 596 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:59:50.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:50 smithi175 bash[21527]: audit 2024-03-26T03:59:49.655263+0000 mon.a (mon.0) 3063 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-118"}]': finished 2024-03-26T03:59:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:50 smithi175 bash[21527]: cluster 2024-03-26T03:59:49.666502+0000 mon.a (mon.0) 3064 : cluster [DBG] osdmap e593: 8 total, 8 up, 8 in 2024-03-26T03:59:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:50 smithi175 bash[21527]: audit 2024-03-26T03:59:49.670704+0000 mon.c (mon.2) 600 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-118", "mode": "writeback"}]: dispatch 2024-03-26T03:59:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:50 smithi175 bash[21527]: audit 2024-03-26T03:59:49.672322+0000 mon.a (mon.0) 3065 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-118", "mode": "writeback"}]: dispatch 2024-03-26T03:59:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:50 smithi175 bash[21527]: audit 2024-03-26T03:59:50.353569+0000 mgr.y (mgr.24431) 596 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:59:50.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:50 smithi175 bash[29284]: audit 2024-03-26T03:59:49.655263+0000 mon.a (mon.0) 3063 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-118"}]': finished 2024-03-26T03:59:50.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:50 smithi175 bash[29284]: cluster 2024-03-26T03:59:49.666502+0000 mon.a (mon.0) 3064 : cluster [DBG] osdmap e593: 8 total, 8 up, 8 in 2024-03-26T03:59:50.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:50 smithi175 bash[29284]: audit 2024-03-26T03:59:49.670704+0000 mon.c (mon.2) 600 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-118", "mode": "writeback"}]: dispatch 2024-03-26T03:59:50.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:50 smithi175 bash[29284]: audit 2024-03-26T03:59:49.672322+0000 mon.a (mon.0) 3065 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-118", "mode": "writeback"}]: dispatch 2024-03-26T03:59:50.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:50 smithi175 bash[29284]: audit 2024-03-26T03:59:50.353569+0000 mgr.y (mgr.24431) 596 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T03:59:51.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:51 smithi190 bash[23392]: cluster 2024-03-26T03:59:50.655857+0000 mon.a (mon.0) 3066 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:59:51.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:51 smithi190 bash[23392]: audit 2024-03-26T03:59:50.658469+0000 mon.a (mon.0) 3067 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-118", "mode": "writeback"}]': finished 2024-03-26T03:59:51.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:51 smithi190 bash[23392]: cluster 2024-03-26T03:59:50.666230+0000 mon.a (mon.0) 3068 : cluster [DBG] osdmap e594: 8 total, 8 up, 8 in 2024-03-26T03:59:51.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:51 smithi190 bash[23392]: audit 2024-03-26T03:59:50.695207+0000 mon.c (mon.2) 601 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:59:51.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:51 smithi190 bash[23392]: cluster 2024-03-26T03:59:50.983210+0000 mgr.y (mgr.24431) 597 : cluster [DBG] pgmap v958: 268 pgs: 268 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-26T03:59:51.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:51 smithi190 bash[23392]: cluster 2024-03-26T03:59:51.628271+0000 mon.a (mon.0) 3069 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:59:51.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:51 smithi175 bash[21527]: cluster 2024-03-26T03:59:50.655857+0000 mon.a (mon.0) 3066 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:59:51.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:51 smithi175 bash[21527]: audit 2024-03-26T03:59:50.658469+0000 mon.a (mon.0) 3067 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-118", "mode": "writeback"}]': finished 2024-03-26T03:59:51.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:51 smithi175 bash[21527]: cluster 2024-03-26T03:59:50.666230+0000 mon.a (mon.0) 3068 : cluster [DBG] osdmap e594: 8 total, 8 up, 8 in 2024-03-26T03:59:51.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:51 smithi175 bash[21527]: audit 2024-03-26T03:59:50.695207+0000 mon.c (mon.2) 601 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:59:51.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:51 smithi175 bash[21527]: cluster 2024-03-26T03:59:50.983210+0000 mgr.y (mgr.24431) 597 : cluster [DBG] pgmap v958: 268 pgs: 268 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-26T03:59:51.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:51 smithi175 bash[21527]: cluster 2024-03-26T03:59:51.628271+0000 mon.a (mon.0) 3069 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:59:51.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:51 smithi175 bash[29284]: cluster 2024-03-26T03:59:50.655857+0000 mon.a (mon.0) 3066 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T03:59:51.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:51 smithi175 bash[29284]: audit 2024-03-26T03:59:50.658469+0000 mon.a (mon.0) 3067 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-118", "mode": "writeback"}]': finished 2024-03-26T03:59:51.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:51 smithi175 bash[29284]: cluster 2024-03-26T03:59:50.666230+0000 mon.a (mon.0) 3068 : cluster [DBG] osdmap e594: 8 total, 8 up, 8 in 2024-03-26T03:59:51.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:51 smithi175 bash[29284]: audit 2024-03-26T03:59:50.695207+0000 mon.c (mon.2) 601 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T03:59:51.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:51 smithi175 bash[29284]: cluster 2024-03-26T03:59:50.983210+0000 mgr.y (mgr.24431) 597 : cluster [DBG] pgmap v958: 268 pgs: 268 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-26T03:59:51.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:51 smithi175 bash[29284]: cluster 2024-03-26T03:59:51.628271+0000 mon.a (mon.0) 3069 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T03:59:52.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:52 smithi190 bash[23392]: cluster 2024-03-26T03:59:51.688980+0000 mon.a (mon.0) 3070 : cluster [DBG] osdmap e595: 8 total, 8 up, 8 in 2024-03-26T03:59:52.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:52 smithi190 bash[23392]: audit 2024-03-26T03:59:51.756046+0000 mon.c (mon.2) 602 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:52.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:52 smithi190 bash[23392]: audit 2024-03-26T03:59:51.756841+0000 mon.a (mon.0) 3071 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:52.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:52 smithi175 bash[21527]: cluster 2024-03-26T03:59:51.688980+0000 mon.a (mon.0) 3070 : cluster [DBG] osdmap e595: 8 total, 8 up, 8 in 2024-03-26T03:59:52.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:52 smithi175 bash[21527]: audit 2024-03-26T03:59:51.756046+0000 mon.c (mon.2) 602 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:52.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:52 smithi175 bash[21527]: audit 2024-03-26T03:59:51.756841+0000 mon.a (mon.0) 3071 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:52.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:52 smithi175 bash[29284]: cluster 2024-03-26T03:59:51.688980+0000 mon.a (mon.0) 3070 : cluster [DBG] osdmap e595: 8 total, 8 up, 8 in 2024-03-26T03:59:52.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:52 smithi175 bash[29284]: audit 2024-03-26T03:59:51.756046+0000 mon.c (mon.2) 602 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:52.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:52 smithi175 bash[29284]: audit 2024-03-26T03:59:51.756841+0000 mon.a (mon.0) 3071 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T03:59:53.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:53 smithi190 bash[23392]: audit 2024-03-26T03:59:52.682467+0000 mon.a (mon.0) 3072 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T03:59:53.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:53 smithi190 bash[23392]: cluster 2024-03-26T03:59:52.690911+0000 mon.a (mon.0) 3073 : cluster [DBG] osdmap e596: 8 total, 8 up, 8 in 2024-03-26T03:59:53.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:53 smithi190 bash[23392]: audit 2024-03-26T03:59:52.692621+0000 mon.c (mon.2) 603 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-118"}]: dispatch 2024-03-26T03:59:53.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:53 smithi190 bash[23392]: audit 2024-03-26T03:59:52.694640+0000 mon.a (mon.0) 3074 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-118"}]: dispatch 2024-03-26T03:59:53.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:53 smithi190 bash[23392]: cluster 2024-03-26T03:59:52.984182+0000 mgr.y (mgr.24431) 598 : cluster [DBG] pgmap v961: 268 pgs: 268 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-26T03:59:53.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:53 smithi190 bash[23392]: cluster 2024-03-26T03:59:53.682832+0000 mon.a (mon.0) 3075 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:59:53.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:53 smithi190 bash[23392]: audit 2024-03-26T03:59:53.685332+0000 mon.a (mon.0) 3076 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-118"}]': finished 2024-03-26T03:59:53.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:53 smithi190 bash[23392]: cluster 2024-03-26T03:59:53.692363+0000 mon.a (mon.0) 3077 : cluster [DBG] osdmap e597: 8 total, 8 up, 8 in 2024-03-26T03:59:53.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:53 smithi175 bash[21527]: audit 2024-03-26T03:59:52.682467+0000 mon.a (mon.0) 3072 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T03:59:53.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:53 smithi175 bash[21527]: cluster 2024-03-26T03:59:52.690911+0000 mon.a (mon.0) 3073 : cluster [DBG] osdmap e596: 8 total, 8 up, 8 in 2024-03-26T03:59:53.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:53 smithi175 bash[21527]: audit 2024-03-26T03:59:52.692621+0000 mon.c (mon.2) 603 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-118"}]: dispatch 2024-03-26T03:59:53.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:53 smithi175 bash[21527]: audit 2024-03-26T03:59:52.694640+0000 mon.a (mon.0) 3074 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-118"}]: dispatch 2024-03-26T03:59:53.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:53 smithi175 bash[21527]: cluster 2024-03-26T03:59:52.984182+0000 mgr.y (mgr.24431) 598 : cluster [DBG] pgmap v961: 268 pgs: 268 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-26T03:59:53.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:53 smithi175 bash[21527]: cluster 2024-03-26T03:59:53.682832+0000 mon.a (mon.0) 3075 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:59:53.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:53 smithi175 bash[21527]: audit 2024-03-26T03:59:53.685332+0000 mon.a (mon.0) 3076 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-118"}]': finished 2024-03-26T03:59:53.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:53 smithi175 bash[21527]: cluster 2024-03-26T03:59:53.692363+0000 mon.a (mon.0) 3077 : cluster [DBG] osdmap e597: 8 total, 8 up, 8 in 2024-03-26T03:59:53.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:53 smithi175 bash[29284]: audit 2024-03-26T03:59:52.682467+0000 mon.a (mon.0) 3072 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T03:59:53.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:53 smithi175 bash[29284]: cluster 2024-03-26T03:59:52.690911+0000 mon.a (mon.0) 3073 : cluster [DBG] osdmap e596: 8 total, 8 up, 8 in 2024-03-26T03:59:53.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:53 smithi175 bash[29284]: audit 2024-03-26T03:59:52.692621+0000 mon.c (mon.2) 603 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-118"}]: dispatch 2024-03-26T03:59:53.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:53 smithi175 bash[29284]: audit 2024-03-26T03:59:52.694640+0000 mon.a (mon.0) 3074 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-118"}]: dispatch 2024-03-26T03:59:53.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:53 smithi175 bash[29284]: cluster 2024-03-26T03:59:52.984182+0000 mgr.y (mgr.24431) 598 : cluster [DBG] pgmap v961: 268 pgs: 268 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-26T03:59:53.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:53 smithi175 bash[29284]: cluster 2024-03-26T03:59:53.682832+0000 mon.a (mon.0) 3075 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T03:59:53.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:53 smithi175 bash[29284]: audit 2024-03-26T03:59:53.685332+0000 mon.a (mon.0) 3076 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-118"}]': finished 2024-03-26T03:59:53.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:53 smithi175 bash[29284]: cluster 2024-03-26T03:59:53.692363+0000 mon.a (mon.0) 3077 : cluster [DBG] osdmap e597: 8 total, 8 up, 8 in 2024-03-26T03:59:55.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:55 smithi190 bash[23392]: cluster 2024-03-26T03:59:54.728956+0000 mon.a (mon.0) 3078 : cluster [DBG] osdmap e598: 8 total, 8 up, 8 in 2024-03-26T03:59:55.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:55 smithi190 bash[23392]: cluster 2024-03-26T03:59:54.986431+0000 mgr.y (mgr.24431) 599 : cluster [DBG] pgmap v964: 236 pgs: 236 active+clean; 582 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:59:55.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:55 smithi175 bash[21527]: cluster 2024-03-26T03:59:54.728956+0000 mon.a (mon.0) 3078 : cluster [DBG] osdmap e598: 8 total, 8 up, 8 in 2024-03-26T03:59:55.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:55 smithi175 bash[21527]: cluster 2024-03-26T03:59:54.986431+0000 mgr.y (mgr.24431) 599 : cluster [DBG] pgmap v964: 236 pgs: 236 active+clean; 582 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:59:55.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:55 smithi175 bash[29284]: cluster 2024-03-26T03:59:54.728956+0000 mon.a (mon.0) 3078 : cluster [DBG] osdmap e598: 8 total, 8 up, 8 in 2024-03-26T03:59:55.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:55 smithi175 bash[29284]: cluster 2024-03-26T03:59:54.986431+0000 mgr.y (mgr.24431) 599 : cluster [DBG] pgmap v964: 236 pgs: 236 active+clean; 582 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:59:56.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:56 smithi190 bash[23392]: cluster 2024-03-26T03:59:55.744507+0000 mon.a (mon.0) 3079 : cluster [DBG] osdmap e599: 8 total, 8 up, 8 in 2024-03-26T03:59:56.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:56 smithi190 bash[23392]: audit 2024-03-26T03:59:55.751289+0000 mon.c (mon.2) 604 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:59:56.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:56 smithi190 bash[23392]: audit 2024-03-26T03:59:55.753646+0000 mon.a (mon.0) 3080 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:59:56.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:56 smithi190 bash[23392]: audit 2024-03-26T03:59:56.633476+0000 mon.a (mon.0) 3081 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:59:56.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:56 smithi190 bash[23392]: cluster 2024-03-26T03:59:56.642356+0000 mon.a (mon.0) 3082 : cluster [DBG] osdmap e600: 8 total, 8 up, 8 in 2024-03-26T03:59:56.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:56 smithi190 bash[23392]: audit 2024-03-26T03:59:56.684054+0000 mon.c (mon.2) 605 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:59:56.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:56 smithi190 bash[23392]: audit 2024-03-26T03:59:56.685332+0000 mon.a (mon.0) 3083 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:59:56.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:56 smithi175 bash[21527]: cluster 2024-03-26T03:59:55.744507+0000 mon.a (mon.0) 3079 : cluster [DBG] osdmap e599: 8 total, 8 up, 8 in 2024-03-26T03:59:56.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:56 smithi175 bash[21527]: audit 2024-03-26T03:59:55.751289+0000 mon.c (mon.2) 604 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:59:56.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:56 smithi175 bash[21527]: audit 2024-03-26T03:59:55.753646+0000 mon.a (mon.0) 3080 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:59:56.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:56 smithi175 bash[21527]: audit 2024-03-26T03:59:56.633476+0000 mon.a (mon.0) 3081 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:59:56.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:56 smithi175 bash[21527]: cluster 2024-03-26T03:59:56.642356+0000 mon.a (mon.0) 3082 : cluster [DBG] osdmap e600: 8 total, 8 up, 8 in 2024-03-26T03:59:56.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:56 smithi175 bash[21527]: audit 2024-03-26T03:59:56.684054+0000 mon.c (mon.2) 605 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:59:56.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:56 smithi175 bash[21527]: audit 2024-03-26T03:59:56.685332+0000 mon.a (mon.0) 3083 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:59:56.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:56 smithi175 bash[29284]: cluster 2024-03-26T03:59:55.744507+0000 mon.a (mon.0) 3079 : cluster [DBG] osdmap e599: 8 total, 8 up, 8 in 2024-03-26T03:59:56.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:56 smithi175 bash[29284]: audit 2024-03-26T03:59:55.751289+0000 mon.c (mon.2) 604 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:59:56.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:56 smithi175 bash[29284]: audit 2024-03-26T03:59:55.753646+0000 mon.a (mon.0) 3080 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T03:59:56.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:56 smithi175 bash[29284]: audit 2024-03-26T03:59:56.633476+0000 mon.a (mon.0) 3081 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T03:59:56.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:56 smithi175 bash[29284]: cluster 2024-03-26T03:59:56.642356+0000 mon.a (mon.0) 3082 : cluster [DBG] osdmap e600: 8 total, 8 up, 8 in 2024-03-26T03:59:56.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:56 smithi175 bash[29284]: audit 2024-03-26T03:59:56.684054+0000 mon.c (mon.2) 605 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:59:56.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:56 smithi175 bash[29284]: audit 2024-03-26T03:59:56.685332+0000 mon.a (mon.0) 3083 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T03:59:57.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:57 smithi190 bash[23392]: cluster 2024-03-26T03:59:56.987492+0000 mgr.y (mgr.24431) 600 : cluster [DBG] pgmap v967: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:59:57.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:57 smithi190 bash[23392]: audit 2024-03-26T03:59:57.636670+0000 mon.a (mon.0) 3084 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-120", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:59:57.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:57 smithi190 bash[23392]: cluster 2024-03-26T03:59:57.643772+0000 mon.a (mon.0) 3085 : cluster [DBG] osdmap e601: 8 total, 8 up, 8 in 2024-03-26T03:59:57.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:57 smithi190 bash[23392]: audit 2024-03-26T03:59:57.652458+0000 mon.c (mon.2) 606 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-120"}]: dispatch 2024-03-26T03:59:57.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:57 smithi190 bash[23392]: audit 2024-03-26T03:59:57.653343+0000 mon.a (mon.0) 3086 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-120"}]: dispatch 2024-03-26T03:59:57.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:57 smithi175 bash[29284]: cluster 2024-03-26T03:59:56.987492+0000 mgr.y (mgr.24431) 600 : cluster [DBG] pgmap v967: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:59:57.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:57 smithi175 bash[29284]: audit 2024-03-26T03:59:57.636670+0000 mon.a (mon.0) 3084 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-120", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:59:57.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:57 smithi175 bash[29284]: cluster 2024-03-26T03:59:57.643772+0000 mon.a (mon.0) 3085 : cluster [DBG] osdmap e601: 8 total, 8 up, 8 in 2024-03-26T03:59:57.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:57 smithi175 bash[29284]: audit 2024-03-26T03:59:57.652458+0000 mon.c (mon.2) 606 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-120"}]: dispatch 2024-03-26T03:59:57.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:57 smithi175 bash[29284]: audit 2024-03-26T03:59:57.653343+0000 mon.a (mon.0) 3086 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-120"}]: dispatch 2024-03-26T03:59:57.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:57 smithi175 bash[21527]: cluster 2024-03-26T03:59:56.987492+0000 mgr.y (mgr.24431) 600 : cluster [DBG] pgmap v967: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T03:59:57.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:57 smithi175 bash[21527]: audit 2024-03-26T03:59:57.636670+0000 mon.a (mon.0) 3084 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-120", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T03:59:57.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:57 smithi175 bash[21527]: cluster 2024-03-26T03:59:57.643772+0000 mon.a (mon.0) 3085 : cluster [DBG] osdmap e601: 8 total, 8 up, 8 in 2024-03-26T03:59:57.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:57 smithi175 bash[21527]: audit 2024-03-26T03:59:57.652458+0000 mon.c (mon.2) 606 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-120"}]: dispatch 2024-03-26T03:59:57.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:57 smithi175 bash[21527]: audit 2024-03-26T03:59:57.653343+0000 mon.a (mon.0) 3086 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-120"}]: dispatch 2024-03-26T03:59:59.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 03:59:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:03:59:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T03:59:59.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:59 smithi190 bash[23392]: audit 2024-03-26T03:59:58.639521+0000 mon.a (mon.0) 3087 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-120"}]': finished 2024-03-26T03:59:59.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:59 smithi190 bash[23392]: cluster 2024-03-26T03:59:58.646519+0000 mon.a (mon.0) 3088 : cluster [DBG] osdmap e602: 8 total, 8 up, 8 in 2024-03-26T03:59:59.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:59 smithi190 bash[23392]: audit 2024-03-26T03:59:58.649552+0000 mon.c (mon.2) 607 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-120", "mode": "writeback"}]: dispatch 2024-03-26T03:59:59.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:59 smithi190 bash[23392]: audit 2024-03-26T03:59:58.651178+0000 mon.a (mon.0) 3089 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-120", "mode": "writeback"}]: dispatch 2024-03-26T03:59:59.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 03:59:59 smithi190 bash[23392]: cluster 2024-03-26T03:59:58.988388+0000 mgr.y (mgr.24431) 601 : cluster [DBG] pgmap v970: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:59:59.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:59 smithi175 bash[21527]: audit 2024-03-26T03:59:58.639521+0000 mon.a (mon.0) 3087 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-120"}]': finished 2024-03-26T03:59:59.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:59 smithi175 bash[21527]: cluster 2024-03-26T03:59:58.646519+0000 mon.a (mon.0) 3088 : cluster [DBG] osdmap e602: 8 total, 8 up, 8 in 2024-03-26T03:59:59.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:59 smithi175 bash[21527]: audit 2024-03-26T03:59:58.649552+0000 mon.c (mon.2) 607 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-120", "mode": "writeback"}]: dispatch 2024-03-26T03:59:59.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:59 smithi175 bash[21527]: audit 2024-03-26T03:59:58.651178+0000 mon.a (mon.0) 3089 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-120", "mode": "writeback"}]: dispatch 2024-03-26T03:59:59.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 03:59:59 smithi175 bash[21527]: cluster 2024-03-26T03:59:58.988388+0000 mgr.y (mgr.24431) 601 : cluster [DBG] pgmap v970: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-03-26T03:59:59.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:59 smithi175 bash[29284]: audit 2024-03-26T03:59:58.639521+0000 mon.a (mon.0) 3087 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-120"}]': finished 2024-03-26T03:59:59.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:59 smithi175 bash[29284]: cluster 2024-03-26T03:59:58.646519+0000 mon.a (mon.0) 3088 : cluster [DBG] osdmap e602: 8 total, 8 up, 8 in 2024-03-26T03:59:59.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:59 smithi175 bash[29284]: audit 2024-03-26T03:59:58.649552+0000 mon.c (mon.2) 607 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-120", "mode": "writeback"}]: dispatch 2024-03-26T03:59:59.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:59 smithi175 bash[29284]: audit 2024-03-26T03:59:58.651178+0000 mon.a (mon.0) 3089 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-120", "mode": "writeback"}]: dispatch 2024-03-26T03:59:59.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 03:59:59 smithi175 bash[29284]: cluster 2024-03-26T03:59:58.988388+0000 mgr.y (mgr.24431) 601 : cluster [DBG] pgmap v970: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-03-26T04:00:00.666 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:00:00 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:00:00.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:00 smithi190 bash[23392]: cluster 2024-03-26T03:59:59.641056+0000 mon.a (mon.0) 3090 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:00:00.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:00 smithi190 bash[23392]: audit 2024-03-26T03:59:59.651171+0000 mon.a (mon.0) 3091 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-120", "mode": "writeback"}]': finished 2024-03-26T04:00:00.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:00 smithi190 bash[23392]: cluster 2024-03-26T03:59:59.663912+0000 mon.a (mon.0) 3092 : cluster [DBG] osdmap e603: 8 total, 8 up, 8 in 2024-03-26T04:00:00.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:00 smithi190 bash[23392]: audit 2024-03-26T03:59:59.814647+0000 mon.c (mon.2) 608 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:00.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:00 smithi190 bash[23392]: audit 2024-03-26T03:59:59.815470+0000 mon.a (mon.0) 3093 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:00.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:00 smithi190 bash[23392]: cluster 2024-03-26T04:00:00.000143+0000 mon.a (mon.0) 3094 : cluster [WRN] Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 4 pool(s) do not have an application enabled 2024-03-26T04:00:00.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:00 smithi190 bash[23392]: cluster 2024-03-26T04:00:00.000193+0000 mon.a (mon.0) 3095 : cluster [WRN] [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets 2024-03-26T04:00:00.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:00 smithi190 bash[23392]: cluster 2024-03-26T04:00:00.000214+0000 mon.a (mon.0) 3096 : cluster [WRN] pool 'test-rados-api-smithi175-64594-120' with cache_mode writeback needs hit_set_type to be set but it is not 2024-03-26T04:00:00.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:00 smithi190 bash[23392]: cluster 2024-03-26T04:00:00.000279+0000 mon.a (mon.0) 3097 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 4 pool(s) do not have an application enabled 2024-03-26T04:00:00.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:00 smithi190 bash[23392]: cluster 2024-03-26T04:00:00.000312+0000 mon.a (mon.0) 3098 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-26T04:00:00.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:00 smithi190 bash[23392]: cluster 2024-03-26T04:00:00.000338+0000 mon.a (mon.0) 3099 : cluster [WRN] application not enabled on pool 'WatchNotifysmithi175-65446-1' 2024-03-26T04:00:00.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:00 smithi190 bash[23392]: cluster 2024-03-26T04:00:00.000373+0000 mon.a (mon.0) 3100 : cluster [WRN] application not enabled on pool 'AssertExistssmithi175-65459-1' 2024-03-26T04:00:00.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:00 smithi190 bash[23392]: cluster 2024-03-26T04:00:00.000397+0000 mon.a (mon.0) 3101 : cluster [WRN] application not enabled on pool 'test-rados-api-smithi175-64594-111' 2024-03-26T04:00:00.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:00 smithi190 bash[23392]: cluster 2024-03-26T04:00:00.000421+0000 mon.a (mon.0) 3102 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-26T04:00:00.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:00 smithi190 bash[23392]: audit 2024-03-26T04:00:00.357854+0000 mgr.y (mgr.24431) 602 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:00:00.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:00 smithi175 bash[21527]: cluster 2024-03-26T03:59:59.641056+0000 mon.a (mon.0) 3090 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:00:00.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:00 smithi175 bash[21527]: audit 2024-03-26T03:59:59.651171+0000 mon.a (mon.0) 3091 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-120", "mode": "writeback"}]': finished 2024-03-26T04:00:00.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:00 smithi175 bash[21527]: cluster 2024-03-26T03:59:59.663912+0000 mon.a (mon.0) 3092 : cluster [DBG] osdmap e603: 8 total, 8 up, 8 in 2024-03-26T04:00:00.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:00 smithi175 bash[21527]: audit 2024-03-26T03:59:59.814647+0000 mon.c (mon.2) 608 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:00.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:00 smithi175 bash[21527]: audit 2024-03-26T03:59:59.815470+0000 mon.a (mon.0) 3093 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:00.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:00 smithi175 bash[21527]: cluster 2024-03-26T04:00:00.000143+0000 mon.a (mon.0) 3094 : cluster [WRN] Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 4 pool(s) do not have an application enabled 2024-03-26T04:00:00.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:00 smithi175 bash[21527]: cluster 2024-03-26T04:00:00.000193+0000 mon.a (mon.0) 3095 : cluster [WRN] [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets 2024-03-26T04:00:00.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:00 smithi175 bash[21527]: cluster 2024-03-26T04:00:00.000214+0000 mon.a (mon.0) 3096 : cluster [WRN] pool 'test-rados-api-smithi175-64594-120' with cache_mode writeback needs hit_set_type to be set but it is not 2024-03-26T04:00:00.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:00 smithi175 bash[21527]: cluster 2024-03-26T04:00:00.000279+0000 mon.a (mon.0) 3097 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 4 pool(s) do not have an application enabled 2024-03-26T04:00:00.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:00 smithi175 bash[21527]: cluster 2024-03-26T04:00:00.000312+0000 mon.a (mon.0) 3098 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-26T04:00:00.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:00 smithi175 bash[21527]: cluster 2024-03-26T04:00:00.000338+0000 mon.a (mon.0) 3099 : cluster [WRN] application not enabled on pool 'WatchNotifysmithi175-65446-1' 2024-03-26T04:00:00.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:00 smithi175 bash[21527]: cluster 2024-03-26T04:00:00.000373+0000 mon.a (mon.0) 3100 : cluster [WRN] application not enabled on pool 'AssertExistssmithi175-65459-1' 2024-03-26T04:00:00.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:00 smithi175 bash[21527]: cluster 2024-03-26T04:00:00.000397+0000 mon.a (mon.0) 3101 : cluster [WRN] application not enabled on pool 'test-rados-api-smithi175-64594-111' 2024-03-26T04:00:00.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:00 smithi175 bash[21527]: cluster 2024-03-26T04:00:00.000421+0000 mon.a (mon.0) 3102 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-26T04:00:00.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:00 smithi175 bash[21527]: audit 2024-03-26T04:00:00.357854+0000 mgr.y (mgr.24431) 602 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:00:00.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:00 smithi175 bash[29284]: cluster 2024-03-26T03:59:59.641056+0000 mon.a (mon.0) 3090 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:00:00.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:00 smithi175 bash[29284]: audit 2024-03-26T03:59:59.651171+0000 mon.a (mon.0) 3091 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-120", "mode": "writeback"}]': finished 2024-03-26T04:00:00.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:00 smithi175 bash[29284]: cluster 2024-03-26T03:59:59.663912+0000 mon.a (mon.0) 3092 : cluster [DBG] osdmap e603: 8 total, 8 up, 8 in 2024-03-26T04:00:00.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:00 smithi175 bash[29284]: audit 2024-03-26T03:59:59.814647+0000 mon.c (mon.2) 608 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:00.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:00 smithi175 bash[29284]: audit 2024-03-26T03:59:59.815470+0000 mon.a (mon.0) 3093 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:00.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:00 smithi175 bash[29284]: cluster 2024-03-26T04:00:00.000143+0000 mon.a (mon.0) 3094 : cluster [WRN] Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 4 pool(s) do not have an application enabled 2024-03-26T04:00:00.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:00 smithi175 bash[29284]: cluster 2024-03-26T04:00:00.000193+0000 mon.a (mon.0) 3095 : cluster [WRN] [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets 2024-03-26T04:00:00.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:00 smithi175 bash[29284]: cluster 2024-03-26T04:00:00.000214+0000 mon.a (mon.0) 3096 : cluster [WRN] pool 'test-rados-api-smithi175-64594-120' with cache_mode writeback needs hit_set_type to be set but it is not 2024-03-26T04:00:00.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:00 smithi175 bash[29284]: cluster 2024-03-26T04:00:00.000279+0000 mon.a (mon.0) 3097 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 4 pool(s) do not have an application enabled 2024-03-26T04:00:00.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:00 smithi175 bash[29284]: cluster 2024-03-26T04:00:00.000312+0000 mon.a (mon.0) 3098 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-26T04:00:00.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:00 smithi175 bash[29284]: cluster 2024-03-26T04:00:00.000338+0000 mon.a (mon.0) 3099 : cluster [WRN] application not enabled on pool 'WatchNotifysmithi175-65446-1' 2024-03-26T04:00:00.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:00 smithi175 bash[29284]: cluster 2024-03-26T04:00:00.000373+0000 mon.a (mon.0) 3100 : cluster [WRN] application not enabled on pool 'AssertExistssmithi175-65459-1' 2024-03-26T04:00:00.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:00 smithi175 bash[29284]: cluster 2024-03-26T04:00:00.000397+0000 mon.a (mon.0) 3101 : cluster [WRN] application not enabled on pool 'test-rados-api-smithi175-64594-111' 2024-03-26T04:00:00.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:00 smithi175 bash[29284]: cluster 2024-03-26T04:00:00.000421+0000 mon.a (mon.0) 3102 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-26T04:00:00.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:00 smithi175 bash[29284]: audit 2024-03-26T04:00:00.357854+0000 mgr.y (mgr.24431) 602 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:00:01.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:01 smithi190 bash[23392]: audit 2024-03-26T04:00:00.654043+0000 mon.a (mon.0) 3103 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:00:01.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:01 smithi190 bash[23392]: cluster 2024-03-26T04:00:00.661116+0000 mon.a (mon.0) 3104 : cluster [DBG] osdmap e604: 8 total, 8 up, 8 in 2024-03-26T04:00:01.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:01 smithi190 bash[23392]: audit 2024-03-26T04:00:00.665229+0000 mon.c (mon.2) 609 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-120"}]: dispatch 2024-03-26T04:00:01.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:01 smithi190 bash[23392]: audit 2024-03-26T04:00:00.679192+0000 mon.a (mon.0) 3105 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-120"}]: dispatch 2024-03-26T04:00:01.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:01 smithi190 bash[23392]: cluster 2024-03-26T04:00:00.989298+0000 mgr.y (mgr.24431) 603 : cluster [DBG] pgmap v973: 268 pgs: 268 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T04:00:01.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:01 smithi175 bash[29284]: audit 2024-03-26T04:00:00.654043+0000 mon.a (mon.0) 3103 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:00:01.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:01 smithi175 bash[29284]: cluster 2024-03-26T04:00:00.661116+0000 mon.a (mon.0) 3104 : cluster [DBG] osdmap e604: 8 total, 8 up, 8 in 2024-03-26T04:00:01.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:01 smithi175 bash[29284]: audit 2024-03-26T04:00:00.665229+0000 mon.c (mon.2) 609 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-120"}]: dispatch 2024-03-26T04:00:01.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:01 smithi175 bash[29284]: audit 2024-03-26T04:00:00.679192+0000 mon.a (mon.0) 3105 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-120"}]: dispatch 2024-03-26T04:00:01.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:01 smithi175 bash[29284]: cluster 2024-03-26T04:00:00.989298+0000 mgr.y (mgr.24431) 603 : cluster [DBG] pgmap v973: 268 pgs: 268 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T04:00:01.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:01 smithi175 bash[21527]: audit 2024-03-26T04:00:00.654043+0000 mon.a (mon.0) 3103 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:00:01.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:01 smithi175 bash[21527]: cluster 2024-03-26T04:00:00.661116+0000 mon.a (mon.0) 3104 : cluster [DBG] osdmap e604: 8 total, 8 up, 8 in 2024-03-26T04:00:01.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:01 smithi175 bash[21527]: audit 2024-03-26T04:00:00.665229+0000 mon.c (mon.2) 609 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-120"}]: dispatch 2024-03-26T04:00:01.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:01 smithi175 bash[21527]: audit 2024-03-26T04:00:00.679192+0000 mon.a (mon.0) 3105 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-120"}]: dispatch 2024-03-26T04:00:01.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:01 smithi175 bash[21527]: cluster 2024-03-26T04:00:00.989298+0000 mgr.y (mgr.24431) 603 : cluster [DBG] pgmap v973: 268 pgs: 268 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T04:00:02.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:02 smithi190 bash[23392]: cluster 2024-03-26T04:00:01.668346+0000 mon.a (mon.0) 3106 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:00:02.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:02 smithi190 bash[23392]: audit 2024-03-26T04:00:01.676056+0000 mon.a (mon.0) 3107 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-120"}]': finished 2024-03-26T04:00:02.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:02 smithi190 bash[23392]: cluster 2024-03-26T04:00:01.688846+0000 mon.a (mon.0) 3108 : cluster [DBG] osdmap e605: 8 total, 8 up, 8 in 2024-03-26T04:00:02.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:02 smithi175 bash[21527]: cluster 2024-03-26T04:00:01.668346+0000 mon.a (mon.0) 3106 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:00:02.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:02 smithi175 bash[21527]: audit 2024-03-26T04:00:01.676056+0000 mon.a (mon.0) 3107 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-120"}]': finished 2024-03-26T04:00:02.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:02 smithi175 bash[21527]: cluster 2024-03-26T04:00:01.688846+0000 mon.a (mon.0) 3108 : cluster [DBG] osdmap e605: 8 total, 8 up, 8 in 2024-03-26T04:00:02.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:02 smithi175 bash[29284]: cluster 2024-03-26T04:00:01.668346+0000 mon.a (mon.0) 3106 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:00:02.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:02 smithi175 bash[29284]: audit 2024-03-26T04:00:01.676056+0000 mon.a (mon.0) 3107 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-120"}]': finished 2024-03-26T04:00:02.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:02 smithi175 bash[29284]: cluster 2024-03-26T04:00:01.688846+0000 mon.a (mon.0) 3108 : cluster [DBG] osdmap e605: 8 total, 8 up, 8 in 2024-03-26T04:00:03.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:03 smithi190 bash[23392]: cluster 2024-03-26T04:00:02.687411+0000 mon.a (mon.0) 3109 : cluster [DBG] osdmap e606: 8 total, 8 up, 8 in 2024-03-26T04:00:03.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:03 smithi190 bash[23392]: cluster 2024-03-26T04:00:02.990158+0000 mgr.y (mgr.24431) 604 : cluster [DBG] pgmap v976: 236 pgs: 236 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T04:00:03.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:03 smithi175 bash[29284]: cluster 2024-03-26T04:00:02.687411+0000 mon.a (mon.0) 3109 : cluster [DBG] osdmap e606: 8 total, 8 up, 8 in 2024-03-26T04:00:03.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:03 smithi175 bash[29284]: cluster 2024-03-26T04:00:02.990158+0000 mgr.y (mgr.24431) 604 : cluster [DBG] pgmap v976: 236 pgs: 236 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T04:00:03.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:03 smithi175 bash[21527]: cluster 2024-03-26T04:00:02.687411+0000 mon.a (mon.0) 3109 : cluster [DBG] osdmap e606: 8 total, 8 up, 8 in 2024-03-26T04:00:03.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:03 smithi175 bash[21527]: cluster 2024-03-26T04:00:02.990158+0000 mgr.y (mgr.24431) 604 : cluster [DBG] pgmap v976: 236 pgs: 236 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T04:00:04.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:04 smithi190 bash[23392]: cluster 2024-03-26T04:00:03.713436+0000 mon.a (mon.0) 3110 : cluster [DBG] osdmap e607: 8 total, 8 up, 8 in 2024-03-26T04:00:04.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:04 smithi190 bash[23392]: audit 2024-03-26T04:00:03.717165+0000 mon.c (mon.2) 610 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:04.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:04 smithi190 bash[23392]: audit 2024-03-26T04:00:03.719570+0000 mon.a (mon.0) 3111 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:04.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:04 smithi175 bash[29284]: cluster 2024-03-26T04:00:03.713436+0000 mon.a (mon.0) 3110 : cluster [DBG] osdmap e607: 8 total, 8 up, 8 in 2024-03-26T04:00:04.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:04 smithi175 bash[29284]: audit 2024-03-26T04:00:03.717165+0000 mon.c (mon.2) 610 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:04.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:04 smithi175 bash[29284]: audit 2024-03-26T04:00:03.719570+0000 mon.a (mon.0) 3111 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:04.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:04 smithi175 bash[21527]: cluster 2024-03-26T04:00:03.713436+0000 mon.a (mon.0) 3110 : cluster [DBG] osdmap e607: 8 total, 8 up, 8 in 2024-03-26T04:00:04.996 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:04 smithi175 bash[21527]: audit 2024-03-26T04:00:03.717165+0000 mon.c (mon.2) 610 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:04.997 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:04 smithi175 bash[21527]: audit 2024-03-26T04:00:03.719570+0000 mon.a (mon.0) 3111 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:05.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:05 smithi190 bash[23392]: audit 2024-03-26T04:00:04.707165+0000 mon.a (mon.0) 3112 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:00:05.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:05 smithi190 bash[23392]: cluster 2024-03-26T04:00:04.716472+0000 mon.a (mon.0) 3113 : cluster [DBG] osdmap e608: 8 total, 8 up, 8 in 2024-03-26T04:00:05.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:05 smithi190 bash[23392]: audit 2024-03-26T04:00:04.764231+0000 mon.c (mon.2) 611 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:05.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:05 smithi190 bash[23392]: audit 2024-03-26T04:00:04.765234+0000 mon.a (mon.0) 3114 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:05.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:05 smithi190 bash[23392]: cluster 2024-03-26T04:00:04.992654+0000 mgr.y (mgr.24431) 605 : cluster [DBG] pgmap v979: 268 pgs: 3 creating+activating, 2 creating+peering, 263 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-26T04:00:05.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:05 smithi175 bash[21527]: audit 2024-03-26T04:00:04.707165+0000 mon.a (mon.0) 3112 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:00:05.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:05 smithi175 bash[21527]: cluster 2024-03-26T04:00:04.716472+0000 mon.a (mon.0) 3113 : cluster [DBG] osdmap e608: 8 total, 8 up, 8 in 2024-03-26T04:00:05.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:05 smithi175 bash[21527]: audit 2024-03-26T04:00:04.764231+0000 mon.c (mon.2) 611 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:05.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:05 smithi175 bash[21527]: audit 2024-03-26T04:00:04.765234+0000 mon.a (mon.0) 3114 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:05.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:05 smithi175 bash[21527]: cluster 2024-03-26T04:00:04.992654+0000 mgr.y (mgr.24431) 605 : cluster [DBG] pgmap v979: 268 pgs: 3 creating+activating, 2 creating+peering, 263 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-26T04:00:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:05 smithi175 bash[29284]: audit 2024-03-26T04:00:04.707165+0000 mon.a (mon.0) 3112 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:00:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:05 smithi175 bash[29284]: cluster 2024-03-26T04:00:04.716472+0000 mon.a (mon.0) 3113 : cluster [DBG] osdmap e608: 8 total, 8 up, 8 in 2024-03-26T04:00:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:05 smithi175 bash[29284]: audit 2024-03-26T04:00:04.764231+0000 mon.c (mon.2) 611 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:05 smithi175 bash[29284]: audit 2024-03-26T04:00:04.765234+0000 mon.a (mon.0) 3114 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:05 smithi175 bash[29284]: cluster 2024-03-26T04:00:04.992654+0000 mgr.y (mgr.24431) 605 : cluster [DBG] pgmap v979: 268 pgs: 3 creating+activating, 2 creating+peering, 263 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-26T04:00:06.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:06 smithi190 bash[23392]: audit 2024-03-26T04:00:05.737566+0000 mon.a (mon.0) 3115 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-122", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:00:06.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:06 smithi190 bash[23392]: cluster 2024-03-26T04:00:05.752214+0000 mon.a (mon.0) 3116 : cluster [DBG] osdmap e609: 8 total, 8 up, 8 in 2024-03-26T04:00:06.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:06 smithi190 bash[23392]: audit 2024-03-26T04:00:05.752765+0000 mon.c (mon.2) 612 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-122"}]: dispatch 2024-03-26T04:00:06.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:06 smithi190 bash[23392]: audit 2024-03-26T04:00:05.757286+0000 mon.a (mon.0) 3117 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-122"}]: dispatch 2024-03-26T04:00:06.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:06 smithi190 bash[23392]: audit 2024-03-26T04:00:05.764644+0000 mon.a (mon.0) 3118 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:00:06.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:06 smithi190 bash[23392]: audit 2024-03-26T04:00:05.768674+0000 mon.c (mon.2) 613 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:00:06.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:06 smithi190 bash[23392]: audit 2024-03-26T04:00:05.822993+0000 mon.c (mon.2) 614 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:00:06.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:06 smithi190 bash[23392]: audit 2024-03-26T04:00:06.550036+0000 mon.c (mon.2) 615 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:00:06.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:06 smithi190 bash[23392]: audit 2024-03-26T04:00:06.551675+0000 mon.c (mon.2) 616 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:00:06.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:06 smithi190 bash[23392]: audit 2024-03-26T04:00:06.562832+0000 mon.a (mon.0) 3119 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:00:06.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:06 smithi175 bash[21527]: audit 2024-03-26T04:00:05.737566+0000 mon.a (mon.0) 3115 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-122", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:00:06.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:06 smithi175 bash[21527]: cluster 2024-03-26T04:00:05.752214+0000 mon.a (mon.0) 3116 : cluster [DBG] osdmap e609: 8 total, 8 up, 8 in 2024-03-26T04:00:06.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:06 smithi175 bash[21527]: audit 2024-03-26T04:00:05.752765+0000 mon.c (mon.2) 612 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-122"}]: dispatch 2024-03-26T04:00:06.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:06 smithi175 bash[21527]: audit 2024-03-26T04:00:05.757286+0000 mon.a (mon.0) 3117 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-122"}]: dispatch 2024-03-26T04:00:06.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:06 smithi175 bash[21527]: audit 2024-03-26T04:00:05.764644+0000 mon.a (mon.0) 3118 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:00:06.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:06 smithi175 bash[21527]: audit 2024-03-26T04:00:05.768674+0000 mon.c (mon.2) 613 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:00:06.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:06 smithi175 bash[21527]: audit 2024-03-26T04:00:05.822993+0000 mon.c (mon.2) 614 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:00:06.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:06 smithi175 bash[21527]: audit 2024-03-26T04:00:06.550036+0000 mon.c (mon.2) 615 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:00:06.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:06 smithi175 bash[21527]: audit 2024-03-26T04:00:06.551675+0000 mon.c (mon.2) 616 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:00:06.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:06 smithi175 bash[21527]: audit 2024-03-26T04:00:06.562832+0000 mon.a (mon.0) 3119 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:00:06.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:06 smithi175 bash[29284]: audit 2024-03-26T04:00:05.737566+0000 mon.a (mon.0) 3115 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-122", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:00:06.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:06 smithi175 bash[29284]: cluster 2024-03-26T04:00:05.752214+0000 mon.a (mon.0) 3116 : cluster [DBG] osdmap e609: 8 total, 8 up, 8 in 2024-03-26T04:00:06.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:06 smithi175 bash[29284]: audit 2024-03-26T04:00:05.752765+0000 mon.c (mon.2) 612 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-122"}]: dispatch 2024-03-26T04:00:06.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:06 smithi175 bash[29284]: audit 2024-03-26T04:00:05.757286+0000 mon.a (mon.0) 3117 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-122"}]: dispatch 2024-03-26T04:00:06.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:06 smithi175 bash[29284]: audit 2024-03-26T04:00:05.764644+0000 mon.a (mon.0) 3118 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:00:06.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:06 smithi175 bash[29284]: audit 2024-03-26T04:00:05.768674+0000 mon.c (mon.2) 613 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:00:06.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:06 smithi175 bash[29284]: audit 2024-03-26T04:00:05.822993+0000 mon.c (mon.2) 614 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:00:06.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:06 smithi175 bash[29284]: audit 2024-03-26T04:00:06.550036+0000 mon.c (mon.2) 615 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:00:06.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:06 smithi175 bash[29284]: audit 2024-03-26T04:00:06.551675+0000 mon.c (mon.2) 616 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:00:06.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:06 smithi175 bash[29284]: audit 2024-03-26T04:00:06.562832+0000 mon.a (mon.0) 3119 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:00:08.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:07 smithi190 bash[23392]: audit 2024-03-26T04:00:06.751116+0000 mon.a (mon.0) 3120 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-122"}]': finished 2024-03-26T04:00:08.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:07 smithi190 bash[23392]: cluster 2024-03-26T04:00:06.759975+0000 mon.a (mon.0) 3121 : cluster [DBG] osdmap e610: 8 total, 8 up, 8 in 2024-03-26T04:00:08.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:07 smithi190 bash[23392]: audit 2024-03-26T04:00:06.762212+0000 mon.c (mon.2) 617 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-122", "mode": "writeback"}]: dispatch 2024-03-26T04:00:08.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:07 smithi190 bash[23392]: audit 2024-03-26T04:00:06.769226+0000 mon.a (mon.0) 3122 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-122", "mode": "writeback"}]: dispatch 2024-03-26T04:00:08.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:07 smithi190 bash[23392]: cluster 2024-03-26T04:00:06.993600+0000 mgr.y (mgr.24431) 606 : cluster [DBG] pgmap v982: 268 pgs: 3 creating+activating, 2 creating+peering, 263 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T04:00:08.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:07 smithi175 bash[21527]: audit 2024-03-26T04:00:06.751116+0000 mon.a (mon.0) 3120 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-122"}]': finished 2024-03-26T04:00:08.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:07 smithi175 bash[21527]: cluster 2024-03-26T04:00:06.759975+0000 mon.a (mon.0) 3121 : cluster [DBG] osdmap e610: 8 total, 8 up, 8 in 2024-03-26T04:00:08.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:07 smithi175 bash[21527]: audit 2024-03-26T04:00:06.762212+0000 mon.c (mon.2) 617 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-122", "mode": "writeback"}]: dispatch 2024-03-26T04:00:08.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:07 smithi175 bash[21527]: audit 2024-03-26T04:00:06.769226+0000 mon.a (mon.0) 3122 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-122", "mode": "writeback"}]: dispatch 2024-03-26T04:00:08.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:07 smithi175 bash[21527]: cluster 2024-03-26T04:00:06.993600+0000 mgr.y (mgr.24431) 606 : cluster [DBG] pgmap v982: 268 pgs: 3 creating+activating, 2 creating+peering, 263 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T04:00:08.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:07 smithi175 bash[29284]: audit 2024-03-26T04:00:06.751116+0000 mon.a (mon.0) 3120 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-122"}]': finished 2024-03-26T04:00:08.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:07 smithi175 bash[29284]: cluster 2024-03-26T04:00:06.759975+0000 mon.a (mon.0) 3121 : cluster [DBG] osdmap e610: 8 total, 8 up, 8 in 2024-03-26T04:00:08.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:07 smithi175 bash[29284]: audit 2024-03-26T04:00:06.762212+0000 mon.c (mon.2) 617 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-122", "mode": "writeback"}]: dispatch 2024-03-26T04:00:08.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:07 smithi175 bash[29284]: audit 2024-03-26T04:00:06.769226+0000 mon.a (mon.0) 3122 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-122", "mode": "writeback"}]: dispatch 2024-03-26T04:00:08.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:07 smithi175 bash[29284]: cluster 2024-03-26T04:00:06.993600+0000 mgr.y (mgr.24431) 606 : cluster [DBG] pgmap v982: 268 pgs: 3 creating+activating, 2 creating+peering, 263 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T04:00:09.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:08 smithi190 bash[23392]: cluster 2024-03-26T04:00:07.752351+0000 mon.a (mon.0) 3123 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:00:09.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:08 smithi190 bash[23392]: audit 2024-03-26T04:00:07.760525+0000 mon.a (mon.0) 3124 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-122", "mode": "writeback"}]': finished 2024-03-26T04:00:09.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:08 smithi190 bash[23392]: cluster 2024-03-26T04:00:07.769627+0000 mon.a (mon.0) 3125 : cluster [DBG] osdmap e611: 8 total, 8 up, 8 in 2024-03-26T04:00:09.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:08 smithi190 bash[23392]: audit 2024-03-26T04:00:07.903807+0000 mon.c (mon.2) 618 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:09.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:08 smithi190 bash[23392]: audit 2024-03-26T04:00:07.904499+0000 mon.a (mon.0) 3126 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:09.243 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:00:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:00:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:00:09.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:08 smithi175 bash[29284]: cluster 2024-03-26T04:00:07.752351+0000 mon.a (mon.0) 3123 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:00:09.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:08 smithi175 bash[29284]: audit 2024-03-26T04:00:07.760525+0000 mon.a (mon.0) 3124 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-122", "mode": "writeback"}]': finished 2024-03-26T04:00:09.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:08 smithi175 bash[29284]: cluster 2024-03-26T04:00:07.769627+0000 mon.a (mon.0) 3125 : cluster [DBG] osdmap e611: 8 total, 8 up, 8 in 2024-03-26T04:00:09.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:08 smithi175 bash[29284]: audit 2024-03-26T04:00:07.903807+0000 mon.c (mon.2) 618 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:09.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:08 smithi175 bash[29284]: audit 2024-03-26T04:00:07.904499+0000 mon.a (mon.0) 3126 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:09.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:08 smithi175 bash[21527]: cluster 2024-03-26T04:00:07.752351+0000 mon.a (mon.0) 3123 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:00:09.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:08 smithi175 bash[21527]: audit 2024-03-26T04:00:07.760525+0000 mon.a (mon.0) 3124 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-122", "mode": "writeback"}]': finished 2024-03-26T04:00:09.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:08 smithi175 bash[21527]: cluster 2024-03-26T04:00:07.769627+0000 mon.a (mon.0) 3125 : cluster [DBG] osdmap e611: 8 total, 8 up, 8 in 2024-03-26T04:00:09.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:08 smithi175 bash[21527]: audit 2024-03-26T04:00:07.903807+0000 mon.c (mon.2) 618 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:09.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:08 smithi175 bash[21527]: audit 2024-03-26T04:00:07.904499+0000 mon.a (mon.0) 3126 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:10.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:09 smithi190 bash[23392]: audit 2024-03-26T04:00:08.763687+0000 mon.a (mon.0) 3127 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:00:10.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:09 smithi190 bash[23392]: cluster 2024-03-26T04:00:08.773440+0000 mon.a (mon.0) 3128 : cluster [DBG] osdmap e612: 8 total, 8 up, 8 in 2024-03-26T04:00:10.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:09 smithi190 bash[23392]: audit 2024-03-26T04:00:08.776600+0000 mon.c (mon.2) 619 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-122"}]: dispatch 2024-03-26T04:00:10.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:09 smithi190 bash[23392]: audit 2024-03-26T04:00:08.790778+0000 mon.a (mon.0) 3129 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-122"}]: dispatch 2024-03-26T04:00:10.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:09 smithi190 bash[23392]: cluster 2024-03-26T04:00:08.994500+0000 mgr.y (mgr.24431) 607 : cluster [DBG] pgmap v985: 268 pgs: 3 creating+activating, 2 creating+peering, 263 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-03-26T04:00:10.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:09 smithi175 bash[21527]: audit 2024-03-26T04:00:08.763687+0000 mon.a (mon.0) 3127 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:00:10.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:09 smithi175 bash[21527]: cluster 2024-03-26T04:00:08.773440+0000 mon.a (mon.0) 3128 : cluster [DBG] osdmap e612: 8 total, 8 up, 8 in 2024-03-26T04:00:10.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:09 smithi175 bash[21527]: audit 2024-03-26T04:00:08.776600+0000 mon.c (mon.2) 619 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-122"}]: dispatch 2024-03-26T04:00:10.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:09 smithi175 bash[21527]: audit 2024-03-26T04:00:08.790778+0000 mon.a (mon.0) 3129 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-122"}]: dispatch 2024-03-26T04:00:10.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:09 smithi175 bash[21527]: cluster 2024-03-26T04:00:08.994500+0000 mgr.y (mgr.24431) 607 : cluster [DBG] pgmap v985: 268 pgs: 3 creating+activating, 2 creating+peering, 263 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-03-26T04:00:10.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:09 smithi175 bash[29284]: audit 2024-03-26T04:00:08.763687+0000 mon.a (mon.0) 3127 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:00:10.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:09 smithi175 bash[29284]: cluster 2024-03-26T04:00:08.773440+0000 mon.a (mon.0) 3128 : cluster [DBG] osdmap e612: 8 total, 8 up, 8 in 2024-03-26T04:00:10.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:09 smithi175 bash[29284]: audit 2024-03-26T04:00:08.776600+0000 mon.c (mon.2) 619 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-122"}]: dispatch 2024-03-26T04:00:10.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:09 smithi175 bash[29284]: audit 2024-03-26T04:00:08.790778+0000 mon.a (mon.0) 3129 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-122"}]: dispatch 2024-03-26T04:00:10.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:09 smithi175 bash[29284]: cluster 2024-03-26T04:00:08.994500+0000 mgr.y (mgr.24431) 607 : cluster [DBG] pgmap v985: 268 pgs: 3 creating+activating, 2 creating+peering, 263 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-03-26T04:00:10.741 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:00:10 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:00:11.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:10 smithi190 bash[23392]: cluster 2024-03-26T04:00:09.779928+0000 mon.a (mon.0) 3130 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:00:11.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:10 smithi190 bash[23392]: audit 2024-03-26T04:00:09.788216+0000 mon.a (mon.0) 3131 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-122"}]': finished 2024-03-26T04:00:11.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:10 smithi190 bash[23392]: cluster 2024-03-26T04:00:09.798319+0000 mon.a (mon.0) 3132 : cluster [DBG] osdmap e613: 8 total, 8 up, 8 in 2024-03-26T04:00:11.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:10 smithi190 bash[23392]: audit 2024-03-26T04:00:10.365725+0000 mgr.y (mgr.24431) 608 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:00:11.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:10 smithi175 bash[21527]: cluster 2024-03-26T04:00:09.779928+0000 mon.a (mon.0) 3130 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:00:11.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:10 smithi175 bash[21527]: audit 2024-03-26T04:00:09.788216+0000 mon.a (mon.0) 3131 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-122"}]': finished 2024-03-26T04:00:11.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:10 smithi175 bash[21527]: cluster 2024-03-26T04:00:09.798319+0000 mon.a (mon.0) 3132 : cluster [DBG] osdmap e613: 8 total, 8 up, 8 in 2024-03-26T04:00:11.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:10 smithi175 bash[21527]: audit 2024-03-26T04:00:10.365725+0000 mgr.y (mgr.24431) 608 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:00:11.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:10 smithi175 bash[29284]: cluster 2024-03-26T04:00:09.779928+0000 mon.a (mon.0) 3130 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:00:11.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:10 smithi175 bash[29284]: audit 2024-03-26T04:00:09.788216+0000 mon.a (mon.0) 3131 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-122"}]': finished 2024-03-26T04:00:11.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:10 smithi175 bash[29284]: cluster 2024-03-26T04:00:09.798319+0000 mon.a (mon.0) 3132 : cluster [DBG] osdmap e613: 8 total, 8 up, 8 in 2024-03-26T04:00:11.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:10 smithi175 bash[29284]: audit 2024-03-26T04:00:10.365725+0000 mgr.y (mgr.24431) 608 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:00:12.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:11 smithi190 bash[23392]: cluster 2024-03-26T04:00:10.806539+0000 mon.a (mon.0) 3133 : cluster [DBG] osdmap e614: 8 total, 8 up, 8 in 2024-03-26T04:00:12.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:11 smithi190 bash[23392]: cluster 2024-03-26T04:00:10.995527+0000 mgr.y (mgr.24431) 609 : cluster [DBG] pgmap v988: 236 pgs: 236 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-26T04:00:12.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:11 smithi175 bash[21527]: cluster 2024-03-26T04:00:10.806539+0000 mon.a (mon.0) 3133 : cluster [DBG] osdmap e614: 8 total, 8 up, 8 in 2024-03-26T04:00:12.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:11 smithi175 bash[21527]: cluster 2024-03-26T04:00:10.995527+0000 mgr.y (mgr.24431) 609 : cluster [DBG] pgmap v988: 236 pgs: 236 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-26T04:00:12.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:11 smithi175 bash[29284]: cluster 2024-03-26T04:00:10.806539+0000 mon.a (mon.0) 3133 : cluster [DBG] osdmap e614: 8 total, 8 up, 8 in 2024-03-26T04:00:12.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:11 smithi175 bash[29284]: cluster 2024-03-26T04:00:10.995527+0000 mgr.y (mgr.24431) 609 : cluster [DBG] pgmap v988: 236 pgs: 236 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-26T04:00:13.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:12 smithi175 bash[29284]: cluster 2024-03-26T04:00:11.823162+0000 mon.a (mon.0) 3134 : cluster [DBG] osdmap e615: 8 total, 8 up, 8 in 2024-03-26T04:00:13.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:12 smithi175 bash[29284]: audit 2024-03-26T04:00:11.828003+0000 mon.c (mon.2) 620 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:13.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:12 smithi175 bash[29284]: audit 2024-03-26T04:00:11.829007+0000 mon.a (mon.0) 3135 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:13.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:12 smithi190 bash[23392]: cluster 2024-03-26T04:00:11.823162+0000 mon.a (mon.0) 3134 : cluster [DBG] osdmap e615: 8 total, 8 up, 8 in 2024-03-26T04:00:13.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:12 smithi190 bash[23392]: audit 2024-03-26T04:00:11.828003+0000 mon.c (mon.2) 620 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:13.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:12 smithi190 bash[23392]: audit 2024-03-26T04:00:11.829007+0000 mon.a (mon.0) 3135 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:13.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:12 smithi175 bash[21527]: cluster 2024-03-26T04:00:11.823162+0000 mon.a (mon.0) 3134 : cluster [DBG] osdmap e615: 8 total, 8 up, 8 in 2024-03-26T04:00:13.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:12 smithi175 bash[21527]: audit 2024-03-26T04:00:11.828003+0000 mon.c (mon.2) 620 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:13.245 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:12 smithi175 bash[21527]: audit 2024-03-26T04:00:11.829007+0000 mon.a (mon.0) 3135 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:14.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:13 smithi190 bash[23392]: audit 2024-03-26T04:00:12.816883+0000 mon.a (mon.0) 3136 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:00:14.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:13 smithi190 bash[23392]: cluster 2024-03-26T04:00:12.824198+0000 mon.a (mon.0) 3137 : cluster [DBG] osdmap e616: 8 total, 8 up, 8 in 2024-03-26T04:00:14.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:13 smithi190 bash[23392]: cluster 2024-03-26T04:00:12.996429+0000 mgr.y (mgr.24431) 610 : cluster [DBG] pgmap v991: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-26T04:00:14.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:13 smithi175 bash[21527]: audit 2024-03-26T04:00:12.816883+0000 mon.a (mon.0) 3136 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:00:14.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:13 smithi175 bash[21527]: cluster 2024-03-26T04:00:12.824198+0000 mon.a (mon.0) 3137 : cluster [DBG] osdmap e616: 8 total, 8 up, 8 in 2024-03-26T04:00:14.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:13 smithi175 bash[21527]: cluster 2024-03-26T04:00:12.996429+0000 mgr.y (mgr.24431) 610 : cluster [DBG] pgmap v991: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-26T04:00:14.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:13 smithi175 bash[29284]: audit 2024-03-26T04:00:12.816883+0000 mon.a (mon.0) 3136 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:00:14.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:13 smithi175 bash[29284]: cluster 2024-03-26T04:00:12.824198+0000 mon.a (mon.0) 3137 : cluster [DBG] osdmap e616: 8 total, 8 up, 8 in 2024-03-26T04:00:14.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:13 smithi175 bash[29284]: cluster 2024-03-26T04:00:12.996429+0000 mgr.y (mgr.24431) 610 : cluster [DBG] pgmap v991: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-26T04:00:15.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:14 smithi190 bash[23392]: cluster 2024-03-26T04:00:13.852015+0000 mon.a (mon.0) 3138 : cluster [DBG] osdmap e617: 8 total, 8 up, 8 in 2024-03-26T04:00:15.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:14 smithi190 bash[23392]: audit 2024-03-26T04:00:13.885466+0000 mon.c (mon.2) 621 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:15.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:14 smithi190 bash[23392]: audit 2024-03-26T04:00:13.886418+0000 mon.a (mon.0) 3139 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:15.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:14 smithi175 bash[21527]: cluster 2024-03-26T04:00:13.852015+0000 mon.a (mon.0) 3138 : cluster [DBG] osdmap e617: 8 total, 8 up, 8 in 2024-03-26T04:00:15.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:14 smithi175 bash[21527]: audit 2024-03-26T04:00:13.885466+0000 mon.c (mon.2) 621 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:15.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:14 smithi175 bash[21527]: audit 2024-03-26T04:00:13.886418+0000 mon.a (mon.0) 3139 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:15.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:14 smithi175 bash[29284]: cluster 2024-03-26T04:00:13.852015+0000 mon.a (mon.0) 3138 : cluster [DBG] osdmap e617: 8 total, 8 up, 8 in 2024-03-26T04:00:15.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:14 smithi175 bash[29284]: audit 2024-03-26T04:00:13.885466+0000 mon.c (mon.2) 621 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:15.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:14 smithi175 bash[29284]: audit 2024-03-26T04:00:13.886418+0000 mon.a (mon.0) 3139 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:16.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:15 smithi190 bash[23392]: audit 2024-03-26T04:00:14.847582+0000 mon.a (mon.0) 3140 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-124", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:00:16.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:15 smithi190 bash[23392]: cluster 2024-03-26T04:00:14.855310+0000 mon.a (mon.0) 3141 : cluster [DBG] osdmap e618: 8 total, 8 up, 8 in 2024-03-26T04:00:16.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:15 smithi190 bash[23392]: audit 2024-03-26T04:00:14.857835+0000 mon.c (mon.2) 622 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-124"}]: dispatch 2024-03-26T04:00:16.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:15 smithi190 bash[23392]: audit 2024-03-26T04:00:14.873073+0000 mon.a (mon.0) 3142 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-124"}]: dispatch 2024-03-26T04:00:16.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:15 smithi190 bash[23392]: cluster 2024-03-26T04:00:14.997902+0000 mgr.y (mgr.24431) 611 : cluster [DBG] pgmap v994: 268 pgs: 268 active+clean; 583 KiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-26T04:00:16.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:15 smithi175 bash[21527]: audit 2024-03-26T04:00:14.847582+0000 mon.a (mon.0) 3140 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-124", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:00:16.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:15 smithi175 bash[21527]: cluster 2024-03-26T04:00:14.855310+0000 mon.a (mon.0) 3141 : cluster [DBG] osdmap e618: 8 total, 8 up, 8 in 2024-03-26T04:00:16.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:15 smithi175 bash[21527]: audit 2024-03-26T04:00:14.857835+0000 mon.c (mon.2) 622 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-124"}]: dispatch 2024-03-26T04:00:16.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:15 smithi175 bash[21527]: audit 2024-03-26T04:00:14.873073+0000 mon.a (mon.0) 3142 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-124"}]: dispatch 2024-03-26T04:00:16.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:15 smithi175 bash[21527]: cluster 2024-03-26T04:00:14.997902+0000 mgr.y (mgr.24431) 611 : cluster [DBG] pgmap v994: 268 pgs: 268 active+clean; 583 KiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-26T04:00:16.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:15 smithi175 bash[29284]: audit 2024-03-26T04:00:14.847582+0000 mon.a (mon.0) 3140 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-124", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:00:16.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:15 smithi175 bash[29284]: cluster 2024-03-26T04:00:14.855310+0000 mon.a (mon.0) 3141 : cluster [DBG] osdmap e618: 8 total, 8 up, 8 in 2024-03-26T04:00:16.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:15 smithi175 bash[29284]: audit 2024-03-26T04:00:14.857835+0000 mon.c (mon.2) 622 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-124"}]: dispatch 2024-03-26T04:00:16.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:15 smithi175 bash[29284]: audit 2024-03-26T04:00:14.873073+0000 mon.a (mon.0) 3142 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-124"}]: dispatch 2024-03-26T04:00:16.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:15 smithi175 bash[29284]: cluster 2024-03-26T04:00:14.997902+0000 mgr.y (mgr.24431) 611 : cluster [DBG] pgmap v994: 268 pgs: 268 active+clean; 583 KiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-26T04:00:17.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:16 smithi190 bash[23392]: audit 2024-03-26T04:00:15.871858+0000 mon.a (mon.0) 3143 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-124"}]': finished 2024-03-26T04:00:17.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:16 smithi190 bash[23392]: cluster 2024-03-26T04:00:15.884511+0000 mon.a (mon.0) 3144 : cluster [DBG] osdmap e619: 8 total, 8 up, 8 in 2024-03-26T04:00:17.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:16 smithi190 bash[23392]: audit 2024-03-26T04:00:15.886923+0000 mon.c (mon.2) 623 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-124", "mode": "writeback"}]: dispatch 2024-03-26T04:00:17.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:16 smithi190 bash[23392]: audit 2024-03-26T04:00:15.888011+0000 mon.a (mon.0) 3145 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-124", "mode": "writeback"}]: dispatch 2024-03-26T04:00:17.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:16 smithi175 bash[29284]: audit 2024-03-26T04:00:15.871858+0000 mon.a (mon.0) 3143 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-124"}]': finished 2024-03-26T04:00:17.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:16 smithi175 bash[29284]: cluster 2024-03-26T04:00:15.884511+0000 mon.a (mon.0) 3144 : cluster [DBG] osdmap e619: 8 total, 8 up, 8 in 2024-03-26T04:00:17.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:16 smithi175 bash[29284]: audit 2024-03-26T04:00:15.886923+0000 mon.c (mon.2) 623 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-124", "mode": "writeback"}]: dispatch 2024-03-26T04:00:17.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:16 smithi175 bash[29284]: audit 2024-03-26T04:00:15.888011+0000 mon.a (mon.0) 3145 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-124", "mode": "writeback"}]: dispatch 2024-03-26T04:00:17.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:16 smithi175 bash[21527]: audit 2024-03-26T04:00:15.871858+0000 mon.a (mon.0) 3143 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-124"}]': finished 2024-03-26T04:00:17.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:16 smithi175 bash[21527]: cluster 2024-03-26T04:00:15.884511+0000 mon.a (mon.0) 3144 : cluster [DBG] osdmap e619: 8 total, 8 up, 8 in 2024-03-26T04:00:17.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:16 smithi175 bash[21527]: audit 2024-03-26T04:00:15.886923+0000 mon.c (mon.2) 623 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-124", "mode": "writeback"}]: dispatch 2024-03-26T04:00:17.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:16 smithi175 bash[21527]: audit 2024-03-26T04:00:15.888011+0000 mon.a (mon.0) 3145 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-124", "mode": "writeback"}]: dispatch 2024-03-26T04:00:18.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:17 smithi190 bash[23392]: cluster 2024-03-26T04:00:16.872343+0000 mon.a (mon.0) 3146 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:00:18.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:17 smithi190 bash[23392]: audit 2024-03-26T04:00:16.874766+0000 mon.a (mon.0) 3147 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-124", "mode": "writeback"}]': finished 2024-03-26T04:00:18.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:17 smithi190 bash[23392]: cluster 2024-03-26T04:00:16.884391+0000 mon.a (mon.0) 3148 : cluster [DBG] osdmap e620: 8 total, 8 up, 8 in 2024-03-26T04:00:18.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:17 smithi190 bash[23392]: cluster 2024-03-26T04:00:16.998803+0000 mgr.y (mgr.24431) 612 : cluster [DBG] pgmap v997: 268 pgs: 268 active+clean; 583 KiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-26T04:00:18.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:17 smithi175 bash[29284]: cluster 2024-03-26T04:00:16.872343+0000 mon.a (mon.0) 3146 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:00:18.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:17 smithi175 bash[29284]: audit 2024-03-26T04:00:16.874766+0000 mon.a (mon.0) 3147 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-124", "mode": "writeback"}]': finished 2024-03-26T04:00:18.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:17 smithi175 bash[29284]: cluster 2024-03-26T04:00:16.884391+0000 mon.a (mon.0) 3148 : cluster [DBG] osdmap e620: 8 total, 8 up, 8 in 2024-03-26T04:00:18.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:17 smithi175 bash[29284]: cluster 2024-03-26T04:00:16.998803+0000 mgr.y (mgr.24431) 612 : cluster [DBG] pgmap v997: 268 pgs: 268 active+clean; 583 KiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-26T04:00:18.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:17 smithi175 bash[21527]: cluster 2024-03-26T04:00:16.872343+0000 mon.a (mon.0) 3146 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:00:18.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:17 smithi175 bash[21527]: audit 2024-03-26T04:00:16.874766+0000 mon.a (mon.0) 3147 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-124", "mode": "writeback"}]': finished 2024-03-26T04:00:18.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:17 smithi175 bash[21527]: cluster 2024-03-26T04:00:16.884391+0000 mon.a (mon.0) 3148 : cluster [DBG] osdmap e620: 8 total, 8 up, 8 in 2024-03-26T04:00:18.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:17 smithi175 bash[21527]: cluster 2024-03-26T04:00:16.998803+0000 mgr.y (mgr.24431) 612 : cluster [DBG] pgmap v997: 268 pgs: 268 active+clean; 583 KiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-26T04:00:19.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:18 smithi190 bash[23392]: cluster 2024-03-26T04:00:17.911259+0000 mon.a (mon.0) 3149 : cluster [DBG] osdmap e621: 8 total, 8 up, 8 in 2024-03-26T04:00:19.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:18 smithi190 bash[23392]: audit 2024-03-26T04:00:17.983934+0000 mon.c (mon.2) 624 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:19.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:18 smithi190 bash[23392]: audit 2024-03-26T04:00:17.984539+0000 mon.a (mon.0) 3150 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:19.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:00:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:00:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:00:19.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:18 smithi175 bash[29284]: cluster 2024-03-26T04:00:17.911259+0000 mon.a (mon.0) 3149 : cluster [DBG] osdmap e621: 8 total, 8 up, 8 in 2024-03-26T04:00:19.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:18 smithi175 bash[29284]: audit 2024-03-26T04:00:17.983934+0000 mon.c (mon.2) 624 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:19.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:18 smithi175 bash[29284]: audit 2024-03-26T04:00:17.984539+0000 mon.a (mon.0) 3150 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:19.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:18 smithi175 bash[21527]: cluster 2024-03-26T04:00:17.911259+0000 mon.a (mon.0) 3149 : cluster [DBG] osdmap e621: 8 total, 8 up, 8 in 2024-03-26T04:00:19.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:18 smithi175 bash[21527]: audit 2024-03-26T04:00:17.983934+0000 mon.c (mon.2) 624 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:19.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:18 smithi175 bash[21527]: audit 2024-03-26T04:00:17.984539+0000 mon.a (mon.0) 3150 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:20.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:19 smithi190 bash[23392]: audit 2024-03-26T04:00:18.904770+0000 mon.a (mon.0) 3151 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:00:20.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:19 smithi190 bash[23392]: cluster 2024-03-26T04:00:18.912450+0000 mon.a (mon.0) 3152 : cluster [DBG] osdmap e622: 8 total, 8 up, 8 in 2024-03-26T04:00:20.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:19 smithi190 bash[23392]: audit 2024-03-26T04:00:18.914449+0000 mon.c (mon.2) 625 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-124"}]: dispatch 2024-03-26T04:00:20.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:19 smithi190 bash[23392]: audit 2024-03-26T04:00:18.917304+0000 mon.a (mon.0) 3153 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-124"}]: dispatch 2024-03-26T04:00:20.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:19 smithi190 bash[23392]: cluster 2024-03-26T04:00:18.999703+0000 mgr.y (mgr.24431) 613 : cluster [DBG] pgmap v1000: 268 pgs: 268 active+clean; 583 KiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-03-26T04:00:20.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:19 smithi190 bash[23392]: cluster 2024-03-26T04:00:19.905243+0000 mon.a (mon.0) 3154 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:00:20.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:19 smithi190 bash[23392]: audit 2024-03-26T04:00:19.907934+0000 mon.a (mon.0) 3155 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-124"}]': finished 2024-03-26T04:00:20.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:19 smithi190 bash[23392]: cluster 2024-03-26T04:00:19.916971+0000 mon.a (mon.0) 3156 : cluster [DBG] osdmap e623: 8 total, 8 up, 8 in 2024-03-26T04:00:20.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:19 smithi175 bash[21527]: audit 2024-03-26T04:00:18.904770+0000 mon.a (mon.0) 3151 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:00:20.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:19 smithi175 bash[21527]: cluster 2024-03-26T04:00:18.912450+0000 mon.a (mon.0) 3152 : cluster [DBG] osdmap e622: 8 total, 8 up, 8 in 2024-03-26T04:00:20.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:19 smithi175 bash[21527]: audit 2024-03-26T04:00:18.914449+0000 mon.c (mon.2) 625 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-124"}]: dispatch 2024-03-26T04:00:20.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:19 smithi175 bash[21527]: audit 2024-03-26T04:00:18.917304+0000 mon.a (mon.0) 3153 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-124"}]: dispatch 2024-03-26T04:00:20.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:19 smithi175 bash[21527]: cluster 2024-03-26T04:00:18.999703+0000 mgr.y (mgr.24431) 613 : cluster [DBG] pgmap v1000: 268 pgs: 268 active+clean; 583 KiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-03-26T04:00:20.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:19 smithi175 bash[21527]: cluster 2024-03-26T04:00:19.905243+0000 mon.a (mon.0) 3154 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:00:20.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:19 smithi175 bash[21527]: audit 2024-03-26T04:00:19.907934+0000 mon.a (mon.0) 3155 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-124"}]': finished 2024-03-26T04:00:20.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:19 smithi175 bash[21527]: cluster 2024-03-26T04:00:19.916971+0000 mon.a (mon.0) 3156 : cluster [DBG] osdmap e623: 8 total, 8 up, 8 in 2024-03-26T04:00:20.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:19 smithi175 bash[29284]: audit 2024-03-26T04:00:18.904770+0000 mon.a (mon.0) 3151 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:00:20.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:19 smithi175 bash[29284]: cluster 2024-03-26T04:00:18.912450+0000 mon.a (mon.0) 3152 : cluster [DBG] osdmap e622: 8 total, 8 up, 8 in 2024-03-26T04:00:20.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:19 smithi175 bash[29284]: audit 2024-03-26T04:00:18.914449+0000 mon.c (mon.2) 625 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-124"}]: dispatch 2024-03-26T04:00:20.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:19 smithi175 bash[29284]: audit 2024-03-26T04:00:18.917304+0000 mon.a (mon.0) 3153 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-124"}]: dispatch 2024-03-26T04:00:20.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:19 smithi175 bash[29284]: cluster 2024-03-26T04:00:18.999703+0000 mgr.y (mgr.24431) 613 : cluster [DBG] pgmap v1000: 268 pgs: 268 active+clean; 583 KiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-03-26T04:00:20.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:19 smithi175 bash[29284]: cluster 2024-03-26T04:00:19.905243+0000 mon.a (mon.0) 3154 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:00:20.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:19 smithi175 bash[29284]: audit 2024-03-26T04:00:19.907934+0000 mon.a (mon.0) 3155 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-124"}]': finished 2024-03-26T04:00:20.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:19 smithi175 bash[29284]: cluster 2024-03-26T04:00:19.916971+0000 mon.a (mon.0) 3156 : cluster [DBG] osdmap e623: 8 total, 8 up, 8 in 2024-03-26T04:00:20.741 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:00:20 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:00:21.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:20 smithi190 bash[23392]: audit 2024-03-26T04:00:20.373700+0000 mgr.y (mgr.24431) 614 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:00:21.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:20 smithi190 bash[23392]: audit 2024-03-26T04:00:20.801787+0000 mon.c (mon.2) 626 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:00:21.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:20 smithi175 bash[21527]: audit 2024-03-26T04:00:20.373700+0000 mgr.y (mgr.24431) 614 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:00:21.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:20 smithi175 bash[21527]: audit 2024-03-26T04:00:20.801787+0000 mon.c (mon.2) 626 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:00:21.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:20 smithi175 bash[29284]: audit 2024-03-26T04:00:20.373700+0000 mgr.y (mgr.24431) 614 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:00:21.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:20 smithi175 bash[29284]: audit 2024-03-26T04:00:20.801787+0000 mon.c (mon.2) 626 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:00:22.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:21 smithi190 bash[23392]: cluster 2024-03-26T04:00:20.957519+0000 mon.a (mon.0) 3157 : cluster [DBG] osdmap e624: 8 total, 8 up, 8 in 2024-03-26T04:00:22.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:21 smithi190 bash[23392]: cluster 2024-03-26T04:00:21.000431+0000 mgr.y (mgr.24431) 615 : cluster [DBG] pgmap v1003: 236 pgs: 236 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-03-26T04:00:22.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:21 smithi175 bash[29284]: cluster 2024-03-26T04:00:20.957519+0000 mon.a (mon.0) 3157 : cluster [DBG] osdmap e624: 8 total, 8 up, 8 in 2024-03-26T04:00:22.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:21 smithi175 bash[29284]: cluster 2024-03-26T04:00:21.000431+0000 mgr.y (mgr.24431) 615 : cluster [DBG] pgmap v1003: 236 pgs: 236 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-03-26T04:00:22.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:21 smithi175 bash[21527]: cluster 2024-03-26T04:00:20.957519+0000 mon.a (mon.0) 3157 : cluster [DBG] osdmap e624: 8 total, 8 up, 8 in 2024-03-26T04:00:22.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:21 smithi175 bash[21527]: cluster 2024-03-26T04:00:21.000431+0000 mgr.y (mgr.24431) 615 : cluster [DBG] pgmap v1003: 236 pgs: 236 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-03-26T04:00:23.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:23 smithi190 bash[23392]: cluster 2024-03-26T04:00:21.974960+0000 mon.a (mon.0) 3158 : cluster [DBG] osdmap e625: 8 total, 8 up, 8 in 2024-03-26T04:00:23.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:23 smithi190 bash[23392]: audit 2024-03-26T04:00:21.978298+0000 mon.c (mon.2) 627 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:23.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:23 smithi190 bash[23392]: audit 2024-03-26T04:00:21.981921+0000 mon.a (mon.0) 3159 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:23.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:23 smithi175 bash[29284]: cluster 2024-03-26T04:00:21.974960+0000 mon.a (mon.0) 3158 : cluster [DBG] osdmap e625: 8 total, 8 up, 8 in 2024-03-26T04:00:23.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:23 smithi175 bash[29284]: audit 2024-03-26T04:00:21.978298+0000 mon.c (mon.2) 627 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:23.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:23 smithi175 bash[29284]: audit 2024-03-26T04:00:21.981921+0000 mon.a (mon.0) 3159 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:23.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:23 smithi175 bash[21527]: cluster 2024-03-26T04:00:21.974960+0000 mon.a (mon.0) 3158 : cluster [DBG] osdmap e625: 8 total, 8 up, 8 in 2024-03-26T04:00:23.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:23 smithi175 bash[21527]: audit 2024-03-26T04:00:21.978298+0000 mon.c (mon.2) 627 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:23.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:23 smithi175 bash[21527]: audit 2024-03-26T04:00:21.981921+0000 mon.a (mon.0) 3159 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:24.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:24 smithi190 bash[23392]: audit 2024-03-26T04:00:22.968682+0000 mon.a (mon.0) 3160 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:00:24.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:24 smithi190 bash[23392]: cluster 2024-03-26T04:00:22.984998+0000 mon.a (mon.0) 3161 : cluster [DBG] osdmap e626: 8 total, 8 up, 8 in 2024-03-26T04:00:24.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:24 smithi190 bash[23392]: audit 2024-03-26T04:00:22.986963+0000 mon.c (mon.2) 628 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:24.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:24 smithi190 bash[23392]: audit 2024-03-26T04:00:22.990322+0000 mon.a (mon.0) 3162 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:24.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:24 smithi190 bash[23392]: cluster 2024-03-26T04:00:23.002389+0000 mgr.y (mgr.24431) 616 : cluster [DBG] pgmap v1006: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-03-26T04:00:24.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:24 smithi190 bash[23392]: audit 2024-03-26T04:00:23.971663+0000 mon.a (mon.0) 3163 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-126", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:00:24.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:24 smithi190 bash[23392]: cluster 2024-03-26T04:00:23.980705+0000 mon.a (mon.0) 3164 : cluster [DBG] osdmap e627: 8 total, 8 up, 8 in 2024-03-26T04:00:24.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:24 smithi190 bash[23392]: audit 2024-03-26T04:00:23.986612+0000 mon.c (mon.2) 629 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-126"}]: dispatch 2024-03-26T04:00:24.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:24 smithi190 bash[23392]: audit 2024-03-26T04:00:23.987516+0000 mon.a (mon.0) 3165 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-126"}]: dispatch 2024-03-26T04:00:24.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:24 smithi175 bash[21527]: audit 2024-03-26T04:00:22.968682+0000 mon.a (mon.0) 3160 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:00:24.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:24 smithi175 bash[21527]: cluster 2024-03-26T04:00:22.984998+0000 mon.a (mon.0) 3161 : cluster [DBG] osdmap e626: 8 total, 8 up, 8 in 2024-03-26T04:00:24.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:24 smithi175 bash[21527]: audit 2024-03-26T04:00:22.986963+0000 mon.c (mon.2) 628 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:24.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:24 smithi175 bash[21527]: audit 2024-03-26T04:00:22.990322+0000 mon.a (mon.0) 3162 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:24.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:24 smithi175 bash[21527]: cluster 2024-03-26T04:00:23.002389+0000 mgr.y (mgr.24431) 616 : cluster [DBG] pgmap v1006: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-03-26T04:00:24.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:24 smithi175 bash[21527]: audit 2024-03-26T04:00:23.971663+0000 mon.a (mon.0) 3163 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-126", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:00:24.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:24 smithi175 bash[21527]: cluster 2024-03-26T04:00:23.980705+0000 mon.a (mon.0) 3164 : cluster [DBG] osdmap e627: 8 total, 8 up, 8 in 2024-03-26T04:00:24.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:24 smithi175 bash[21527]: audit 2024-03-26T04:00:23.986612+0000 mon.c (mon.2) 629 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-126"}]: dispatch 2024-03-26T04:00:24.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:24 smithi175 bash[21527]: audit 2024-03-26T04:00:23.987516+0000 mon.a (mon.0) 3165 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-126"}]: dispatch 2024-03-26T04:00:24.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:24 smithi175 bash[29284]: audit 2024-03-26T04:00:22.968682+0000 mon.a (mon.0) 3160 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:00:24.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:24 smithi175 bash[29284]: cluster 2024-03-26T04:00:22.984998+0000 mon.a (mon.0) 3161 : cluster [DBG] osdmap e626: 8 total, 8 up, 8 in 2024-03-26T04:00:24.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:24 smithi175 bash[29284]: audit 2024-03-26T04:00:22.986963+0000 mon.c (mon.2) 628 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:24.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:24 smithi175 bash[29284]: audit 2024-03-26T04:00:22.990322+0000 mon.a (mon.0) 3162 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:24.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:24 smithi175 bash[29284]: cluster 2024-03-26T04:00:23.002389+0000 mgr.y (mgr.24431) 616 : cluster [DBG] pgmap v1006: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-03-26T04:00:24.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:24 smithi175 bash[29284]: audit 2024-03-26T04:00:23.971663+0000 mon.a (mon.0) 3163 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-126", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:00:24.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:24 smithi175 bash[29284]: cluster 2024-03-26T04:00:23.980705+0000 mon.a (mon.0) 3164 : cluster [DBG] osdmap e627: 8 total, 8 up, 8 in 2024-03-26T04:00:24.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:24 smithi175 bash[29284]: audit 2024-03-26T04:00:23.986612+0000 mon.c (mon.2) 629 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-126"}]: dispatch 2024-03-26T04:00:24.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:24 smithi175 bash[29284]: audit 2024-03-26T04:00:23.987516+0000 mon.a (mon.0) 3165 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-126"}]: dispatch 2024-03-26T04:00:26.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:25 smithi190 bash[23392]: audit 2024-03-26T04:00:24.974403+0000 mon.a (mon.0) 3166 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-126"}]': finished 2024-03-26T04:00:26.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:25 smithi190 bash[23392]: cluster 2024-03-26T04:00:24.982333+0000 mon.a (mon.0) 3167 : cluster [DBG] osdmap e628: 8 total, 8 up, 8 in 2024-03-26T04:00:26.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:25 smithi190 bash[23392]: audit 2024-03-26T04:00:24.985341+0000 mon.c (mon.2) 630 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-126", "mode": "writeback"}]: dispatch 2024-03-26T04:00:26.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:25 smithi190 bash[23392]: audit 2024-03-26T04:00:24.986177+0000 mon.a (mon.0) 3168 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-126", "mode": "writeback"}]: dispatch 2024-03-26T04:00:26.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:25 smithi190 bash[23392]: cluster 2024-03-26T04:00:25.003281+0000 mgr.y (mgr.24431) 617 : cluster [DBG] pgmap v1009: 268 pgs: 268 active+clean; 582 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:00:26.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:25 smithi175 bash[29284]: audit 2024-03-26T04:00:24.974403+0000 mon.a (mon.0) 3166 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-126"}]': finished 2024-03-26T04:00:26.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:25 smithi175 bash[29284]: cluster 2024-03-26T04:00:24.982333+0000 mon.a (mon.0) 3167 : cluster [DBG] osdmap e628: 8 total, 8 up, 8 in 2024-03-26T04:00:26.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:25 smithi175 bash[29284]: audit 2024-03-26T04:00:24.985341+0000 mon.c (mon.2) 630 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-126", "mode": "writeback"}]: dispatch 2024-03-26T04:00:26.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:25 smithi175 bash[29284]: audit 2024-03-26T04:00:24.986177+0000 mon.a (mon.0) 3168 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-126", "mode": "writeback"}]: dispatch 2024-03-26T04:00:26.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:25 smithi175 bash[29284]: cluster 2024-03-26T04:00:25.003281+0000 mgr.y (mgr.24431) 617 : cluster [DBG] pgmap v1009: 268 pgs: 268 active+clean; 582 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:00:26.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:25 smithi175 bash[21527]: audit 2024-03-26T04:00:24.974403+0000 mon.a (mon.0) 3166 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-126"}]': finished 2024-03-26T04:00:26.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:25 smithi175 bash[21527]: cluster 2024-03-26T04:00:24.982333+0000 mon.a (mon.0) 3167 : cluster [DBG] osdmap e628: 8 total, 8 up, 8 in 2024-03-26T04:00:26.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:25 smithi175 bash[21527]: audit 2024-03-26T04:00:24.985341+0000 mon.c (mon.2) 630 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-126", "mode": "writeback"}]: dispatch 2024-03-26T04:00:26.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:25 smithi175 bash[21527]: audit 2024-03-26T04:00:24.986177+0000 mon.a (mon.0) 3168 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-126", "mode": "writeback"}]: dispatch 2024-03-26T04:00:26.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:25 smithi175 bash[21527]: cluster 2024-03-26T04:00:25.003281+0000 mgr.y (mgr.24431) 617 : cluster [DBG] pgmap v1009: 268 pgs: 268 active+clean; 582 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:00:27.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:26 smithi190 bash[23392]: cluster 2024-03-26T04:00:25.975548+0000 mon.a (mon.0) 3169 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:00:27.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:26 smithi190 bash[23392]: audit 2024-03-26T04:00:25.983296+0000 mon.a (mon.0) 3170 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-126", "mode": "writeback"}]': finished 2024-03-26T04:00:27.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:26 smithi190 bash[23392]: cluster 2024-03-26T04:00:25.993043+0000 mon.a (mon.0) 3171 : cluster [DBG] osdmap e629: 8 total, 8 up, 8 in 2024-03-26T04:00:27.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:26 smithi190 bash[23392]: audit 2024-03-26T04:00:26.137387+0000 mon.c (mon.2) 631 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:27.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:26 smithi190 bash[23392]: audit 2024-03-26T04:00:26.138118+0000 mon.a (mon.0) 3172 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:27.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:26 smithi190 bash[23392]: audit 2024-03-26T04:00:26.644554+0000 mon.a (mon.0) 3173 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:00:27.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:26 smithi190 bash[23392]: cluster 2024-03-26T04:00:26.652992+0000 mon.a (mon.0) 3174 : cluster [DBG] osdmap e630: 8 total, 8 up, 8 in 2024-03-26T04:00:27.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:26 smithi190 bash[23392]: audit 2024-03-26T04:00:26.657403+0000 mon.c (mon.2) 632 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-126"}]: dispatch 2024-03-26T04:00:27.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:26 smithi190 bash[23392]: audit 2024-03-26T04:00:26.659706+0000 mon.a (mon.0) 3175 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-126"}]: dispatch 2024-03-26T04:00:27.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:26 smithi175 bash[21527]: cluster 2024-03-26T04:00:25.975548+0000 mon.a (mon.0) 3169 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:00:27.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:26 smithi175 bash[21527]: audit 2024-03-26T04:00:25.983296+0000 mon.a (mon.0) 3170 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-126", "mode": "writeback"}]': finished 2024-03-26T04:00:27.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:26 smithi175 bash[21527]: cluster 2024-03-26T04:00:25.993043+0000 mon.a (mon.0) 3171 : cluster [DBG] osdmap e629: 8 total, 8 up, 8 in 2024-03-26T04:00:27.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:26 smithi175 bash[21527]: audit 2024-03-26T04:00:26.137387+0000 mon.c (mon.2) 631 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:27.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:26 smithi175 bash[21527]: audit 2024-03-26T04:00:26.138118+0000 mon.a (mon.0) 3172 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:27.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:26 smithi175 bash[21527]: audit 2024-03-26T04:00:26.644554+0000 mon.a (mon.0) 3173 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:00:27.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:26 smithi175 bash[21527]: cluster 2024-03-26T04:00:26.652992+0000 mon.a (mon.0) 3174 : cluster [DBG] osdmap e630: 8 total, 8 up, 8 in 2024-03-26T04:00:27.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:26 smithi175 bash[21527]: audit 2024-03-26T04:00:26.657403+0000 mon.c (mon.2) 632 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-126"}]: dispatch 2024-03-26T04:00:27.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:26 smithi175 bash[21527]: audit 2024-03-26T04:00:26.659706+0000 mon.a (mon.0) 3175 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-126"}]: dispatch 2024-03-26T04:00:27.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:26 smithi175 bash[29284]: cluster 2024-03-26T04:00:25.975548+0000 mon.a (mon.0) 3169 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:00:27.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:26 smithi175 bash[29284]: audit 2024-03-26T04:00:25.983296+0000 mon.a (mon.0) 3170 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-126", "mode": "writeback"}]': finished 2024-03-26T04:00:27.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:26 smithi175 bash[29284]: cluster 2024-03-26T04:00:25.993043+0000 mon.a (mon.0) 3171 : cluster [DBG] osdmap e629: 8 total, 8 up, 8 in 2024-03-26T04:00:27.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:26 smithi175 bash[29284]: audit 2024-03-26T04:00:26.137387+0000 mon.c (mon.2) 631 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:27.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:26 smithi175 bash[29284]: audit 2024-03-26T04:00:26.138118+0000 mon.a (mon.0) 3172 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:27.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:26 smithi175 bash[29284]: audit 2024-03-26T04:00:26.644554+0000 mon.a (mon.0) 3173 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:00:27.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:26 smithi175 bash[29284]: cluster 2024-03-26T04:00:26.652992+0000 mon.a (mon.0) 3174 : cluster [DBG] osdmap e630: 8 total, 8 up, 8 in 2024-03-26T04:00:27.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:26 smithi175 bash[29284]: audit 2024-03-26T04:00:26.657403+0000 mon.c (mon.2) 632 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-126"}]: dispatch 2024-03-26T04:00:27.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:26 smithi175 bash[29284]: audit 2024-03-26T04:00:26.659706+0000 mon.a (mon.0) 3175 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-126"}]: dispatch 2024-03-26T04:00:28.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:27 smithi175 bash[29284]: cluster 2024-03-26T04:00:27.004132+0000 mgr.y (mgr.24431) 618 : cluster [DBG] pgmap v1012: 268 pgs: 268 active+clean; 582 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:00:28.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:27 smithi175 bash[29284]: cluster 2024-03-26T04:00:27.645004+0000 mon.a (mon.0) 3176 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:00:28.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:27 smithi175 bash[29284]: audit 2024-03-26T04:00:27.647508+0000 mon.a (mon.0) 3177 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-126"}]': finished 2024-03-26T04:00:28.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:27 smithi175 bash[29284]: cluster 2024-03-26T04:00:27.655020+0000 mon.a (mon.0) 3178 : cluster [DBG] osdmap e631: 8 total, 8 up, 8 in 2024-03-26T04:00:28.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:27 smithi175 bash[21527]: cluster 2024-03-26T04:00:27.004132+0000 mgr.y (mgr.24431) 618 : cluster [DBG] pgmap v1012: 268 pgs: 268 active+clean; 582 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:00:28.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:27 smithi175 bash[21527]: cluster 2024-03-26T04:00:27.645004+0000 mon.a (mon.0) 3176 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:00:28.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:27 smithi175 bash[21527]: audit 2024-03-26T04:00:27.647508+0000 mon.a (mon.0) 3177 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-126"}]': finished 2024-03-26T04:00:28.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:27 smithi175 bash[21527]: cluster 2024-03-26T04:00:27.655020+0000 mon.a (mon.0) 3178 : cluster [DBG] osdmap e631: 8 total, 8 up, 8 in 2024-03-26T04:00:28.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:28 smithi190 bash[23392]: cluster 2024-03-26T04:00:27.004132+0000 mgr.y (mgr.24431) 618 : cluster [DBG] pgmap v1012: 268 pgs: 268 active+clean; 582 KiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:00:28.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:28 smithi190 bash[23392]: cluster 2024-03-26T04:00:27.645004+0000 mon.a (mon.0) 3176 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:00:28.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:28 smithi190 bash[23392]: audit 2024-03-26T04:00:27.647508+0000 mon.a (mon.0) 3177 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-126"}]': finished 2024-03-26T04:00:28.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:28 smithi190 bash[23392]: cluster 2024-03-26T04:00:27.655020+0000 mon.a (mon.0) 3178 : cluster [DBG] osdmap e631: 8 total, 8 up, 8 in 2024-03-26T04:00:28.658 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: require_osd_release = squid 2024-03-26T04:00:28.659 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (9036 ms) 2024-03-26T04:00:28.659 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-03-26T04:00:28.659 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: require_osd_release = squid 2024-03-26T04:00:28.659 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (6037 ms) 2024-03-26T04:00:28.659 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2024-03-26T04:00:28.659 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: require_osd_release = squid 2024-03-26T04:00:28.659 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollback (5042 ms) 2024-03-26T04:00:28.659 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-03-26T04:00:28.659 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: require_osd_release = squid 2024-03-26T04:00:28.659 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (26109 ms) 2024-03-26T04:00:28.659 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-03-26T04:00:28.659 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: require_osd_release = squid 2024-03-26T04:00:28.659 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictRollback (12480 ms) 2024-03-26T04:00:28.659 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-03-26T04:00:28.660 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (12123 ms) 2024-03-26T04:00:28.660 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-03-26T04:00:28.660 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-03-26T04:00:28.660 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: 00000010 69 73 |is| 2024-03-26T04:00:28.660 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: 00000012 2024-03-26T04:00:28.660 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (12094 ms) 2024-03-26T04:00:28.660 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-03-26T04:00:28.660 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: require_osd_release = squid 2024-03-26T04:00:28.660 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (6067 ms) 2024-03-26T04:00:28.660 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP (555874 ms total) 2024-03-26T04:00:28.660 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: 2024-03-26T04:00:28.660 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2024-03-26T04:00:28.660 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2024-03-26T04:00:28.660 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1079 ms) 2024-03-26T04:00:28.660 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-03-26T04:00:28.661 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (11069 ms) 2024-03-26T04:00:28.661 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2024-03-26T04:00:28.661 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (18200 ms) 2024-03-26T04:00:28.661 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2024-03-26T04:00:28.661 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (3 ms) 2024-03-26T04:00:28.661 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (30351 ms total) 2024-03-26T04:00:28.661 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: 2024-03-26T04:00:28.661 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2024-03-26T04:00:28.661 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-03-26T04:00:28.661 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (7078 ms) 2024-03-26T04:00:28.661 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2024-03-26T04:00:28.661 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (8111 ms) 2024-03-26T04:00:28.661 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-03-26T04:00:28.661 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: waiting for scrub... 2024-03-26T04:00:28.661 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: done waiting 2024-03-26T04:00:28.662 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (25192 ms) 2024-03-26T04:00:28.662 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-03-26T04:00:28.662 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (9630 ms) 2024-03-26T04:00:28.662 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-03-26T04:00:28.662 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (7959 ms) 2024-03-26T04:00:28.662 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2024-03-26T04:00:28.662 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (8118 ms) 2024-03-26T04:00:28.662 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-03-26T04:00:28.662 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (10165 ms) 2024-03-26T04:00:28.662 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-03-26T04:00:29.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:00:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:00:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:00:29.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:29 smithi190 bash[23392]: cluster 2024-03-26T04:00:28.658019+0000 mon.a (mon.0) 3179 : cluster [DBG] osdmap e632: 8 total, 8 up, 8 in 2024-03-26T04:00:29.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:29 smithi190 bash[23392]: cluster 2024-03-26T04:00:29.004954+0000 mgr.y (mgr.24431) 619 : cluster [DBG] pgmap v1015: 236 pgs: 236 active+clean; 582 KiB data, 662 MiB used, 715 GiB / 715 GiB avail 2024-03-26T04:00:29.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:29 smithi175 bash[21527]: cluster 2024-03-26T04:00:28.658019+0000 mon.a (mon.0) 3179 : cluster [DBG] osdmap e632: 8 total, 8 up, 8 in 2024-03-26T04:00:29.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:29 smithi175 bash[21527]: cluster 2024-03-26T04:00:29.004954+0000 mgr.y (mgr.24431) 619 : cluster [DBG] pgmap v1015: 236 pgs: 236 active+clean; 582 KiB data, 662 MiB used, 715 GiB / 715 GiB avail 2024-03-26T04:00:29.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:29 smithi175 bash[29284]: cluster 2024-03-26T04:00:28.658019+0000 mon.a (mon.0) 3179 : cluster [DBG] osdmap e632: 8 total, 8 up, 8 in 2024-03-26T04:00:29.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:29 smithi175 bash[29284]: cluster 2024-03-26T04:00:29.004954+0000 mgr.y (mgr.24431) 619 : cluster [DBG] pgmap v1015: 236 pgs: 236 active+clean; 582 KiB data, 662 MiB used, 715 GiB / 715 GiB avail 2024-03-26T04:00:30.664 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:00:30 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:00:30.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:30 smithi190 bash[23392]: cluster 2024-03-26T04:00:29.672847+0000 mon.a (mon.0) 3180 : cluster [DBG] osdmap e633: 8 total, 8 up, 8 in 2024-03-26T04:00:30.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:30 smithi190 bash[23392]: audit 2024-03-26T04:00:29.682394+0000 mon.c (mon.2) 633 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:30.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:30 smithi190 bash[23392]: audit 2024-03-26T04:00:29.685663+0000 mon.a (mon.0) 3181 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:30.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:30 smithi190 bash[23392]: audit 2024-03-26T04:00:30.381667+0000 mgr.y (mgr.24431) 620 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:00:30.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:30 smithi175 bash[29284]: cluster 2024-03-26T04:00:29.672847+0000 mon.a (mon.0) 3180 : cluster [DBG] osdmap e633: 8 total, 8 up, 8 in 2024-03-26T04:00:30.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:30 smithi175 bash[29284]: audit 2024-03-26T04:00:29.682394+0000 mon.c (mon.2) 633 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:30.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:30 smithi175 bash[29284]: audit 2024-03-26T04:00:29.685663+0000 mon.a (mon.0) 3181 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:30.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:30 smithi175 bash[29284]: audit 2024-03-26T04:00:30.381667+0000 mgr.y (mgr.24431) 620 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:00:30.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:30 smithi175 bash[21527]: cluster 2024-03-26T04:00:29.672847+0000 mon.a (mon.0) 3180 : cluster [DBG] osdmap e633: 8 total, 8 up, 8 in 2024-03-26T04:00:30.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:30 smithi175 bash[21527]: audit 2024-03-26T04:00:29.682394+0000 mon.c (mon.2) 633 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:30.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:30 smithi175 bash[21527]: audit 2024-03-26T04:00:29.685663+0000 mon.a (mon.0) 3181 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:30.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:30 smithi175 bash[21527]: audit 2024-03-26T04:00:30.381667+0000 mgr.y (mgr.24431) 620 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:00:31.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:31 smithi190 bash[23392]: audit 2024-03-26T04:00:30.674828+0000 mon.a (mon.0) 3182 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:00:31.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:31 smithi190 bash[23392]: cluster 2024-03-26T04:00:30.684294+0000 mon.a (mon.0) 3183 : cluster [DBG] osdmap e634: 8 total, 8 up, 8 in 2024-03-26T04:00:31.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:31 smithi190 bash[23392]: audit 2024-03-26T04:00:30.698938+0000 mon.c (mon.2) 634 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:31.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:31 smithi190 bash[23392]: audit 2024-03-26T04:00:30.704321+0000 mon.a (mon.0) 3184 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:31.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:31 smithi190 bash[23392]: cluster 2024-03-26T04:00:31.005851+0000 mgr.y (mgr.24431) 621 : cluster [DBG] pgmap v1018: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T04:00:31.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:31 smithi175 bash[29284]: audit 2024-03-26T04:00:30.674828+0000 mon.a (mon.0) 3182 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:00:31.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:31 smithi175 bash[29284]: cluster 2024-03-26T04:00:30.684294+0000 mon.a (mon.0) 3183 : cluster [DBG] osdmap e634: 8 total, 8 up, 8 in 2024-03-26T04:00:31.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:31 smithi175 bash[29284]: audit 2024-03-26T04:00:30.698938+0000 mon.c (mon.2) 634 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:31.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:31 smithi175 bash[29284]: audit 2024-03-26T04:00:30.704321+0000 mon.a (mon.0) 3184 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:31.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:31 smithi175 bash[29284]: cluster 2024-03-26T04:00:31.005851+0000 mgr.y (mgr.24431) 621 : cluster [DBG] pgmap v1018: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T04:00:31.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:31 smithi175 bash[21527]: audit 2024-03-26T04:00:30.674828+0000 mon.a (mon.0) 3182 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:00:31.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:31 smithi175 bash[21527]: cluster 2024-03-26T04:00:30.684294+0000 mon.a (mon.0) 3183 : cluster [DBG] osdmap e634: 8 total, 8 up, 8 in 2024-03-26T04:00:31.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:31 smithi175 bash[21527]: audit 2024-03-26T04:00:30.698938+0000 mon.c (mon.2) 634 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:31.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:31 smithi175 bash[21527]: audit 2024-03-26T04:00:30.704321+0000 mon.a (mon.0) 3184 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:31.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:31 smithi175 bash[21527]: cluster 2024-03-26T04:00:31.005851+0000 mgr.y (mgr.24431) 621 : cluster [DBG] pgmap v1018: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T04:00:32.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:32 smithi190 bash[23392]: audit 2024-03-26T04:00:31.685376+0000 mon.a (mon.0) 3185 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-128", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:00:32.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:32 smithi190 bash[23392]: cluster 2024-03-26T04:00:31.698911+0000 mon.a (mon.0) 3186 : cluster [DBG] osdmap e635: 8 total, 8 up, 8 in 2024-03-26T04:00:32.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:32 smithi190 bash[23392]: audit 2024-03-26T04:00:31.701432+0000 mon.c (mon.2) 635 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-128"}]: dispatch 2024-03-26T04:00:32.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:32 smithi190 bash[23392]: audit 2024-03-26T04:00:31.704089+0000 mon.a (mon.0) 3187 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-128"}]: dispatch 2024-03-26T04:00:32.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:32 smithi175 bash[29284]: audit 2024-03-26T04:00:31.685376+0000 mon.a (mon.0) 3185 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-128", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:00:32.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:32 smithi175 bash[29284]: cluster 2024-03-26T04:00:31.698911+0000 mon.a (mon.0) 3186 : cluster [DBG] osdmap e635: 8 total, 8 up, 8 in 2024-03-26T04:00:32.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:32 smithi175 bash[29284]: audit 2024-03-26T04:00:31.701432+0000 mon.c (mon.2) 635 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-128"}]: dispatch 2024-03-26T04:00:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:32 smithi175 bash[29284]: audit 2024-03-26T04:00:31.704089+0000 mon.a (mon.0) 3187 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-128"}]: dispatch 2024-03-26T04:00:32.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:32 smithi175 bash[21527]: audit 2024-03-26T04:00:31.685376+0000 mon.a (mon.0) 3185 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-128", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:00:32.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:32 smithi175 bash[21527]: cluster 2024-03-26T04:00:31.698911+0000 mon.a (mon.0) 3186 : cluster [DBG] osdmap e635: 8 total, 8 up, 8 in 2024-03-26T04:00:32.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:32 smithi175 bash[21527]: audit 2024-03-26T04:00:31.701432+0000 mon.c (mon.2) 635 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-128"}]: dispatch 2024-03-26T04:00:32.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:32 smithi175 bash[21527]: audit 2024-03-26T04:00:31.704089+0000 mon.a (mon.0) 3187 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-128"}]: dispatch 2024-03-26T04:00:33.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:33 smithi190 bash[23392]: audit 2024-03-26T04:00:32.687975+0000 mon.a (mon.0) 3188 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-128"}]': finished 2024-03-26T04:00:33.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:33 smithi190 bash[23392]: cluster 2024-03-26T04:00:32.695612+0000 mon.a (mon.0) 3189 : cluster [DBG] osdmap e636: 8 total, 8 up, 8 in 2024-03-26T04:00:33.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:33 smithi190 bash[23392]: audit 2024-03-26T04:00:32.698310+0000 mon.c (mon.2) 636 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-128", "mode": "writeback"}]: dispatch 2024-03-26T04:00:33.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:33 smithi190 bash[23392]: audit 2024-03-26T04:00:32.713459+0000 mon.a (mon.0) 3190 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-128", "mode": "writeback"}]: dispatch 2024-03-26T04:00:33.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:33 smithi190 bash[23392]: cluster 2024-03-26T04:00:33.006679+0000 mgr.y (mgr.24431) 622 : cluster [DBG] pgmap v1021: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T04:00:33.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:33 smithi175 bash[29284]: audit 2024-03-26T04:00:32.687975+0000 mon.a (mon.0) 3188 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-128"}]': finished 2024-03-26T04:00:33.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:33 smithi175 bash[29284]: cluster 2024-03-26T04:00:32.695612+0000 mon.a (mon.0) 3189 : cluster [DBG] osdmap e636: 8 total, 8 up, 8 in 2024-03-26T04:00:33.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:33 smithi175 bash[29284]: audit 2024-03-26T04:00:32.698310+0000 mon.c (mon.2) 636 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-128", "mode": "writeback"}]: dispatch 2024-03-26T04:00:33.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:33 smithi175 bash[29284]: audit 2024-03-26T04:00:32.713459+0000 mon.a (mon.0) 3190 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-128", "mode": "writeback"}]: dispatch 2024-03-26T04:00:33.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:33 smithi175 bash[29284]: cluster 2024-03-26T04:00:33.006679+0000 mgr.y (mgr.24431) 622 : cluster [DBG] pgmap v1021: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T04:00:33.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:33 smithi175 bash[21527]: audit 2024-03-26T04:00:32.687975+0000 mon.a (mon.0) 3188 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-128"}]': finished 2024-03-26T04:00:33.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:33 smithi175 bash[21527]: cluster 2024-03-26T04:00:32.695612+0000 mon.a (mon.0) 3189 : cluster [DBG] osdmap e636: 8 total, 8 up, 8 in 2024-03-26T04:00:33.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:33 smithi175 bash[21527]: audit 2024-03-26T04:00:32.698310+0000 mon.c (mon.2) 636 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-128", "mode": "writeback"}]: dispatch 2024-03-26T04:00:33.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:33 smithi175 bash[21527]: audit 2024-03-26T04:00:32.713459+0000 mon.a (mon.0) 3190 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-128", "mode": "writeback"}]: dispatch 2024-03-26T04:00:33.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:33 smithi175 bash[21527]: cluster 2024-03-26T04:00:33.006679+0000 mgr.y (mgr.24431) 622 : cluster [DBG] pgmap v1021: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T04:00:34.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:34 smithi190 bash[23392]: cluster 2024-03-26T04:00:33.703807+0000 mon.a (mon.0) 3191 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:00:34.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:34 smithi190 bash[23392]: audit 2024-03-26T04:00:33.711851+0000 mon.a (mon.0) 3192 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-128", "mode": "writeback"}]': finished 2024-03-26T04:00:34.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:34 smithi190 bash[23392]: cluster 2024-03-26T04:00:33.720935+0000 mon.a (mon.0) 3193 : cluster [DBG] osdmap e637: 8 total, 8 up, 8 in 2024-03-26T04:00:34.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:34 smithi175 bash[21527]: cluster 2024-03-26T04:00:33.703807+0000 mon.a (mon.0) 3191 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:00:34.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:34 smithi175 bash[21527]: audit 2024-03-26T04:00:33.711851+0000 mon.a (mon.0) 3192 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-128", "mode": "writeback"}]': finished 2024-03-26T04:00:34.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:34 smithi175 bash[21527]: cluster 2024-03-26T04:00:33.720935+0000 mon.a (mon.0) 3193 : cluster [DBG] osdmap e637: 8 total, 8 up, 8 in 2024-03-26T04:00:34.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:34 smithi175 bash[29284]: cluster 2024-03-26T04:00:33.703807+0000 mon.a (mon.0) 3191 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:00:34.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:34 smithi175 bash[29284]: audit 2024-03-26T04:00:33.711851+0000 mon.a (mon.0) 3192 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-128", "mode": "writeback"}]': finished 2024-03-26T04:00:34.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:34 smithi175 bash[29284]: cluster 2024-03-26T04:00:33.720935+0000 mon.a (mon.0) 3193 : cluster [DBG] osdmap e637: 8 total, 8 up, 8 in 2024-03-26T04:00:35.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:35 smithi190 bash[23392]: cluster 2024-03-26T04:00:35.009228+0000 mgr.y (mgr.24431) 623 : cluster [DBG] pgmap v1023: 268 pgs: 268 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 472 B/s wr, 1 op/s 2024-03-26T04:00:35.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:35 smithi175 bash[21527]: cluster 2024-03-26T04:00:35.009228+0000 mgr.y (mgr.24431) 623 : cluster [DBG] pgmap v1023: 268 pgs: 268 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 472 B/s wr, 1 op/s 2024-03-26T04:00:35.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:35 smithi175 bash[29284]: cluster 2024-03-26T04:00:35.009228+0000 mgr.y (mgr.24431) 623 : cluster [DBG] pgmap v1023: 268 pgs: 268 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 472 B/s wr, 1 op/s 2024-03-26T04:00:37.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:36 smithi190 bash[23392]: audit 2024-03-26T04:00:35.843025+0000 mon.a (mon.0) 3194 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:00:37.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:36 smithi190 bash[23392]: audit 2024-03-26T04:00:35.846132+0000 mon.c (mon.2) 637 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:00:37.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:36 smithi175 bash[29284]: audit 2024-03-26T04:00:35.843025+0000 mon.a (mon.0) 3194 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:00:37.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:36 smithi175 bash[29284]: audit 2024-03-26T04:00:35.846132+0000 mon.c (mon.2) 637 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:00:37.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:36 smithi175 bash[21527]: audit 2024-03-26T04:00:35.843025+0000 mon.a (mon.0) 3194 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:00:37.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:36 smithi175 bash[21527]: audit 2024-03-26T04:00:35.846132+0000 mon.c (mon.2) 637 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:00:38.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:37 smithi190 bash[23392]: cluster 2024-03-26T04:00:37.010223+0000 mgr.y (mgr.24431) 624 : cluster [DBG] pgmap v1024: 268 pgs: 268 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2024-03-26T04:00:38.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:37 smithi175 bash[21527]: cluster 2024-03-26T04:00:37.010223+0000 mgr.y (mgr.24431) 624 : cluster [DBG] pgmap v1024: 268 pgs: 268 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2024-03-26T04:00:38.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:37 smithi175 bash[29284]: cluster 2024-03-26T04:00:37.010223+0000 mgr.y (mgr.24431) 624 : cluster [DBG] pgmap v1024: 268 pgs: 268 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2024-03-26T04:00:39.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:38 smithi190 bash[23392]: audit 2024-03-26T04:00:38.896140+0000 mon.c (mon.2) 638 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:39.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:38 smithi190 bash[23392]: audit 2024-03-26T04:00:38.896953+0000 mon.a (mon.0) 3195 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:39.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:00:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:00:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:00:39.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:38 smithi175 bash[21527]: audit 2024-03-26T04:00:38.896140+0000 mon.c (mon.2) 638 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:39.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:38 smithi175 bash[21527]: audit 2024-03-26T04:00:38.896953+0000 mon.a (mon.0) 3195 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:39.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:38 smithi175 bash[29284]: audit 2024-03-26T04:00:38.896140+0000 mon.c (mon.2) 638 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:39.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:38 smithi175 bash[29284]: audit 2024-03-26T04:00:38.896953+0000 mon.a (mon.0) 3195 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:40.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:39 smithi190 bash[23392]: audit 2024-03-26T04:00:38.950028+0000 mon.a (mon.0) 3196 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:00:40.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:39 smithi190 bash[23392]: cluster 2024-03-26T04:00:38.958586+0000 mon.a (mon.0) 3197 : cluster [DBG] osdmap e638: 8 total, 8 up, 8 in 2024-03-26T04:00:40.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:39 smithi190 bash[23392]: audit 2024-03-26T04:00:38.960319+0000 mon.c (mon.2) 639 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-128"}]: dispatch 2024-03-26T04:00:40.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:39 smithi190 bash[23392]: audit 2024-03-26T04:00:38.962282+0000 mon.a (mon.0) 3198 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-128"}]: dispatch 2024-03-26T04:00:40.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:39 smithi190 bash[23392]: cluster 2024-03-26T04:00:39.011153+0000 mgr.y (mgr.24431) 625 : cluster [DBG] pgmap v1026: 268 pgs: 268 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 972 B/s rd, 324 B/s wr, 1 op/s 2024-03-26T04:00:40.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:39 smithi190 bash[23392]: cluster 2024-03-26T04:00:39.950487+0000 mon.a (mon.0) 3199 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:00:40.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:39 smithi190 bash[23392]: audit 2024-03-26T04:00:39.952870+0000 mon.a (mon.0) 3200 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-128"}]': finished 2024-03-26T04:00:40.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:39 smithi190 bash[23392]: cluster 2024-03-26T04:00:39.960873+0000 mon.a (mon.0) 3201 : cluster [DBG] osdmap e639: 8 total, 8 up, 8 in 2024-03-26T04:00:40.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:39 smithi175 bash[29284]: audit 2024-03-26T04:00:38.950028+0000 mon.a (mon.0) 3196 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:00:40.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:39 smithi175 bash[29284]: cluster 2024-03-26T04:00:38.958586+0000 mon.a (mon.0) 3197 : cluster [DBG] osdmap e638: 8 total, 8 up, 8 in 2024-03-26T04:00:40.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:39 smithi175 bash[29284]: audit 2024-03-26T04:00:38.960319+0000 mon.c (mon.2) 639 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-128"}]: dispatch 2024-03-26T04:00:40.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:39 smithi175 bash[29284]: audit 2024-03-26T04:00:38.962282+0000 mon.a (mon.0) 3198 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-128"}]: dispatch 2024-03-26T04:00:40.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:39 smithi175 bash[29284]: cluster 2024-03-26T04:00:39.011153+0000 mgr.y (mgr.24431) 625 : cluster [DBG] pgmap v1026: 268 pgs: 268 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 972 B/s rd, 324 B/s wr, 1 op/s 2024-03-26T04:00:40.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:39 smithi175 bash[29284]: cluster 2024-03-26T04:00:39.950487+0000 mon.a (mon.0) 3199 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:00:40.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:39 smithi175 bash[29284]: audit 2024-03-26T04:00:39.952870+0000 mon.a (mon.0) 3200 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-128"}]': finished 2024-03-26T04:00:40.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:39 smithi175 bash[29284]: cluster 2024-03-26T04:00:39.960873+0000 mon.a (mon.0) 3201 : cluster [DBG] osdmap e639: 8 total, 8 up, 8 in 2024-03-26T04:00:40.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:39 smithi175 bash[21527]: audit 2024-03-26T04:00:38.950028+0000 mon.a (mon.0) 3196 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:00:40.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:39 smithi175 bash[21527]: cluster 2024-03-26T04:00:38.958586+0000 mon.a (mon.0) 3197 : cluster [DBG] osdmap e638: 8 total, 8 up, 8 in 2024-03-26T04:00:40.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:39 smithi175 bash[21527]: audit 2024-03-26T04:00:38.960319+0000 mon.c (mon.2) 639 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-128"}]: dispatch 2024-03-26T04:00:40.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:39 smithi175 bash[21527]: audit 2024-03-26T04:00:38.962282+0000 mon.a (mon.0) 3198 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-128"}]: dispatch 2024-03-26T04:00:40.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:39 smithi175 bash[21527]: cluster 2024-03-26T04:00:39.011153+0000 mgr.y (mgr.24431) 625 : cluster [DBG] pgmap v1026: 268 pgs: 268 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 972 B/s rd, 324 B/s wr, 1 op/s 2024-03-26T04:00:40.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:39 smithi175 bash[21527]: cluster 2024-03-26T04:00:39.950487+0000 mon.a (mon.0) 3199 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:00:40.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:39 smithi175 bash[21527]: audit 2024-03-26T04:00:39.952870+0000 mon.a (mon.0) 3200 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-128"}]': finished 2024-03-26T04:00:40.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:39 smithi175 bash[21527]: cluster 2024-03-26T04:00:39.960873+0000 mon.a (mon.0) 3201 : cluster [DBG] osdmap e639: 8 total, 8 up, 8 in 2024-03-26T04:00:40.741 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:00:40 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:00:41.241 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:40 smithi190 bash[23392]: audit 2024-03-26T04:00:40.393013+0000 mgr.y (mgr.24431) 626 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:00:41.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:40 smithi175 bash[29284]: audit 2024-03-26T04:00:40.393013+0000 mgr.y (mgr.24431) 626 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:00:41.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:40 smithi175 bash[21527]: audit 2024-03-26T04:00:40.393013+0000 mgr.y (mgr.24431) 626 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:00:42.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:42 smithi190 bash[23392]: cluster 2024-03-26T04:00:40.997169+0000 mon.a (mon.0) 3202 : cluster [DBG] osdmap e640: 8 total, 8 up, 8 in 2024-03-26T04:00:42.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:42 smithi190 bash[23392]: cluster 2024-03-26T04:00:41.012309+0000 mgr.y (mgr.24431) 627 : cluster [DBG] pgmap v1029: 236 pgs: 236 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-03-26T04:00:42.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:42 smithi175 bash[29284]: cluster 2024-03-26T04:00:40.997169+0000 mon.a (mon.0) 3202 : cluster [DBG] osdmap e640: 8 total, 8 up, 8 in 2024-03-26T04:00:42.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:42 smithi175 bash[29284]: cluster 2024-03-26T04:00:41.012309+0000 mgr.y (mgr.24431) 627 : cluster [DBG] pgmap v1029: 236 pgs: 236 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-03-26T04:00:42.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:42 smithi175 bash[21527]: cluster 2024-03-26T04:00:40.997169+0000 mon.a (mon.0) 3202 : cluster [DBG] osdmap e640: 8 total, 8 up, 8 in 2024-03-26T04:00:42.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:42 smithi175 bash[21527]: cluster 2024-03-26T04:00:41.012309+0000 mgr.y (mgr.24431) 627 : cluster [DBG] pgmap v1029: 236 pgs: 236 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-03-26T04:00:43.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:43 smithi190 bash[23392]: cluster 2024-03-26T04:00:42.015119+0000 mon.a (mon.0) 3203 : cluster [DBG] osdmap e641: 8 total, 8 up, 8 in 2024-03-26T04:00:43.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:43 smithi190 bash[23392]: audit 2024-03-26T04:00:42.019006+0000 mon.c (mon.2) 640 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:43.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:43 smithi190 bash[23392]: audit 2024-03-26T04:00:42.020704+0000 mon.a (mon.0) 3204 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:43.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:43 smithi175 bash[21527]: cluster 2024-03-26T04:00:42.015119+0000 mon.a (mon.0) 3203 : cluster [DBG] osdmap e641: 8 total, 8 up, 8 in 2024-03-26T04:00:43.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:43 smithi175 bash[21527]: audit 2024-03-26T04:00:42.019006+0000 mon.c (mon.2) 640 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:43.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:43 smithi175 bash[21527]: audit 2024-03-26T04:00:42.020704+0000 mon.a (mon.0) 3204 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:43.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:43 smithi175 bash[29284]: cluster 2024-03-26T04:00:42.015119+0000 mon.a (mon.0) 3203 : cluster [DBG] osdmap e641: 8 total, 8 up, 8 in 2024-03-26T04:00:43.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:43 smithi175 bash[29284]: audit 2024-03-26T04:00:42.019006+0000 mon.c (mon.2) 640 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:43.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:43 smithi175 bash[29284]: audit 2024-03-26T04:00:42.020704+0000 mon.a (mon.0) 3204 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:44.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:44 smithi190 bash[23392]: audit 2024-03-26T04:00:43.006987+0000 mon.a (mon.0) 3205 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:00:44.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:44 smithi190 bash[23392]: cluster 2024-03-26T04:00:43.013534+0000 mgr.y (mgr.24431) 628 : cluster [DBG] pgmap v1032: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T04:00:44.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:44 smithi190 bash[23392]: cluster 2024-03-26T04:00:43.014058+0000 mon.a (mon.0) 3206 : cluster [DBG] osdmap e642: 8 total, 8 up, 8 in 2024-03-26T04:00:44.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:44 smithi190 bash[23392]: audit 2024-03-26T04:00:43.020351+0000 mon.c (mon.2) 641 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:44.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:44 smithi190 bash[23392]: audit 2024-03-26T04:00:43.027896+0000 mon.a (mon.0) 3207 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:44.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:44 smithi190 bash[23392]: audit 2024-03-26T04:00:44.009842+0000 mon.a (mon.0) 3208 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-130", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:00:44.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:44 smithi190 bash[23392]: cluster 2024-03-26T04:00:44.019201+0000 mon.a (mon.0) 3209 : cluster [DBG] osdmap e643: 8 total, 8 up, 8 in 2024-03-26T04:00:44.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:44 smithi190 bash[23392]: audit 2024-03-26T04:00:44.020677+0000 mon.c (mon.2) 642 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-130"}]: dispatch 2024-03-26T04:00:44.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:44 smithi190 bash[23392]: audit 2024-03-26T04:00:44.025123+0000 mon.a (mon.0) 3210 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-130"}]: dispatch 2024-03-26T04:00:44.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:44 smithi175 bash[21527]: audit 2024-03-26T04:00:43.006987+0000 mon.a (mon.0) 3205 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:00:44.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:44 smithi175 bash[21527]: cluster 2024-03-26T04:00:43.013534+0000 mgr.y (mgr.24431) 628 : cluster [DBG] pgmap v1032: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T04:00:44.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:44 smithi175 bash[21527]: cluster 2024-03-26T04:00:43.014058+0000 mon.a (mon.0) 3206 : cluster [DBG] osdmap e642: 8 total, 8 up, 8 in 2024-03-26T04:00:44.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:44 smithi175 bash[21527]: audit 2024-03-26T04:00:43.020351+0000 mon.c (mon.2) 641 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:44.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:44 smithi175 bash[21527]: audit 2024-03-26T04:00:43.027896+0000 mon.a (mon.0) 3207 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:44.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:44 smithi175 bash[21527]: audit 2024-03-26T04:00:44.009842+0000 mon.a (mon.0) 3208 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-130", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:00:44.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:44 smithi175 bash[21527]: cluster 2024-03-26T04:00:44.019201+0000 mon.a (mon.0) 3209 : cluster [DBG] osdmap e643: 8 total, 8 up, 8 in 2024-03-26T04:00:44.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:44 smithi175 bash[21527]: audit 2024-03-26T04:00:44.020677+0000 mon.c (mon.2) 642 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-130"}]: dispatch 2024-03-26T04:00:44.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:44 smithi175 bash[21527]: audit 2024-03-26T04:00:44.025123+0000 mon.a (mon.0) 3210 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-130"}]: dispatch 2024-03-26T04:00:44.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:44 smithi175 bash[29284]: audit 2024-03-26T04:00:43.006987+0000 mon.a (mon.0) 3205 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:00:44.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:44 smithi175 bash[29284]: cluster 2024-03-26T04:00:43.013534+0000 mgr.y (mgr.24431) 628 : cluster [DBG] pgmap v1032: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T04:00:44.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:44 smithi175 bash[29284]: cluster 2024-03-26T04:00:43.014058+0000 mon.a (mon.0) 3206 : cluster [DBG] osdmap e642: 8 total, 8 up, 8 in 2024-03-26T04:00:44.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:44 smithi175 bash[29284]: audit 2024-03-26T04:00:43.020351+0000 mon.c (mon.2) 641 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:44.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:44 smithi175 bash[29284]: audit 2024-03-26T04:00:43.027896+0000 mon.a (mon.0) 3207 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:44.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:44 smithi175 bash[29284]: audit 2024-03-26T04:00:44.009842+0000 mon.a (mon.0) 3208 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-130", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:00:44.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:44 smithi175 bash[29284]: cluster 2024-03-26T04:00:44.019201+0000 mon.a (mon.0) 3209 : cluster [DBG] osdmap e643: 8 total, 8 up, 8 in 2024-03-26T04:00:44.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:44 smithi175 bash[29284]: audit 2024-03-26T04:00:44.020677+0000 mon.c (mon.2) 642 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-130"}]: dispatch 2024-03-26T04:00:44.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:44 smithi175 bash[29284]: audit 2024-03-26T04:00:44.025123+0000 mon.a (mon.0) 3210 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-130"}]: dispatch 2024-03-26T04:00:46.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:46 smithi190 bash[23392]: audit 2024-03-26T04:00:45.012701+0000 mon.a (mon.0) 3211 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-130"}]': finished 2024-03-26T04:00:46.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:46 smithi190 bash[23392]: cluster 2024-03-26T04:00:45.016842+0000 mgr.y (mgr.24431) 629 : cluster [DBG] pgmap v1034: 268 pgs: 268 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:00:46.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:46 smithi190 bash[23392]: cluster 2024-03-26T04:00:45.020171+0000 mon.a (mon.0) 3212 : cluster [DBG] osdmap e644: 8 total, 8 up, 8 in 2024-03-26T04:00:46.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:46 smithi190 bash[23392]: audit 2024-03-26T04:00:45.023467+0000 mon.c (mon.2) 643 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-130", "mode": "writeback"}]: dispatch 2024-03-26T04:00:46.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:46 smithi190 bash[23392]: audit 2024-03-26T04:00:45.027545+0000 mon.a (mon.0) 3213 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-130", "mode": "writeback"}]: dispatch 2024-03-26T04:00:46.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:46 smithi175 bash[21527]: audit 2024-03-26T04:00:45.012701+0000 mon.a (mon.0) 3211 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-130"}]': finished 2024-03-26T04:00:46.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:46 smithi175 bash[21527]: cluster 2024-03-26T04:00:45.016842+0000 mgr.y (mgr.24431) 629 : cluster [DBG] pgmap v1034: 268 pgs: 268 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:00:46.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:46 smithi175 bash[21527]: cluster 2024-03-26T04:00:45.020171+0000 mon.a (mon.0) 3212 : cluster [DBG] osdmap e644: 8 total, 8 up, 8 in 2024-03-26T04:00:46.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:46 smithi175 bash[21527]: audit 2024-03-26T04:00:45.023467+0000 mon.c (mon.2) 643 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-130", "mode": "writeback"}]: dispatch 2024-03-26T04:00:46.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:46 smithi175 bash[21527]: audit 2024-03-26T04:00:45.027545+0000 mon.a (mon.0) 3213 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-130", "mode": "writeback"}]: dispatch 2024-03-26T04:00:46.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:46 smithi175 bash[29284]: audit 2024-03-26T04:00:45.012701+0000 mon.a (mon.0) 3211 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-130"}]': finished 2024-03-26T04:00:46.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:46 smithi175 bash[29284]: cluster 2024-03-26T04:00:45.016842+0000 mgr.y (mgr.24431) 629 : cluster [DBG] pgmap v1034: 268 pgs: 268 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:00:46.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:46 smithi175 bash[29284]: cluster 2024-03-26T04:00:45.020171+0000 mon.a (mon.0) 3212 : cluster [DBG] osdmap e644: 8 total, 8 up, 8 in 2024-03-26T04:00:46.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:46 smithi175 bash[29284]: audit 2024-03-26T04:00:45.023467+0000 mon.c (mon.2) 643 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-130", "mode": "writeback"}]: dispatch 2024-03-26T04:00:46.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:46 smithi175 bash[29284]: audit 2024-03-26T04:00:45.027545+0000 mon.a (mon.0) 3213 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-130", "mode": "writeback"}]: dispatch 2024-03-26T04:00:47.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:47 smithi190 bash[23392]: cluster 2024-03-26T04:00:46.013124+0000 mon.a (mon.0) 3214 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:00:47.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:47 smithi190 bash[23392]: audit 2024-03-26T04:00:46.015479+0000 mon.a (mon.0) 3215 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-130", "mode": "writeback"}]': finished 2024-03-26T04:00:47.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:47 smithi190 bash[23392]: cluster 2024-03-26T04:00:46.023240+0000 mon.a (mon.0) 3216 : cluster [DBG] osdmap e645: 8 total, 8 up, 8 in 2024-03-26T04:00:47.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:47 smithi190 bash[23392]: audit 2024-03-26T04:00:46.239300+0000 mon.c (mon.2) 644 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:47.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:47 smithi190 bash[23392]: audit 2024-03-26T04:00:46.240042+0000 mon.a (mon.0) 3217 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:47.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:47 smithi175 bash[21527]: cluster 2024-03-26T04:00:46.013124+0000 mon.a (mon.0) 3214 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:00:47.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:47 smithi175 bash[21527]: audit 2024-03-26T04:00:46.015479+0000 mon.a (mon.0) 3215 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-130", "mode": "writeback"}]': finished 2024-03-26T04:00:47.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:47 smithi175 bash[21527]: cluster 2024-03-26T04:00:46.023240+0000 mon.a (mon.0) 3216 : cluster [DBG] osdmap e645: 8 total, 8 up, 8 in 2024-03-26T04:00:47.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:47 smithi175 bash[21527]: audit 2024-03-26T04:00:46.239300+0000 mon.c (mon.2) 644 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:47.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:47 smithi175 bash[21527]: audit 2024-03-26T04:00:46.240042+0000 mon.a (mon.0) 3217 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:47.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:47 smithi175 bash[29284]: cluster 2024-03-26T04:00:46.013124+0000 mon.a (mon.0) 3214 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:00:47.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:47 smithi175 bash[29284]: audit 2024-03-26T04:00:46.015479+0000 mon.a (mon.0) 3215 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-130", "mode": "writeback"}]': finished 2024-03-26T04:00:47.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:47 smithi175 bash[29284]: cluster 2024-03-26T04:00:46.023240+0000 mon.a (mon.0) 3216 : cluster [DBG] osdmap e645: 8 total, 8 up, 8 in 2024-03-26T04:00:47.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:47 smithi175 bash[29284]: audit 2024-03-26T04:00:46.239300+0000 mon.c (mon.2) 644 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:47.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:47 smithi175 bash[29284]: audit 2024-03-26T04:00:46.240042+0000 mon.a (mon.0) 3217 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:48.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:48 smithi190 bash[23392]: cluster 2024-03-26T04:00:47.017831+0000 mgr.y (mgr.24431) 630 : cluster [DBG] pgmap v1037: 268 pgs: 268 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:00:48.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:48 smithi190 bash[23392]: audit 2024-03-26T04:00:47.040475+0000 mon.a (mon.0) 3218 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:00:48.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:48 smithi190 bash[23392]: cluster 2024-03-26T04:00:47.047619+0000 mon.a (mon.0) 3219 : cluster [DBG] osdmap e646: 8 total, 8 up, 8 in 2024-03-26T04:00:48.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:48 smithi190 bash[23392]: audit 2024-03-26T04:00:47.050428+0000 mon.c (mon.2) 645 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-130"}]: dispatch 2024-03-26T04:00:48.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:48 smithi190 bash[23392]: audit 2024-03-26T04:00:47.051547+0000 mon.a (mon.0) 3220 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-130"}]: dispatch 2024-03-26T04:00:48.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:48 smithi175 bash[29284]: cluster 2024-03-26T04:00:47.017831+0000 mgr.y (mgr.24431) 630 : cluster [DBG] pgmap v1037: 268 pgs: 268 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:00:48.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:48 smithi175 bash[29284]: audit 2024-03-26T04:00:47.040475+0000 mon.a (mon.0) 3218 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:00:48.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:48 smithi175 bash[29284]: cluster 2024-03-26T04:00:47.047619+0000 mon.a (mon.0) 3219 : cluster [DBG] osdmap e646: 8 total, 8 up, 8 in 2024-03-26T04:00:48.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:48 smithi175 bash[29284]: audit 2024-03-26T04:00:47.050428+0000 mon.c (mon.2) 645 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-130"}]: dispatch 2024-03-26T04:00:48.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:48 smithi175 bash[29284]: audit 2024-03-26T04:00:47.051547+0000 mon.a (mon.0) 3220 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-130"}]: dispatch 2024-03-26T04:00:48.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:48 smithi175 bash[21527]: cluster 2024-03-26T04:00:47.017831+0000 mgr.y (mgr.24431) 630 : cluster [DBG] pgmap v1037: 268 pgs: 268 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:00:48.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:48 smithi175 bash[21527]: audit 2024-03-26T04:00:47.040475+0000 mon.a (mon.0) 3218 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:00:48.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:48 smithi175 bash[21527]: cluster 2024-03-26T04:00:47.047619+0000 mon.a (mon.0) 3219 : cluster [DBG] osdmap e646: 8 total, 8 up, 8 in 2024-03-26T04:00:48.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:48 smithi175 bash[21527]: audit 2024-03-26T04:00:47.050428+0000 mon.c (mon.2) 645 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-130"}]: dispatch 2024-03-26T04:00:48.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:48 smithi175 bash[21527]: audit 2024-03-26T04:00:47.051547+0000 mon.a (mon.0) 3220 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-130"}]: dispatch 2024-03-26T04:00:49.091 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:00:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:00:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:00:49.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:49 smithi190 bash[23392]: cluster 2024-03-26T04:00:48.040780+0000 mon.a (mon.0) 3221 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:00:49.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:49 smithi190 bash[23392]: audit 2024-03-26T04:00:48.043240+0000 mon.a (mon.0) 3222 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-130"}]': finished 2024-03-26T04:00:49.499 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:49 smithi190 bash[23392]: cluster 2024-03-26T04:00:48.050649+0000 mon.a (mon.0) 3223 : cluster [DBG] osdmap e647: 8 total, 8 up, 8 in 2024-03-26T04:00:49.499 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:49 smithi175 bash[21527]: cluster 2024-03-26T04:00:48.040780+0000 mon.a (mon.0) 3221 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:00:49.500 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:49 smithi175 bash[21527]: audit 2024-03-26T04:00:48.043240+0000 mon.a (mon.0) 3222 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-130"}]': finished 2024-03-26T04:00:49.500 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:49 smithi175 bash[21527]: cluster 2024-03-26T04:00:48.050649+0000 mon.a (mon.0) 3223 : cluster [DBG] osdmap e647: 8 total, 8 up, 8 in 2024-03-26T04:00:49.500 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:49 smithi175 bash[29284]: cluster 2024-03-26T04:00:48.040780+0000 mon.a (mon.0) 3221 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:00:49.500 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:49 smithi175 bash[29284]: audit 2024-03-26T04:00:48.043240+0000 mon.a (mon.0) 3222 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-130"}]': finished 2024-03-26T04:00:49.500 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:49 smithi175 bash[29284]: cluster 2024-03-26T04:00:48.050649+0000 mon.a (mon.0) 3223 : cluster [DBG] osdmap e647: 8 total, 8 up, 8 in 2024-03-26T04:00:50.403 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:50 smithi190 bash[23392]: cluster 2024-03-26T04:00:49.018756+0000 mgr.y (mgr.24431) 631 : cluster [DBG] pgmap v1040: 268 pgs: 268 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail 2024-03-26T04:00:50.403 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:50 smithi190 bash[23392]: cluster 2024-03-26T04:00:49.057058+0000 mon.a (mon.0) 3224 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:00:50.403 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:50 smithi190 bash[23392]: cluster 2024-03-26T04:00:49.065492+0000 mon.a (mon.0) 3225 : cluster [DBG] osdmap e648: 8 total, 8 up, 8 in 2024-03-26T04:00:50.403 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:50 smithi190 bash[23392]: cluster 2024-03-26T04:00:50.071423+0000 mon.a (mon.0) 3226 : cluster [DBG] osdmap e649: 8 total, 8 up, 8 in 2024-03-26T04:00:50.403 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:50 smithi190 bash[23392]: audit 2024-03-26T04:00:50.074517+0000 mon.c (mon.2) 646 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:50.403 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:50 smithi190 bash[23392]: audit 2024-03-26T04:00:50.077219+0000 mon.a (mon.0) 3227 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:50.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:50 smithi175 bash[29284]: cluster 2024-03-26T04:00:49.018756+0000 mgr.y (mgr.24431) 631 : cluster [DBG] pgmap v1040: 268 pgs: 268 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail 2024-03-26T04:00:50.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:50 smithi175 bash[29284]: cluster 2024-03-26T04:00:49.057058+0000 mon.a (mon.0) 3224 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:00:50.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:50 smithi175 bash[29284]: cluster 2024-03-26T04:00:49.065492+0000 mon.a (mon.0) 3225 : cluster [DBG] osdmap e648: 8 total, 8 up, 8 in 2024-03-26T04:00:50.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:50 smithi175 bash[29284]: cluster 2024-03-26T04:00:50.071423+0000 mon.a (mon.0) 3226 : cluster [DBG] osdmap e649: 8 total, 8 up, 8 in 2024-03-26T04:00:50.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:50 smithi175 bash[29284]: audit 2024-03-26T04:00:50.074517+0000 mon.c (mon.2) 646 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:50.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:50 smithi175 bash[29284]: audit 2024-03-26T04:00:50.077219+0000 mon.a (mon.0) 3227 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:50.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:50 smithi175 bash[21527]: cluster 2024-03-26T04:00:49.018756+0000 mgr.y (mgr.24431) 631 : cluster [DBG] pgmap v1040: 268 pgs: 268 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail 2024-03-26T04:00:50.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:50 smithi175 bash[21527]: cluster 2024-03-26T04:00:49.057058+0000 mon.a (mon.0) 3224 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:00:50.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:50 smithi175 bash[21527]: cluster 2024-03-26T04:00:49.065492+0000 mon.a (mon.0) 3225 : cluster [DBG] osdmap e648: 8 total, 8 up, 8 in 2024-03-26T04:00:50.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:50 smithi175 bash[21527]: cluster 2024-03-26T04:00:50.071423+0000 mon.a (mon.0) 3226 : cluster [DBG] osdmap e649: 8 total, 8 up, 8 in 2024-03-26T04:00:50.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:50 smithi175 bash[21527]: audit 2024-03-26T04:00:50.074517+0000 mon.c (mon.2) 646 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:50.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:50 smithi175 bash[21527]: audit 2024-03-26T04:00:50.077219+0000 mon.a (mon.0) 3227 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:00:50.741 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:00:50 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:00:51.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:51 smithi190 bash[23392]: audit 2024-03-26T04:00:50.404457+0000 mgr.y (mgr.24431) 632 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:00:51.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:51 smithi190 bash[23392]: audit 2024-03-26T04:00:50.889574+0000 mon.a (mon.0) 3228 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:00:51.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:51 smithi190 bash[23392]: audit 2024-03-26T04:00:50.892775+0000 mon.c (mon.2) 647 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:00:51.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:51 smithi190 bash[23392]: audit 2024-03-26T04:00:51.065852+0000 mon.a (mon.0) 3229 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:00:51.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:51 smithi190 bash[23392]: cluster 2024-03-26T04:00:51.073722+0000 mon.a (mon.0) 3230 : cluster [DBG] osdmap e650: 8 total, 8 up, 8 in 2024-03-26T04:00:51.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:51 smithi190 bash[23392]: audit 2024-03-26T04:00:51.091319+0000 mon.c (mon.2) 648 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:51.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:51 smithi190 bash[23392]: audit 2024-03-26T04:00:51.092373+0000 mon.a (mon.0) 3231 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:51.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:51 smithi175 bash[21527]: audit 2024-03-26T04:00:50.404457+0000 mgr.y (mgr.24431) 632 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:00:51.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:51 smithi175 bash[21527]: audit 2024-03-26T04:00:50.889574+0000 mon.a (mon.0) 3228 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:00:51.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:51 smithi175 bash[21527]: audit 2024-03-26T04:00:50.892775+0000 mon.c (mon.2) 647 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:00:51.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:51 smithi175 bash[21527]: audit 2024-03-26T04:00:51.065852+0000 mon.a (mon.0) 3229 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:00:51.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:51 smithi175 bash[21527]: cluster 2024-03-26T04:00:51.073722+0000 mon.a (mon.0) 3230 : cluster [DBG] osdmap e650: 8 total, 8 up, 8 in 2024-03-26T04:00:51.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:51 smithi175 bash[21527]: audit 2024-03-26T04:00:51.091319+0000 mon.c (mon.2) 648 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:51.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:51 smithi175 bash[21527]: audit 2024-03-26T04:00:51.092373+0000 mon.a (mon.0) 3231 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:51.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:51 smithi175 bash[29284]: audit 2024-03-26T04:00:50.404457+0000 mgr.y (mgr.24431) 632 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:00:51.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:51 smithi175 bash[29284]: audit 2024-03-26T04:00:50.889574+0000 mon.a (mon.0) 3228 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:00:51.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:51 smithi175 bash[29284]: audit 2024-03-26T04:00:50.892775+0000 mon.c (mon.2) 647 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:00:51.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:51 smithi175 bash[29284]: audit 2024-03-26T04:00:51.065852+0000 mon.a (mon.0) 3229 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:00:51.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:51 smithi175 bash[29284]: cluster 2024-03-26T04:00:51.073722+0000 mon.a (mon.0) 3230 : cluster [DBG] osdmap e650: 8 total, 8 up, 8 in 2024-03-26T04:00:51.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:51 smithi175 bash[29284]: audit 2024-03-26T04:00:51.091319+0000 mon.c (mon.2) 648 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:51.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:51 smithi175 bash[29284]: audit 2024-03-26T04:00:51.092373+0000 mon.a (mon.0) 3231 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:00:52.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:52 smithi190 bash[23392]: cluster 2024-03-26T04:00:51.019638+0000 mgr.y (mgr.24431) 633 : cluster [DBG] pgmap v1043: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 669 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T04:00:52.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:52 smithi190 bash[23392]: audit 2024-03-26T04:00:52.068611+0000 mon.a (mon.0) 3232 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-132", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:00:52.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:52 smithi190 bash[23392]: cluster 2024-03-26T04:00:52.076304+0000 mon.a (mon.0) 3233 : cluster [DBG] osdmap e651: 8 total, 8 up, 8 in 2024-03-26T04:00:52.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:52 smithi190 bash[23392]: audit 2024-03-26T04:00:52.077419+0000 mon.c (mon.2) 649 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-132"}]: dispatch 2024-03-26T04:00:52.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:52 smithi190 bash[23392]: audit 2024-03-26T04:00:52.079701+0000 mon.a (mon.0) 3234 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-132"}]: dispatch 2024-03-26T04:00:52.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:52 smithi175 bash[29284]: cluster 2024-03-26T04:00:51.019638+0000 mgr.y (mgr.24431) 633 : cluster [DBG] pgmap v1043: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 669 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T04:00:52.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:52 smithi175 bash[29284]: audit 2024-03-26T04:00:52.068611+0000 mon.a (mon.0) 3232 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-132", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:00:52.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:52 smithi175 bash[29284]: cluster 2024-03-26T04:00:52.076304+0000 mon.a (mon.0) 3233 : cluster [DBG] osdmap e651: 8 total, 8 up, 8 in 2024-03-26T04:00:52.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:52 smithi175 bash[29284]: audit 2024-03-26T04:00:52.077419+0000 mon.c (mon.2) 649 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-132"}]: dispatch 2024-03-26T04:00:52.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:52 smithi175 bash[29284]: audit 2024-03-26T04:00:52.079701+0000 mon.a (mon.0) 3234 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-132"}]: dispatch 2024-03-26T04:00:52.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:52 smithi175 bash[21527]: cluster 2024-03-26T04:00:51.019638+0000 mgr.y (mgr.24431) 633 : cluster [DBG] pgmap v1043: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 669 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T04:00:52.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:52 smithi175 bash[21527]: audit 2024-03-26T04:00:52.068611+0000 mon.a (mon.0) 3232 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-132", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:00:52.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:52 smithi175 bash[21527]: cluster 2024-03-26T04:00:52.076304+0000 mon.a (mon.0) 3233 : cluster [DBG] osdmap e651: 8 total, 8 up, 8 in 2024-03-26T04:00:52.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:52 smithi175 bash[21527]: audit 2024-03-26T04:00:52.077419+0000 mon.c (mon.2) 649 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-132"}]: dispatch 2024-03-26T04:00:52.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:52 smithi175 bash[21527]: audit 2024-03-26T04:00:52.079701+0000 mon.a (mon.0) 3234 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-132"}]: dispatch 2024-03-26T04:00:54.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:54 smithi190 bash[23392]: cluster 2024-03-26T04:00:53.020639+0000 mgr.y (mgr.24431) 634 : cluster [DBG] pgmap v1046: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 669 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T04:00:54.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:54 smithi190 bash[23392]: audit 2024-03-26T04:00:53.071302+0000 mon.a (mon.0) 3235 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-132"}]': finished 2024-03-26T04:00:54.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:54 smithi190 bash[23392]: cluster 2024-03-26T04:00:53.078537+0000 mon.a (mon.0) 3236 : cluster [DBG] osdmap e652: 8 total, 8 up, 8 in 2024-03-26T04:00:54.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:54 smithi190 bash[23392]: audit 2024-03-26T04:00:53.081980+0000 mon.c (mon.2) 650 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-132", "mode": "writeback"}]: dispatch 2024-03-26T04:00:54.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:54 smithi190 bash[23392]: audit 2024-03-26T04:00:53.087539+0000 mon.a (mon.0) 3237 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-132", "mode": "writeback"}]: dispatch 2024-03-26T04:00:54.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:54 smithi175 bash[29284]: cluster 2024-03-26T04:00:53.020639+0000 mgr.y (mgr.24431) 634 : cluster [DBG] pgmap v1046: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 669 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T04:00:54.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:54 smithi175 bash[29284]: audit 2024-03-26T04:00:53.071302+0000 mon.a (mon.0) 3235 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-132"}]': finished 2024-03-26T04:00:54.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:54 smithi175 bash[29284]: cluster 2024-03-26T04:00:53.078537+0000 mon.a (mon.0) 3236 : cluster [DBG] osdmap e652: 8 total, 8 up, 8 in 2024-03-26T04:00:54.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:54 smithi175 bash[29284]: audit 2024-03-26T04:00:53.081980+0000 mon.c (mon.2) 650 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-132", "mode": "writeback"}]: dispatch 2024-03-26T04:00:54.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:54 smithi175 bash[29284]: audit 2024-03-26T04:00:53.087539+0000 mon.a (mon.0) 3237 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-132", "mode": "writeback"}]: dispatch 2024-03-26T04:00:54.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:54 smithi175 bash[21527]: cluster 2024-03-26T04:00:53.020639+0000 mgr.y (mgr.24431) 634 : cluster [DBG] pgmap v1046: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 669 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T04:00:54.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:54 smithi175 bash[21527]: audit 2024-03-26T04:00:53.071302+0000 mon.a (mon.0) 3235 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-132"}]': finished 2024-03-26T04:00:54.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:54 smithi175 bash[21527]: cluster 2024-03-26T04:00:53.078537+0000 mon.a (mon.0) 3236 : cluster [DBG] osdmap e652: 8 total, 8 up, 8 in 2024-03-26T04:00:54.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:54 smithi175 bash[21527]: audit 2024-03-26T04:00:53.081980+0000 mon.c (mon.2) 650 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-132", "mode": "writeback"}]: dispatch 2024-03-26T04:00:54.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:54 smithi175 bash[21527]: audit 2024-03-26T04:00:53.087539+0000 mon.a (mon.0) 3237 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-132", "mode": "writeback"}]: dispatch 2024-03-26T04:00:55.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:55 smithi190 bash[23392]: cluster 2024-03-26T04:00:54.071718+0000 mon.a (mon.0) 3238 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:00:55.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:55 smithi190 bash[23392]: audit 2024-03-26T04:00:54.075983+0000 mon.a (mon.0) 3239 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-132", "mode": "writeback"}]': finished 2024-03-26T04:00:55.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:55 smithi190 bash[23392]: cluster 2024-03-26T04:00:54.083531+0000 mon.a (mon.0) 3240 : cluster [DBG] osdmap e653: 8 total, 8 up, 8 in 2024-03-26T04:00:55.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:55 smithi175 bash[29284]: cluster 2024-03-26T04:00:54.071718+0000 mon.a (mon.0) 3238 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:00:55.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:55 smithi175 bash[29284]: audit 2024-03-26T04:00:54.075983+0000 mon.a (mon.0) 3239 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-132", "mode": "writeback"}]': finished 2024-03-26T04:00:55.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:55 smithi175 bash[29284]: cluster 2024-03-26T04:00:54.083531+0000 mon.a (mon.0) 3240 : cluster [DBG] osdmap e653: 8 total, 8 up, 8 in 2024-03-26T04:00:55.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:55 smithi175 bash[21527]: cluster 2024-03-26T04:00:54.071718+0000 mon.a (mon.0) 3238 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:00:55.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:55 smithi175 bash[21527]: audit 2024-03-26T04:00:54.075983+0000 mon.a (mon.0) 3239 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-132", "mode": "writeback"}]': finished 2024-03-26T04:00:55.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:55 smithi175 bash[21527]: cluster 2024-03-26T04:00:54.083531+0000 mon.a (mon.0) 3240 : cluster [DBG] osdmap e653: 8 total, 8 up, 8 in 2024-03-26T04:00:56.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:56 smithi190 bash[23392]: cluster 2024-03-26T04:00:55.023045+0000 mgr.y (mgr.24431) 635 : cluster [DBG] pgmap v1049: 268 pgs: 268 active+clean; 582 KiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T04:00:56.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:56 smithi190 bash[23392]: cluster 2024-03-26T04:00:55.110024+0000 mon.a (mon.0) 3241 : cluster [DBG] osdmap e654: 8 total, 8 up, 8 in 2024-03-26T04:00:56.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:56 smithi175 bash[21527]: cluster 2024-03-26T04:00:55.023045+0000 mgr.y (mgr.24431) 635 : cluster [DBG] pgmap v1049: 268 pgs: 268 active+clean; 582 KiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T04:00:56.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:56 smithi175 bash[21527]: cluster 2024-03-26T04:00:55.110024+0000 mon.a (mon.0) 3241 : cluster [DBG] osdmap e654: 8 total, 8 up, 8 in 2024-03-26T04:00:56.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:56 smithi175 bash[29284]: cluster 2024-03-26T04:00:55.023045+0000 mgr.y (mgr.24431) 635 : cluster [DBG] pgmap v1049: 268 pgs: 268 active+clean; 582 KiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T04:00:56.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:56 smithi175 bash[29284]: cluster 2024-03-26T04:00:55.110024+0000 mon.a (mon.0) 3241 : cluster [DBG] osdmap e654: 8 total, 8 up, 8 in 2024-03-26T04:00:57.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:57 smithi190 bash[23392]: cluster 2024-03-26T04:00:56.118253+0000 mon.a (mon.0) 3242 : cluster [DBG] osdmap e655: 8 total, 8 up, 8 in 2024-03-26T04:00:57.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:57 smithi190 bash[23392]: audit 2024-03-26T04:00:56.219817+0000 mon.c (mon.2) 651 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:57.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:57 smithi190 bash[23392]: audit 2024-03-26T04:00:56.220583+0000 mon.a (mon.0) 3243 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:57.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:57 smithi190 bash[23392]: cluster 2024-03-26T04:00:56.655947+0000 mon.a (mon.0) 3244 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:00:57.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:57 smithi175 bash[21527]: cluster 2024-03-26T04:00:56.118253+0000 mon.a (mon.0) 3242 : cluster [DBG] osdmap e655: 8 total, 8 up, 8 in 2024-03-26T04:00:57.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:57 smithi175 bash[21527]: audit 2024-03-26T04:00:56.219817+0000 mon.c (mon.2) 651 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:57.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:57 smithi175 bash[21527]: audit 2024-03-26T04:00:56.220583+0000 mon.a (mon.0) 3243 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:57.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:57 smithi175 bash[21527]: cluster 2024-03-26T04:00:56.655947+0000 mon.a (mon.0) 3244 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:00:57.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:57 smithi175 bash[29284]: cluster 2024-03-26T04:00:56.118253+0000 mon.a (mon.0) 3242 : cluster [DBG] osdmap e655: 8 total, 8 up, 8 in 2024-03-26T04:00:57.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:57 smithi175 bash[29284]: audit 2024-03-26T04:00:56.219817+0000 mon.c (mon.2) 651 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:57.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:57 smithi175 bash[29284]: audit 2024-03-26T04:00:56.220583+0000 mon.a (mon.0) 3243 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:00:57.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:57 smithi175 bash[29284]: cluster 2024-03-26T04:00:56.655947+0000 mon.a (mon.0) 3244 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:00:58.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:58 smithi190 bash[23392]: cluster 2024-03-26T04:00:57.024065+0000 mgr.y (mgr.24431) 636 : cluster [DBG] pgmap v1052: 268 pgs: 268 active+clean; 582 KiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T04:00:58.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:58 smithi190 bash[23392]: audit 2024-03-26T04:00:57.120110+0000 mon.a (mon.0) 3245 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:00:58.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:58 smithi190 bash[23392]: cluster 2024-03-26T04:00:57.129676+0000 mon.a (mon.0) 3246 : cluster [DBG] osdmap e656: 8 total, 8 up, 8 in 2024-03-26T04:00:58.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:58 smithi190 bash[23392]: audit 2024-03-26T04:00:57.155046+0000 mon.c (mon.2) 652 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-132"}]: dispatch 2024-03-26T04:00:58.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:58 smithi190 bash[23392]: audit 2024-03-26T04:00:57.155774+0000 mon.a (mon.0) 3247 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-132"}]: dispatch 2024-03-26T04:00:58.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:58 smithi175 bash[29284]: cluster 2024-03-26T04:00:57.024065+0000 mgr.y (mgr.24431) 636 : cluster [DBG] pgmap v1052: 268 pgs: 268 active+clean; 582 KiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T04:00:58.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:58 smithi175 bash[29284]: audit 2024-03-26T04:00:57.120110+0000 mon.a (mon.0) 3245 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:00:58.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:58 smithi175 bash[29284]: cluster 2024-03-26T04:00:57.129676+0000 mon.a (mon.0) 3246 : cluster [DBG] osdmap e656: 8 total, 8 up, 8 in 2024-03-26T04:00:58.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:58 smithi175 bash[29284]: audit 2024-03-26T04:00:57.155046+0000 mon.c (mon.2) 652 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-132"}]: dispatch 2024-03-26T04:00:58.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:58 smithi175 bash[29284]: audit 2024-03-26T04:00:57.155774+0000 mon.a (mon.0) 3247 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-132"}]: dispatch 2024-03-26T04:00:58.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:58 smithi175 bash[21527]: cluster 2024-03-26T04:00:57.024065+0000 mgr.y (mgr.24431) 636 : cluster [DBG] pgmap v1052: 268 pgs: 268 active+clean; 582 KiB data, 709 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T04:00:58.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:58 smithi175 bash[21527]: audit 2024-03-26T04:00:57.120110+0000 mon.a (mon.0) 3245 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:00:58.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:58 smithi175 bash[21527]: cluster 2024-03-26T04:00:57.129676+0000 mon.a (mon.0) 3246 : cluster [DBG] osdmap e656: 8 total, 8 up, 8 in 2024-03-26T04:00:58.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:58 smithi175 bash[21527]: audit 2024-03-26T04:00:57.155046+0000 mon.c (mon.2) 652 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-132"}]: dispatch 2024-03-26T04:00:58.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:58 smithi175 bash[21527]: audit 2024-03-26T04:00:57.155774+0000 mon.a (mon.0) 3247 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-132"}]: dispatch 2024-03-26T04:00:59.143 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:00:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:00:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:00:59.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:59 smithi190 bash[23392]: audit 2024-03-26T04:00:58.122780+0000 mon.a (mon.0) 3248 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-132"}]': finished 2024-03-26T04:00:59.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:00:59 smithi190 bash[23392]: cluster 2024-03-26T04:00:58.130442+0000 mon.a (mon.0) 3249 : cluster [DBG] osdmap e657: 8 total, 8 up, 8 in 2024-03-26T04:00:59.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:59 smithi175 bash[29284]: audit 2024-03-26T04:00:58.122780+0000 mon.a (mon.0) 3248 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-132"}]': finished 2024-03-26T04:00:59.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:00:59 smithi175 bash[29284]: cluster 2024-03-26T04:00:58.130442+0000 mon.a (mon.0) 3249 : cluster [DBG] osdmap e657: 8 total, 8 up, 8 in 2024-03-26T04:00:59.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:59 smithi175 bash[21527]: audit 2024-03-26T04:00:58.122780+0000 mon.a (mon.0) 3248 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-132"}]': finished 2024-03-26T04:00:59.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:00:59 smithi175 bash[21527]: cluster 2024-03-26T04:00:58.130442+0000 mon.a (mon.0) 3249 : cluster [DBG] osdmap e657: 8 total, 8 up, 8 in 2024-03-26T04:01:00.410 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:00 smithi190 bash[23392]: cluster 2024-03-26T04:00:59.024966+0000 mgr.y (mgr.24431) 637 : cluster [DBG] pgmap v1055: 268 pgs: 268 active+clean; 582 KiB data, 709 MiB used, 715 GiB / 715 GiB avail 2024-03-26T04:01:00.410 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:00 smithi190 bash[23392]: cluster 2024-03-26T04:00:59.152680+0000 mon.a (mon.0) 3250 : cluster [DBG] osdmap e658: 8 total, 8 up, 8 in 2024-03-26T04:01:00.410 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:00 smithi190 bash[23392]: audit 2024-03-26T04:00:59.225296+0000 mon.c (mon.2) 653 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:00.410 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:00 smithi190 bash[23392]: audit 2024-03-26T04:00:59.225968+0000 mon.a (mon.0) 3251 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:00.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:00 smithi175 bash[29284]: cluster 2024-03-26T04:00:59.024966+0000 mgr.y (mgr.24431) 637 : cluster [DBG] pgmap v1055: 268 pgs: 268 active+clean; 582 KiB data, 709 MiB used, 715 GiB / 715 GiB avail 2024-03-26T04:01:00.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:00 smithi175 bash[29284]: cluster 2024-03-26T04:00:59.152680+0000 mon.a (mon.0) 3250 : cluster [DBG] osdmap e658: 8 total, 8 up, 8 in 2024-03-26T04:01:00.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:00 smithi175 bash[29284]: audit 2024-03-26T04:00:59.225296+0000 mon.c (mon.2) 653 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:00.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:00 smithi175 bash[29284]: audit 2024-03-26T04:00:59.225968+0000 mon.a (mon.0) 3251 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:00.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:00 smithi175 bash[21527]: cluster 2024-03-26T04:00:59.024966+0000 mgr.y (mgr.24431) 637 : cluster [DBG] pgmap v1055: 268 pgs: 268 active+clean; 582 KiB data, 709 MiB used, 715 GiB / 715 GiB avail 2024-03-26T04:01:00.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:00 smithi175 bash[21527]: cluster 2024-03-26T04:00:59.152680+0000 mon.a (mon.0) 3250 : cluster [DBG] osdmap e658: 8 total, 8 up, 8 in 2024-03-26T04:01:00.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:00 smithi175 bash[21527]: audit 2024-03-26T04:00:59.225296+0000 mon.c (mon.2) 653 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:00.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:00 smithi175 bash[21527]: audit 2024-03-26T04:00:59.225968+0000 mon.a (mon.0) 3251 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:00.741 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:01:00 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:01:01.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:01 smithi190 bash[23392]: audit 2024-03-26T04:01:00.147229+0000 mon.a (mon.0) 3252 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:01:01.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:01 smithi190 bash[23392]: audit 2024-03-26T04:01:00.151713+0000 mon.c (mon.2) 654 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-132"}]: dispatch 2024-03-26T04:01:01.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:01 smithi190 bash[23392]: cluster 2024-03-26T04:01:00.154329+0000 mon.a (mon.0) 3253 : cluster [DBG] osdmap e659: 8 total, 8 up, 8 in 2024-03-26T04:01:01.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:01 smithi190 bash[23392]: audit 2024-03-26T04:01:00.156204+0000 mon.a (mon.0) 3254 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-132"}]: dispatch 2024-03-26T04:01:01.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:01 smithi190 bash[23392]: audit 2024-03-26T04:01:00.411620+0000 mgr.y (mgr.24431) 638 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:01:01.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:01 smithi190 bash[23392]: cluster 2024-03-26T04:01:01.147645+0000 mon.a (mon.0) 3255 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:01:01.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:01 smithi190 bash[23392]: audit 2024-03-26T04:01:01.151891+0000 mon.a (mon.0) 3256 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-132"}]': finished 2024-03-26T04:01:01.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:01 smithi190 bash[23392]: cluster 2024-03-26T04:01:01.161654+0000 mon.a (mon.0) 3257 : cluster [DBG] osdmap e660: 8 total, 8 up, 8 in 2024-03-26T04:01:01.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:01 smithi175 bash[21527]: audit 2024-03-26T04:01:00.147229+0000 mon.a (mon.0) 3252 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:01:01.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:01 smithi175 bash[21527]: audit 2024-03-26T04:01:00.151713+0000 mon.c (mon.2) 654 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-132"}]: dispatch 2024-03-26T04:01:01.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:01 smithi175 bash[21527]: cluster 2024-03-26T04:01:00.154329+0000 mon.a (mon.0) 3253 : cluster [DBG] osdmap e659: 8 total, 8 up, 8 in 2024-03-26T04:01:01.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:01 smithi175 bash[21527]: audit 2024-03-26T04:01:00.156204+0000 mon.a (mon.0) 3254 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-132"}]: dispatch 2024-03-26T04:01:01.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:01 smithi175 bash[21527]: audit 2024-03-26T04:01:00.411620+0000 mgr.y (mgr.24431) 638 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:01:01.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:01 smithi175 bash[21527]: cluster 2024-03-26T04:01:01.147645+0000 mon.a (mon.0) 3255 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:01:01.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:01 smithi175 bash[21527]: audit 2024-03-26T04:01:01.151891+0000 mon.a (mon.0) 3256 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-132"}]': finished 2024-03-26T04:01:01.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:01 smithi175 bash[21527]: cluster 2024-03-26T04:01:01.161654+0000 mon.a (mon.0) 3257 : cluster [DBG] osdmap e660: 8 total, 8 up, 8 in 2024-03-26T04:01:01.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:01 smithi175 bash[29284]: audit 2024-03-26T04:01:00.147229+0000 mon.a (mon.0) 3252 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:01:01.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:01 smithi175 bash[29284]: audit 2024-03-26T04:01:00.151713+0000 mon.c (mon.2) 654 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-132"}]: dispatch 2024-03-26T04:01:01.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:01 smithi175 bash[29284]: cluster 2024-03-26T04:01:00.154329+0000 mon.a (mon.0) 3253 : cluster [DBG] osdmap e659: 8 total, 8 up, 8 in 2024-03-26T04:01:01.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:01 smithi175 bash[29284]: audit 2024-03-26T04:01:00.156204+0000 mon.a (mon.0) 3254 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-132"}]: dispatch 2024-03-26T04:01:01.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:01 smithi175 bash[29284]: audit 2024-03-26T04:01:00.411620+0000 mgr.y (mgr.24431) 638 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:01:01.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:01 smithi175 bash[29284]: cluster 2024-03-26T04:01:01.147645+0000 mon.a (mon.0) 3255 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:01:01.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:01 smithi175 bash[29284]: audit 2024-03-26T04:01:01.151891+0000 mon.a (mon.0) 3256 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-132"}]': finished 2024-03-26T04:01:01.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:01 smithi175 bash[29284]: cluster 2024-03-26T04:01:01.161654+0000 mon.a (mon.0) 3257 : cluster [DBG] osdmap e660: 8 total, 8 up, 8 in 2024-03-26T04:01:02.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:02 smithi190 bash[23392]: cluster 2024-03-26T04:01:01.025870+0000 mgr.y (mgr.24431) 639 : cluster [DBG] pgmap v1058: 268 pgs: 268 active+clean; 582 KiB data, 710 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-26T04:01:02.491 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:02 smithi190 bash[23392]: cluster 2024-03-26T04:01:01.669517+0000 mon.a (mon.0) 3258 : cluster [DBG] osdmap e661: 8 total, 8 up, 8 in 2024-03-26T04:01:02.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:02 smithi175 bash[21527]: cluster 2024-03-26T04:01:01.025870+0000 mgr.y (mgr.24431) 639 : cluster [DBG] pgmap v1058: 268 pgs: 268 active+clean; 582 KiB data, 710 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-26T04:01:02.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:02 smithi175 bash[21527]: cluster 2024-03-26T04:01:01.669517+0000 mon.a (mon.0) 3258 : cluster [DBG] osdmap e661: 8 total, 8 up, 8 in 2024-03-26T04:01:02.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:02 smithi175 bash[29284]: cluster 2024-03-26T04:01:01.025870+0000 mgr.y (mgr.24431) 639 : cluster [DBG] pgmap v1058: 268 pgs: 268 active+clean; 582 KiB data, 710 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-26T04:01:02.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:02 smithi175 bash[29284]: cluster 2024-03-26T04:01:01.669517+0000 mon.a (mon.0) 3258 : cluster [DBG] osdmap e661: 8 total, 8 up, 8 in 2024-03-26T04:01:03.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:03 smithi190 bash[23392]: cluster 2024-03-26T04:01:02.672003+0000 mon.a (mon.0) 3259 : cluster [DBG] osdmap e662: 8 total, 8 up, 8 in 2024-03-26T04:01:03.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:03 smithi190 bash[23392]: audit 2024-03-26T04:01:02.681903+0000 mon.c (mon.2) 655 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:01:03.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:03 smithi190 bash[23392]: audit 2024-03-26T04:01:02.684138+0000 mon.a (mon.0) 3260 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:01:03.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:03 smithi190 bash[23392]: cluster 2024-03-26T04:01:03.026789+0000 mgr.y (mgr.24431) 640 : cluster [DBG] pgmap v1062: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 710 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 528 B/s wr, 2 op/s 2024-03-26T04:01:03.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:03 smithi175 bash[21527]: cluster 2024-03-26T04:01:02.672003+0000 mon.a (mon.0) 3259 : cluster [DBG] osdmap e662: 8 total, 8 up, 8 in 2024-03-26T04:01:03.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:03 smithi175 bash[21527]: audit 2024-03-26T04:01:02.681903+0000 mon.c (mon.2) 655 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:01:03.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:03 smithi175 bash[21527]: audit 2024-03-26T04:01:02.684138+0000 mon.a (mon.0) 3260 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:01:03.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:03 smithi175 bash[21527]: cluster 2024-03-26T04:01:03.026789+0000 mgr.y (mgr.24431) 640 : cluster [DBG] pgmap v1062: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 710 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 528 B/s wr, 2 op/s 2024-03-26T04:01:03.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:03 smithi175 bash[29284]: cluster 2024-03-26T04:01:02.672003+0000 mon.a (mon.0) 3259 : cluster [DBG] osdmap e662: 8 total, 8 up, 8 in 2024-03-26T04:01:03.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:03 smithi175 bash[29284]: audit 2024-03-26T04:01:02.681903+0000 mon.c (mon.2) 655 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:01:03.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:03 smithi175 bash[29284]: audit 2024-03-26T04:01:02.684138+0000 mon.a (mon.0) 3260 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:01:03.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:03 smithi175 bash[29284]: cluster 2024-03-26T04:01:03.026789+0000 mgr.y (mgr.24431) 640 : cluster [DBG] pgmap v1062: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 710 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 528 B/s wr, 2 op/s 2024-03-26T04:01:04.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:04 smithi190 bash[23392]: cluster 2024-03-26T04:01:03.664191+0000 mon.a (mon.0) 3261 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:01:04.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:04 smithi190 bash[23392]: audit 2024-03-26T04:01:03.675809+0000 mon.a (mon.0) 3262 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:01:04.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:04 smithi190 bash[23392]: cluster 2024-03-26T04:01:03.687785+0000 mon.a (mon.0) 3263 : cluster [DBG] osdmap e663: 8 total, 8 up, 8 in 2024-03-26T04:01:04.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:04 smithi190 bash[23392]: audit 2024-03-26T04:01:03.709296+0000 mon.c (mon.2) 656 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:01:04.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:04 smithi190 bash[23392]: audit 2024-03-26T04:01:03.710540+0000 mon.a (mon.0) 3264 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:01:04.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:04 smithi175 bash[21527]: cluster 2024-03-26T04:01:03.664191+0000 mon.a (mon.0) 3261 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:01:04.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:04 smithi175 bash[21527]: audit 2024-03-26T04:01:03.675809+0000 mon.a (mon.0) 3262 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:01:04.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:04 smithi175 bash[21527]: cluster 2024-03-26T04:01:03.687785+0000 mon.a (mon.0) 3263 : cluster [DBG] osdmap e663: 8 total, 8 up, 8 in 2024-03-26T04:01:04.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:04 smithi175 bash[21527]: audit 2024-03-26T04:01:03.709296+0000 mon.c (mon.2) 656 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:01:04.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:04 smithi175 bash[21527]: audit 2024-03-26T04:01:03.710540+0000 mon.a (mon.0) 3264 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:01:04.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:04 smithi175 bash[29284]: cluster 2024-03-26T04:01:03.664191+0000 mon.a (mon.0) 3261 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:01:04.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:04 smithi175 bash[29284]: audit 2024-03-26T04:01:03.675809+0000 mon.a (mon.0) 3262 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:01:04.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:04 smithi175 bash[29284]: cluster 2024-03-26T04:01:03.687785+0000 mon.a (mon.0) 3263 : cluster [DBG] osdmap e663: 8 total, 8 up, 8 in 2024-03-26T04:01:04.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:04 smithi175 bash[29284]: audit 2024-03-26T04:01:03.709296+0000 mon.c (mon.2) 656 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:01:04.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:04 smithi175 bash[29284]: audit 2024-03-26T04:01:03.710540+0000 mon.a (mon.0) 3264 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:01:05.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:05 smithi190 bash[23392]: audit 2024-03-26T04:01:04.687848+0000 mon.a (mon.0) 3265 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-134", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:01:05.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:05 smithi190 bash[23392]: cluster 2024-03-26T04:01:04.697030+0000 mon.a (mon.0) 3266 : cluster [DBG] osdmap e664: 8 total, 8 up, 8 in 2024-03-26T04:01:05.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:05 smithi190 bash[23392]: audit 2024-03-26T04:01:04.698782+0000 mon.c (mon.2) 657 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-134"}]: dispatch 2024-03-26T04:01:05.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:05 smithi190 bash[23392]: audit 2024-03-26T04:01:04.699823+0000 mon.a (mon.0) 3267 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-134"}]: dispatch 2024-03-26T04:01:05.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:05 smithi190 bash[23392]: cluster 2024-03-26T04:01:05.029134+0000 mgr.y (mgr.24431) 641 : cluster [DBG] pgmap v1065: 268 pgs: 268 active+clean; 582 KiB data, 710 MiB used, 715 GiB / 715 GiB avail; 529 B/s rd, 0 op/s 2024-03-26T04:01:05.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:05 smithi175 bash[29284]: audit 2024-03-26T04:01:04.687848+0000 mon.a (mon.0) 3265 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-134", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:01:05.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:05 smithi175 bash[29284]: cluster 2024-03-26T04:01:04.697030+0000 mon.a (mon.0) 3266 : cluster [DBG] osdmap e664: 8 total, 8 up, 8 in 2024-03-26T04:01:05.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:05 smithi175 bash[29284]: audit 2024-03-26T04:01:04.698782+0000 mon.c (mon.2) 657 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-134"}]: dispatch 2024-03-26T04:01:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:05 smithi175 bash[29284]: audit 2024-03-26T04:01:04.699823+0000 mon.a (mon.0) 3267 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-134"}]: dispatch 2024-03-26T04:01:05.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:05 smithi175 bash[29284]: cluster 2024-03-26T04:01:05.029134+0000 mgr.y (mgr.24431) 641 : cluster [DBG] pgmap v1065: 268 pgs: 268 active+clean; 582 KiB data, 710 MiB used, 715 GiB / 715 GiB avail; 529 B/s rd, 0 op/s 2024-03-26T04:01:05.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:05 smithi175 bash[21527]: audit 2024-03-26T04:01:04.687848+0000 mon.a (mon.0) 3265 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-134", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:01:05.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:05 smithi175 bash[21527]: cluster 2024-03-26T04:01:04.697030+0000 mon.a (mon.0) 3266 : cluster [DBG] osdmap e664: 8 total, 8 up, 8 in 2024-03-26T04:01:05.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:05 smithi175 bash[21527]: audit 2024-03-26T04:01:04.698782+0000 mon.c (mon.2) 657 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-134"}]: dispatch 2024-03-26T04:01:05.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:05 smithi175 bash[21527]: audit 2024-03-26T04:01:04.699823+0000 mon.a (mon.0) 3267 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-134"}]: dispatch 2024-03-26T04:01:05.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:05 smithi175 bash[21527]: cluster 2024-03-26T04:01:05.029134+0000 mgr.y (mgr.24431) 641 : cluster [DBG] pgmap v1065: 268 pgs: 268 active+clean; 582 KiB data, 710 MiB used, 715 GiB / 715 GiB avail; 529 B/s rd, 0 op/s 2024-03-26T04:01:06.991 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:06 smithi190 bash[23392]: audit 2024-03-26T04:01:05.699174+0000 mon.a (mon.0) 3268 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-134"}]': finished 2024-03-26T04:01:06.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:06 smithi190 bash[23392]: cluster 2024-03-26T04:01:05.712468+0000 mon.a (mon.0) 3269 : cluster [DBG] osdmap e665: 8 total, 8 up, 8 in 2024-03-26T04:01:06.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:06 smithi190 bash[23392]: audit 2024-03-26T04:01:05.713215+0000 mon.c (mon.2) 658 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-134", "mode": "writeback"}]: dispatch 2024-03-26T04:01:06.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:06 smithi190 bash[23392]: audit 2024-03-26T04:01:05.716042+0000 mon.a (mon.0) 3270 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-134", "mode": "writeback"}]: dispatch 2024-03-26T04:01:06.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:06 smithi190 bash[23392]: audit 2024-03-26T04:01:05.927621+0000 mon.c (mon.2) 659 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:01:06.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:06 smithi190 bash[23392]: audit 2024-03-26T04:01:06.655726+0000 mon.c (mon.2) 660 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:01:06.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:06 smithi175 bash[21527]: audit 2024-03-26T04:01:05.699174+0000 mon.a (mon.0) 3268 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-134"}]': finished 2024-03-26T04:01:06.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:06 smithi175 bash[21527]: cluster 2024-03-26T04:01:05.712468+0000 mon.a (mon.0) 3269 : cluster [DBG] osdmap e665: 8 total, 8 up, 8 in 2024-03-26T04:01:06.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:06 smithi175 bash[21527]: audit 2024-03-26T04:01:05.713215+0000 mon.c (mon.2) 658 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-134", "mode": "writeback"}]: dispatch 2024-03-26T04:01:06.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:06 smithi175 bash[21527]: audit 2024-03-26T04:01:05.716042+0000 mon.a (mon.0) 3270 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-134", "mode": "writeback"}]: dispatch 2024-03-26T04:01:06.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:06 smithi175 bash[21527]: audit 2024-03-26T04:01:05.927621+0000 mon.c (mon.2) 659 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:01:06.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:06 smithi175 bash[21527]: audit 2024-03-26T04:01:06.655726+0000 mon.c (mon.2) 660 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:01:06.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:06 smithi175 bash[29284]: audit 2024-03-26T04:01:05.699174+0000 mon.a (mon.0) 3268 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-134"}]': finished 2024-03-26T04:01:06.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:06 smithi175 bash[29284]: cluster 2024-03-26T04:01:05.712468+0000 mon.a (mon.0) 3269 : cluster [DBG] osdmap e665: 8 total, 8 up, 8 in 2024-03-26T04:01:06.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:06 smithi175 bash[29284]: audit 2024-03-26T04:01:05.713215+0000 mon.c (mon.2) 658 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-134", "mode": "writeback"}]: dispatch 2024-03-26T04:01:06.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:06 smithi175 bash[29284]: audit 2024-03-26T04:01:05.716042+0000 mon.a (mon.0) 3270 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-134", "mode": "writeback"}]: dispatch 2024-03-26T04:01:06.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:06 smithi175 bash[29284]: audit 2024-03-26T04:01:05.927621+0000 mon.c (mon.2) 659 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:01:06.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:06 smithi175 bash[29284]: audit 2024-03-26T04:01:06.655726+0000 mon.c (mon.2) 660 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:01:07.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:07 smithi190 bash[23392]: cluster 2024-03-26T04:01:06.699410+0000 mon.a (mon.0) 3271 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:01:07.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:07 smithi190 bash[23392]: audit 2024-03-26T04:01:06.701938+0000 mon.a (mon.0) 3272 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-134", "mode": "writeback"}]': finished 2024-03-26T04:01:07.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:07 smithi190 bash[23392]: cluster 2024-03-26T04:01:06.712207+0000 mon.a (mon.0) 3273 : cluster [DBG] osdmap e666: 8 total, 8 up, 8 in 2024-03-26T04:01:07.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:07 smithi190 bash[23392]: audit 2024-03-26T04:01:06.794433+0000 mon.c (mon.2) 661 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:07.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:07 smithi190 bash[23392]: audit 2024-03-26T04:01:06.794881+0000 mon.a (mon.0) 3274 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:07.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:07 smithi190 bash[23392]: cluster 2024-03-26T04:01:07.029768+0000 mgr.y (mgr.24431) 642 : cluster [DBG] pgmap v1068: 268 pgs: 268 active+clean; 582 KiB data, 710 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:01:07.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:07 smithi190 bash[23392]: audit 2024-03-26T04:01:07.392910+0000 mon.c (mon.2) 662 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:01:07.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:07 smithi190 bash[23392]: audit 2024-03-26T04:01:07.394635+0000 mon.c (mon.2) 663 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:01:07.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:07 smithi190 bash[23392]: audit 2024-03-26T04:01:07.404986+0000 mon.a (mon.0) 3275 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:01:07.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:07 smithi175 bash[21527]: cluster 2024-03-26T04:01:06.699410+0000 mon.a (mon.0) 3271 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:01:07.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:07 smithi175 bash[21527]: audit 2024-03-26T04:01:06.701938+0000 mon.a (mon.0) 3272 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-134", "mode": "writeback"}]': finished 2024-03-26T04:01:07.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:07 smithi175 bash[21527]: cluster 2024-03-26T04:01:06.712207+0000 mon.a (mon.0) 3273 : cluster [DBG] osdmap e666: 8 total, 8 up, 8 in 2024-03-26T04:01:07.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:07 smithi175 bash[21527]: audit 2024-03-26T04:01:06.794433+0000 mon.c (mon.2) 661 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:07.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:07 smithi175 bash[21527]: audit 2024-03-26T04:01:06.794881+0000 mon.a (mon.0) 3274 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:07.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:07 smithi175 bash[21527]: cluster 2024-03-26T04:01:07.029768+0000 mgr.y (mgr.24431) 642 : cluster [DBG] pgmap v1068: 268 pgs: 268 active+clean; 582 KiB data, 710 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:01:07.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:07 smithi175 bash[21527]: audit 2024-03-26T04:01:07.392910+0000 mon.c (mon.2) 662 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:01:07.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:07 smithi175 bash[21527]: audit 2024-03-26T04:01:07.394635+0000 mon.c (mon.2) 663 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:01:07.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:07 smithi175 bash[21527]: audit 2024-03-26T04:01:07.404986+0000 mon.a (mon.0) 3275 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:01:07.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:07 smithi175 bash[29284]: cluster 2024-03-26T04:01:06.699410+0000 mon.a (mon.0) 3271 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:01:07.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:07 smithi175 bash[29284]: audit 2024-03-26T04:01:06.701938+0000 mon.a (mon.0) 3272 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-134", "mode": "writeback"}]': finished 2024-03-26T04:01:07.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:07 smithi175 bash[29284]: cluster 2024-03-26T04:01:06.712207+0000 mon.a (mon.0) 3273 : cluster [DBG] osdmap e666: 8 total, 8 up, 8 in 2024-03-26T04:01:07.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:07 smithi175 bash[29284]: audit 2024-03-26T04:01:06.794433+0000 mon.c (mon.2) 661 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:07.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:07 smithi175 bash[29284]: audit 2024-03-26T04:01:06.794881+0000 mon.a (mon.0) 3274 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:07.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:07 smithi175 bash[29284]: cluster 2024-03-26T04:01:07.029768+0000 mgr.y (mgr.24431) 642 : cluster [DBG] pgmap v1068: 268 pgs: 268 active+clean; 582 KiB data, 710 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:01:07.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:07 smithi175 bash[29284]: audit 2024-03-26T04:01:07.392910+0000 mon.c (mon.2) 662 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:01:07.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:07 smithi175 bash[29284]: audit 2024-03-26T04:01:07.394635+0000 mon.c (mon.2) 663 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:01:07.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:07 smithi175 bash[29284]: audit 2024-03-26T04:01:07.404986+0000 mon.a (mon.0) 3275 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:01:08.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:08 smithi190 bash[23392]: audit 2024-03-26T04:01:07.726975+0000 mon.a (mon.0) 3276 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:01:08.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:08 smithi190 bash[23392]: cluster 2024-03-26T04:01:07.739803+0000 mon.a (mon.0) 3277 : cluster [DBG] osdmap e667: 8 total, 8 up, 8 in 2024-03-26T04:01:08.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:08 smithi190 bash[23392]: audit 2024-03-26T04:01:07.742947+0000 mon.c (mon.2) 664 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-134"}]: dispatch 2024-03-26T04:01:08.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:08 smithi190 bash[23392]: audit 2024-03-26T04:01:07.745138+0000 mon.a (mon.0) 3278 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-134"}]: dispatch 2024-03-26T04:01:08.993 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:01:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:01:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:01:08.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:08 smithi175 bash[21527]: audit 2024-03-26T04:01:07.726975+0000 mon.a (mon.0) 3276 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:01:08.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:08 smithi175 bash[21527]: cluster 2024-03-26T04:01:07.739803+0000 mon.a (mon.0) 3277 : cluster [DBG] osdmap e667: 8 total, 8 up, 8 in 2024-03-26T04:01:08.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:08 smithi175 bash[21527]: audit 2024-03-26T04:01:07.742947+0000 mon.c (mon.2) 664 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-134"}]: dispatch 2024-03-26T04:01:08.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:08 smithi175 bash[21527]: audit 2024-03-26T04:01:07.745138+0000 mon.a (mon.0) 3278 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-134"}]: dispatch 2024-03-26T04:01:08.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:08 smithi175 bash[29284]: audit 2024-03-26T04:01:07.726975+0000 mon.a (mon.0) 3276 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:01:08.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:08 smithi175 bash[29284]: cluster 2024-03-26T04:01:07.739803+0000 mon.a (mon.0) 3277 : cluster [DBG] osdmap e667: 8 total, 8 up, 8 in 2024-03-26T04:01:08.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:08 smithi175 bash[29284]: audit 2024-03-26T04:01:07.742947+0000 mon.c (mon.2) 664 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-134"}]: dispatch 2024-03-26T04:01:08.995 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:08 smithi175 bash[29284]: audit 2024-03-26T04:01:07.745138+0000 mon.a (mon.0) 3278 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-134"}]: dispatch 2024-03-26T04:01:09.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:09 smithi190 bash[23392]: cluster 2024-03-26T04:01:08.728014+0000 mon.a (mon.0) 3279 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:01:09.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:09 smithi190 bash[23392]: audit 2024-03-26T04:01:08.738803+0000 mon.a (mon.0) 3280 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-134"}]': finished 2024-03-26T04:01:09.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:09 smithi190 bash[23392]: cluster 2024-03-26T04:01:08.746436+0000 mon.a (mon.0) 3281 : cluster [DBG] osdmap e668: 8 total, 8 up, 8 in 2024-03-26T04:01:09.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:09 smithi190 bash[23392]: cluster 2024-03-26T04:01:09.030650+0000 mgr.y (mgr.24431) 643 : cluster [DBG] pgmap v1071: 268 pgs: 268 active+clean; 582 KiB data, 710 MiB used, 715 GiB / 715 GiB avail 2024-03-26T04:01:09.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:09 smithi175 bash[29284]: cluster 2024-03-26T04:01:08.728014+0000 mon.a (mon.0) 3279 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:01:09.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:09 smithi175 bash[29284]: audit 2024-03-26T04:01:08.738803+0000 mon.a (mon.0) 3280 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-134"}]': finished 2024-03-26T04:01:09.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:09 smithi175 bash[29284]: cluster 2024-03-26T04:01:08.746436+0000 mon.a (mon.0) 3281 : cluster [DBG] osdmap e668: 8 total, 8 up, 8 in 2024-03-26T04:01:09.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:09 smithi175 bash[29284]: cluster 2024-03-26T04:01:09.030650+0000 mgr.y (mgr.24431) 643 : cluster [DBG] pgmap v1071: 268 pgs: 268 active+clean; 582 KiB data, 710 MiB used, 715 GiB / 715 GiB avail 2024-03-26T04:01:09.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:09 smithi175 bash[21527]: cluster 2024-03-26T04:01:08.728014+0000 mon.a (mon.0) 3279 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:01:09.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:09 smithi175 bash[21527]: audit 2024-03-26T04:01:08.738803+0000 mon.a (mon.0) 3280 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-134"}]': finished 2024-03-26T04:01:09.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:09 smithi175 bash[21527]: cluster 2024-03-26T04:01:08.746436+0000 mon.a (mon.0) 3281 : cluster [DBG] osdmap e668: 8 total, 8 up, 8 in 2024-03-26T04:01:09.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:09 smithi175 bash[21527]: cluster 2024-03-26T04:01:09.030650+0000 mgr.y (mgr.24431) 643 : cluster [DBG] pgmap v1071: 268 pgs: 268 active+clean; 582 KiB data, 710 MiB used, 715 GiB / 715 GiB avail 2024-03-26T04:01:10.732 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:01:10 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:01:10.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:10 smithi190 bash[23392]: cluster 2024-03-26T04:01:09.738772+0000 mon.a (mon.0) 3282 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:01:10.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:10 smithi190 bash[23392]: cluster 2024-03-26T04:01:09.755367+0000 mon.a (mon.0) 3283 : cluster [DBG] osdmap e669: 8 total, 8 up, 8 in 2024-03-26T04:01:10.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:10 smithi190 bash[23392]: audit 2024-03-26T04:01:10.421916+0000 mgr.y (mgr.24431) 644 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:01:10.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:10 smithi175 bash[21527]: cluster 2024-03-26T04:01:09.738772+0000 mon.a (mon.0) 3282 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:01:10.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:10 smithi175 bash[21527]: cluster 2024-03-26T04:01:09.755367+0000 mon.a (mon.0) 3283 : cluster [DBG] osdmap e669: 8 total, 8 up, 8 in 2024-03-26T04:01:10.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:10 smithi175 bash[21527]: audit 2024-03-26T04:01:10.421916+0000 mgr.y (mgr.24431) 644 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:01:10.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:10 smithi175 bash[29284]: cluster 2024-03-26T04:01:09.738772+0000 mon.a (mon.0) 3282 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:01:10.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:10 smithi175 bash[29284]: cluster 2024-03-26T04:01:09.755367+0000 mon.a (mon.0) 3283 : cluster [DBG] osdmap e669: 8 total, 8 up, 8 in 2024-03-26T04:01:10.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:10 smithi175 bash[29284]: audit 2024-03-26T04:01:10.421916+0000 mgr.y (mgr.24431) 644 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:01:12.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:11 smithi190 bash[23392]: cluster 2024-03-26T04:01:10.760284+0000 mon.a (mon.0) 3284 : cluster [DBG] osdmap e670: 8 total, 8 up, 8 in 2024-03-26T04:01:12.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:11 smithi190 bash[23392]: audit 2024-03-26T04:01:10.769838+0000 mon.c (mon.2) 665 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:01:12.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:11 smithi190 bash[23392]: audit 2024-03-26T04:01:10.770985+0000 mon.a (mon.0) 3285 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:01:12.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:11 smithi190 bash[23392]: cluster 2024-03-26T04:01:11.031632+0000 mgr.y (mgr.24431) 645 : cluster [DBG] pgmap v1074: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 711 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-26T04:01:12.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:11 smithi175 bash[21527]: cluster 2024-03-26T04:01:10.760284+0000 mon.a (mon.0) 3284 : cluster [DBG] osdmap e670: 8 total, 8 up, 8 in 2024-03-26T04:01:12.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:11 smithi175 bash[21527]: audit 2024-03-26T04:01:10.769838+0000 mon.c (mon.2) 665 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:01:12.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:11 smithi175 bash[21527]: audit 2024-03-26T04:01:10.770985+0000 mon.a (mon.0) 3285 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:01:12.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:11 smithi175 bash[21527]: cluster 2024-03-26T04:01:11.031632+0000 mgr.y (mgr.24431) 645 : cluster [DBG] pgmap v1074: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 711 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-26T04:01:12.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:11 smithi175 bash[29284]: cluster 2024-03-26T04:01:10.760284+0000 mon.a (mon.0) 3284 : cluster [DBG] osdmap e670: 8 total, 8 up, 8 in 2024-03-26T04:01:12.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:11 smithi175 bash[29284]: audit 2024-03-26T04:01:10.769838+0000 mon.c (mon.2) 665 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:01:12.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:11 smithi175 bash[29284]: audit 2024-03-26T04:01:10.770985+0000 mon.a (mon.0) 3285 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:01:12.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:11 smithi175 bash[29284]: cluster 2024-03-26T04:01:11.031632+0000 mgr.y (mgr.24431) 645 : cluster [DBG] pgmap v1074: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 711 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-26T04:01:13.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:12 smithi190 bash[23392]: audit 2024-03-26T04:01:11.761260+0000 mon.a (mon.0) 3286 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:01:13.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:12 smithi190 bash[23392]: cluster 2024-03-26T04:01:11.777433+0000 mon.a (mon.0) 3287 : cluster [DBG] osdmap e671: 8 total, 8 up, 8 in 2024-03-26T04:01:13.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:12 smithi190 bash[23392]: audit 2024-03-26T04:01:11.778369+0000 mon.c (mon.2) 666 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:01:13.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:12 smithi190 bash[23392]: audit 2024-03-26T04:01:11.791519+0000 mon.a (mon.0) 3288 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:01:13.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:12 smithi175 bash[21527]: audit 2024-03-26T04:01:11.761260+0000 mon.a (mon.0) 3286 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:01:13.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:12 smithi175 bash[21527]: cluster 2024-03-26T04:01:11.777433+0000 mon.a (mon.0) 3287 : cluster [DBG] osdmap e671: 8 total, 8 up, 8 in 2024-03-26T04:01:13.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:12 smithi175 bash[21527]: audit 2024-03-26T04:01:11.778369+0000 mon.c (mon.2) 666 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:01:13.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:12 smithi175 bash[21527]: audit 2024-03-26T04:01:11.791519+0000 mon.a (mon.0) 3288 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:01:13.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:12 smithi175 bash[29284]: audit 2024-03-26T04:01:11.761260+0000 mon.a (mon.0) 3286 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:01:13.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:12 smithi175 bash[29284]: cluster 2024-03-26T04:01:11.777433+0000 mon.a (mon.0) 3287 : cluster [DBG] osdmap e671: 8 total, 8 up, 8 in 2024-03-26T04:01:13.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:12 smithi175 bash[29284]: audit 2024-03-26T04:01:11.778369+0000 mon.c (mon.2) 666 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:01:13.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:12 smithi175 bash[29284]: audit 2024-03-26T04:01:11.791519+0000 mon.a (mon.0) 3288 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:01:14.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:13 smithi190 bash[23392]: audit 2024-03-26T04:01:12.764172+0000 mon.a (mon.0) 3289 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-136", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:01:14.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:13 smithi190 bash[23392]: cluster 2024-03-26T04:01:12.771942+0000 mon.a (mon.0) 3290 : cluster [DBG] osdmap e672: 8 total, 8 up, 8 in 2024-03-26T04:01:14.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:13 smithi190 bash[23392]: audit 2024-03-26T04:01:12.775921+0000 mon.c (mon.2) 667 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-136"}]: dispatch 2024-03-26T04:01:14.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:13 smithi190 bash[23392]: audit 2024-03-26T04:01:12.788055+0000 mon.a (mon.0) 3291 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-136"}]: dispatch 2024-03-26T04:01:14.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:13 smithi190 bash[23392]: cluster 2024-03-26T04:01:13.032528+0000 mgr.y (mgr.24431) 646 : cluster [DBG] pgmap v1077: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 711 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-26T04:01:14.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:13 smithi175 bash[29284]: audit 2024-03-26T04:01:12.764172+0000 mon.a (mon.0) 3289 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-136", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:01:14.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:13 smithi175 bash[29284]: cluster 2024-03-26T04:01:12.771942+0000 mon.a (mon.0) 3290 : cluster [DBG] osdmap e672: 8 total, 8 up, 8 in 2024-03-26T04:01:14.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:13 smithi175 bash[29284]: audit 2024-03-26T04:01:12.775921+0000 mon.c (mon.2) 667 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-136"}]: dispatch 2024-03-26T04:01:14.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:13 smithi175 bash[29284]: audit 2024-03-26T04:01:12.788055+0000 mon.a (mon.0) 3291 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-136"}]: dispatch 2024-03-26T04:01:14.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:13 smithi175 bash[29284]: cluster 2024-03-26T04:01:13.032528+0000 mgr.y (mgr.24431) 646 : cluster [DBG] pgmap v1077: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 711 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-26T04:01:14.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:13 smithi175 bash[21527]: audit 2024-03-26T04:01:12.764172+0000 mon.a (mon.0) 3289 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-136", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:01:14.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:13 smithi175 bash[21527]: cluster 2024-03-26T04:01:12.771942+0000 mon.a (mon.0) 3290 : cluster [DBG] osdmap e672: 8 total, 8 up, 8 in 2024-03-26T04:01:14.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:13 smithi175 bash[21527]: audit 2024-03-26T04:01:12.775921+0000 mon.c (mon.2) 667 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-136"}]: dispatch 2024-03-26T04:01:14.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:13 smithi175 bash[21527]: audit 2024-03-26T04:01:12.788055+0000 mon.a (mon.0) 3291 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-136"}]: dispatch 2024-03-26T04:01:14.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:13 smithi175 bash[21527]: cluster 2024-03-26T04:01:13.032528+0000 mgr.y (mgr.24431) 646 : cluster [DBG] pgmap v1077: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 711 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-26T04:01:15.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:14 smithi190 bash[23392]: audit 2024-03-26T04:01:13.778753+0000 mon.a (mon.0) 3292 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-136"}]': finished 2024-03-26T04:01:15.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:14 smithi190 bash[23392]: cluster 2024-03-26T04:01:13.785956+0000 mon.a (mon.0) 3293 : cluster [DBG] osdmap e673: 8 total, 8 up, 8 in 2024-03-26T04:01:15.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:14 smithi190 bash[23392]: audit 2024-03-26T04:01:13.788372+0000 mon.c (mon.2) 668 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-136", "mode": "writeback"}]: dispatch 2024-03-26T04:01:15.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:14 smithi190 bash[23392]: audit 2024-03-26T04:01:13.791892+0000 mon.a (mon.0) 3294 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-136", "mode": "writeback"}]: dispatch 2024-03-26T04:01:15.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:14 smithi190 bash[23392]: cluster 2024-03-26T04:01:14.779197+0000 mon.a (mon.0) 3295 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:01:15.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:14 smithi190 bash[23392]: audit 2024-03-26T04:01:14.781581+0000 mon.a (mon.0) 3296 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-136", "mode": "writeback"}]': finished 2024-03-26T04:01:15.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:14 smithi190 bash[23392]: cluster 2024-03-26T04:01:14.789150+0000 mon.a (mon.0) 3297 : cluster [DBG] osdmap e674: 8 total, 8 up, 8 in 2024-03-26T04:01:15.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:14 smithi175 bash[21527]: audit 2024-03-26T04:01:13.778753+0000 mon.a (mon.0) 3292 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-136"}]': finished 2024-03-26T04:01:15.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:14 smithi175 bash[21527]: cluster 2024-03-26T04:01:13.785956+0000 mon.a (mon.0) 3293 : cluster [DBG] osdmap e673: 8 total, 8 up, 8 in 2024-03-26T04:01:15.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:14 smithi175 bash[21527]: audit 2024-03-26T04:01:13.788372+0000 mon.c (mon.2) 668 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-136", "mode": "writeback"}]: dispatch 2024-03-26T04:01:15.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:14 smithi175 bash[21527]: audit 2024-03-26T04:01:13.791892+0000 mon.a (mon.0) 3294 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-136", "mode": "writeback"}]: dispatch 2024-03-26T04:01:15.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:14 smithi175 bash[21527]: cluster 2024-03-26T04:01:14.779197+0000 mon.a (mon.0) 3295 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:01:15.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:14 smithi175 bash[21527]: audit 2024-03-26T04:01:14.781581+0000 mon.a (mon.0) 3296 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-136", "mode": "writeback"}]': finished 2024-03-26T04:01:15.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:14 smithi175 bash[21527]: cluster 2024-03-26T04:01:14.789150+0000 mon.a (mon.0) 3297 : cluster [DBG] osdmap e674: 8 total, 8 up, 8 in 2024-03-26T04:01:15.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:14 smithi175 bash[29284]: audit 2024-03-26T04:01:13.778753+0000 mon.a (mon.0) 3292 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-136"}]': finished 2024-03-26T04:01:15.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:14 smithi175 bash[29284]: cluster 2024-03-26T04:01:13.785956+0000 mon.a (mon.0) 3293 : cluster [DBG] osdmap e673: 8 total, 8 up, 8 in 2024-03-26T04:01:15.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:14 smithi175 bash[29284]: audit 2024-03-26T04:01:13.788372+0000 mon.c (mon.2) 668 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-136", "mode": "writeback"}]: dispatch 2024-03-26T04:01:15.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:14 smithi175 bash[29284]: audit 2024-03-26T04:01:13.791892+0000 mon.a (mon.0) 3294 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-136", "mode": "writeback"}]: dispatch 2024-03-26T04:01:15.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:14 smithi175 bash[29284]: cluster 2024-03-26T04:01:14.779197+0000 mon.a (mon.0) 3295 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:01:15.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:14 smithi175 bash[29284]: audit 2024-03-26T04:01:14.781581+0000 mon.a (mon.0) 3296 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-136", "mode": "writeback"}]': finished 2024-03-26T04:01:15.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:14 smithi175 bash[29284]: cluster 2024-03-26T04:01:14.789150+0000 mon.a (mon.0) 3297 : cluster [DBG] osdmap e674: 8 total, 8 up, 8 in 2024-03-26T04:01:16.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:15 smithi190 bash[23392]: audit 2024-03-26T04:01:14.920465+0000 mon.c (mon.2) 669 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:16.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:15 smithi190 bash[23392]: audit 2024-03-26T04:01:14.921290+0000 mon.a (mon.0) 3298 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:16.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:15 smithi190 bash[23392]: cluster 2024-03-26T04:01:15.034369+0000 mgr.y (mgr.24431) 647 : cluster [DBG] pgmap v1080: 268 pgs: 268 active+clean; 4.4 MiB data, 716 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-26T04:01:16.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:15 smithi175 bash[21527]: audit 2024-03-26T04:01:14.920465+0000 mon.c (mon.2) 669 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:16.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:15 smithi175 bash[21527]: audit 2024-03-26T04:01:14.921290+0000 mon.a (mon.0) 3298 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:16.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:15 smithi175 bash[21527]: cluster 2024-03-26T04:01:15.034369+0000 mgr.y (mgr.24431) 647 : cluster [DBG] pgmap v1080: 268 pgs: 268 active+clean; 4.4 MiB data, 716 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-26T04:01:16.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:15 smithi175 bash[29284]: audit 2024-03-26T04:01:14.920465+0000 mon.c (mon.2) 669 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:16.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:15 smithi175 bash[29284]: audit 2024-03-26T04:01:14.921290+0000 mon.a (mon.0) 3298 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:16.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:15 smithi175 bash[29284]: cluster 2024-03-26T04:01:15.034369+0000 mgr.y (mgr.24431) 647 : cluster [DBG] pgmap v1080: 268 pgs: 268 active+clean; 4.4 MiB data, 716 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-26T04:01:17.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:16 smithi190 bash[23392]: audit 2024-03-26T04:01:15.809419+0000 mon.a (mon.0) 3299 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:01:17.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:16 smithi190 bash[23392]: cluster 2024-03-26T04:01:15.817420+0000 mon.a (mon.0) 3300 : cluster [DBG] osdmap e675: 8 total, 8 up, 8 in 2024-03-26T04:01:17.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:16 smithi190 bash[23392]: audit 2024-03-26T04:01:15.824376+0000 mon.c (mon.2) 670 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-136"}]: dispatch 2024-03-26T04:01:17.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:16 smithi190 bash[23392]: audit 2024-03-26T04:01:15.827796+0000 mon.a (mon.0) 3301 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-136"}]: dispatch 2024-03-26T04:01:17.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:16 smithi190 bash[23392]: cluster 2024-03-26T04:01:16.662051+0000 mon.a (mon.0) 3302 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:01:17.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:16 smithi175 bash[21527]: audit 2024-03-26T04:01:15.809419+0000 mon.a (mon.0) 3299 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:01:17.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:16 smithi175 bash[21527]: cluster 2024-03-26T04:01:15.817420+0000 mon.a (mon.0) 3300 : cluster [DBG] osdmap e675: 8 total, 8 up, 8 in 2024-03-26T04:01:17.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:16 smithi175 bash[21527]: audit 2024-03-26T04:01:15.824376+0000 mon.c (mon.2) 670 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-136"}]: dispatch 2024-03-26T04:01:17.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:16 smithi175 bash[21527]: audit 2024-03-26T04:01:15.827796+0000 mon.a (mon.0) 3301 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-136"}]: dispatch 2024-03-26T04:01:17.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:16 smithi175 bash[21527]: cluster 2024-03-26T04:01:16.662051+0000 mon.a (mon.0) 3302 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:01:17.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:16 smithi175 bash[29284]: audit 2024-03-26T04:01:15.809419+0000 mon.a (mon.0) 3299 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:01:17.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:16 smithi175 bash[29284]: cluster 2024-03-26T04:01:15.817420+0000 mon.a (mon.0) 3300 : cluster [DBG] osdmap e675: 8 total, 8 up, 8 in 2024-03-26T04:01:17.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:16 smithi175 bash[29284]: audit 2024-03-26T04:01:15.824376+0000 mon.c (mon.2) 670 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-136"}]: dispatch 2024-03-26T04:01:17.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:16 smithi175 bash[29284]: audit 2024-03-26T04:01:15.827796+0000 mon.a (mon.0) 3301 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-136"}]: dispatch 2024-03-26T04:01:17.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:16 smithi175 bash[29284]: cluster 2024-03-26T04:01:16.662051+0000 mon.a (mon.0) 3302 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:01:18.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:17 smithi190 bash[23392]: cluster 2024-03-26T04:01:16.809717+0000 mon.a (mon.0) 3303 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:01:18.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:17 smithi190 bash[23392]: audit 2024-03-26T04:01:16.812115+0000 mon.a (mon.0) 3304 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-136"}]': finished 2024-03-26T04:01:18.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:17 smithi190 bash[23392]: cluster 2024-03-26T04:01:16.820548+0000 mon.a (mon.0) 3305 : cluster [DBG] osdmap e676: 8 total, 8 up, 8 in 2024-03-26T04:01:18.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:17 smithi190 bash[23392]: cluster 2024-03-26T04:01:17.035341+0000 mgr.y (mgr.24431) 648 : cluster [DBG] pgmap v1083: 268 pgs: 268 active+clean; 4.4 MiB data, 716 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-26T04:01:18.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:17 smithi175 bash[29284]: cluster 2024-03-26T04:01:16.809717+0000 mon.a (mon.0) 3303 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:01:18.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:17 smithi175 bash[29284]: audit 2024-03-26T04:01:16.812115+0000 mon.a (mon.0) 3304 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-136"}]': finished 2024-03-26T04:01:18.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:17 smithi175 bash[29284]: cluster 2024-03-26T04:01:16.820548+0000 mon.a (mon.0) 3305 : cluster [DBG] osdmap e676: 8 total, 8 up, 8 in 2024-03-26T04:01:18.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:17 smithi175 bash[29284]: cluster 2024-03-26T04:01:17.035341+0000 mgr.y (mgr.24431) 648 : cluster [DBG] pgmap v1083: 268 pgs: 268 active+clean; 4.4 MiB data, 716 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-26T04:01:18.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:17 smithi175 bash[21527]: cluster 2024-03-26T04:01:16.809717+0000 mon.a (mon.0) 3303 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:01:18.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:17 smithi175 bash[21527]: audit 2024-03-26T04:01:16.812115+0000 mon.a (mon.0) 3304 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-136"}]': finished 2024-03-26T04:01:18.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:17 smithi175 bash[21527]: cluster 2024-03-26T04:01:16.820548+0000 mon.a (mon.0) 3305 : cluster [DBG] osdmap e676: 8 total, 8 up, 8 in 2024-03-26T04:01:18.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:17 smithi175 bash[21527]: cluster 2024-03-26T04:01:17.035341+0000 mgr.y (mgr.24431) 648 : cluster [DBG] pgmap v1083: 268 pgs: 268 active+clean; 4.4 MiB data, 716 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-26T04:01:19.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:18 smithi190 bash[23392]: cluster 2024-03-26T04:01:17.843889+0000 mon.a (mon.0) 3306 : cluster [DBG] osdmap e677: 8 total, 8 up, 8 in 2024-03-26T04:01:19.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:18 smithi175 bash[29284]: cluster 2024-03-26T04:01:17.843889+0000 mon.a (mon.0) 3306 : cluster [DBG] osdmap e677: 8 total, 8 up, 8 in 2024-03-26T04:01:19.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:18 smithi175 bash[21527]: cluster 2024-03-26T04:01:17.843889+0000 mon.a (mon.0) 3306 : cluster [DBG] osdmap e677: 8 total, 8 up, 8 in 2024-03-26T04:01:19.243 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:01:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:01:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:01:20.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:19 smithi190 bash[23392]: cluster 2024-03-26T04:01:18.850071+0000 mon.a (mon.0) 3307 : cluster [DBG] osdmap e678: 8 total, 8 up, 8 in 2024-03-26T04:01:20.292 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:19 smithi190 bash[23392]: audit 2024-03-26T04:01:18.855213+0000 mon.c (mon.2) 671 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:01:20.292 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:19 smithi190 bash[23392]: audit 2024-03-26T04:01:18.879768+0000 mon.a (mon.0) 3308 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:01:20.292 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:19 smithi190 bash[23392]: cluster 2024-03-26T04:01:19.036223+0000 mgr.y (mgr.24431) 649 : cluster [DBG] pgmap v1086: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 716 MiB used, 715 GiB / 715 GiB avail 2024-03-26T04:01:20.292 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:19 smithi190 bash[23392]: audit 2024-03-26T04:01:19.842097+0000 mon.a (mon.0) 3309 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:01:20.292 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:19 smithi190 bash[23392]: cluster 2024-03-26T04:01:19.850197+0000 mon.a (mon.0) 3310 : cluster [DBG] osdmap e679: 8 total, 8 up, 8 in 2024-03-26T04:01:20.293 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:19 smithi175 bash[21527]: cluster 2024-03-26T04:01:18.850071+0000 mon.a (mon.0) 3307 : cluster [DBG] osdmap e678: 8 total, 8 up, 8 in 2024-03-26T04:01:20.293 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:19 smithi175 bash[21527]: audit 2024-03-26T04:01:18.855213+0000 mon.c (mon.2) 671 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:01:20.293 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:19 smithi175 bash[21527]: audit 2024-03-26T04:01:18.879768+0000 mon.a (mon.0) 3308 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:01:20.293 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:19 smithi175 bash[21527]: cluster 2024-03-26T04:01:19.036223+0000 mgr.y (mgr.24431) 649 : cluster [DBG] pgmap v1086: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 716 MiB used, 715 GiB / 715 GiB avail 2024-03-26T04:01:20.293 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:19 smithi175 bash[21527]: audit 2024-03-26T04:01:19.842097+0000 mon.a (mon.0) 3309 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:01:20.293 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:19 smithi175 bash[21527]: cluster 2024-03-26T04:01:19.850197+0000 mon.a (mon.0) 3310 : cluster [DBG] osdmap e679: 8 total, 8 up, 8 in 2024-03-26T04:01:20.293 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:19 smithi175 bash[29284]: cluster 2024-03-26T04:01:18.850071+0000 mon.a (mon.0) 3307 : cluster [DBG] osdmap e678: 8 total, 8 up, 8 in 2024-03-26T04:01:20.294 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:19 smithi175 bash[29284]: audit 2024-03-26T04:01:18.855213+0000 mon.c (mon.2) 671 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:01:20.294 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:19 smithi175 bash[29284]: audit 2024-03-26T04:01:18.879768+0000 mon.a (mon.0) 3308 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:01:20.294 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:19 smithi175 bash[29284]: cluster 2024-03-26T04:01:19.036223+0000 mgr.y (mgr.24431) 649 : cluster [DBG] pgmap v1086: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 716 MiB used, 715 GiB / 715 GiB avail 2024-03-26T04:01:20.294 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:19 smithi175 bash[29284]: audit 2024-03-26T04:01:19.842097+0000 mon.a (mon.0) 3309 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:01:20.294 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:19 smithi175 bash[29284]: cluster 2024-03-26T04:01:19.850197+0000 mon.a (mon.0) 3310 : cluster [DBG] osdmap e679: 8 total, 8 up, 8 in 2024-03-26T04:01:20.742 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:01:20 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:01:21.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:20 smithi190 bash[23392]: audit 2024-03-26T04:01:19.862713+0000 mon.c (mon.2) 672 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:01:21.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:20 smithi190 bash[23392]: audit 2024-03-26T04:01:19.882541+0000 mon.a (mon.0) 3311 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:01:21.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:20 smithi190 bash[23392]: audit 2024-03-26T04:01:20.426407+0000 mgr.y (mgr.24431) 650 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:01:21.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:20 smithi190 bash[23392]: audit 2024-03-26T04:01:20.868798+0000 mon.a (mon.0) 3312 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-138", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:01:21.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:20 smithi190 bash[23392]: cluster 2024-03-26T04:01:20.876001+0000 mon.a (mon.0) 3313 : cluster [DBG] osdmap e680: 8 total, 8 up, 8 in 2024-03-26T04:01:21.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:20 smithi190 bash[23392]: audit 2024-03-26T04:01:20.879727+0000 mon.c (mon.2) 673 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-138","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T04:01:21.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:20 smithi190 bash[23392]: audit 2024-03-26T04:01:20.880768+0000 mon.a (mon.0) 3314 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-138","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T04:01:21.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:20 smithi175 bash[21527]: audit 2024-03-26T04:01:19.862713+0000 mon.c (mon.2) 672 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:01:21.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:20 smithi175 bash[21527]: audit 2024-03-26T04:01:19.882541+0000 mon.a (mon.0) 3311 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:01:21.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:20 smithi175 bash[21527]: audit 2024-03-26T04:01:20.426407+0000 mgr.y (mgr.24431) 650 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:01:21.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:20 smithi175 bash[21527]: audit 2024-03-26T04:01:20.868798+0000 mon.a (mon.0) 3312 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-138", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:01:21.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:20 smithi175 bash[21527]: cluster 2024-03-26T04:01:20.876001+0000 mon.a (mon.0) 3313 : cluster [DBG] osdmap e680: 8 total, 8 up, 8 in 2024-03-26T04:01:21.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:20 smithi175 bash[21527]: audit 2024-03-26T04:01:20.879727+0000 mon.c (mon.2) 673 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-138","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T04:01:21.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:20 smithi175 bash[21527]: audit 2024-03-26T04:01:20.880768+0000 mon.a (mon.0) 3314 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-138","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T04:01:21.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:20 smithi175 bash[29284]: audit 2024-03-26T04:01:19.862713+0000 mon.c (mon.2) 672 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:01:21.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:20 smithi175 bash[29284]: audit 2024-03-26T04:01:19.882541+0000 mon.a (mon.0) 3311 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:01:21.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:20 smithi175 bash[29284]: audit 2024-03-26T04:01:20.426407+0000 mgr.y (mgr.24431) 650 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:01:21.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:20 smithi175 bash[29284]: audit 2024-03-26T04:01:20.868798+0000 mon.a (mon.0) 3312 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-138", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:01:21.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:20 smithi175 bash[29284]: cluster 2024-03-26T04:01:20.876001+0000 mon.a (mon.0) 3313 : cluster [DBG] osdmap e680: 8 total, 8 up, 8 in 2024-03-26T04:01:21.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:20 smithi175 bash[29284]: audit 2024-03-26T04:01:20.879727+0000 mon.c (mon.2) 673 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-138","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T04:01:21.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:20 smithi175 bash[29284]: audit 2024-03-26T04:01:20.880768+0000 mon.a (mon.0) 3314 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-138","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T04:01:22.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:21 smithi190 bash[23392]: audit 2024-03-26T04:01:20.961768+0000 mon.c (mon.2) 674 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:01:22.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:21 smithi190 bash[23392]: cluster 2024-03-26T04:01:21.037125+0000 mgr.y (mgr.24431) 651 : cluster [DBG] pgmap v1089: 268 pgs: 3 creating+activating, 26 creating+peering, 239 active+clean; 4.4 MiB data, 722 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:01:22.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:21 smithi190 bash[23392]: cluster 2024-03-26T04:01:21.663270+0000 mon.a (mon.0) 3315 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:01:22.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:21 smithi190 bash[23392]: audit 2024-03-26T04:01:21.871827+0000 mon.a (mon.0) 3316 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-138","var": "hit_set_count","val": "2"}]': finished 2024-03-26T04:01:22.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:21 smithi190 bash[23392]: cluster 2024-03-26T04:01:21.879186+0000 mon.a (mon.0) 3317 : cluster [DBG] osdmap e681: 8 total, 8 up, 8 in 2024-03-26T04:01:22.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:21 smithi190 bash[23392]: audit 2024-03-26T04:01:21.884465+0000 mon.c (mon.2) 675 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-138","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T04:01:22.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:21 smithi175 bash[21527]: audit 2024-03-26T04:01:20.961768+0000 mon.c (mon.2) 674 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:01:22.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:21 smithi175 bash[21527]: cluster 2024-03-26T04:01:21.037125+0000 mgr.y (mgr.24431) 651 : cluster [DBG] pgmap v1089: 268 pgs: 3 creating+activating, 26 creating+peering, 239 active+clean; 4.4 MiB data, 722 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:01:22.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:21 smithi175 bash[21527]: cluster 2024-03-26T04:01:21.663270+0000 mon.a (mon.0) 3315 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:01:22.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:21 smithi175 bash[21527]: audit 2024-03-26T04:01:21.871827+0000 mon.a (mon.0) 3316 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-138","var": "hit_set_count","val": "2"}]': finished 2024-03-26T04:01:22.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:21 smithi175 bash[21527]: cluster 2024-03-26T04:01:21.879186+0000 mon.a (mon.0) 3317 : cluster [DBG] osdmap e681: 8 total, 8 up, 8 in 2024-03-26T04:01:22.244 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:21 smithi175 bash[21527]: audit 2024-03-26T04:01:21.884465+0000 mon.c (mon.2) 675 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-138","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T04:01:22.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:21 smithi175 bash[29284]: audit 2024-03-26T04:01:20.961768+0000 mon.c (mon.2) 674 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:01:22.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:21 smithi175 bash[29284]: cluster 2024-03-26T04:01:21.037125+0000 mgr.y (mgr.24431) 651 : cluster [DBG] pgmap v1089: 268 pgs: 3 creating+activating, 26 creating+peering, 239 active+clean; 4.4 MiB data, 722 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:01:22.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:21 smithi175 bash[29284]: cluster 2024-03-26T04:01:21.663270+0000 mon.a (mon.0) 3315 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:01:22.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:21 smithi175 bash[29284]: audit 2024-03-26T04:01:21.871827+0000 mon.a (mon.0) 3316 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-138","var": "hit_set_count","val": "2"}]': finished 2024-03-26T04:01:22.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:21 smithi175 bash[29284]: cluster 2024-03-26T04:01:21.879186+0000 mon.a (mon.0) 3317 : cluster [DBG] osdmap e681: 8 total, 8 up, 8 in 2024-03-26T04:01:22.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:21 smithi175 bash[29284]: audit 2024-03-26T04:01:21.884465+0000 mon.c (mon.2) 675 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-138","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T04:01:23.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:22 smithi190 bash[23392]: audit 2024-03-26T04:01:21.887652+0000 mon.a (mon.0) 3318 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-138","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T04:01:23.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:22 smithi190 bash[23392]: audit 2024-03-26T04:01:22.874414+0000 mon.a (mon.0) 3319 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-138","var": "hit_set_period","val": "600"}]': finished 2024-03-26T04:01:23.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:22 smithi190 bash[23392]: cluster 2024-03-26T04:01:22.881564+0000 mon.a (mon.0) 3320 : cluster [DBG] osdmap e682: 8 total, 8 up, 8 in 2024-03-26T04:01:23.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:22 smithi190 bash[23392]: audit 2024-03-26T04:01:22.884444+0000 mon.c (mon.2) 676 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-26T04:01:23.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:22 smithi190 bash[23392]: audit 2024-03-26T04:01:22.885874+0000 mon.a (mon.0) 3321 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-26T04:01:23.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:22 smithi175 bash[21527]: audit 2024-03-26T04:01:21.887652+0000 mon.a (mon.0) 3318 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-138","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T04:01:23.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:22 smithi175 bash[21527]: audit 2024-03-26T04:01:22.874414+0000 mon.a (mon.0) 3319 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-138","var": "hit_set_period","val": "600"}]': finished 2024-03-26T04:01:23.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:22 smithi175 bash[21527]: cluster 2024-03-26T04:01:22.881564+0000 mon.a (mon.0) 3320 : cluster [DBG] osdmap e682: 8 total, 8 up, 8 in 2024-03-26T04:01:23.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:22 smithi175 bash[21527]: audit 2024-03-26T04:01:22.884444+0000 mon.c (mon.2) 676 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-26T04:01:23.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:22 smithi175 bash[21527]: audit 2024-03-26T04:01:22.885874+0000 mon.a (mon.0) 3321 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-26T04:01:23.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:22 smithi175 bash[29284]: audit 2024-03-26T04:01:21.887652+0000 mon.a (mon.0) 3318 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-138","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T04:01:23.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:22 smithi175 bash[29284]: audit 2024-03-26T04:01:22.874414+0000 mon.a (mon.0) 3319 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-138","var": "hit_set_period","val": "600"}]': finished 2024-03-26T04:01:23.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:22 smithi175 bash[29284]: cluster 2024-03-26T04:01:22.881564+0000 mon.a (mon.0) 3320 : cluster [DBG] osdmap e682: 8 total, 8 up, 8 in 2024-03-26T04:01:23.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:22 smithi175 bash[29284]: audit 2024-03-26T04:01:22.884444+0000 mon.c (mon.2) 676 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-26T04:01:23.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:22 smithi175 bash[29284]: audit 2024-03-26T04:01:22.885874+0000 mon.a (mon.0) 3321 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-26T04:01:24.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:23 smithi190 bash[23392]: cluster 2024-03-26T04:01:23.038013+0000 mgr.y (mgr.24431) 652 : cluster [DBG] pgmap v1092: 268 pgs: 3 creating+activating, 26 creating+peering, 239 active+clean; 4.4 MiB data, 722 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-26T04:01:24.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:23 smithi190 bash[23392]: audit 2024-03-26T04:01:23.877125+0000 mon.a (mon.0) 3322 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-26T04:01:24.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:23 smithi190 bash[23392]: cluster 2024-03-26T04:01:23.885667+0000 mon.a (mon.0) 3323 : cluster [DBG] osdmap e683: 8 total, 8 up, 8 in 2024-03-26T04:01:24.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:23 smithi175 bash[21527]: cluster 2024-03-26T04:01:23.038013+0000 mgr.y (mgr.24431) 652 : cluster [DBG] pgmap v1092: 268 pgs: 3 creating+activating, 26 creating+peering, 239 active+clean; 4.4 MiB data, 722 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-26T04:01:24.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:23 smithi175 bash[21527]: audit 2024-03-26T04:01:23.877125+0000 mon.a (mon.0) 3322 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-26T04:01:24.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:23 smithi175 bash[21527]: cluster 2024-03-26T04:01:23.885667+0000 mon.a (mon.0) 3323 : cluster [DBG] osdmap e683: 8 total, 8 up, 8 in 2024-03-26T04:01:24.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:23 smithi175 bash[29284]: cluster 2024-03-26T04:01:23.038013+0000 mgr.y (mgr.24431) 652 : cluster [DBG] pgmap v1092: 268 pgs: 3 creating+activating, 26 creating+peering, 239 active+clean; 4.4 MiB data, 722 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-26T04:01:24.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:23 smithi175 bash[29284]: audit 2024-03-26T04:01:23.877125+0000 mon.a (mon.0) 3322 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-26T04:01:24.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:23 smithi175 bash[29284]: cluster 2024-03-26T04:01:23.885667+0000 mon.a (mon.0) 3323 : cluster [DBG] osdmap e683: 8 total, 8 up, 8 in 2024-03-26T04:01:25.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:24 smithi190 bash[23392]: audit 2024-03-26T04:01:24.937765+0000 mon.c (mon.2) 677 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:25.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:24 smithi190 bash[23392]: audit 2024-03-26T04:01:24.938505+0000 mon.a (mon.0) 3324 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:25.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:24 smithi190 bash[23392]: audit 2024-03-26T04:01:24.939814+0000 mon.c (mon.2) 678 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-138"}]: dispatch 2024-03-26T04:01:25.242 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:24 smithi190 bash[23392]: audit 2024-03-26T04:01:24.940417+0000 mon.a (mon.0) 3325 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-138"}]: dispatch 2024-03-26T04:01:25.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:24 smithi175 bash[29284]: audit 2024-03-26T04:01:24.937765+0000 mon.c (mon.2) 677 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:25.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:24 smithi175 bash[29284]: audit 2024-03-26T04:01:24.938505+0000 mon.a (mon.0) 3324 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:25.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:24 smithi175 bash[29284]: audit 2024-03-26T04:01:24.939814+0000 mon.c (mon.2) 678 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-138"}]: dispatch 2024-03-26T04:01:25.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:24 smithi175 bash[29284]: audit 2024-03-26T04:01:24.940417+0000 mon.a (mon.0) 3325 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-138"}]: dispatch 2024-03-26T04:01:25.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:24 smithi175 bash[21527]: audit 2024-03-26T04:01:24.937765+0000 mon.c (mon.2) 677 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:25.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:24 smithi175 bash[21527]: audit 2024-03-26T04:01:24.938505+0000 mon.a (mon.0) 3324 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:25.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:24 smithi175 bash[21527]: audit 2024-03-26T04:01:24.939814+0000 mon.c (mon.2) 678 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-138"}]: dispatch 2024-03-26T04:01:25.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:24 smithi175 bash[21527]: audit 2024-03-26T04:01:24.940417+0000 mon.a (mon.0) 3325 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-138"}]: dispatch 2024-03-26T04:01:26.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:26 smithi190 bash[23392]: audit 2024-03-26T04:01:25.003964+0000 mon.a (mon.0) 3326 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-138"}]': finished 2024-03-26T04:01:26.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:26 smithi190 bash[23392]: cluster 2024-03-26T04:01:25.012184+0000 mon.a (mon.0) 3327 : cluster [DBG] osdmap e684: 8 total, 8 up, 8 in 2024-03-26T04:01:26.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:26 smithi190 bash[23392]: cluster 2024-03-26T04:01:25.039727+0000 mgr.y (mgr.24431) 653 : cluster [DBG] pgmap v1095: 268 pgs: 268 active+clean; 4.4 MiB data, 723 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:01:26.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:26 smithi175 bash[21527]: audit 2024-03-26T04:01:25.003964+0000 mon.a (mon.0) 3326 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-138"}]': finished 2024-03-26T04:01:26.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:26 smithi175 bash[21527]: cluster 2024-03-26T04:01:25.012184+0000 mon.a (mon.0) 3327 : cluster [DBG] osdmap e684: 8 total, 8 up, 8 in 2024-03-26T04:01:26.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:26 smithi175 bash[21527]: cluster 2024-03-26T04:01:25.039727+0000 mgr.y (mgr.24431) 653 : cluster [DBG] pgmap v1095: 268 pgs: 268 active+clean; 4.4 MiB data, 723 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:01:26.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:26 smithi175 bash[29284]: audit 2024-03-26T04:01:25.003964+0000 mon.a (mon.0) 3326 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-138"}]': finished 2024-03-26T04:01:26.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:26 smithi175 bash[29284]: cluster 2024-03-26T04:01:25.012184+0000 mon.a (mon.0) 3327 : cluster [DBG] osdmap e684: 8 total, 8 up, 8 in 2024-03-26T04:01:26.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:26 smithi175 bash[29284]: cluster 2024-03-26T04:01:25.039727+0000 mgr.y (mgr.24431) 653 : cluster [DBG] pgmap v1095: 268 pgs: 268 active+clean; 4.4 MiB data, 723 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:01:27.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:27 smithi190 bash[23392]: cluster 2024-03-26T04:01:26.018229+0000 mon.a (mon.0) 3328 : cluster [DBG] osdmap e685: 8 total, 8 up, 8 in 2024-03-26T04:01:27.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:27 smithi190 bash[23392]: cluster 2024-03-26T04:01:27.022301+0000 mon.a (mon.0) 3329 : cluster [DBG] osdmap e686: 8 total, 8 up, 8 in 2024-03-26T04:01:27.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:27 smithi190 bash[23392]: audit 2024-03-26T04:01:27.030289+0000 mon.c (mon.2) 679 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:01:27.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:27 smithi190 bash[23392]: audit 2024-03-26T04:01:27.032934+0000 mon.a (mon.0) 3330 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:01:27.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:27 smithi175 bash[21527]: cluster 2024-03-26T04:01:26.018229+0000 mon.a (mon.0) 3328 : cluster [DBG] osdmap e685: 8 total, 8 up, 8 in 2024-03-26T04:01:27.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:27 smithi175 bash[21527]: cluster 2024-03-26T04:01:27.022301+0000 mon.a (mon.0) 3329 : cluster [DBG] osdmap e686: 8 total, 8 up, 8 in 2024-03-26T04:01:27.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:27 smithi175 bash[21527]: audit 2024-03-26T04:01:27.030289+0000 mon.c (mon.2) 679 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:01:27.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:27 smithi175 bash[21527]: audit 2024-03-26T04:01:27.032934+0000 mon.a (mon.0) 3330 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:01:27.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:27 smithi175 bash[29284]: cluster 2024-03-26T04:01:26.018229+0000 mon.a (mon.0) 3328 : cluster [DBG] osdmap e685: 8 total, 8 up, 8 in 2024-03-26T04:01:27.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:27 smithi175 bash[29284]: cluster 2024-03-26T04:01:27.022301+0000 mon.a (mon.0) 3329 : cluster [DBG] osdmap e686: 8 total, 8 up, 8 in 2024-03-26T04:01:27.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:27 smithi175 bash[29284]: audit 2024-03-26T04:01:27.030289+0000 mon.c (mon.2) 679 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:01:27.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:27 smithi175 bash[29284]: audit 2024-03-26T04:01:27.032934+0000 mon.a (mon.0) 3330 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:01:28.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:28 smithi190 bash[23392]: cluster 2024-03-26T04:01:27.040944+0000 mgr.y (mgr.24431) 654 : cluster [DBG] pgmap v1098: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 723 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:01:28.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:28 smithi190 bash[23392]: audit 2024-03-26T04:01:28.014095+0000 mon.a (mon.0) 3331 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:01:28.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:28 smithi190 bash[23392]: cluster 2024-03-26T04:01:28.021814+0000 mon.a (mon.0) 3332 : cluster [DBG] osdmap e687: 8 total, 8 up, 8 in 2024-03-26T04:01:28.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:28 smithi190 bash[23392]: audit 2024-03-26T04:01:28.043083+0000 mon.c (mon.2) 680 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:01:28.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:28 smithi190 bash[23392]: audit 2024-03-26T04:01:28.045644+0000 mon.a (mon.0) 3333 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:01:28.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:28 smithi175 bash[29284]: cluster 2024-03-26T04:01:27.040944+0000 mgr.y (mgr.24431) 654 : cluster [DBG] pgmap v1098: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 723 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:01:28.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:28 smithi175 bash[29284]: audit 2024-03-26T04:01:28.014095+0000 mon.a (mon.0) 3331 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:01:28.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:28 smithi175 bash[29284]: cluster 2024-03-26T04:01:28.021814+0000 mon.a (mon.0) 3332 : cluster [DBG] osdmap e687: 8 total, 8 up, 8 in 2024-03-26T04:01:28.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:28 smithi175 bash[29284]: audit 2024-03-26T04:01:28.043083+0000 mon.c (mon.2) 680 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:01:28.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:28 smithi175 bash[29284]: audit 2024-03-26T04:01:28.045644+0000 mon.a (mon.0) 3333 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:01:28.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:28 smithi175 bash[21527]: cluster 2024-03-26T04:01:27.040944+0000 mgr.y (mgr.24431) 654 : cluster [DBG] pgmap v1098: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 723 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:01:28.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:28 smithi175 bash[21527]: audit 2024-03-26T04:01:28.014095+0000 mon.a (mon.0) 3331 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:01:28.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:28 smithi175 bash[21527]: cluster 2024-03-26T04:01:28.021814+0000 mon.a (mon.0) 3332 : cluster [DBG] osdmap e687: 8 total, 8 up, 8 in 2024-03-26T04:01:28.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:28 smithi175 bash[21527]: audit 2024-03-26T04:01:28.043083+0000 mon.c (mon.2) 680 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:01:28.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:28 smithi175 bash[21527]: audit 2024-03-26T04:01:28.045644+0000 mon.a (mon.0) 3333 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:01:29.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:01:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:01:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:01:30.435 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:30 smithi190 bash[23392]: audit 2024-03-26T04:01:29.016833+0000 mon.a (mon.0) 3334 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-140", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:01:30.436 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:30 smithi190 bash[23392]: cluster 2024-03-26T04:01:29.024889+0000 mon.a (mon.0) 3335 : cluster [DBG] osdmap e688: 8 total, 8 up, 8 in 2024-03-26T04:01:30.436 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:30 smithi190 bash[23392]: audit 2024-03-26T04:01:29.027022+0000 mon.c (mon.2) 681 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_count","val": "3"}]: dispatch 2024-03-26T04:01:30.436 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:30 smithi190 bash[23392]: audit 2024-03-26T04:01:29.028954+0000 mon.a (mon.0) 3336 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_count","val": "3"}]: dispatch 2024-03-26T04:01:30.436 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:30 smithi190 bash[23392]: cluster 2024-03-26T04:01:29.041825+0000 mgr.y (mgr.24431) 655 : cluster [DBG] pgmap v1101: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 723 MiB used, 715 GiB / 715 GiB avail 2024-03-26T04:01:30.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:30 smithi175 bash[29284]: audit 2024-03-26T04:01:29.016833+0000 mon.a (mon.0) 3334 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-140", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:01:30.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:30 smithi175 bash[29284]: cluster 2024-03-26T04:01:29.024889+0000 mon.a (mon.0) 3335 : cluster [DBG] osdmap e688: 8 total, 8 up, 8 in 2024-03-26T04:01:30.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:30 smithi175 bash[29284]: audit 2024-03-26T04:01:29.027022+0000 mon.c (mon.2) 681 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_count","val": "3"}]: dispatch 2024-03-26T04:01:30.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:30 smithi175 bash[29284]: audit 2024-03-26T04:01:29.028954+0000 mon.a (mon.0) 3336 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_count","val": "3"}]: dispatch 2024-03-26T04:01:30.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:30 smithi175 bash[29284]: cluster 2024-03-26T04:01:29.041825+0000 mgr.y (mgr.24431) 655 : cluster [DBG] pgmap v1101: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 723 MiB used, 715 GiB / 715 GiB avail 2024-03-26T04:01:30.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:30 smithi175 bash[21527]: audit 2024-03-26T04:01:29.016833+0000 mon.a (mon.0) 3334 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-140", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:01:30.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:30 smithi175 bash[21527]: cluster 2024-03-26T04:01:29.024889+0000 mon.a (mon.0) 3335 : cluster [DBG] osdmap e688: 8 total, 8 up, 8 in 2024-03-26T04:01:30.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:30 smithi175 bash[21527]: audit 2024-03-26T04:01:29.027022+0000 mon.c (mon.2) 681 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_count","val": "3"}]: dispatch 2024-03-26T04:01:30.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:30 smithi175 bash[21527]: audit 2024-03-26T04:01:29.028954+0000 mon.a (mon.0) 3336 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_count","val": "3"}]: dispatch 2024-03-26T04:01:30.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:30 smithi175 bash[21527]: cluster 2024-03-26T04:01:29.041825+0000 mgr.y (mgr.24431) 655 : cluster [DBG] pgmap v1101: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 723 MiB used, 715 GiB / 715 GiB avail 2024-03-26T04:01:30.742 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:01:30 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:01:31.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:31 smithi190 bash[23392]: cluster 2024-03-26T04:01:30.016723+0000 mon.a (mon.0) 3337 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:01:31.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:31 smithi190 bash[23392]: audit 2024-03-26T04:01:30.028248+0000 mon.a (mon.0) 3338 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_count","val": "3"}]': finished 2024-03-26T04:01:31.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:31 smithi190 bash[23392]: audit 2024-03-26T04:01:30.040206+0000 mon.c (mon.2) 682 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_period","val": "3"}]: dispatch 2024-03-26T04:01:31.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:31 smithi190 bash[23392]: cluster 2024-03-26T04:01:30.041267+0000 mon.a (mon.0) 3339 : cluster [DBG] osdmap e689: 8 total, 8 up, 8 in 2024-03-26T04:01:31.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:31 smithi190 bash[23392]: audit 2024-03-26T04:01:30.043628+0000 mon.a (mon.0) 3340 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_period","val": "3"}]: dispatch 2024-03-26T04:01:31.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:31 smithi190 bash[23392]: audit 2024-03-26T04:01:30.432949+0000 mgr.y (mgr.24431) 656 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:01:31.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:31 smithi175 bash[29284]: cluster 2024-03-26T04:01:30.016723+0000 mon.a (mon.0) 3337 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:01:31.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:31 smithi175 bash[29284]: audit 2024-03-26T04:01:30.028248+0000 mon.a (mon.0) 3338 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_count","val": "3"}]': finished 2024-03-26T04:01:31.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:31 smithi175 bash[29284]: audit 2024-03-26T04:01:30.040206+0000 mon.c (mon.2) 682 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_period","val": "3"}]: dispatch 2024-03-26T04:01:31.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:31 smithi175 bash[29284]: cluster 2024-03-26T04:01:30.041267+0000 mon.a (mon.0) 3339 : cluster [DBG] osdmap e689: 8 total, 8 up, 8 in 2024-03-26T04:01:31.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:31 smithi175 bash[29284]: audit 2024-03-26T04:01:30.043628+0000 mon.a (mon.0) 3340 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_period","val": "3"}]: dispatch 2024-03-26T04:01:31.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:31 smithi175 bash[29284]: audit 2024-03-26T04:01:30.432949+0000 mgr.y (mgr.24431) 656 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:01:31.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:31 smithi175 bash[21527]: cluster 2024-03-26T04:01:30.016723+0000 mon.a (mon.0) 3337 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:01:31.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:31 smithi175 bash[21527]: audit 2024-03-26T04:01:30.028248+0000 mon.a (mon.0) 3338 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_count","val": "3"}]': finished 2024-03-26T04:01:31.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:31 smithi175 bash[21527]: audit 2024-03-26T04:01:30.040206+0000 mon.c (mon.2) 682 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_period","val": "3"}]: dispatch 2024-03-26T04:01:31.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:31 smithi175 bash[21527]: cluster 2024-03-26T04:01:30.041267+0000 mon.a (mon.0) 3339 : cluster [DBG] osdmap e689: 8 total, 8 up, 8 in 2024-03-26T04:01:31.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:31 smithi175 bash[21527]: audit 2024-03-26T04:01:30.043628+0000 mon.a (mon.0) 3340 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_period","val": "3"}]: dispatch 2024-03-26T04:01:31.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:31 smithi175 bash[21527]: audit 2024-03-26T04:01:30.432949+0000 mgr.y (mgr.24431) 656 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:01:32.406 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:32 smithi190 bash[23392]: audit 2024-03-26T04:01:31.031105+0000 mon.a (mon.0) 3341 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_period","val": "3"}]': finished 2024-03-26T04:01:32.406 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:32 smithi190 bash[23392]: cluster 2024-03-26T04:01:31.039499+0000 mon.a (mon.0) 3342 : cluster [DBG] osdmap e690: 8 total, 8 up, 8 in 2024-03-26T04:01:32.406 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:32 smithi190 bash[23392]: audit 2024-03-26T04:01:31.042168+0000 mon.c (mon.2) 683 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-26T04:01:32.406 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:32 smithi190 bash[23392]: cluster 2024-03-26T04:01:31.042979+0000 mgr.y (mgr.24431) 657 : cluster [DBG] pgmap v1104: 268 pgs: 268 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:01:32.406 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:32 smithi190 bash[23392]: audit 2024-03-26T04:01:31.056531+0000 mon.a (mon.0) 3343 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-26T04:01:32.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:32 smithi175 bash[21527]: audit 2024-03-26T04:01:31.031105+0000 mon.a (mon.0) 3341 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_period","val": "3"}]': finished 2024-03-26T04:01:32.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:32 smithi175 bash[21527]: cluster 2024-03-26T04:01:31.039499+0000 mon.a (mon.0) 3342 : cluster [DBG] osdmap e690: 8 total, 8 up, 8 in 2024-03-26T04:01:32.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:32 smithi175 bash[21527]: audit 2024-03-26T04:01:31.042168+0000 mon.c (mon.2) 683 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-26T04:01:32.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:32 smithi175 bash[21527]: cluster 2024-03-26T04:01:31.042979+0000 mgr.y (mgr.24431) 657 : cluster [DBG] pgmap v1104: 268 pgs: 268 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:01:32.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:32 smithi175 bash[21527]: audit 2024-03-26T04:01:31.056531+0000 mon.a (mon.0) 3343 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-26T04:01:32.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:32 smithi175 bash[29284]: audit 2024-03-26T04:01:31.031105+0000 mon.a (mon.0) 3341 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_period","val": "3"}]': finished 2024-03-26T04:01:32.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:32 smithi175 bash[29284]: cluster 2024-03-26T04:01:31.039499+0000 mon.a (mon.0) 3342 : cluster [DBG] osdmap e690: 8 total, 8 up, 8 in 2024-03-26T04:01:32.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:32 smithi175 bash[29284]: audit 2024-03-26T04:01:31.042168+0000 mon.c (mon.2) 683 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-26T04:01:32.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:32 smithi175 bash[29284]: cluster 2024-03-26T04:01:31.042979+0000 mgr.y (mgr.24431) 657 : cluster [DBG] pgmap v1104: 268 pgs: 268 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:01:32.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:32 smithi175 bash[29284]: audit 2024-03-26T04:01:31.056531+0000 mon.a (mon.0) 3343 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-26T04:01:32.742 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 04:01:32 smithi190 bash[50860]: logger=sqlstore.transactions t=2024-03-26T04:01:32.370377233Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-03-26T04:01:33.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:33 smithi190 bash[23392]: audit 2024-03-26T04:01:32.047668+0000 mon.a (mon.0) 3344 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_type","val": "bloom"}]': finished 2024-03-26T04:01:33.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:33 smithi190 bash[23392]: cluster 2024-03-26T04:01:32.055666+0000 mon.a (mon.0) 3345 : cluster [DBG] osdmap e691: 8 total, 8 up, 8 in 2024-03-26T04:01:33.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:33 smithi190 bash[23392]: audit 2024-03-26T04:01:32.058782+0000 mon.c (mon.2) 684 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-26T04:01:33.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:33 smithi190 bash[23392]: audit 2024-03-26T04:01:32.071146+0000 mon.a (mon.0) 3346 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-26T04:01:33.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:33 smithi175 bash[21527]: audit 2024-03-26T04:01:32.047668+0000 mon.a (mon.0) 3344 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_type","val": "bloom"}]': finished 2024-03-26T04:01:33.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:33 smithi175 bash[21527]: cluster 2024-03-26T04:01:32.055666+0000 mon.a (mon.0) 3345 : cluster [DBG] osdmap e691: 8 total, 8 up, 8 in 2024-03-26T04:01:33.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:33 smithi175 bash[21527]: audit 2024-03-26T04:01:32.058782+0000 mon.c (mon.2) 684 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-26T04:01:33.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:33 smithi175 bash[21527]: audit 2024-03-26T04:01:32.071146+0000 mon.a (mon.0) 3346 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-26T04:01:33.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:33 smithi175 bash[29284]: audit 2024-03-26T04:01:32.047668+0000 mon.a (mon.0) 3344 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_type","val": "bloom"}]': finished 2024-03-26T04:01:33.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:33 smithi175 bash[29284]: cluster 2024-03-26T04:01:32.055666+0000 mon.a (mon.0) 3345 : cluster [DBG] osdmap e691: 8 total, 8 up, 8 in 2024-03-26T04:01:33.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:33 smithi175 bash[29284]: audit 2024-03-26T04:01:32.058782+0000 mon.c (mon.2) 684 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-26T04:01:33.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:33 smithi175 bash[29284]: audit 2024-03-26T04:01:32.071146+0000 mon.a (mon.0) 3346 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-26T04:01:34.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:34 smithi190 bash[23392]: cluster 2024-03-26T04:01:33.043929+0000 mgr.y (mgr.24431) 658 : cluster [DBG] pgmap v1106: 268 pgs: 268 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:01:34.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:34 smithi190 bash[23392]: audit 2024-03-26T04:01:33.070409+0000 mon.a (mon.0) 3347 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-03-26T04:01:34.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:34 smithi190 bash[23392]: cluster 2024-03-26T04:01:33.084068+0000 mon.a (mon.0) 3348 : cluster [DBG] osdmap e692: 8 total, 8 up, 8 in 2024-03-26T04:01:34.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:34 smithi175 bash[21527]: cluster 2024-03-26T04:01:33.043929+0000 mgr.y (mgr.24431) 658 : cluster [DBG] pgmap v1106: 268 pgs: 268 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:01:34.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:34 smithi175 bash[21527]: audit 2024-03-26T04:01:33.070409+0000 mon.a (mon.0) 3347 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-03-26T04:01:34.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:34 smithi175 bash[21527]: cluster 2024-03-26T04:01:33.084068+0000 mon.a (mon.0) 3348 : cluster [DBG] osdmap e692: 8 total, 8 up, 8 in 2024-03-26T04:01:34.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:34 smithi175 bash[29284]: cluster 2024-03-26T04:01:33.043929+0000 mgr.y (mgr.24431) 658 : cluster [DBG] pgmap v1106: 268 pgs: 268 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:01:34.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:34 smithi175 bash[29284]: audit 2024-03-26T04:01:33.070409+0000 mon.a (mon.0) 3347 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-03-26T04:01:34.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:34 smithi175 bash[29284]: cluster 2024-03-26T04:01:33.084068+0000 mon.a (mon.0) 3348 : cluster [DBG] osdmap e692: 8 total, 8 up, 8 in 2024-03-26T04:01:36.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:36 smithi190 bash[23392]: cluster 2024-03-26T04:01:35.046417+0000 mgr.y (mgr.24431) 659 : cluster [DBG] pgmap v1108: 268 pgs: 268 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 1021 B/s rd, 3.2 KiB/s wr, 1 op/s 2024-03-26T04:01:36.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:36 smithi190 bash[23392]: audit 2024-03-26T04:01:35.996320+0000 mon.c (mon.2) 685 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:01:36.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:36 smithi175 bash[29284]: cluster 2024-03-26T04:01:35.046417+0000 mgr.y (mgr.24431) 659 : cluster [DBG] pgmap v1108: 268 pgs: 268 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 1021 B/s rd, 3.2 KiB/s wr, 1 op/s 2024-03-26T04:01:36.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:36 smithi175 bash[29284]: audit 2024-03-26T04:01:35.996320+0000 mon.c (mon.2) 685 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:01:36.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:36 smithi175 bash[21527]: cluster 2024-03-26T04:01:35.046417+0000 mgr.y (mgr.24431) 659 : cluster [DBG] pgmap v1108: 268 pgs: 268 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 1021 B/s rd, 3.2 KiB/s wr, 1 op/s 2024-03-26T04:01:36.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:36 smithi175 bash[21527]: audit 2024-03-26T04:01:35.996320+0000 mon.c (mon.2) 685 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:01:38.492 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:38 smithi190 bash[23392]: cluster 2024-03-26T04:01:37.047244+0000 mgr.y (mgr.24431) 660 : cluster [DBG] pgmap v1109: 268 pgs: 268 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 851 B/s rd, 2.7 KiB/s wr, 1 op/s 2024-03-26T04:01:38.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:38 smithi175 bash[21527]: cluster 2024-03-26T04:01:37.047244+0000 mgr.y (mgr.24431) 660 : cluster [DBG] pgmap v1109: 268 pgs: 268 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 851 B/s rd, 2.7 KiB/s wr, 1 op/s 2024-03-26T04:01:38.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:38 smithi175 bash[29284]: cluster 2024-03-26T04:01:37.047244+0000 mgr.y (mgr.24431) 660 : cluster [DBG] pgmap v1109: 268 pgs: 268 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 851 B/s rd, 2.7 KiB/s wr, 1 op/s 2024-03-26T04:01:39.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:01:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:01:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:01:40.441 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:40 smithi190 bash[23392]: cluster 2024-03-26T04:01:39.048114+0000 mgr.y (mgr.24431) 661 : cluster [DBG] pgmap v1110: 268 pgs: 268 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-03-26T04:01:40.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:40 smithi175 bash[29284]: cluster 2024-03-26T04:01:39.048114+0000 mgr.y (mgr.24431) 661 : cluster [DBG] pgmap v1110: 268 pgs: 268 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-03-26T04:01:40.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:40 smithi175 bash[21527]: cluster 2024-03-26T04:01:39.048114+0000 mgr.y (mgr.24431) 661 : cluster [DBG] pgmap v1110: 268 pgs: 268 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-03-26T04:01:40.742 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:01:40 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:01:41.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:41 smithi175 bash[29284]: audit 2024-03-26T04:01:40.442580+0000 mgr.y (mgr.24431) 662 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:01:41.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:41 smithi190 bash[23392]: audit 2024-03-26T04:01:40.442580+0000 mgr.y (mgr.24431) 662 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:01:41.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:41 smithi175 bash[21527]: audit 2024-03-26T04:01:40.442580+0000 mgr.y (mgr.24431) 662 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:01:42.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:42 smithi175 bash[21527]: cluster 2024-03-26T04:01:41.050710+0000 mgr.y (mgr.24431) 663 : cluster [DBG] pgmap v1111: 268 pgs: 268 active+clean; 4.4 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 6.4 KiB/s wr, 2 op/s 2024-03-26T04:01:42.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:42 smithi190 bash[23392]: cluster 2024-03-26T04:01:41.050710+0000 mgr.y (mgr.24431) 663 : cluster [DBG] pgmap v1111: 268 pgs: 268 active+clean; 4.4 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 6.4 KiB/s wr, 2 op/s 2024-03-26T04:01:42.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:42 smithi175 bash[29284]: cluster 2024-03-26T04:01:41.050710+0000 mgr.y (mgr.24431) 663 : cluster [DBG] pgmap v1111: 268 pgs: 268 active+clean; 4.4 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 6.4 KiB/s wr, 2 op/s 2024-03-26T04:01:44.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:44 smithi175 bash[21527]: cluster 2024-03-26T04:01:43.051544+0000 mgr.y (mgr.24431) 664 : cluster [DBG] pgmap v1112: 268 pgs: 268 active+clean; 4.4 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 5.8 KiB/s wr, 1 op/s 2024-03-26T04:01:44.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:44 smithi175 bash[29284]: cluster 2024-03-26T04:01:43.051544+0000 mgr.y (mgr.24431) 664 : cluster [DBG] pgmap v1112: 268 pgs: 268 active+clean; 4.4 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 5.8 KiB/s wr, 1 op/s 2024-03-26T04:01:44.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:44 smithi190 bash[23392]: cluster 2024-03-26T04:01:43.051544+0000 mgr.y (mgr.24431) 664 : cluster [DBG] pgmap v1112: 268 pgs: 268 active+clean; 4.4 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 5.8 KiB/s wr, 1 op/s 2024-03-26T04:01:46.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:46 smithi190 bash[23392]: cluster 2024-03-26T04:01:45.053997+0000 mgr.y (mgr.24431) 665 : cluster [DBG] pgmap v1113: 268 pgs: 268 active+clean; 4.5 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 8.3 KiB/s wr, 2 op/s 2024-03-26T04:01:46.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:46 smithi190 bash[23392]: audit 2024-03-26T04:01:45.185207+0000 mon.c (mon.2) 686 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:46.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:46 smithi190 bash[23392]: audit 2024-03-26T04:01:45.185900+0000 mon.a (mon.0) 3349 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:46.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:46 smithi190 bash[23392]: audit 2024-03-26T04:01:45.187072+0000 mon.c (mon.2) 687 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-140"}]: dispatch 2024-03-26T04:01:46.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:46 smithi190 bash[23392]: audit 2024-03-26T04:01:45.187647+0000 mon.a (mon.0) 3350 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-140"}]: dispatch 2024-03-26T04:01:46.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:46 smithi175 bash[21527]: cluster 2024-03-26T04:01:45.053997+0000 mgr.y (mgr.24431) 665 : cluster [DBG] pgmap v1113: 268 pgs: 268 active+clean; 4.5 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 8.3 KiB/s wr, 2 op/s 2024-03-26T04:01:46.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:46 smithi175 bash[21527]: audit 2024-03-26T04:01:45.185207+0000 mon.c (mon.2) 686 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:46.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:46 smithi175 bash[21527]: audit 2024-03-26T04:01:45.185900+0000 mon.a (mon.0) 3349 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:46.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:46 smithi175 bash[21527]: audit 2024-03-26T04:01:45.187072+0000 mon.c (mon.2) 687 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-140"}]: dispatch 2024-03-26T04:01:46.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:46 smithi175 bash[21527]: audit 2024-03-26T04:01:45.187647+0000 mon.a (mon.0) 3350 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-140"}]: dispatch 2024-03-26T04:01:46.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:46 smithi175 bash[29284]: cluster 2024-03-26T04:01:45.053997+0000 mgr.y (mgr.24431) 665 : cluster [DBG] pgmap v1113: 268 pgs: 268 active+clean; 4.5 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 8.3 KiB/s wr, 2 op/s 2024-03-26T04:01:46.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:46 smithi175 bash[29284]: audit 2024-03-26T04:01:45.185207+0000 mon.c (mon.2) 686 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:46.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:46 smithi175 bash[29284]: audit 2024-03-26T04:01:45.185900+0000 mon.a (mon.0) 3349 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:46.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:46 smithi175 bash[29284]: audit 2024-03-26T04:01:45.187072+0000 mon.c (mon.2) 687 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-140"}]: dispatch 2024-03-26T04:01:46.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:46 smithi175 bash[29284]: audit 2024-03-26T04:01:45.187647+0000 mon.a (mon.0) 3350 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-140"}]: dispatch 2024-03-26T04:01:47.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:47 smithi190 bash[23392]: audit 2024-03-26T04:01:46.107289+0000 mon.a (mon.0) 3351 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-140"}]': finished 2024-03-26T04:01:47.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:47 smithi190 bash[23392]: cluster 2024-03-26T04:01:46.116993+0000 mon.a (mon.0) 3352 : cluster [DBG] osdmap e693: 8 total, 8 up, 8 in 2024-03-26T04:01:47.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:47 smithi175 bash[21527]: audit 2024-03-26T04:01:46.107289+0000 mon.a (mon.0) 3351 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-140"}]': finished 2024-03-26T04:01:47.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:47 smithi175 bash[21527]: cluster 2024-03-26T04:01:46.116993+0000 mon.a (mon.0) 3352 : cluster [DBG] osdmap e693: 8 total, 8 up, 8 in 2024-03-26T04:01:47.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:47 smithi175 bash[29284]: audit 2024-03-26T04:01:46.107289+0000 mon.a (mon.0) 3351 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-140"}]': finished 2024-03-26T04:01:47.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:47 smithi175 bash[29284]: cluster 2024-03-26T04:01:46.116993+0000 mon.a (mon.0) 3352 : cluster [DBG] osdmap e693: 8 total, 8 up, 8 in 2024-03-26T04:01:48.495 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:48 smithi190 bash[23392]: cluster 2024-03-26T04:01:47.054900+0000 mgr.y (mgr.24431) 666 : cluster [DBG] pgmap v1115: 268 pgs: 268 active+clean; 4.5 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-03-26T04:01:48.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:48 smithi190 bash[23392]: cluster 2024-03-26T04:01:47.147845+0000 mon.a (mon.0) 3353 : cluster [DBG] osdmap e694: 8 total, 8 up, 8 in 2024-03-26T04:01:48.496 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:48 smithi190 bash[23392]: cluster 2024-03-26T04:01:47.673172+0000 mon.a (mon.0) 3354 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:01:48.496 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:48 smithi175 bash[29284]: cluster 2024-03-26T04:01:47.054900+0000 mgr.y (mgr.24431) 666 : cluster [DBG] pgmap v1115: 268 pgs: 268 active+clean; 4.5 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-03-26T04:01:48.496 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:48 smithi175 bash[29284]: cluster 2024-03-26T04:01:47.147845+0000 mon.a (mon.0) 3353 : cluster [DBG] osdmap e694: 8 total, 8 up, 8 in 2024-03-26T04:01:48.496 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:48 smithi175 bash[29284]: cluster 2024-03-26T04:01:47.673172+0000 mon.a (mon.0) 3354 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:01:48.496 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:48 smithi175 bash[21527]: cluster 2024-03-26T04:01:47.054900+0000 mgr.y (mgr.24431) 666 : cluster [DBG] pgmap v1115: 268 pgs: 268 active+clean; 4.5 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-03-26T04:01:48.496 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:48 smithi175 bash[21527]: cluster 2024-03-26T04:01:47.147845+0000 mon.a (mon.0) 3353 : cluster [DBG] osdmap e694: 8 total, 8 up, 8 in 2024-03-26T04:01:48.496 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:48 smithi175 bash[21527]: cluster 2024-03-26T04:01:47.673172+0000 mon.a (mon.0) 3354 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:01:49.158 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:01:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:01:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:01:49.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:49 smithi175 bash[21527]: cluster 2024-03-26T04:01:48.156196+0000 mon.a (mon.0) 3355 : cluster [DBG] osdmap e695: 8 total, 8 up, 8 in 2024-03-26T04:01:49.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:49 smithi175 bash[21527]: audit 2024-03-26T04:01:48.162338+0000 mon.c (mon.2) 688 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:01:49.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:49 smithi175 bash[21527]: audit 2024-03-26T04:01:48.168874+0000 mon.a (mon.0) 3356 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:01:49.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:49 smithi175 bash[29284]: cluster 2024-03-26T04:01:48.156196+0000 mon.a (mon.0) 3355 : cluster [DBG] osdmap e695: 8 total, 8 up, 8 in 2024-03-26T04:01:49.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:49 smithi175 bash[29284]: audit 2024-03-26T04:01:48.162338+0000 mon.c (mon.2) 688 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:01:49.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:49 smithi175 bash[29284]: audit 2024-03-26T04:01:48.168874+0000 mon.a (mon.0) 3356 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:01:49.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:49 smithi190 bash[23392]: cluster 2024-03-26T04:01:48.156196+0000 mon.a (mon.0) 3355 : cluster [DBG] osdmap e695: 8 total, 8 up, 8 in 2024-03-26T04:01:49.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:49 smithi190 bash[23392]: audit 2024-03-26T04:01:48.162338+0000 mon.c (mon.2) 688 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:01:49.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:49 smithi190 bash[23392]: audit 2024-03-26T04:01:48.168874+0000 mon.a (mon.0) 3356 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:01:50.453 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:50 smithi190 bash[23392]: cluster 2024-03-26T04:01:49.055784+0000 mgr.y (mgr.24431) 667 : cluster [DBG] pgmap v1118: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:01:50.453 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:50 smithi190 bash[23392]: audit 2024-03-26T04:01:49.158827+0000 mon.a (mon.0) 3357 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:01:50.453 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:50 smithi190 bash[23392]: cluster 2024-03-26T04:01:49.168957+0000 mon.a (mon.0) 3358 : cluster [DBG] osdmap e696: 8 total, 8 up, 8 in 2024-03-26T04:01:50.453 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:50 smithi190 bash[23392]: audit 2024-03-26T04:01:49.275135+0000 mon.c (mon.2) 689 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:01:50.453 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:50 smithi190 bash[23392]: audit 2024-03-26T04:01:49.275828+0000 mon.a (mon.0) 3359 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:01:50.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:50 smithi175 bash[21527]: cluster 2024-03-26T04:01:49.055784+0000 mgr.y (mgr.24431) 667 : cluster [DBG] pgmap v1118: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:01:50.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:50 smithi175 bash[21527]: audit 2024-03-26T04:01:49.158827+0000 mon.a (mon.0) 3357 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:01:50.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:50 smithi175 bash[21527]: cluster 2024-03-26T04:01:49.168957+0000 mon.a (mon.0) 3358 : cluster [DBG] osdmap e696: 8 total, 8 up, 8 in 2024-03-26T04:01:50.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:50 smithi175 bash[21527]: audit 2024-03-26T04:01:49.275135+0000 mon.c (mon.2) 689 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:01:50.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:50 smithi175 bash[21527]: audit 2024-03-26T04:01:49.275828+0000 mon.a (mon.0) 3359 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:01:50.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:50 smithi175 bash[29284]: cluster 2024-03-26T04:01:49.055784+0000 mgr.y (mgr.24431) 667 : cluster [DBG] pgmap v1118: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:01:50.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:50 smithi175 bash[29284]: audit 2024-03-26T04:01:49.158827+0000 mon.a (mon.0) 3357 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:01:50.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:50 smithi175 bash[29284]: cluster 2024-03-26T04:01:49.168957+0000 mon.a (mon.0) 3358 : cluster [DBG] osdmap e696: 8 total, 8 up, 8 in 2024-03-26T04:01:50.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:50 smithi175 bash[29284]: audit 2024-03-26T04:01:49.275135+0000 mon.c (mon.2) 689 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:01:50.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:50 smithi175 bash[29284]: audit 2024-03-26T04:01:49.275828+0000 mon.a (mon.0) 3359 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:01:50.742 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:01:50 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:01:51.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:51 smithi190 bash[23392]: audit 2024-03-26T04:01:50.169493+0000 mon.a (mon.0) 3360 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-142", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:01:51.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:51 smithi190 bash[23392]: cluster 2024-03-26T04:01:50.176382+0000 mon.a (mon.0) 3361 : cluster [DBG] osdmap e697: 8 total, 8 up, 8 in 2024-03-26T04:01:51.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:51 smithi190 bash[23392]: audit 2024-03-26T04:01:50.179342+0000 mon.c (mon.2) 690 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-142"}]: dispatch 2024-03-26T04:01:51.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:51 smithi190 bash[23392]: audit 2024-03-26T04:01:50.182432+0000 mon.a (mon.0) 3362 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-142"}]: dispatch 2024-03-26T04:01:51.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:51 smithi190 bash[23392]: audit 2024-03-26T04:01:50.454272+0000 mgr.y (mgr.24431) 668 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:01:51.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:51 smithi190 bash[23392]: audit 2024-03-26T04:01:51.030639+0000 mon.c (mon.2) 691 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:01:51.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:51 smithi175 bash[29284]: audit 2024-03-26T04:01:50.169493+0000 mon.a (mon.0) 3360 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-142", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:01:51.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:51 smithi175 bash[29284]: cluster 2024-03-26T04:01:50.176382+0000 mon.a (mon.0) 3361 : cluster [DBG] osdmap e697: 8 total, 8 up, 8 in 2024-03-26T04:01:51.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:51 smithi175 bash[29284]: audit 2024-03-26T04:01:50.179342+0000 mon.c (mon.2) 690 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-142"}]: dispatch 2024-03-26T04:01:51.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:51 smithi175 bash[29284]: audit 2024-03-26T04:01:50.182432+0000 mon.a (mon.0) 3362 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-142"}]: dispatch 2024-03-26T04:01:51.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:51 smithi175 bash[29284]: audit 2024-03-26T04:01:50.454272+0000 mgr.y (mgr.24431) 668 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:01:51.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:51 smithi175 bash[29284]: audit 2024-03-26T04:01:51.030639+0000 mon.c (mon.2) 691 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:01:51.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:51 smithi175 bash[21527]: audit 2024-03-26T04:01:50.169493+0000 mon.a (mon.0) 3360 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-142", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:01:51.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:51 smithi175 bash[21527]: cluster 2024-03-26T04:01:50.176382+0000 mon.a (mon.0) 3361 : cluster [DBG] osdmap e697: 8 total, 8 up, 8 in 2024-03-26T04:01:51.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:51 smithi175 bash[21527]: audit 2024-03-26T04:01:50.179342+0000 mon.c (mon.2) 690 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-142"}]: dispatch 2024-03-26T04:01:51.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:51 smithi175 bash[21527]: audit 2024-03-26T04:01:50.182432+0000 mon.a (mon.0) 3362 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-142"}]: dispatch 2024-03-26T04:01:51.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:51 smithi175 bash[21527]: audit 2024-03-26T04:01:50.454272+0000 mgr.y (mgr.24431) 668 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:01:51.495 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:51 smithi175 bash[21527]: audit 2024-03-26T04:01:51.030639+0000 mon.c (mon.2) 691 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:01:52.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:52 smithi175 bash[21527]: cluster 2024-03-26T04:01:51.056748+0000 mgr.y (mgr.24431) 669 : cluster [DBG] pgmap v1121: 268 pgs: 268 active+clean; 4.4 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-26T04:01:52.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:52 smithi175 bash[21527]: audit 2024-03-26T04:01:51.172328+0000 mon.a (mon.0) 3363 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-142"}]': finished 2024-03-26T04:01:52.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:52 smithi175 bash[21527]: cluster 2024-03-26T04:01:51.179720+0000 mon.a (mon.0) 3364 : cluster [DBG] osdmap e698: 8 total, 8 up, 8 in 2024-03-26T04:01:52.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:52 smithi175 bash[21527]: audit 2024-03-26T04:01:51.182237+0000 mon.c (mon.2) 692 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-142", "mode": "writeback"}]: dispatch 2024-03-26T04:01:52.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:52 smithi175 bash[21527]: audit 2024-03-26T04:01:51.183235+0000 mon.a (mon.0) 3365 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-142", "mode": "writeback"}]: dispatch 2024-03-26T04:01:52.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:52 smithi175 bash[21527]: cluster 2024-03-26T04:01:52.172812+0000 mon.a (mon.0) 3366 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:01:52.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:52 smithi175 bash[21527]: audit 2024-03-26T04:01:52.175267+0000 mon.a (mon.0) 3367 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-142", "mode": "writeback"}]': finished 2024-03-26T04:01:52.494 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:52 smithi175 bash[21527]: cluster 2024-03-26T04:01:52.182249+0000 mon.a (mon.0) 3368 : cluster [DBG] osdmap e699: 8 total, 8 up, 8 in 2024-03-26T04:01:52.495 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:52 smithi175 bash[29284]: cluster 2024-03-26T04:01:51.056748+0000 mgr.y (mgr.24431) 669 : cluster [DBG] pgmap v1121: 268 pgs: 268 active+clean; 4.4 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-26T04:01:52.495 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:52 smithi175 bash[29284]: audit 2024-03-26T04:01:51.172328+0000 mon.a (mon.0) 3363 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-142"}]': finished 2024-03-26T04:01:52.495 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:52 smithi175 bash[29284]: cluster 2024-03-26T04:01:51.179720+0000 mon.a (mon.0) 3364 : cluster [DBG] osdmap e698: 8 total, 8 up, 8 in 2024-03-26T04:01:52.495 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:52 smithi175 bash[29284]: audit 2024-03-26T04:01:51.182237+0000 mon.c (mon.2) 692 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-142", "mode": "writeback"}]: dispatch 2024-03-26T04:01:52.495 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:52 smithi175 bash[29284]: audit 2024-03-26T04:01:51.183235+0000 mon.a (mon.0) 3365 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-142", "mode": "writeback"}]: dispatch 2024-03-26T04:01:52.496 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:52 smithi175 bash[29284]: cluster 2024-03-26T04:01:52.172812+0000 mon.a (mon.0) 3366 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:01:52.496 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:52 smithi175 bash[29284]: audit 2024-03-26T04:01:52.175267+0000 mon.a (mon.0) 3367 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-142", "mode": "writeback"}]': finished 2024-03-26T04:01:52.496 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:52 smithi175 bash[29284]: cluster 2024-03-26T04:01:52.182249+0000 mon.a (mon.0) 3368 : cluster [DBG] osdmap e699: 8 total, 8 up, 8 in 2024-03-26T04:01:52.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:52 smithi190 bash[23392]: cluster 2024-03-26T04:01:51.056748+0000 mgr.y (mgr.24431) 669 : cluster [DBG] pgmap v1121: 268 pgs: 268 active+clean; 4.4 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-26T04:01:52.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:52 smithi190 bash[23392]: audit 2024-03-26T04:01:51.172328+0000 mon.a (mon.0) 3363 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-142"}]': finished 2024-03-26T04:01:52.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:52 smithi190 bash[23392]: cluster 2024-03-26T04:01:51.179720+0000 mon.a (mon.0) 3364 : cluster [DBG] osdmap e698: 8 total, 8 up, 8 in 2024-03-26T04:01:52.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:52 smithi190 bash[23392]: audit 2024-03-26T04:01:51.182237+0000 mon.c (mon.2) 692 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-142", "mode": "writeback"}]: dispatch 2024-03-26T04:01:52.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:52 smithi190 bash[23392]: audit 2024-03-26T04:01:51.183235+0000 mon.a (mon.0) 3365 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-142", "mode": "writeback"}]: dispatch 2024-03-26T04:01:52.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:52 smithi190 bash[23392]: cluster 2024-03-26T04:01:52.172812+0000 mon.a (mon.0) 3366 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:01:52.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:52 smithi190 bash[23392]: audit 2024-03-26T04:01:52.175267+0000 mon.a (mon.0) 3367 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-142", "mode": "writeback"}]': finished 2024-03-26T04:01:52.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:52 smithi190 bash[23392]: cluster 2024-03-26T04:01:52.182249+0000 mon.a (mon.0) 3368 : cluster [DBG] osdmap e699: 8 total, 8 up, 8 in 2024-03-26T04:01:53.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:53 smithi175 bash[21527]: audit 2024-03-26T04:01:52.186178+0000 mon.c (mon.2) 693 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T04:01:53.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:53 smithi175 bash[21527]: audit 2024-03-26T04:01:52.188388+0000 mon.a (mon.0) 3369 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T04:01:53.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:53 smithi175 bash[21527]: audit 2024-03-26T04:01:53.178147+0000 mon.a (mon.0) 3370 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_count","val": "2"}]': finished 2024-03-26T04:01:53.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:53 smithi175 bash[21527]: cluster 2024-03-26T04:01:53.185429+0000 mon.a (mon.0) 3371 : cluster [DBG] osdmap e700: 8 total, 8 up, 8 in 2024-03-26T04:01:53.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:53 smithi175 bash[29284]: audit 2024-03-26T04:01:52.186178+0000 mon.c (mon.2) 693 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T04:01:53.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:53 smithi175 bash[29284]: audit 2024-03-26T04:01:52.188388+0000 mon.a (mon.0) 3369 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T04:01:53.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:53 smithi175 bash[29284]: audit 2024-03-26T04:01:53.178147+0000 mon.a (mon.0) 3370 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_count","val": "2"}]': finished 2024-03-26T04:01:53.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:53 smithi175 bash[29284]: cluster 2024-03-26T04:01:53.185429+0000 mon.a (mon.0) 3371 : cluster [DBG] osdmap e700: 8 total, 8 up, 8 in 2024-03-26T04:01:53.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:53 smithi190 bash[23392]: audit 2024-03-26T04:01:52.186178+0000 mon.c (mon.2) 693 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T04:01:53.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:53 smithi190 bash[23392]: audit 2024-03-26T04:01:52.188388+0000 mon.a (mon.0) 3369 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T04:01:53.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:53 smithi190 bash[23392]: audit 2024-03-26T04:01:53.178147+0000 mon.a (mon.0) 3370 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_count","val": "2"}]': finished 2024-03-26T04:01:53.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:53 smithi190 bash[23392]: cluster 2024-03-26T04:01:53.185429+0000 mon.a (mon.0) 3371 : cluster [DBG] osdmap e700: 8 total, 8 up, 8 in 2024-03-26T04:01:54.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:54 smithi175 bash[21527]: cluster 2024-03-26T04:01:53.057643+0000 mgr.y (mgr.24431) 670 : cluster [DBG] pgmap v1124: 268 pgs: 268 active+clean; 4.4 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-26T04:01:54.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:54 smithi175 bash[21527]: audit 2024-03-26T04:01:53.191185+0000 mon.c (mon.2) 694 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T04:01:54.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:54 smithi175 bash[21527]: audit 2024-03-26T04:01:53.192821+0000 mon.a (mon.0) 3372 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T04:01:54.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:54 smithi175 bash[21527]: audit 2024-03-26T04:01:54.180993+0000 mon.a (mon.0) 3373 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_period","val": "600"}]': finished 2024-03-26T04:01:54.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:54 smithi175 bash[21527]: cluster 2024-03-26T04:01:54.189276+0000 mon.a (mon.0) 3374 : cluster [DBG] osdmap e701: 8 total, 8 up, 8 in 2024-03-26T04:01:54.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:54 smithi175 bash[21527]: audit 2024-03-26T04:01:54.195932+0000 mon.c (mon.2) 695 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-26T04:01:54.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:54 smithi175 bash[21527]: audit 2024-03-26T04:01:54.196900+0000 mon.a (mon.0) 3375 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-26T04:01:54.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:54 smithi175 bash[29284]: cluster 2024-03-26T04:01:53.057643+0000 mgr.y (mgr.24431) 670 : cluster [DBG] pgmap v1124: 268 pgs: 268 active+clean; 4.4 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-26T04:01:54.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:54 smithi175 bash[29284]: audit 2024-03-26T04:01:53.191185+0000 mon.c (mon.2) 694 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T04:01:54.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:54 smithi175 bash[29284]: audit 2024-03-26T04:01:53.192821+0000 mon.a (mon.0) 3372 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T04:01:54.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:54 smithi175 bash[29284]: audit 2024-03-26T04:01:54.180993+0000 mon.a (mon.0) 3373 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_period","val": "600"}]': finished 2024-03-26T04:01:54.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:54 smithi175 bash[29284]: cluster 2024-03-26T04:01:54.189276+0000 mon.a (mon.0) 3374 : cluster [DBG] osdmap e701: 8 total, 8 up, 8 in 2024-03-26T04:01:54.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:54 smithi175 bash[29284]: audit 2024-03-26T04:01:54.195932+0000 mon.c (mon.2) 695 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-26T04:01:54.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:54 smithi175 bash[29284]: audit 2024-03-26T04:01:54.196900+0000 mon.a (mon.0) 3375 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-26T04:01:54.495 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:54 smithi190 bash[23392]: cluster 2024-03-26T04:01:53.057643+0000 mgr.y (mgr.24431) 670 : cluster [DBG] pgmap v1124: 268 pgs: 268 active+clean; 4.4 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-26T04:01:54.495 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:54 smithi190 bash[23392]: audit 2024-03-26T04:01:53.191185+0000 mon.c (mon.2) 694 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T04:01:54.495 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:54 smithi190 bash[23392]: audit 2024-03-26T04:01:53.192821+0000 mon.a (mon.0) 3372 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T04:01:54.495 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:54 smithi190 bash[23392]: audit 2024-03-26T04:01:54.180993+0000 mon.a (mon.0) 3373 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_period","val": "600"}]': finished 2024-03-26T04:01:54.495 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:54 smithi190 bash[23392]: cluster 2024-03-26T04:01:54.189276+0000 mon.a (mon.0) 3374 : cluster [DBG] osdmap e701: 8 total, 8 up, 8 in 2024-03-26T04:01:54.495 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:54 smithi190 bash[23392]: audit 2024-03-26T04:01:54.195932+0000 mon.c (mon.2) 695 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-26T04:01:54.495 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:54 smithi190 bash[23392]: audit 2024-03-26T04:01:54.196900+0000 mon.a (mon.0) 3375 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-26T04:01:55.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:55 smithi175 bash[21527]: cluster 2024-03-26T04:01:55.181463+0000 mon.a (mon.0) 3376 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:01:55.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:55 smithi175 bash[21527]: audit 2024-03-26T04:01:55.184085+0000 mon.a (mon.0) 3377 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_type","val": "bloom"}]': finished 2024-03-26T04:01:55.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:55 smithi175 bash[21527]: cluster 2024-03-26T04:01:55.191671+0000 mon.a (mon.0) 3378 : cluster [DBG] osdmap e702: 8 total, 8 up, 8 in 2024-03-26T04:01:55.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:55 smithi175 bash[21527]: audit 2024-03-26T04:01:55.196578+0000 mon.c (mon.2) 696 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-26T04:01:55.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:55 smithi175 bash[21527]: audit 2024-03-26T04:01:55.197902+0000 mon.a (mon.0) 3379 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-26T04:01:55.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:55 smithi175 bash[29284]: cluster 2024-03-26T04:01:55.181463+0000 mon.a (mon.0) 3376 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:01:55.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:55 smithi175 bash[29284]: audit 2024-03-26T04:01:55.184085+0000 mon.a (mon.0) 3377 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_type","val": "bloom"}]': finished 2024-03-26T04:01:55.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:55 smithi175 bash[29284]: cluster 2024-03-26T04:01:55.191671+0000 mon.a (mon.0) 3378 : cluster [DBG] osdmap e702: 8 total, 8 up, 8 in 2024-03-26T04:01:55.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:55 smithi175 bash[29284]: audit 2024-03-26T04:01:55.196578+0000 mon.c (mon.2) 696 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-26T04:01:55.494 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:55 smithi175 bash[29284]: audit 2024-03-26T04:01:55.197902+0000 mon.a (mon.0) 3379 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-26T04:01:55.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:55 smithi190 bash[23392]: cluster 2024-03-26T04:01:55.181463+0000 mon.a (mon.0) 3376 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:01:55.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:55 smithi190 bash[23392]: audit 2024-03-26T04:01:55.184085+0000 mon.a (mon.0) 3377 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_type","val": "bloom"}]': finished 2024-03-26T04:01:55.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:55 smithi190 bash[23392]: cluster 2024-03-26T04:01:55.191671+0000 mon.a (mon.0) 3378 : cluster [DBG] osdmap e702: 8 total, 8 up, 8 in 2024-03-26T04:01:55.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:55 smithi190 bash[23392]: audit 2024-03-26T04:01:55.196578+0000 mon.c (mon.2) 696 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-26T04:01:55.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:55 smithi190 bash[23392]: audit 2024-03-26T04:01:55.197902+0000 mon.a (mon.0) 3379 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-26T04:01:56.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:56 smithi175 bash[21527]: cluster 2024-03-26T04:01:55.060159+0000 mgr.y (mgr.24431) 671 : cluster [DBG] pgmap v1127: 268 pgs: 268 active+clean; 4.4 MiB data, 747 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:01:56.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:56 smithi175 bash[21527]: audit 2024-03-26T04:01:56.186855+0000 mon.a (mon.0) 3380 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-26T04:01:56.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:56 smithi175 bash[21527]: cluster 2024-03-26T04:01:56.194164+0000 mon.a (mon.0) 3381 : cluster [DBG] osdmap e703: 8 total, 8 up, 8 in 2024-03-26T04:01:56.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:56 smithi175 bash[21527]: audit 2024-03-26T04:01:56.196645+0000 mon.c (mon.2) 697 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-26T04:01:56.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:56 smithi175 bash[21527]: audit 2024-03-26T04:01:56.199684+0000 mon.a (mon.0) 3382 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-26T04:01:56.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:56 smithi175 bash[29284]: cluster 2024-03-26T04:01:55.060159+0000 mgr.y (mgr.24431) 671 : cluster [DBG] pgmap v1127: 268 pgs: 268 active+clean; 4.4 MiB data, 747 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:01:56.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:56 smithi175 bash[29284]: audit 2024-03-26T04:01:56.186855+0000 mon.a (mon.0) 3380 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-26T04:01:56.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:56 smithi175 bash[29284]: cluster 2024-03-26T04:01:56.194164+0000 mon.a (mon.0) 3381 : cluster [DBG] osdmap e703: 8 total, 8 up, 8 in 2024-03-26T04:01:56.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:56 smithi175 bash[29284]: audit 2024-03-26T04:01:56.196645+0000 mon.c (mon.2) 697 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-26T04:01:56.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:56 smithi175 bash[29284]: audit 2024-03-26T04:01:56.199684+0000 mon.a (mon.0) 3382 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-26T04:01:56.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:56 smithi190 bash[23392]: cluster 2024-03-26T04:01:55.060159+0000 mgr.y (mgr.24431) 671 : cluster [DBG] pgmap v1127: 268 pgs: 268 active+clean; 4.4 MiB data, 747 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:01:56.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:56 smithi190 bash[23392]: audit 2024-03-26T04:01:56.186855+0000 mon.a (mon.0) 3380 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-26T04:01:56.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:56 smithi190 bash[23392]: cluster 2024-03-26T04:01:56.194164+0000 mon.a (mon.0) 3381 : cluster [DBG] osdmap e703: 8 total, 8 up, 8 in 2024-03-26T04:01:56.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:56 smithi190 bash[23392]: audit 2024-03-26T04:01:56.196645+0000 mon.c (mon.2) 697 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-26T04:01:56.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:56 smithi190 bash[23392]: audit 2024-03-26T04:01:56.199684+0000 mon.a (mon.0) 3382 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-26T04:01:57.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:57 smithi175 bash[29284]: cluster 2024-03-26T04:01:56.673697+0000 mon.a (mon.0) 3383 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:01:57.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:57 smithi175 bash[29284]: audit 2024-03-26T04:01:57.189562+0000 mon.a (mon.0) 3384 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-03-26T04:01:57.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:57 smithi175 bash[29284]: cluster 2024-03-26T04:01:57.197426+0000 mon.a (mon.0) 3385 : cluster [DBG] osdmap e704: 8 total, 8 up, 8 in 2024-03-26T04:01:57.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:57 smithi175 bash[29284]: audit 2024-03-26T04:01:57.208117+0000 mon.c (mon.2) 698 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-26T04:01:57.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:57 smithi175 bash[29284]: audit 2024-03-26T04:01:57.209825+0000 mon.a (mon.0) 3386 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-26T04:01:57.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:57 smithi175 bash[21527]: cluster 2024-03-26T04:01:56.673697+0000 mon.a (mon.0) 3383 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:01:57.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:57 smithi175 bash[21527]: audit 2024-03-26T04:01:57.189562+0000 mon.a (mon.0) 3384 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-03-26T04:01:57.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:57 smithi175 bash[21527]: cluster 2024-03-26T04:01:57.197426+0000 mon.a (mon.0) 3385 : cluster [DBG] osdmap e704: 8 total, 8 up, 8 in 2024-03-26T04:01:57.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:57 smithi175 bash[21527]: audit 2024-03-26T04:01:57.208117+0000 mon.c (mon.2) 698 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-26T04:01:57.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:57 smithi175 bash[21527]: audit 2024-03-26T04:01:57.209825+0000 mon.a (mon.0) 3386 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-26T04:01:57.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:57 smithi190 bash[23392]: cluster 2024-03-26T04:01:56.673697+0000 mon.a (mon.0) 3383 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:01:57.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:57 smithi190 bash[23392]: audit 2024-03-26T04:01:57.189562+0000 mon.a (mon.0) 3384 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-03-26T04:01:57.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:57 smithi190 bash[23392]: cluster 2024-03-26T04:01:57.197426+0000 mon.a (mon.0) 3385 : cluster [DBG] osdmap e704: 8 total, 8 up, 8 in 2024-03-26T04:01:57.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:57 smithi190 bash[23392]: audit 2024-03-26T04:01:57.208117+0000 mon.c (mon.2) 698 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-26T04:01:57.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:57 smithi190 bash[23392]: audit 2024-03-26T04:01:57.209825+0000 mon.a (mon.0) 3386 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-26T04:01:58.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:58 smithi175 bash[29284]: cluster 2024-03-26T04:01:57.061150+0000 mgr.y (mgr.24431) 672 : cluster [DBG] pgmap v1130: 268 pgs: 268 active+clean; 4.4 MiB data, 747 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:01:58.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:58 smithi175 bash[29284]: audit 2024-03-26T04:01:58.192413+0000 mon.a (mon.0) 3387 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-03-26T04:01:58.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:58 smithi175 bash[29284]: cluster 2024-03-26T04:01:58.200161+0000 mon.a (mon.0) 3388 : cluster [DBG] osdmap e705: 8 total, 8 up, 8 in 2024-03-26T04:01:58.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:58 smithi175 bash[21527]: cluster 2024-03-26T04:01:57.061150+0000 mgr.y (mgr.24431) 672 : cluster [DBG] pgmap v1130: 268 pgs: 268 active+clean; 4.4 MiB data, 747 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:01:58.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:58 smithi175 bash[21527]: audit 2024-03-26T04:01:58.192413+0000 mon.a (mon.0) 3387 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-03-26T04:01:58.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:58 smithi175 bash[21527]: cluster 2024-03-26T04:01:58.200161+0000 mon.a (mon.0) 3388 : cluster [DBG] osdmap e705: 8 total, 8 up, 8 in 2024-03-26T04:01:58.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:58 smithi190 bash[23392]: cluster 2024-03-26T04:01:57.061150+0000 mgr.y (mgr.24431) 672 : cluster [DBG] pgmap v1130: 268 pgs: 268 active+clean; 4.4 MiB data, 747 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:01:58.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:58 smithi190 bash[23392]: audit 2024-03-26T04:01:58.192413+0000 mon.a (mon.0) 3387 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-03-26T04:01:58.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:58 smithi190 bash[23392]: cluster 2024-03-26T04:01:58.200161+0000 mon.a (mon.0) 3388 : cluster [DBG] osdmap e705: 8 total, 8 up, 8 in 2024-03-26T04:01:59.234 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:01:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:01:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:01:59.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:59 smithi175 bash[21527]: audit 2024-03-26T04:01:58.305704+0000 mon.c (mon.2) 699 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:59.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:01:59 smithi175 bash[21527]: audit 2024-03-26T04:01:58.306444+0000 mon.a (mon.0) 3389 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:59.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:59 smithi175 bash[29284]: audit 2024-03-26T04:01:58.305704+0000 mon.c (mon.2) 699 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:59.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:01:59 smithi175 bash[29284]: audit 2024-03-26T04:01:58.306444+0000 mon.a (mon.0) 3389 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:59.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:59 smithi190 bash[23392]: audit 2024-03-26T04:01:58.305704+0000 mon.c (mon.2) 699 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:01:59.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:01:59 smithi190 bash[23392]: audit 2024-03-26T04:01:58.306444+0000 mon.a (mon.0) 3389 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:00.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:00 smithi175 bash[21527]: cluster 2024-03-26T04:01:59.062126+0000 mgr.y (mgr.24431) 673 : cluster [DBG] pgmap v1133: 268 pgs: 268 active+clean; 4.4 MiB data, 747 MiB used, 714 GiB / 715 GiB avail 2024-03-26T04:02:00.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:00 smithi175 bash[21527]: audit 2024-03-26T04:01:59.247922+0000 mon.a (mon.0) 3390 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:02:00.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:00 smithi175 bash[21527]: cluster 2024-03-26T04:01:59.257530+0000 mon.a (mon.0) 3391 : cluster [DBG] osdmap e706: 8 total, 8 up, 8 in 2024-03-26T04:02:00.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:00 smithi175 bash[21527]: audit 2024-03-26T04:01:59.261818+0000 mon.c (mon.2) 700 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-142"}]: dispatch 2024-03-26T04:02:00.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:00 smithi175 bash[21527]: audit 2024-03-26T04:01:59.263628+0000 mon.a (mon.0) 3392 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-142"}]: dispatch 2024-03-26T04:02:00.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:00 smithi175 bash[29284]: cluster 2024-03-26T04:01:59.062126+0000 mgr.y (mgr.24431) 673 : cluster [DBG] pgmap v1133: 268 pgs: 268 active+clean; 4.4 MiB data, 747 MiB used, 714 GiB / 715 GiB avail 2024-03-26T04:02:00.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:00 smithi175 bash[29284]: audit 2024-03-26T04:01:59.247922+0000 mon.a (mon.0) 3390 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:02:00.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:00 smithi175 bash[29284]: cluster 2024-03-26T04:01:59.257530+0000 mon.a (mon.0) 3391 : cluster [DBG] osdmap e706: 8 total, 8 up, 8 in 2024-03-26T04:02:00.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:00 smithi175 bash[29284]: audit 2024-03-26T04:01:59.261818+0000 mon.c (mon.2) 700 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-142"}]: dispatch 2024-03-26T04:02:00.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:00 smithi175 bash[29284]: audit 2024-03-26T04:01:59.263628+0000 mon.a (mon.0) 3392 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-142"}]: dispatch 2024-03-26T04:02:00.744 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:02:00 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:02:00.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:00 smithi190 bash[23392]: cluster 2024-03-26T04:01:59.062126+0000 mgr.y (mgr.24431) 673 : cluster [DBG] pgmap v1133: 268 pgs: 268 active+clean; 4.4 MiB data, 747 MiB used, 714 GiB / 715 GiB avail 2024-03-26T04:02:00.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:00 smithi190 bash[23392]: audit 2024-03-26T04:01:59.247922+0000 mon.a (mon.0) 3390 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:02:00.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:00 smithi190 bash[23392]: cluster 2024-03-26T04:01:59.257530+0000 mon.a (mon.0) 3391 : cluster [DBG] osdmap e706: 8 total, 8 up, 8 in 2024-03-26T04:02:00.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:00 smithi190 bash[23392]: audit 2024-03-26T04:01:59.261818+0000 mon.c (mon.2) 700 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-142"}]: dispatch 2024-03-26T04:02:00.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:00 smithi190 bash[23392]: audit 2024-03-26T04:01:59.263628+0000 mon.a (mon.0) 3392 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-142"}]: dispatch 2024-03-26T04:02:01.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:01 smithi175 bash[21527]: audit 2024-03-26T04:02:00.250909+0000 mon.a (mon.0) 3393 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-142"}]': finished 2024-03-26T04:02:01.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:01 smithi175 bash[21527]: cluster 2024-03-26T04:02:00.258776+0000 mon.a (mon.0) 3394 : cluster [DBG] osdmap e707: 8 total, 8 up, 8 in 2024-03-26T04:02:01.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:01 smithi175 bash[21527]: audit 2024-03-26T04:02:00.330207+0000 mon.c (mon.2) 701 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:01.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:01 smithi175 bash[21527]: audit 2024-03-26T04:02:00.331063+0000 mon.a (mon.0) 3395 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:01.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:01 smithi175 bash[21527]: audit 2024-03-26T04:02:00.332878+0000 mon.c (mon.2) 702 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-142"}]: dispatch 2024-03-26T04:02:01.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:01 smithi175 bash[21527]: audit 2024-03-26T04:02:00.333801+0000 mon.a (mon.0) 3396 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-142"}]: dispatch 2024-03-26T04:02:01.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:01 smithi175 bash[21527]: audit 2024-03-26T04:02:00.465969+0000 mgr.y (mgr.24431) 674 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:02:01.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:01 smithi175 bash[29284]: audit 2024-03-26T04:02:00.250909+0000 mon.a (mon.0) 3393 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-142"}]': finished 2024-03-26T04:02:01.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:01 smithi175 bash[29284]: cluster 2024-03-26T04:02:00.258776+0000 mon.a (mon.0) 3394 : cluster [DBG] osdmap e707: 8 total, 8 up, 8 in 2024-03-26T04:02:01.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:01 smithi175 bash[29284]: audit 2024-03-26T04:02:00.330207+0000 mon.c (mon.2) 701 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:01.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:01 smithi175 bash[29284]: audit 2024-03-26T04:02:00.331063+0000 mon.a (mon.0) 3395 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:01.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:01 smithi175 bash[29284]: audit 2024-03-26T04:02:00.332878+0000 mon.c (mon.2) 702 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-142"}]: dispatch 2024-03-26T04:02:01.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:01 smithi175 bash[29284]: audit 2024-03-26T04:02:00.333801+0000 mon.a (mon.0) 3396 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-142"}]: dispatch 2024-03-26T04:02:01.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:01 smithi175 bash[29284]: audit 2024-03-26T04:02:00.465969+0000 mgr.y (mgr.24431) 674 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:02:01.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:01 smithi190 bash[23392]: audit 2024-03-26T04:02:00.250909+0000 mon.a (mon.0) 3393 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-142"}]': finished 2024-03-26T04:02:01.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:01 smithi190 bash[23392]: cluster 2024-03-26T04:02:00.258776+0000 mon.a (mon.0) 3394 : cluster [DBG] osdmap e707: 8 total, 8 up, 8 in 2024-03-26T04:02:01.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:01 smithi190 bash[23392]: audit 2024-03-26T04:02:00.330207+0000 mon.c (mon.2) 701 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:01.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:01 smithi190 bash[23392]: audit 2024-03-26T04:02:00.331063+0000 mon.a (mon.0) 3395 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:01.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:01 smithi190 bash[23392]: audit 2024-03-26T04:02:00.332878+0000 mon.c (mon.2) 702 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-142"}]: dispatch 2024-03-26T04:02:01.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:01 smithi190 bash[23392]: audit 2024-03-26T04:02:00.333801+0000 mon.a (mon.0) 3396 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-142"}]: dispatch 2024-03-26T04:02:01.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:01 smithi190 bash[23392]: audit 2024-03-26T04:02:00.465969+0000 mgr.y (mgr.24431) 674 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:02:02.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:02 smithi175 bash[21527]: cluster 2024-03-26T04:02:01.063203+0000 mgr.y (mgr.24431) 675 : cluster [DBG] pgmap v1136: 268 pgs: 268 active+clean; 4.4 MiB data, 747 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-26T04:02:02.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:02 smithi175 bash[21527]: cluster 2024-03-26T04:02:01.281846+0000 mon.a (mon.0) 3397 : cluster [DBG] osdmap e708: 8 total, 8 up, 8 in 2024-03-26T04:02:02.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:02 smithi175 bash[21527]: cluster 2024-03-26T04:02:01.674937+0000 mon.a (mon.0) 3398 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:02:02.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:02 smithi175 bash[29284]: cluster 2024-03-26T04:02:01.063203+0000 mgr.y (mgr.24431) 675 : cluster [DBG] pgmap v1136: 268 pgs: 268 active+clean; 4.4 MiB data, 747 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-26T04:02:02.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:02 smithi175 bash[29284]: cluster 2024-03-26T04:02:01.281846+0000 mon.a (mon.0) 3397 : cluster [DBG] osdmap e708: 8 total, 8 up, 8 in 2024-03-26T04:02:02.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:02 smithi175 bash[29284]: cluster 2024-03-26T04:02:01.674937+0000 mon.a (mon.0) 3398 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:02:02.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:02 smithi190 bash[23392]: cluster 2024-03-26T04:02:01.063203+0000 mgr.y (mgr.24431) 675 : cluster [DBG] pgmap v1136: 268 pgs: 268 active+clean; 4.4 MiB data, 747 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-26T04:02:02.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:02 smithi190 bash[23392]: cluster 2024-03-26T04:02:01.281846+0000 mon.a (mon.0) 3397 : cluster [DBG] osdmap e708: 8 total, 8 up, 8 in 2024-03-26T04:02:02.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:02 smithi190 bash[23392]: cluster 2024-03-26T04:02:01.674937+0000 mon.a (mon.0) 3398 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:02:03.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:03 smithi175 bash[21527]: cluster 2024-03-26T04:02:02.286618+0000 mon.a (mon.0) 3399 : cluster [DBG] osdmap e709: 8 total, 8 up, 8 in 2024-03-26T04:02:03.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:03 smithi175 bash[21527]: audit 2024-03-26T04:02:02.292297+0000 mon.c (mon.2) 703 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:03.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:03 smithi175 bash[21527]: audit 2024-03-26T04:02:02.310330+0000 mon.a (mon.0) 3400 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:03.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:03 smithi175 bash[21527]: audit 2024-03-26T04:02:03.279835+0000 mon.a (mon.0) 3401 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:02:03.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:03 smithi175 bash[21527]: cluster 2024-03-26T04:02:03.290755+0000 mon.a (mon.0) 3402 : cluster [DBG] osdmap e710: 8 total, 8 up, 8 in 2024-03-26T04:02:03.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:03 smithi175 bash[29284]: cluster 2024-03-26T04:02:02.286618+0000 mon.a (mon.0) 3399 : cluster [DBG] osdmap e709: 8 total, 8 up, 8 in 2024-03-26T04:02:03.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:03 smithi175 bash[29284]: audit 2024-03-26T04:02:02.292297+0000 mon.c (mon.2) 703 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:03.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:03 smithi175 bash[29284]: audit 2024-03-26T04:02:02.310330+0000 mon.a (mon.0) 3400 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:03.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:03 smithi175 bash[29284]: audit 2024-03-26T04:02:03.279835+0000 mon.a (mon.0) 3401 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:02:03.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:03 smithi175 bash[29284]: cluster 2024-03-26T04:02:03.290755+0000 mon.a (mon.0) 3402 : cluster [DBG] osdmap e710: 8 total, 8 up, 8 in 2024-03-26T04:02:03.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:03 smithi190 bash[23392]: cluster 2024-03-26T04:02:02.286618+0000 mon.a (mon.0) 3399 : cluster [DBG] osdmap e709: 8 total, 8 up, 8 in 2024-03-26T04:02:03.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:03 smithi190 bash[23392]: audit 2024-03-26T04:02:02.292297+0000 mon.c (mon.2) 703 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:03.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:03 smithi190 bash[23392]: audit 2024-03-26T04:02:02.310330+0000 mon.a (mon.0) 3400 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:03.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:03 smithi190 bash[23392]: audit 2024-03-26T04:02:03.279835+0000 mon.a (mon.0) 3401 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:02:03.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:03 smithi190 bash[23392]: cluster 2024-03-26T04:02:03.290755+0000 mon.a (mon.0) 3402 : cluster [DBG] osdmap e710: 8 total, 8 up, 8 in 2024-03-26T04:02:04.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:04 smithi175 bash[29284]: cluster 2024-03-26T04:02:03.064107+0000 mgr.y (mgr.24431) 676 : cluster [DBG] pgmap v1139: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 747 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-26T04:02:04.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:04 smithi175 bash[29284]: audit 2024-03-26T04:02:03.347121+0000 mon.c (mon.2) 704 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:02:04.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:04 smithi175 bash[29284]: audit 2024-03-26T04:02:03.347995+0000 mon.a (mon.0) 3403 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:02:04.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:04 smithi175 bash[29284]: audit 2024-03-26T04:02:04.304906+0000 mon.a (mon.0) 3404 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-144", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:02:04.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:04 smithi175 bash[29284]: cluster 2024-03-26T04:02:04.312977+0000 mon.a (mon.0) 3405 : cluster [DBG] osdmap e711: 8 total, 8 up, 8 in 2024-03-26T04:02:04.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:04 smithi175 bash[29284]: audit 2024-03-26T04:02:04.314859+0000 mon.c (mon.2) 705 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-144"}]: dispatch 2024-03-26T04:02:04.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:04 smithi175 bash[29284]: audit 2024-03-26T04:02:04.316338+0000 mon.a (mon.0) 3406 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-144"}]: dispatch 2024-03-26T04:02:04.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:04 smithi175 bash[21527]: cluster 2024-03-26T04:02:03.064107+0000 mgr.y (mgr.24431) 676 : cluster [DBG] pgmap v1139: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 747 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-26T04:02:04.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:04 smithi175 bash[21527]: audit 2024-03-26T04:02:03.347121+0000 mon.c (mon.2) 704 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:02:04.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:04 smithi175 bash[21527]: audit 2024-03-26T04:02:03.347995+0000 mon.a (mon.0) 3403 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:02:04.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:04 smithi175 bash[21527]: audit 2024-03-26T04:02:04.304906+0000 mon.a (mon.0) 3404 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-144", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:02:04.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:04 smithi175 bash[21527]: cluster 2024-03-26T04:02:04.312977+0000 mon.a (mon.0) 3405 : cluster [DBG] osdmap e711: 8 total, 8 up, 8 in 2024-03-26T04:02:04.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:04 smithi175 bash[21527]: audit 2024-03-26T04:02:04.314859+0000 mon.c (mon.2) 705 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-144"}]: dispatch 2024-03-26T04:02:04.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:04 smithi175 bash[21527]: audit 2024-03-26T04:02:04.316338+0000 mon.a (mon.0) 3406 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-144"}]: dispatch 2024-03-26T04:02:04.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:04 smithi190 bash[23392]: cluster 2024-03-26T04:02:03.064107+0000 mgr.y (mgr.24431) 676 : cluster [DBG] pgmap v1139: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 747 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-26T04:02:04.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:04 smithi190 bash[23392]: audit 2024-03-26T04:02:03.347121+0000 mon.c (mon.2) 704 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:02:04.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:04 smithi190 bash[23392]: audit 2024-03-26T04:02:03.347995+0000 mon.a (mon.0) 3403 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:02:04.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:04 smithi190 bash[23392]: audit 2024-03-26T04:02:04.304906+0000 mon.a (mon.0) 3404 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-144", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:02:04.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:04 smithi190 bash[23392]: cluster 2024-03-26T04:02:04.312977+0000 mon.a (mon.0) 3405 : cluster [DBG] osdmap e711: 8 total, 8 up, 8 in 2024-03-26T04:02:04.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:04 smithi190 bash[23392]: audit 2024-03-26T04:02:04.314859+0000 mon.c (mon.2) 705 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-144"}]: dispatch 2024-03-26T04:02:04.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:04 smithi190 bash[23392]: audit 2024-03-26T04:02:04.316338+0000 mon.a (mon.0) 3406 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-144"}]: dispatch 2024-03-26T04:02:06.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:06 smithi175 bash[21527]: cluster 2024-03-26T04:02:05.066590+0000 mgr.y (mgr.24431) 677 : cluster [DBG] pgmap v1142: 268 pgs: 268 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T04:02:06.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:06 smithi175 bash[21527]: audit 2024-03-26T04:02:05.308009+0000 mon.a (mon.0) 3407 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-144"}]': finished 2024-03-26T04:02:06.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:06 smithi175 bash[21527]: cluster 2024-03-26T04:02:05.315226+0000 mon.a (mon.0) 3408 : cluster [DBG] osdmap e712: 8 total, 8 up, 8 in 2024-03-26T04:02:06.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:06 smithi175 bash[21527]: audit 2024-03-26T04:02:05.317994+0000 mon.c (mon.2) 706 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-144", "mode": "readproxy"}]: dispatch 2024-03-26T04:02:06.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:06 smithi175 bash[21527]: audit 2024-03-26T04:02:05.320828+0000 mon.a (mon.0) 3409 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-144", "mode": "readproxy"}]: dispatch 2024-03-26T04:02:06.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:06 smithi175 bash[21527]: audit 2024-03-26T04:02:06.065482+0000 mon.c (mon.2) 707 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:02:06.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:06 smithi175 bash[29284]: cluster 2024-03-26T04:02:05.066590+0000 mgr.y (mgr.24431) 677 : cluster [DBG] pgmap v1142: 268 pgs: 268 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T04:02:06.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:06 smithi175 bash[29284]: audit 2024-03-26T04:02:05.308009+0000 mon.a (mon.0) 3407 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-144"}]': finished 2024-03-26T04:02:06.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:06 smithi175 bash[29284]: cluster 2024-03-26T04:02:05.315226+0000 mon.a (mon.0) 3408 : cluster [DBG] osdmap e712: 8 total, 8 up, 8 in 2024-03-26T04:02:06.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:06 smithi175 bash[29284]: audit 2024-03-26T04:02:05.317994+0000 mon.c (mon.2) 706 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-144", "mode": "readproxy"}]: dispatch 2024-03-26T04:02:06.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:06 smithi175 bash[29284]: audit 2024-03-26T04:02:05.320828+0000 mon.a (mon.0) 3409 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-144", "mode": "readproxy"}]: dispatch 2024-03-26T04:02:06.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:06 smithi175 bash[29284]: audit 2024-03-26T04:02:06.065482+0000 mon.c (mon.2) 707 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:02:06.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:06 smithi190 bash[23392]: cluster 2024-03-26T04:02:05.066590+0000 mgr.y (mgr.24431) 677 : cluster [DBG] pgmap v1142: 268 pgs: 268 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T04:02:06.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:06 smithi190 bash[23392]: audit 2024-03-26T04:02:05.308009+0000 mon.a (mon.0) 3407 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-144"}]': finished 2024-03-26T04:02:06.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:06 smithi190 bash[23392]: cluster 2024-03-26T04:02:05.315226+0000 mon.a (mon.0) 3408 : cluster [DBG] osdmap e712: 8 total, 8 up, 8 in 2024-03-26T04:02:06.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:06 smithi190 bash[23392]: audit 2024-03-26T04:02:05.317994+0000 mon.c (mon.2) 706 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-144", "mode": "readproxy"}]: dispatch 2024-03-26T04:02:06.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:06 smithi190 bash[23392]: audit 2024-03-26T04:02:05.320828+0000 mon.a (mon.0) 3409 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-144", "mode": "readproxy"}]: dispatch 2024-03-26T04:02:06.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:06 smithi190 bash[23392]: audit 2024-03-26T04:02:06.065482+0000 mon.c (mon.2) 707 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:02:07.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:07 smithi175 bash[21527]: cluster 2024-03-26T04:02:06.308192+0000 mon.a (mon.0) 3410 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:02:07.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:07 smithi175 bash[21527]: audit 2024-03-26T04:02:06.310649+0000 mon.a (mon.0) 3411 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-144", "mode": "readproxy"}]': finished 2024-03-26T04:02:07.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:07 smithi175 bash[21527]: cluster 2024-03-26T04:02:06.318688+0000 mon.a (mon.0) 3412 : cluster [DBG] osdmap e713: 8 total, 8 up, 8 in 2024-03-26T04:02:07.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:07 smithi175 bash[21527]: cluster 2024-03-26T04:02:06.676885+0000 mon.a (mon.0) 3413 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:02:07.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:07 smithi175 bash[29284]: cluster 2024-03-26T04:02:06.308192+0000 mon.a (mon.0) 3410 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:02:07.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:07 smithi175 bash[29284]: audit 2024-03-26T04:02:06.310649+0000 mon.a (mon.0) 3411 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-144", "mode": "readproxy"}]': finished 2024-03-26T04:02:07.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:07 smithi175 bash[29284]: cluster 2024-03-26T04:02:06.318688+0000 mon.a (mon.0) 3412 : cluster [DBG] osdmap e713: 8 total, 8 up, 8 in 2024-03-26T04:02:07.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:07 smithi175 bash[29284]: cluster 2024-03-26T04:02:06.676885+0000 mon.a (mon.0) 3413 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:02:07.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:07 smithi190 bash[23392]: cluster 2024-03-26T04:02:06.308192+0000 mon.a (mon.0) 3410 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:02:07.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:07 smithi190 bash[23392]: audit 2024-03-26T04:02:06.310649+0000 mon.a (mon.0) 3411 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-144", "mode": "readproxy"}]': finished 2024-03-26T04:02:07.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:07 smithi190 bash[23392]: cluster 2024-03-26T04:02:06.318688+0000 mon.a (mon.0) 3412 : cluster [DBG] osdmap e713: 8 total, 8 up, 8 in 2024-03-26T04:02:07.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:07 smithi190 bash[23392]: cluster 2024-03-26T04:02:06.676885+0000 mon.a (mon.0) 3413 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:02:08.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:08 smithi175 bash[21527]: cluster 2024-03-26T04:02:07.067652+0000 mgr.y (mgr.24431) 678 : cluster [DBG] pgmap v1145: 268 pgs: 268 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T04:02:08.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:08 smithi175 bash[21527]: audit 2024-03-26T04:02:07.489921+0000 mon.c (mon.2) 708 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:02:08.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:08 smithi175 bash[21527]: audit 2024-03-26T04:02:08.218135+0000 mon.c (mon.2) 709 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:02:08.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:08 smithi175 bash[21527]: audit 2024-03-26T04:02:08.219589+0000 mon.c (mon.2) 710 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:02:08.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:08 smithi175 bash[21527]: audit 2024-03-26T04:02:08.229636+0000 mon.a (mon.0) 3414 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:02:08.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:08 smithi175 bash[29284]: cluster 2024-03-26T04:02:07.067652+0000 mgr.y (mgr.24431) 678 : cluster [DBG] pgmap v1145: 268 pgs: 268 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T04:02:08.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:08 smithi175 bash[29284]: audit 2024-03-26T04:02:07.489921+0000 mon.c (mon.2) 708 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:02:08.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:08 smithi175 bash[29284]: audit 2024-03-26T04:02:08.218135+0000 mon.c (mon.2) 709 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:02:08.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:08 smithi175 bash[29284]: audit 2024-03-26T04:02:08.219589+0000 mon.c (mon.2) 710 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:02:08.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:08 smithi175 bash[29284]: audit 2024-03-26T04:02:08.229636+0000 mon.a (mon.0) 3414 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:02:08.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:08 smithi190 bash[23392]: cluster 2024-03-26T04:02:07.067652+0000 mgr.y (mgr.24431) 678 : cluster [DBG] pgmap v1145: 268 pgs: 268 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-26T04:02:08.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:08 smithi190 bash[23392]: audit 2024-03-26T04:02:07.489921+0000 mon.c (mon.2) 708 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:02:08.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:08 smithi190 bash[23392]: audit 2024-03-26T04:02:08.218135+0000 mon.c (mon.2) 709 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:02:08.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:08 smithi190 bash[23392]: audit 2024-03-26T04:02:08.219589+0000 mon.c (mon.2) 710 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:02:08.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:08 smithi190 bash[23392]: audit 2024-03-26T04:02:08.229636+0000 mon.a (mon.0) 3414 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:02:09.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:02:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:02:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:02:10.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:10 smithi175 bash[21527]: cluster 2024-03-26T04:02:09.068525+0000 mgr.y (mgr.24431) 679 : cluster [DBG] pgmap v1146: 268 pgs: 268 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 885 B/s rd, 177 B/s wr, 1 op/s 2024-03-26T04:02:10.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:10 smithi175 bash[29284]: cluster 2024-03-26T04:02:09.068525+0000 mgr.y (mgr.24431) 679 : cluster [DBG] pgmap v1146: 268 pgs: 268 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 885 B/s rd, 177 B/s wr, 1 op/s 2024-03-26T04:02:10.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:10 smithi190 bash[23392]: cluster 2024-03-26T04:02:09.068525+0000 mgr.y (mgr.24431) 679 : cluster [DBG] pgmap v1146: 268 pgs: 268 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 885 B/s rd, 177 B/s wr, 1 op/s 2024-03-26T04:02:10.743 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:02:10 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:02:11.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:11 smithi175 bash[29284]: audit 2024-03-26T04:02:10.477505+0000 mgr.y (mgr.24431) 680 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:02:11.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:11 smithi175 bash[21527]: audit 2024-03-26T04:02:10.477505+0000 mgr.y (mgr.24431) 680 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:02:11.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:11 smithi190 bash[23392]: audit 2024-03-26T04:02:10.477505+0000 mgr.y (mgr.24431) 680 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:02:12.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:12 smithi175 bash[29284]: cluster 2024-03-26T04:02:11.072035+0000 mgr.y (mgr.24431) 681 : cluster [DBG] pgmap v1147: 268 pgs: 268 active+clean; 4.4 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 151 B/s wr, 1 op/s 2024-03-26T04:02:12.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:12 smithi175 bash[21527]: cluster 2024-03-26T04:02:11.072035+0000 mgr.y (mgr.24431) 681 : cluster [DBG] pgmap v1147: 268 pgs: 268 active+clean; 4.4 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 151 B/s wr, 1 op/s 2024-03-26T04:02:12.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:12 smithi190 bash[23392]: cluster 2024-03-26T04:02:11.072035+0000 mgr.y (mgr.24431) 681 : cluster [DBG] pgmap v1147: 268 pgs: 268 active+clean; 4.4 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 151 B/s wr, 1 op/s 2024-03-26T04:02:14.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:14 smithi175 bash[21527]: cluster 2024-03-26T04:02:13.072917+0000 mgr.y (mgr.24431) 682 : cluster [DBG] pgmap v1148: 268 pgs: 268 active+clean; 4.4 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T04:02:14.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:14 smithi175 bash[29284]: cluster 2024-03-26T04:02:13.072917+0000 mgr.y (mgr.24431) 682 : cluster [DBG] pgmap v1148: 268 pgs: 268 active+clean; 4.4 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T04:02:14.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:14 smithi190 bash[23392]: cluster 2024-03-26T04:02:13.072917+0000 mgr.y (mgr.24431) 682 : cluster [DBG] pgmap v1148: 268 pgs: 268 active+clean; 4.4 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T04:02:16.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:16 smithi175 bash[21527]: cluster 2024-03-26T04:02:15.075419+0000 mgr.y (mgr.24431) 683 : cluster [DBG] pgmap v1149: 268 pgs: 268 active+clean; 4.4 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-26T04:02:16.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:16 smithi175 bash[29284]: cluster 2024-03-26T04:02:15.075419+0000 mgr.y (mgr.24431) 683 : cluster [DBG] pgmap v1149: 268 pgs: 268 active+clean; 4.4 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-26T04:02:16.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:16 smithi190 bash[23392]: cluster 2024-03-26T04:02:15.075419+0000 mgr.y (mgr.24431) 683 : cluster [DBG] pgmap v1149: 268 pgs: 268 active+clean; 4.4 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-26T04:02:17.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:17 smithi175 bash[21527]: audit 2024-03-26T04:02:16.635929+0000 mon.c (mon.2) 711 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:17.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:17 smithi175 bash[21527]: audit 2024-03-26T04:02:16.636602+0000 mon.a (mon.0) 3415 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:17.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:17 smithi175 bash[29284]: audit 2024-03-26T04:02:16.635929+0000 mon.c (mon.2) 711 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:17.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:17 smithi175 bash[29284]: audit 2024-03-26T04:02:16.636602+0000 mon.a (mon.0) 3415 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:17.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:17 smithi190 bash[23392]: audit 2024-03-26T04:02:16.635929+0000 mon.c (mon.2) 711 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:17.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:17 smithi190 bash[23392]: audit 2024-03-26T04:02:16.636602+0000 mon.a (mon.0) 3415 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:18.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:18 smithi175 bash[21527]: cluster 2024-03-26T04:02:17.076396+0000 mgr.y (mgr.24431) 684 : cluster [DBG] pgmap v1150: 268 pgs: 268 active+clean; 4.4 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-26T04:02:18.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:18 smithi175 bash[21527]: audit 2024-03-26T04:02:17.374644+0000 mon.a (mon.0) 3416 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:02:18.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:18 smithi175 bash[21527]: cluster 2024-03-26T04:02:17.382089+0000 mon.a (mon.0) 3417 : cluster [DBG] osdmap e714: 8 total, 8 up, 8 in 2024-03-26T04:02:18.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:18 smithi175 bash[21527]: audit 2024-03-26T04:02:17.384160+0000 mon.c (mon.2) 712 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-144"}]: dispatch 2024-03-26T04:02:18.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:18 smithi175 bash[21527]: audit 2024-03-26T04:02:17.389069+0000 mon.a (mon.0) 3418 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-144"}]: dispatch 2024-03-26T04:02:18.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:18 smithi175 bash[29284]: cluster 2024-03-26T04:02:17.076396+0000 mgr.y (mgr.24431) 684 : cluster [DBG] pgmap v1150: 268 pgs: 268 active+clean; 4.4 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-26T04:02:18.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:18 smithi175 bash[29284]: audit 2024-03-26T04:02:17.374644+0000 mon.a (mon.0) 3416 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:02:18.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:18 smithi175 bash[29284]: cluster 2024-03-26T04:02:17.382089+0000 mon.a (mon.0) 3417 : cluster [DBG] osdmap e714: 8 total, 8 up, 8 in 2024-03-26T04:02:18.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:18 smithi175 bash[29284]: audit 2024-03-26T04:02:17.384160+0000 mon.c (mon.2) 712 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-144"}]: dispatch 2024-03-26T04:02:18.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:18 smithi175 bash[29284]: audit 2024-03-26T04:02:17.389069+0000 mon.a (mon.0) 3418 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-144"}]: dispatch 2024-03-26T04:02:18.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:18 smithi190 bash[23392]: cluster 2024-03-26T04:02:17.076396+0000 mgr.y (mgr.24431) 684 : cluster [DBG] pgmap v1150: 268 pgs: 268 active+clean; 4.4 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-26T04:02:18.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:18 smithi190 bash[23392]: audit 2024-03-26T04:02:17.374644+0000 mon.a (mon.0) 3416 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:02:18.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:18 smithi190 bash[23392]: cluster 2024-03-26T04:02:17.382089+0000 mon.a (mon.0) 3417 : cluster [DBG] osdmap e714: 8 total, 8 up, 8 in 2024-03-26T04:02:18.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:18 smithi190 bash[23392]: audit 2024-03-26T04:02:17.384160+0000 mon.c (mon.2) 712 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-144"}]: dispatch 2024-03-26T04:02:18.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:18 smithi190 bash[23392]: audit 2024-03-26T04:02:17.389069+0000 mon.a (mon.0) 3418 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-144"}]: dispatch 2024-03-26T04:02:19.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:02:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:02:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:02:19.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:19 smithi175 bash[21527]: cluster 2024-03-26T04:02:18.375849+0000 mon.a (mon.0) 3419 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:02:19.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:19 smithi175 bash[21527]: audit 2024-03-26T04:02:18.386163+0000 mon.a (mon.0) 3420 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-144"}]': finished 2024-03-26T04:02:19.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:19 smithi175 bash[21527]: cluster 2024-03-26T04:02:18.391312+0000 mon.a (mon.0) 3421 : cluster [DBG] osdmap e715: 8 total, 8 up, 8 in 2024-03-26T04:02:19.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:19 smithi175 bash[21527]: audit 2024-03-26T04:02:18.462210+0000 mon.c (mon.2) 713 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:19.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:19 smithi175 bash[21527]: audit 2024-03-26T04:02:18.463073+0000 mon.a (mon.0) 3422 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:19.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:19 smithi175 bash[21527]: audit 2024-03-26T04:02:18.464390+0000 mon.c (mon.2) 714 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-144"}]: dispatch 2024-03-26T04:02:19.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:19 smithi175 bash[21527]: audit 2024-03-26T04:02:18.465058+0000 mon.a (mon.0) 3423 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-144"}]: dispatch 2024-03-26T04:02:19.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:19 smithi175 bash[29284]: cluster 2024-03-26T04:02:18.375849+0000 mon.a (mon.0) 3419 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:02:19.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:19 smithi175 bash[29284]: audit 2024-03-26T04:02:18.386163+0000 mon.a (mon.0) 3420 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-144"}]': finished 2024-03-26T04:02:19.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:19 smithi175 bash[29284]: cluster 2024-03-26T04:02:18.391312+0000 mon.a (mon.0) 3421 : cluster [DBG] osdmap e715: 8 total, 8 up, 8 in 2024-03-26T04:02:19.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:19 smithi175 bash[29284]: audit 2024-03-26T04:02:18.462210+0000 mon.c (mon.2) 713 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:19.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:19 smithi175 bash[29284]: audit 2024-03-26T04:02:18.463073+0000 mon.a (mon.0) 3422 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:19.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:19 smithi175 bash[29284]: audit 2024-03-26T04:02:18.464390+0000 mon.c (mon.2) 714 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-144"}]: dispatch 2024-03-26T04:02:19.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:19 smithi175 bash[29284]: audit 2024-03-26T04:02:18.465058+0000 mon.a (mon.0) 3423 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-144"}]: dispatch 2024-03-26T04:02:19.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:19 smithi190 bash[23392]: cluster 2024-03-26T04:02:18.375849+0000 mon.a (mon.0) 3419 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:02:19.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:19 smithi190 bash[23392]: audit 2024-03-26T04:02:18.386163+0000 mon.a (mon.0) 3420 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-144"}]': finished 2024-03-26T04:02:19.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:19 smithi190 bash[23392]: cluster 2024-03-26T04:02:18.391312+0000 mon.a (mon.0) 3421 : cluster [DBG] osdmap e715: 8 total, 8 up, 8 in 2024-03-26T04:02:19.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:19 smithi190 bash[23392]: audit 2024-03-26T04:02:18.462210+0000 mon.c (mon.2) 713 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:19.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:19 smithi190 bash[23392]: audit 2024-03-26T04:02:18.463073+0000 mon.a (mon.0) 3422 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:19.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:19 smithi190 bash[23392]: audit 2024-03-26T04:02:18.464390+0000 mon.c (mon.2) 714 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-144"}]: dispatch 2024-03-26T04:02:19.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:19 smithi190 bash[23392]: audit 2024-03-26T04:02:18.465058+0000 mon.a (mon.0) 3423 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-144"}]: dispatch 2024-03-26T04:02:20.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:20 smithi175 bash[21527]: cluster 2024-03-26T04:02:19.077302+0000 mgr.y (mgr.24431) 685 : cluster [DBG] pgmap v1153: 268 pgs: 268 active+clean; 4.4 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-26T04:02:20.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:20 smithi175 bash[21527]: cluster 2024-03-26T04:02:19.386254+0000 mon.a (mon.0) 3424 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:02:20.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:20 smithi175 bash[21527]: cluster 2024-03-26T04:02:19.402094+0000 mon.a (mon.0) 3425 : cluster [DBG] osdmap e716: 8 total, 8 up, 8 in 2024-03-26T04:02:20.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:20 smithi175 bash[29284]: cluster 2024-03-26T04:02:19.077302+0000 mgr.y (mgr.24431) 685 : cluster [DBG] pgmap v1153: 268 pgs: 268 active+clean; 4.4 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-26T04:02:20.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:20 smithi175 bash[29284]: cluster 2024-03-26T04:02:19.386254+0000 mon.a (mon.0) 3424 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:02:20.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:20 smithi175 bash[29284]: cluster 2024-03-26T04:02:19.402094+0000 mon.a (mon.0) 3425 : cluster [DBG] osdmap e716: 8 total, 8 up, 8 in 2024-03-26T04:02:20.743 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:02:20 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:02:20.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:20 smithi190 bash[23392]: cluster 2024-03-26T04:02:19.077302+0000 mgr.y (mgr.24431) 685 : cluster [DBG] pgmap v1153: 268 pgs: 268 active+clean; 4.4 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-26T04:02:20.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:20 smithi190 bash[23392]: cluster 2024-03-26T04:02:19.386254+0000 mon.a (mon.0) 3424 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:02:20.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:20 smithi190 bash[23392]: cluster 2024-03-26T04:02:19.402094+0000 mon.a (mon.0) 3425 : cluster [DBG] osdmap e716: 8 total, 8 up, 8 in 2024-03-26T04:02:21.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:21 smithi175 bash[21527]: cluster 2024-03-26T04:02:20.406677+0000 mon.a (mon.0) 3426 : cluster [DBG] osdmap e717: 8 total, 8 up, 8 in 2024-03-26T04:02:21.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:21 smithi175 bash[21527]: audit 2024-03-26T04:02:20.412626+0000 mon.c (mon.2) 715 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:21.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:21 smithi175 bash[21527]: audit 2024-03-26T04:02:20.413822+0000 mon.a (mon.0) 3427 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:21.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:21 smithi175 bash[21527]: audit 2024-03-26T04:02:20.488615+0000 mgr.y (mgr.24431) 686 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:02:21.746 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:21 smithi175 bash[21527]: audit 2024-03-26T04:02:21.099894+0000 mon.c (mon.2) 716 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:02:21.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:21 smithi175 bash[29284]: cluster 2024-03-26T04:02:20.406677+0000 mon.a (mon.0) 3426 : cluster [DBG] osdmap e717: 8 total, 8 up, 8 in 2024-03-26T04:02:21.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:21 smithi175 bash[29284]: audit 2024-03-26T04:02:20.412626+0000 mon.c (mon.2) 715 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:21.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:21 smithi175 bash[29284]: audit 2024-03-26T04:02:20.413822+0000 mon.a (mon.0) 3427 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:21.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:21 smithi175 bash[29284]: audit 2024-03-26T04:02:20.488615+0000 mgr.y (mgr.24431) 686 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:02:21.746 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:21 smithi175 bash[29284]: audit 2024-03-26T04:02:21.099894+0000 mon.c (mon.2) 716 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:02:21.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:21 smithi190 bash[23392]: cluster 2024-03-26T04:02:20.406677+0000 mon.a (mon.0) 3426 : cluster [DBG] osdmap e717: 8 total, 8 up, 8 in 2024-03-26T04:02:21.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:21 smithi190 bash[23392]: audit 2024-03-26T04:02:20.412626+0000 mon.c (mon.2) 715 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:21.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:21 smithi190 bash[23392]: audit 2024-03-26T04:02:20.413822+0000 mon.a (mon.0) 3427 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:21.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:21 smithi190 bash[23392]: audit 2024-03-26T04:02:20.488615+0000 mgr.y (mgr.24431) 686 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:02:21.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:21 smithi190 bash[23392]: audit 2024-03-26T04:02:21.099894+0000 mon.c (mon.2) 716 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:02:22.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:22 smithi175 bash[21527]: cluster 2024-03-26T04:02:21.078158+0000 mgr.y (mgr.24431) 687 : cluster [DBG] pgmap v1156: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:02:22.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:22 smithi175 bash[21527]: audit 2024-03-26T04:02:21.408738+0000 mon.a (mon.0) 3428 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:02:22.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:22 smithi175 bash[21527]: cluster 2024-03-26T04:02:21.418534+0000 mon.a (mon.0) 3429 : cluster [DBG] osdmap e718: 8 total, 8 up, 8 in 2024-03-26T04:02:22.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:22 smithi175 bash[21527]: audit 2024-03-26T04:02:21.482180+0000 mon.c (mon.2) 717 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:02:22.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:22 smithi175 bash[21527]: audit 2024-03-26T04:02:21.482664+0000 mon.a (mon.0) 3430 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:02:22.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:22 smithi175 bash[29284]: cluster 2024-03-26T04:02:21.078158+0000 mgr.y (mgr.24431) 687 : cluster [DBG] pgmap v1156: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:02:22.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:22 smithi175 bash[29284]: audit 2024-03-26T04:02:21.408738+0000 mon.a (mon.0) 3428 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:02:22.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:22 smithi175 bash[29284]: cluster 2024-03-26T04:02:21.418534+0000 mon.a (mon.0) 3429 : cluster [DBG] osdmap e718: 8 total, 8 up, 8 in 2024-03-26T04:02:22.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:22 smithi175 bash[29284]: audit 2024-03-26T04:02:21.482180+0000 mon.c (mon.2) 717 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:02:22.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:22 smithi175 bash[29284]: audit 2024-03-26T04:02:21.482664+0000 mon.a (mon.0) 3430 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:02:22.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:22 smithi190 bash[23392]: cluster 2024-03-26T04:02:21.078158+0000 mgr.y (mgr.24431) 687 : cluster [DBG] pgmap v1156: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:02:22.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:22 smithi190 bash[23392]: audit 2024-03-26T04:02:21.408738+0000 mon.a (mon.0) 3428 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:02:22.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:22 smithi190 bash[23392]: cluster 2024-03-26T04:02:21.418534+0000 mon.a (mon.0) 3429 : cluster [DBG] osdmap e718: 8 total, 8 up, 8 in 2024-03-26T04:02:22.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:22 smithi190 bash[23392]: audit 2024-03-26T04:02:21.482180+0000 mon.c (mon.2) 717 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:02:22.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:22 smithi190 bash[23392]: audit 2024-03-26T04:02:21.482664+0000 mon.a (mon.0) 3430 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:02:23.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:23 smithi175 bash[29284]: audit 2024-03-26T04:02:22.411391+0000 mon.a (mon.0) 3431 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-146", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:02:23.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:23 smithi175 bash[29284]: cluster 2024-03-26T04:02:22.418601+0000 mon.a (mon.0) 3432 : cluster [DBG] osdmap e719: 8 total, 8 up, 8 in 2024-03-26T04:02:23.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:23 smithi175 bash[29284]: audit 2024-03-26T04:02:22.422866+0000 mon.c (mon.2) 718 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-146"}]: dispatch 2024-03-26T04:02:23.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:23 smithi175 bash[29284]: audit 2024-03-26T04:02:22.440597+0000 mon.a (mon.0) 3433 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-146"}]: dispatch 2024-03-26T04:02:23.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:23 smithi175 bash[21527]: audit 2024-03-26T04:02:22.411391+0000 mon.a (mon.0) 3431 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-146", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:02:23.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:23 smithi175 bash[21527]: cluster 2024-03-26T04:02:22.418601+0000 mon.a (mon.0) 3432 : cluster [DBG] osdmap e719: 8 total, 8 up, 8 in 2024-03-26T04:02:23.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:23 smithi175 bash[21527]: audit 2024-03-26T04:02:22.422866+0000 mon.c (mon.2) 718 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-146"}]: dispatch 2024-03-26T04:02:23.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:23 smithi175 bash[21527]: audit 2024-03-26T04:02:22.440597+0000 mon.a (mon.0) 3433 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-146"}]: dispatch 2024-03-26T04:02:23.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:23 smithi190 bash[23392]: audit 2024-03-26T04:02:22.411391+0000 mon.a (mon.0) 3431 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-146", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:02:23.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:23 smithi190 bash[23392]: cluster 2024-03-26T04:02:22.418601+0000 mon.a (mon.0) 3432 : cluster [DBG] osdmap e719: 8 total, 8 up, 8 in 2024-03-26T04:02:23.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:23 smithi190 bash[23392]: audit 2024-03-26T04:02:22.422866+0000 mon.c (mon.2) 718 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-146"}]: dispatch 2024-03-26T04:02:23.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:23 smithi190 bash[23392]: audit 2024-03-26T04:02:22.440597+0000 mon.a (mon.0) 3433 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-146"}]: dispatch 2024-03-26T04:02:24.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:24 smithi175 bash[21527]: cluster 2024-03-26T04:02:23.079062+0000 mgr.y (mgr.24431) 688 : cluster [DBG] pgmap v1159: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:02:24.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:24 smithi175 bash[21527]: audit 2024-03-26T04:02:23.427276+0000 mon.a (mon.0) 3434 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-146"}]': finished 2024-03-26T04:02:24.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:24 smithi175 bash[21527]: cluster 2024-03-26T04:02:23.434922+0000 mon.a (mon.0) 3435 : cluster [DBG] osdmap e720: 8 total, 8 up, 8 in 2024-03-26T04:02:24.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:24 smithi175 bash[21527]: audit 2024-03-26T04:02:23.436669+0000 mon.c (mon.2) 719 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-146", "mode": "writeback"}]: dispatch 2024-03-26T04:02:24.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:24 smithi175 bash[21527]: audit 2024-03-26T04:02:23.440498+0000 mon.a (mon.0) 3436 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-146", "mode": "writeback"}]: dispatch 2024-03-26T04:02:24.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:24 smithi175 bash[21527]: cluster 2024-03-26T04:02:24.427738+0000 mon.a (mon.0) 3437 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:02:24.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:24 smithi175 bash[21527]: audit 2024-03-26T04:02:24.430236+0000 mon.a (mon.0) 3438 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-146", "mode": "writeback"}]': finished 2024-03-26T04:02:24.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:24 smithi175 bash[21527]: cluster 2024-03-26T04:02:24.437664+0000 mon.a (mon.0) 3439 : cluster [DBG] osdmap e721: 8 total, 8 up, 8 in 2024-03-26T04:02:24.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:24 smithi175 bash[29284]: cluster 2024-03-26T04:02:23.079062+0000 mgr.y (mgr.24431) 688 : cluster [DBG] pgmap v1159: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:02:24.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:24 smithi175 bash[29284]: audit 2024-03-26T04:02:23.427276+0000 mon.a (mon.0) 3434 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-146"}]': finished 2024-03-26T04:02:24.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:24 smithi175 bash[29284]: cluster 2024-03-26T04:02:23.434922+0000 mon.a (mon.0) 3435 : cluster [DBG] osdmap e720: 8 total, 8 up, 8 in 2024-03-26T04:02:24.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:24 smithi175 bash[29284]: audit 2024-03-26T04:02:23.436669+0000 mon.c (mon.2) 719 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-146", "mode": "writeback"}]: dispatch 2024-03-26T04:02:24.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:24 smithi175 bash[29284]: audit 2024-03-26T04:02:23.440498+0000 mon.a (mon.0) 3436 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-146", "mode": "writeback"}]: dispatch 2024-03-26T04:02:24.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:24 smithi175 bash[29284]: cluster 2024-03-26T04:02:24.427738+0000 mon.a (mon.0) 3437 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:02:24.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:24 smithi175 bash[29284]: audit 2024-03-26T04:02:24.430236+0000 mon.a (mon.0) 3438 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-146", "mode": "writeback"}]': finished 2024-03-26T04:02:24.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:24 smithi175 bash[29284]: cluster 2024-03-26T04:02:24.437664+0000 mon.a (mon.0) 3439 : cluster [DBG] osdmap e721: 8 total, 8 up, 8 in 2024-03-26T04:02:24.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:24 smithi190 bash[23392]: cluster 2024-03-26T04:02:23.079062+0000 mgr.y (mgr.24431) 688 : cluster [DBG] pgmap v1159: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:02:24.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:24 smithi190 bash[23392]: audit 2024-03-26T04:02:23.427276+0000 mon.a (mon.0) 3434 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi175-64594-111", "overlaypool": "test-rados-api-smithi175-64594-146"}]': finished 2024-03-26T04:02:24.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:24 smithi190 bash[23392]: cluster 2024-03-26T04:02:23.434922+0000 mon.a (mon.0) 3435 : cluster [DBG] osdmap e720: 8 total, 8 up, 8 in 2024-03-26T04:02:24.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:24 smithi190 bash[23392]: audit 2024-03-26T04:02:23.436669+0000 mon.c (mon.2) 719 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-146", "mode": "writeback"}]: dispatch 2024-03-26T04:02:24.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:24 smithi190 bash[23392]: audit 2024-03-26T04:02:23.440498+0000 mon.a (mon.0) 3436 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-146", "mode": "writeback"}]: dispatch 2024-03-26T04:02:24.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:24 smithi190 bash[23392]: cluster 2024-03-26T04:02:24.427738+0000 mon.a (mon.0) 3437 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-26T04:02:24.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:24 smithi190 bash[23392]: audit 2024-03-26T04:02:24.430236+0000 mon.a (mon.0) 3438 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi175-64594-146", "mode": "writeback"}]': finished 2024-03-26T04:02:24.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:24 smithi190 bash[23392]: cluster 2024-03-26T04:02:24.437664+0000 mon.a (mon.0) 3439 : cluster [DBG] osdmap e721: 8 total, 8 up, 8 in 2024-03-26T04:02:25.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:25 smithi175 bash[21527]: audit 2024-03-26T04:02:24.560959+0000 mon.c (mon.2) 720 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T04:02:25.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:25 smithi175 bash[21527]: audit 2024-03-26T04:02:24.561788+0000 mon.a (mon.0) 3440 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T04:02:25.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:25 smithi175 bash[29284]: audit 2024-03-26T04:02:24.560959+0000 mon.c (mon.2) 720 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T04:02:25.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:25 smithi175 bash[29284]: audit 2024-03-26T04:02:24.561788+0000 mon.a (mon.0) 3440 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T04:02:25.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:25 smithi190 bash[23392]: audit 2024-03-26T04:02:24.560959+0000 mon.c (mon.2) 720 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T04:02:25.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:25 smithi190 bash[23392]: audit 2024-03-26T04:02:24.561788+0000 mon.a (mon.0) 3440 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "hit_set_count","val": "2"}]: dispatch 2024-03-26T04:02:26.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:26 smithi175 bash[21527]: cluster 2024-03-26T04:02:25.081480+0000 mgr.y (mgr.24431) 689 : cluster [DBG] pgmap v1162: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-03-26T04:02:26.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:26 smithi175 bash[21527]: audit 2024-03-26T04:02:25.473102+0000 mon.a (mon.0) 3441 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "hit_set_count","val": "2"}]': finished 2024-03-26T04:02:26.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:26 smithi175 bash[21527]: cluster 2024-03-26T04:02:25.480620+0000 mon.a (mon.0) 3442 : cluster [DBG] osdmap e722: 8 total, 8 up, 8 in 2024-03-26T04:02:26.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:26 smithi175 bash[21527]: audit 2024-03-26T04:02:25.485592+0000 mon.c (mon.2) 721 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T04:02:26.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:26 smithi175 bash[21527]: audit 2024-03-26T04:02:25.486585+0000 mon.a (mon.0) 3443 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T04:02:26.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:26 smithi175 bash[29284]: cluster 2024-03-26T04:02:25.081480+0000 mgr.y (mgr.24431) 689 : cluster [DBG] pgmap v1162: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-03-26T04:02:26.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:26 smithi175 bash[29284]: audit 2024-03-26T04:02:25.473102+0000 mon.a (mon.0) 3441 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "hit_set_count","val": "2"}]': finished 2024-03-26T04:02:26.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:26 smithi175 bash[29284]: cluster 2024-03-26T04:02:25.480620+0000 mon.a (mon.0) 3442 : cluster [DBG] osdmap e722: 8 total, 8 up, 8 in 2024-03-26T04:02:26.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:26 smithi175 bash[29284]: audit 2024-03-26T04:02:25.485592+0000 mon.c (mon.2) 721 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T04:02:26.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:26 smithi175 bash[29284]: audit 2024-03-26T04:02:25.486585+0000 mon.a (mon.0) 3443 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T04:02:26.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:26 smithi190 bash[23392]: cluster 2024-03-26T04:02:25.081480+0000 mgr.y (mgr.24431) 689 : cluster [DBG] pgmap v1162: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-03-26T04:02:26.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:26 smithi190 bash[23392]: audit 2024-03-26T04:02:25.473102+0000 mon.a (mon.0) 3441 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "hit_set_count","val": "2"}]': finished 2024-03-26T04:02:26.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:26 smithi190 bash[23392]: cluster 2024-03-26T04:02:25.480620+0000 mon.a (mon.0) 3442 : cluster [DBG] osdmap e722: 8 total, 8 up, 8 in 2024-03-26T04:02:26.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:26 smithi190 bash[23392]: audit 2024-03-26T04:02:25.485592+0000 mon.c (mon.2) 721 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T04:02:26.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:26 smithi190 bash[23392]: audit 2024-03-26T04:02:25.486585+0000 mon.a (mon.0) 3443 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "hit_set_period","val": "600"}]: dispatch 2024-03-26T04:02:27.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:27 smithi190 bash[23392]: audit 2024-03-26T04:02:26.483922+0000 mon.a (mon.0) 3444 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "hit_set_period","val": "600"}]': finished 2024-03-26T04:02:27.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:27 smithi190 bash[23392]: cluster 2024-03-26T04:02:26.490737+0000 mon.a (mon.0) 3445 : cluster [DBG] osdmap e723: 8 total, 8 up, 8 in 2024-03-26T04:02:27.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:27 smithi190 bash[23392]: audit 2024-03-26T04:02:26.494608+0000 mon.c (mon.2) 722 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-26T04:02:27.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:27 smithi190 bash[23392]: audit 2024-03-26T04:02:26.495798+0000 mon.a (mon.0) 3446 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-26T04:02:27.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:27 smithi190 bash[23392]: cluster 2024-03-26T04:02:26.687215+0000 mon.a (mon.0) 3447 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:02:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:27 smithi175 bash[21527]: audit 2024-03-26T04:02:26.483922+0000 mon.a (mon.0) 3444 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "hit_set_period","val": "600"}]': finished 2024-03-26T04:02:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:27 smithi175 bash[21527]: cluster 2024-03-26T04:02:26.490737+0000 mon.a (mon.0) 3445 : cluster [DBG] osdmap e723: 8 total, 8 up, 8 in 2024-03-26T04:02:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:27 smithi175 bash[21527]: audit 2024-03-26T04:02:26.494608+0000 mon.c (mon.2) 722 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-26T04:02:27.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:27 smithi175 bash[21527]: audit 2024-03-26T04:02:26.495798+0000 mon.a (mon.0) 3446 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-26T04:02:27.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:27 smithi175 bash[21527]: cluster 2024-03-26T04:02:26.687215+0000 mon.a (mon.0) 3447 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:02:27.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:27 smithi175 bash[29284]: audit 2024-03-26T04:02:26.483922+0000 mon.a (mon.0) 3444 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "hit_set_period","val": "600"}]': finished 2024-03-26T04:02:27.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:27 smithi175 bash[29284]: cluster 2024-03-26T04:02:26.490737+0000 mon.a (mon.0) 3445 : cluster [DBG] osdmap e723: 8 total, 8 up, 8 in 2024-03-26T04:02:27.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:27 smithi175 bash[29284]: audit 2024-03-26T04:02:26.494608+0000 mon.c (mon.2) 722 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-26T04:02:27.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:27 smithi175 bash[29284]: audit 2024-03-26T04:02:26.495798+0000 mon.a (mon.0) 3446 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-26T04:02:27.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:27 smithi175 bash[29284]: cluster 2024-03-26T04:02:26.687215+0000 mon.a (mon.0) 3447 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:02:28.771 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:28 smithi175 bash[29284]: cluster 2024-03-26T04:02:27.082361+0000 mgr.y (mgr.24431) 690 : cluster [DBG] pgmap v1165: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-03-26T04:02:28.771 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:28 smithi175 bash[29284]: cluster 2024-03-26T04:02:27.484980+0000 mon.a (mon.0) 3448 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:02:28.771 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:28 smithi175 bash[29284]: audit 2024-03-26T04:02:27.493072+0000 mon.a (mon.0) 3449 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "hit_set_type","val": "bloom"}]': finished 2024-03-26T04:02:28.771 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:28 smithi175 bash[29284]: cluster 2024-03-26T04:02:27.503934+0000 mon.a (mon.0) 3450 : cluster [DBG] osdmap e724: 8 total, 8 up, 8 in 2024-03-26T04:02:28.771 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:28 smithi175 bash[29284]: audit 2024-03-26T04:02:27.505154+0000 mon.c (mon.2) 723 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-26T04:02:28.771 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:28 smithi175 bash[29284]: audit 2024-03-26T04:02:27.506670+0000 mon.a (mon.0) 3451 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-26T04:02:28.771 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:28 smithi175 bash[21527]: cluster 2024-03-26T04:02:27.082361+0000 mgr.y (mgr.24431) 690 : cluster [DBG] pgmap v1165: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-03-26T04:02:28.772 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:28 smithi175 bash[21527]: cluster 2024-03-26T04:02:27.484980+0000 mon.a (mon.0) 3448 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:02:28.772 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:28 smithi175 bash[21527]: audit 2024-03-26T04:02:27.493072+0000 mon.a (mon.0) 3449 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "hit_set_type","val": "bloom"}]': finished 2024-03-26T04:02:28.772 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:28 smithi175 bash[21527]: cluster 2024-03-26T04:02:27.503934+0000 mon.a (mon.0) 3450 : cluster [DBG] osdmap e724: 8 total, 8 up, 8 in 2024-03-26T04:02:28.772 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:28 smithi175 bash[21527]: audit 2024-03-26T04:02:27.505154+0000 mon.c (mon.2) 723 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-26T04:02:28.772 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:28 smithi175 bash[21527]: audit 2024-03-26T04:02:27.506670+0000 mon.a (mon.0) 3451 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-26T04:02:28.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:28 smithi190 bash[23392]: cluster 2024-03-26T04:02:27.082361+0000 mgr.y (mgr.24431) 690 : cluster [DBG] pgmap v1165: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-03-26T04:02:28.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:28 smithi190 bash[23392]: cluster 2024-03-26T04:02:27.484980+0000 mon.a (mon.0) 3448 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-26T04:02:28.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:28 smithi190 bash[23392]: audit 2024-03-26T04:02:27.493072+0000 mon.a (mon.0) 3449 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "hit_set_type","val": "bloom"}]': finished 2024-03-26T04:02:28.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:28 smithi190 bash[23392]: cluster 2024-03-26T04:02:27.503934+0000 mon.a (mon.0) 3450 : cluster [DBG] osdmap e724: 8 total, 8 up, 8 in 2024-03-26T04:02:28.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:28 smithi190 bash[23392]: audit 2024-03-26T04:02:27.505154+0000 mon.c (mon.2) 723 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-26T04:02:28.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:28 smithi190 bash[23392]: audit 2024-03-26T04:02:27.506670+0000 mon.a (mon.0) 3451 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-26T04:02:29.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:02:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:02:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:02:29.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:29 smithi175 bash[21527]: audit 2024-03-26T04:02:28.507973+0000 mon.a (mon.0) 3452 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-26T04:02:29.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:29 smithi175 bash[21527]: cluster 2024-03-26T04:02:28.514310+0000 mon.a (mon.0) 3453 : cluster [DBG] osdmap e725: 8 total, 8 up, 8 in 2024-03-26T04:02:29.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:29 smithi175 bash[21527]: audit 2024-03-26T04:02:28.520774+0000 mon.c (mon.2) 724 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "target_max_objects","val": "1"}]: dispatch 2024-03-26T04:02:29.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:29 smithi175 bash[21527]: audit 2024-03-26T04:02:28.525322+0000 mon.a (mon.0) 3454 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "target_max_objects","val": "1"}]: dispatch 2024-03-26T04:02:29.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:29 smithi175 bash[29284]: audit 2024-03-26T04:02:28.507973+0000 mon.a (mon.0) 3452 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-26T04:02:29.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:29 smithi175 bash[29284]: cluster 2024-03-26T04:02:28.514310+0000 mon.a (mon.0) 3453 : cluster [DBG] osdmap e725: 8 total, 8 up, 8 in 2024-03-26T04:02:29.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:29 smithi175 bash[29284]: audit 2024-03-26T04:02:28.520774+0000 mon.c (mon.2) 724 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "target_max_objects","val": "1"}]: dispatch 2024-03-26T04:02:29.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:29 smithi175 bash[29284]: audit 2024-03-26T04:02:28.525322+0000 mon.a (mon.0) 3454 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "target_max_objects","val": "1"}]: dispatch 2024-03-26T04:02:29.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:29 smithi190 bash[23392]: audit 2024-03-26T04:02:28.507973+0000 mon.a (mon.0) 3452 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-26T04:02:29.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:29 smithi190 bash[23392]: cluster 2024-03-26T04:02:28.514310+0000 mon.a (mon.0) 3453 : cluster [DBG] osdmap e725: 8 total, 8 up, 8 in 2024-03-26T04:02:29.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:29 smithi190 bash[23392]: audit 2024-03-26T04:02:28.520774+0000 mon.c (mon.2) 724 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "target_max_objects","val": "1"}]: dispatch 2024-03-26T04:02:29.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:29 smithi190 bash[23392]: audit 2024-03-26T04:02:28.525322+0000 mon.a (mon.0) 3454 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "target_max_objects","val": "1"}]: dispatch 2024-03-26T04:02:30.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:30 smithi175 bash[29284]: cluster 2024-03-26T04:02:29.083297+0000 mgr.y (mgr.24431) 691 : cluster [DBG] pgmap v1168: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail 2024-03-26T04:02:30.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:30 smithi175 bash[29284]: audit 2024-03-26T04:02:29.521639+0000 mon.a (mon.0) 3455 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "target_max_objects","val": "1"}]': finished 2024-03-26T04:02:30.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:30 smithi175 bash[29284]: cluster 2024-03-26T04:02:29.530738+0000 mon.a (mon.0) 3456 : cluster [DBG] osdmap e726: 8 total, 8 up, 8 in 2024-03-26T04:02:30.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:30 smithi175 bash[21527]: cluster 2024-03-26T04:02:29.083297+0000 mgr.y (mgr.24431) 691 : cluster [DBG] pgmap v1168: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail 2024-03-26T04:02:30.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:30 smithi175 bash[21527]: audit 2024-03-26T04:02:29.521639+0000 mon.a (mon.0) 3455 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "target_max_objects","val": "1"}]': finished 2024-03-26T04:02:30.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:30 smithi175 bash[21527]: cluster 2024-03-26T04:02:29.530738+0000 mon.a (mon.0) 3456 : cluster [DBG] osdmap e726: 8 total, 8 up, 8 in 2024-03-26T04:02:30.993 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:02:30 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:02:30.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:30 smithi190 bash[23392]: cluster 2024-03-26T04:02:29.083297+0000 mgr.y (mgr.24431) 691 : cluster [DBG] pgmap v1168: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail 2024-03-26T04:02:30.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:30 smithi190 bash[23392]: audit 2024-03-26T04:02:29.521639+0000 mon.a (mon.0) 3455 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-146","var": "target_max_objects","val": "1"}]': finished 2024-03-26T04:02:30.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:30 smithi190 bash[23392]: cluster 2024-03-26T04:02:29.530738+0000 mon.a (mon.0) 3456 : cluster [DBG] osdmap e726: 8 total, 8 up, 8 in 2024-03-26T04:02:31.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:31 smithi175 bash[21527]: audit 2024-03-26T04:02:30.500165+0000 mgr.y (mgr.24431) 692 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:02:31.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:31 smithi175 bash[29284]: audit 2024-03-26T04:02:30.500165+0000 mgr.y (mgr.24431) 692 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:02:31.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:31 smithi190 bash[23392]: audit 2024-03-26T04:02:30.500165+0000 mgr.y (mgr.24431) 692 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:02:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:32 smithi175 bash[21527]: cluster 2024-03-26T04:02:31.085885+0000 mgr.y (mgr.24431) 693 : cluster [DBG] pgmap v1170: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-26T04:02:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:32 smithi175 bash[21527]: cluster 2024-03-26T04:02:31.523598+0000 mon.a (mon.0) 3457 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-03-26T04:02:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:32 smithi175 bash[29284]: cluster 2024-03-26T04:02:31.085885+0000 mgr.y (mgr.24431) 693 : cluster [DBG] pgmap v1170: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-26T04:02:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:32 smithi175 bash[29284]: cluster 2024-03-26T04:02:31.523598+0000 mon.a (mon.0) 3457 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-03-26T04:02:32.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:32 smithi190 bash[23392]: cluster 2024-03-26T04:02:31.085885+0000 mgr.y (mgr.24431) 693 : cluster [DBG] pgmap v1170: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-26T04:02:32.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:32 smithi190 bash[23392]: cluster 2024-03-26T04:02:31.523598+0000 mon.a (mon.0) 3457 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-03-26T04:02:34.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:34 smithi175 bash[21527]: cluster 2024-03-26T04:02:33.086810+0000 mgr.y (mgr.24431) 694 : cluster [DBG] pgmap v1171: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:02:34.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:34 smithi175 bash[29284]: cluster 2024-03-26T04:02:33.086810+0000 mgr.y (mgr.24431) 694 : cluster [DBG] pgmap v1171: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:02:34.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:34 smithi190 bash[23392]: cluster 2024-03-26T04:02:33.086810+0000 mgr.y (mgr.24431) 694 : cluster [DBG] pgmap v1171: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:02:36.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:36 smithi175 bash[29284]: cluster 2024-03-26T04:02:35.089344+0000 mgr.y (mgr.24431) 695 : cluster [DBG] pgmap v1172: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T04:02:36.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:36 smithi175 bash[29284]: audit 2024-03-26T04:02:36.140274+0000 mon.a (mon.0) 3458 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:02:36.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:36 smithi175 bash[29284]: audit 2024-03-26T04:02:36.145258+0000 mon.c (mon.2) 725 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:02:36.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:36 smithi175 bash[21527]: cluster 2024-03-26T04:02:35.089344+0000 mgr.y (mgr.24431) 695 : cluster [DBG] pgmap v1172: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T04:02:36.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:36 smithi175 bash[21527]: audit 2024-03-26T04:02:36.140274+0000 mon.a (mon.0) 3458 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:02:36.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:36 smithi175 bash[21527]: audit 2024-03-26T04:02:36.145258+0000 mon.c (mon.2) 725 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:02:36.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:36 smithi190 bash[23392]: cluster 2024-03-26T04:02:35.089344+0000 mgr.y (mgr.24431) 695 : cluster [DBG] pgmap v1172: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T04:02:36.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:36 smithi190 bash[23392]: audit 2024-03-26T04:02:36.140274+0000 mon.a (mon.0) 3458 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:02:36.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:36 smithi190 bash[23392]: audit 2024-03-26T04:02:36.145258+0000 mon.c (mon.2) 725 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:02:38.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:38 smithi175 bash[29284]: cluster 2024-03-26T04:02:37.090256+0000 mgr.y (mgr.24431) 696 : cluster [DBG] pgmap v1173: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T04:02:38.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:38 smithi175 bash[21527]: cluster 2024-03-26T04:02:37.090256+0000 mgr.y (mgr.24431) 696 : cluster [DBG] pgmap v1173: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T04:02:38.993 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:02:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:02:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:02:38.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:38 smithi190 bash[23392]: cluster 2024-03-26T04:02:37.090256+0000 mgr.y (mgr.24431) 696 : cluster [DBG] pgmap v1173: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T04:02:40.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:40 smithi175 bash[29284]: cluster 2024-03-26T04:02:39.091164+0000 mgr.y (mgr.24431) 697 : cluster [DBG] pgmap v1174: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-26T04:02:40.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:40 smithi175 bash[29284]: audit 2024-03-26T04:02:39.566920+0000 mon.c (mon.2) 726 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:40.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:40 smithi175 bash[29284]: audit 2024-03-26T04:02:39.567540+0000 mon.a (mon.0) 3459 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:40.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:40 smithi175 bash[21527]: cluster 2024-03-26T04:02:39.091164+0000 mgr.y (mgr.24431) 697 : cluster [DBG] pgmap v1174: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-26T04:02:40.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:40 smithi175 bash[21527]: audit 2024-03-26T04:02:39.566920+0000 mon.c (mon.2) 726 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:40.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:40 smithi175 bash[21527]: audit 2024-03-26T04:02:39.567540+0000 mon.a (mon.0) 3459 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:40.993 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:02:40 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:02:40.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:40 smithi190 bash[23392]: cluster 2024-03-26T04:02:39.091164+0000 mgr.y (mgr.24431) 697 : cluster [DBG] pgmap v1174: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-26T04:02:40.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:40 smithi190 bash[23392]: audit 2024-03-26T04:02:39.566920+0000 mon.c (mon.2) 726 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:40.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:40 smithi190 bash[23392]: audit 2024-03-26T04:02:39.567540+0000 mon.a (mon.0) 3459 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:41 smithi175 bash[29284]: audit 2024-03-26T04:02:40.509671+0000 mgr.y (mgr.24431) 698 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:02:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:41 smithi175 bash[29284]: audit 2024-03-26T04:02:40.561668+0000 mon.a (mon.0) 3460 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:02:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:41 smithi175 bash[29284]: cluster 2024-03-26T04:02:40.570455+0000 mon.a (mon.0) 3461 : cluster [DBG] osdmap e727: 8 total, 8 up, 8 in 2024-03-26T04:02:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:41 smithi175 bash[29284]: audit 2024-03-26T04:02:40.572383+0000 mon.c (mon.2) 727 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-146"}]: dispatch 2024-03-26T04:02:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:41 smithi175 bash[29284]: audit 2024-03-26T04:02:40.575906+0000 mon.a (mon.0) 3462 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-146"}]: dispatch 2024-03-26T04:02:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:41 smithi175 bash[29284]: audit 2024-03-26T04:02:41.564746+0000 mon.a (mon.0) 3463 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-146"}]': finished 2024-03-26T04:02:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:41 smithi175 bash[29284]: cluster 2024-03-26T04:02:41.572011+0000 mon.a (mon.0) 3464 : cluster [DBG] osdmap e728: 8 total, 8 up, 8 in 2024-03-26T04:02:41.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:41 smithi175 bash[21527]: audit 2024-03-26T04:02:40.509671+0000 mgr.y (mgr.24431) 698 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:02:41.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:41 smithi175 bash[21527]: audit 2024-03-26T04:02:40.561668+0000 mon.a (mon.0) 3460 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:02:41.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:41 smithi175 bash[21527]: cluster 2024-03-26T04:02:40.570455+0000 mon.a (mon.0) 3461 : cluster [DBG] osdmap e727: 8 total, 8 up, 8 in 2024-03-26T04:02:41.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:41 smithi175 bash[21527]: audit 2024-03-26T04:02:40.572383+0000 mon.c (mon.2) 727 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-146"}]: dispatch 2024-03-26T04:02:41.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:41 smithi175 bash[21527]: audit 2024-03-26T04:02:40.575906+0000 mon.a (mon.0) 3462 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-146"}]: dispatch 2024-03-26T04:02:41.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:41 smithi175 bash[21527]: audit 2024-03-26T04:02:41.564746+0000 mon.a (mon.0) 3463 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-146"}]': finished 2024-03-26T04:02:41.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:41 smithi175 bash[21527]: cluster 2024-03-26T04:02:41.572011+0000 mon.a (mon.0) 3464 : cluster [DBG] osdmap e728: 8 total, 8 up, 8 in 2024-03-26T04:02:41.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:41 smithi190 bash[23392]: audit 2024-03-26T04:02:40.509671+0000 mgr.y (mgr.24431) 698 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:02:41.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:41 smithi190 bash[23392]: audit 2024-03-26T04:02:40.561668+0000 mon.a (mon.0) 3460 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:02:41.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:41 smithi190 bash[23392]: cluster 2024-03-26T04:02:40.570455+0000 mon.a (mon.0) 3461 : cluster [DBG] osdmap e727: 8 total, 8 up, 8 in 2024-03-26T04:02:41.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:41 smithi190 bash[23392]: audit 2024-03-26T04:02:40.572383+0000 mon.c (mon.2) 727 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-146"}]: dispatch 2024-03-26T04:02:41.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:41 smithi190 bash[23392]: audit 2024-03-26T04:02:40.575906+0000 mon.a (mon.0) 3462 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-146"}]: dispatch 2024-03-26T04:02:41.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:41 smithi190 bash[23392]: audit 2024-03-26T04:02:41.564746+0000 mon.a (mon.0) 3463 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-146"}]': finished 2024-03-26T04:02:41.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:41 smithi190 bash[23392]: cluster 2024-03-26T04:02:41.572011+0000 mon.a (mon.0) 3464 : cluster [DBG] osdmap e728: 8 total, 8 up, 8 in 2024-03-26T04:02:42.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:42 smithi175 bash[29284]: cluster 2024-03-26T04:02:41.092127+0000 mgr.y (mgr.24431) 699 : cluster [DBG] pgmap v1176: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T04:02:42.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:42 smithi175 bash[29284]: audit 2024-03-26T04:02:41.647907+0000 mon.c (mon.2) 728 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:42.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:42 smithi175 bash[29284]: audit 2024-03-26T04:02:41.648553+0000 mon.a (mon.0) 3465 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:42.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:42 smithi175 bash[29284]: audit 2024-03-26T04:02:41.649694+0000 mon.c (mon.2) 729 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-146"}]: dispatch 2024-03-26T04:02:42.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:42 smithi175 bash[29284]: audit 2024-03-26T04:02:41.650286+0000 mon.a (mon.0) 3466 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-146"}]: dispatch 2024-03-26T04:02:42.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:42 smithi175 bash[21527]: cluster 2024-03-26T04:02:41.092127+0000 mgr.y (mgr.24431) 699 : cluster [DBG] pgmap v1176: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T04:02:42.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:42 smithi175 bash[21527]: audit 2024-03-26T04:02:41.647907+0000 mon.c (mon.2) 728 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:42.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:42 smithi175 bash[21527]: audit 2024-03-26T04:02:41.648553+0000 mon.a (mon.0) 3465 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:42.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:42 smithi175 bash[21527]: audit 2024-03-26T04:02:41.649694+0000 mon.c (mon.2) 729 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-146"}]: dispatch 2024-03-26T04:02:42.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:42 smithi175 bash[21527]: audit 2024-03-26T04:02:41.650286+0000 mon.a (mon.0) 3466 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-146"}]: dispatch 2024-03-26T04:02:42.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:42 smithi190 bash[23392]: cluster 2024-03-26T04:02:41.092127+0000 mgr.y (mgr.24431) 699 : cluster [DBG] pgmap v1176: 268 pgs: 268 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-26T04:02:42.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:42 smithi190 bash[23392]: audit 2024-03-26T04:02:41.647907+0000 mon.c (mon.2) 728 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:42.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:42 smithi190 bash[23392]: audit 2024-03-26T04:02:41.648553+0000 mon.a (mon.0) 3465 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:42.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:42 smithi190 bash[23392]: audit 2024-03-26T04:02:41.649694+0000 mon.c (mon.2) 729 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-146"}]: dispatch 2024-03-26T04:02:42.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:42 smithi190 bash[23392]: audit 2024-03-26T04:02:41.650286+0000 mon.a (mon.0) 3466 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-146"}]: dispatch 2024-03-26T04:02:43.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:43 smithi175 bash[21527]: cluster 2024-03-26T04:02:42.610216+0000 mon.a (mon.0) 3467 : cluster [DBG] osdmap e729: 8 total, 8 up, 8 in 2024-03-26T04:02:43.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:43 smithi175 bash[29284]: cluster 2024-03-26T04:02:42.610216+0000 mon.a (mon.0) 3467 : cluster [DBG] osdmap e729: 8 total, 8 up, 8 in 2024-03-26T04:02:43.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:43 smithi190 bash[23392]: cluster 2024-03-26T04:02:42.610216+0000 mon.a (mon.0) 3467 : cluster [DBG] osdmap e729: 8 total, 8 up, 8 in 2024-03-26T04:02:44.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:44 smithi175 bash[29284]: cluster 2024-03-26T04:02:43.093107+0000 mgr.y (mgr.24431) 700 : cluster [DBG] pgmap v1179: 236 pgs: 236 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-03-26T04:02:44.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:44 smithi175 bash[29284]: cluster 2024-03-26T04:02:43.603558+0000 mon.a (mon.0) 3468 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-03-26T04:02:44.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:44 smithi175 bash[29284]: cluster 2024-03-26T04:02:43.624842+0000 mon.a (mon.0) 3469 : cluster [DBG] osdmap e730: 8 total, 8 up, 8 in 2024-03-26T04:02:44.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:44 smithi175 bash[29284]: audit 2024-03-26T04:02:43.628862+0000 mon.c (mon.2) 730 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:44.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:44 smithi175 bash[29284]: audit 2024-03-26T04:02:43.630452+0000 mon.a (mon.0) 3470 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:44.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:44 smithi175 bash[21527]: cluster 2024-03-26T04:02:43.093107+0000 mgr.y (mgr.24431) 700 : cluster [DBG] pgmap v1179: 236 pgs: 236 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-03-26T04:02:44.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:44 smithi175 bash[21527]: cluster 2024-03-26T04:02:43.603558+0000 mon.a (mon.0) 3468 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-03-26T04:02:44.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:44 smithi175 bash[21527]: cluster 2024-03-26T04:02:43.624842+0000 mon.a (mon.0) 3469 : cluster [DBG] osdmap e730: 8 total, 8 up, 8 in 2024-03-26T04:02:44.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:44 smithi175 bash[21527]: audit 2024-03-26T04:02:43.628862+0000 mon.c (mon.2) 730 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:44.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:44 smithi175 bash[21527]: audit 2024-03-26T04:02:43.630452+0000 mon.a (mon.0) 3470 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:44.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:44 smithi190 bash[23392]: cluster 2024-03-26T04:02:43.093107+0000 mgr.y (mgr.24431) 700 : cluster [DBG] pgmap v1179: 236 pgs: 236 active+clean; 4.4 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-03-26T04:02:44.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:44 smithi190 bash[23392]: cluster 2024-03-26T04:02:43.603558+0000 mon.a (mon.0) 3468 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-03-26T04:02:44.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:44 smithi190 bash[23392]: cluster 2024-03-26T04:02:43.624842+0000 mon.a (mon.0) 3469 : cluster [DBG] osdmap e730: 8 total, 8 up, 8 in 2024-03-26T04:02:44.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:44 smithi190 bash[23392]: audit 2024-03-26T04:02:43.628862+0000 mon.c (mon.2) 730 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:44.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:44 smithi190 bash[23392]: audit 2024-03-26T04:02:43.630452+0000 mon.a (mon.0) 3470 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:45.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:45 smithi175 bash[21527]: audit 2024-03-26T04:02:44.619847+0000 mon.a (mon.0) 3471 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:02:45.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:45 smithi175 bash[21527]: cluster 2024-03-26T04:02:44.628562+0000 mon.a (mon.0) 3472 : cluster [DBG] osdmap e731: 8 total, 8 up, 8 in 2024-03-26T04:02:45.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:45 smithi175 bash[21527]: audit 2024-03-26T04:02:44.686338+0000 mon.c (mon.2) 731 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:02:45.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:45 smithi175 bash[21527]: audit 2024-03-26T04:02:44.686848+0000 mon.a (mon.0) 3473 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:02:45.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:45 smithi175 bash[29284]: audit 2024-03-26T04:02:44.619847+0000 mon.a (mon.0) 3471 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:02:45.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:45 smithi175 bash[29284]: cluster 2024-03-26T04:02:44.628562+0000 mon.a (mon.0) 3472 : cluster [DBG] osdmap e731: 8 total, 8 up, 8 in 2024-03-26T04:02:45.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:45 smithi175 bash[29284]: audit 2024-03-26T04:02:44.686338+0000 mon.c (mon.2) 731 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:02:45.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:45 smithi175 bash[29284]: audit 2024-03-26T04:02:44.686848+0000 mon.a (mon.0) 3473 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:02:45.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:45 smithi190 bash[23392]: audit 2024-03-26T04:02:44.619847+0000 mon.a (mon.0) 3471 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:02:45.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:45 smithi190 bash[23392]: cluster 2024-03-26T04:02:44.628562+0000 mon.a (mon.0) 3472 : cluster [DBG] osdmap e731: 8 total, 8 up, 8 in 2024-03-26T04:02:45.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:45 smithi190 bash[23392]: audit 2024-03-26T04:02:44.686338+0000 mon.c (mon.2) 731 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:02:45.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:45 smithi190 bash[23392]: audit 2024-03-26T04:02:44.686848+0000 mon.a (mon.0) 3473 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-26T04:02:46.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:46 smithi175 bash[29284]: cluster 2024-03-26T04:02:45.095571+0000 mgr.y (mgr.24431) 701 : cluster [DBG] pgmap v1182: 268 pgs: 268 active+clean; 4.4 MiB data, 794 MiB used, 714 GiB / 715 GiB avail 2024-03-26T04:02:46.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:46 smithi175 bash[29284]: audit 2024-03-26T04:02:45.638117+0000 mon.a (mon.0) 3474 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-148", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:02:46.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:46 smithi175 bash[29284]: cluster 2024-03-26T04:02:45.648232+0000 mon.a (mon.0) 3475 : cluster [DBG] osdmap e732: 8 total, 8 up, 8 in 2024-03-26T04:02:46.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:46 smithi175 bash[29284]: audit 2024-03-26T04:02:45.687170+0000 mon.c (mon.2) 732 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-148"}]: dispatch 2024-03-26T04:02:46.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:46 smithi175 bash[29284]: audit 2024-03-26T04:02:45.688611+0000 mon.a (mon.0) 3476 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-148"}]: dispatch 2024-03-26T04:02:46.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:46 smithi175 bash[21527]: cluster 2024-03-26T04:02:45.095571+0000 mgr.y (mgr.24431) 701 : cluster [DBG] pgmap v1182: 268 pgs: 268 active+clean; 4.4 MiB data, 794 MiB used, 714 GiB / 715 GiB avail 2024-03-26T04:02:46.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:46 smithi175 bash[21527]: audit 2024-03-26T04:02:45.638117+0000 mon.a (mon.0) 3474 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-148", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:02:46.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:46 smithi175 bash[21527]: cluster 2024-03-26T04:02:45.648232+0000 mon.a (mon.0) 3475 : cluster [DBG] osdmap e732: 8 total, 8 up, 8 in 2024-03-26T04:02:46.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:46 smithi175 bash[21527]: audit 2024-03-26T04:02:45.687170+0000 mon.c (mon.2) 732 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-148"}]: dispatch 2024-03-26T04:02:46.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:46 smithi175 bash[21527]: audit 2024-03-26T04:02:45.688611+0000 mon.a (mon.0) 3476 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-148"}]: dispatch 2024-03-26T04:02:46.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:46 smithi190 bash[23392]: cluster 2024-03-26T04:02:45.095571+0000 mgr.y (mgr.24431) 701 : cluster [DBG] pgmap v1182: 268 pgs: 268 active+clean; 4.4 MiB data, 794 MiB used, 714 GiB / 715 GiB avail 2024-03-26T04:02:46.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:46 smithi190 bash[23392]: audit 2024-03-26T04:02:45.638117+0000 mon.a (mon.0) 3474 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-148", "force_nonempty": "--force-nonempty" }]': finished 2024-03-26T04:02:46.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:46 smithi190 bash[23392]: cluster 2024-03-26T04:02:45.648232+0000 mon.a (mon.0) 3475 : cluster [DBG] osdmap e732: 8 total, 8 up, 8 in 2024-03-26T04:02:46.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:46 smithi190 bash[23392]: audit 2024-03-26T04:02:45.687170+0000 mon.c (mon.2) 732 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-148"}]: dispatch 2024-03-26T04:02:46.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:46 smithi190 bash[23392]: audit 2024-03-26T04:02:45.688611+0000 mon.a (mon.0) 3476 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-148"}]: dispatch 2024-03-26T04:02:47.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:47 smithi175 bash[29284]: audit 2024-03-26T04:02:46.670020+0000 mon.a (mon.0) 3477 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-148"}]': finished 2024-03-26T04:02:47.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:47 smithi175 bash[29284]: cluster 2024-03-26T04:02:46.681369+0000 mon.a (mon.0) 3478 : cluster [DBG] osdmap e733: 8 total, 8 up, 8 in 2024-03-26T04:02:47.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:47 smithi175 bash[29284]: audit 2024-03-26T04:02:46.757737+0000 mon.c (mon.2) 733 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:47.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:47 smithi175 bash[29284]: audit 2024-03-26T04:02:46.758613+0000 mon.a (mon.0) 3479 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:47.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:47 smithi175 bash[29284]: audit 2024-03-26T04:02:46.760018+0000 mon.c (mon.2) 734 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-148"}]: dispatch 2024-03-26T04:02:47.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:47 smithi175 bash[29284]: audit 2024-03-26T04:02:46.760694+0000 mon.a (mon.0) 3480 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-148"}]: dispatch 2024-03-26T04:02:47.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:47 smithi175 bash[29284]: cluster 2024-03-26T04:02:47.096556+0000 mgr.y (mgr.24431) 702 : cluster [DBG] pgmap v1185: 268 pgs: 268 active+clean; 4.4 MiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-26T04:02:47.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:47 smithi175 bash[21527]: audit 2024-03-26T04:02:46.670020+0000 mon.a (mon.0) 3477 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-148"}]': finished 2024-03-26T04:02:47.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:47 smithi175 bash[21527]: cluster 2024-03-26T04:02:46.681369+0000 mon.a (mon.0) 3478 : cluster [DBG] osdmap e733: 8 total, 8 up, 8 in 2024-03-26T04:02:47.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:47 smithi175 bash[21527]: audit 2024-03-26T04:02:46.757737+0000 mon.c (mon.2) 733 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:47.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:47 smithi175 bash[21527]: audit 2024-03-26T04:02:46.758613+0000 mon.a (mon.0) 3479 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:47.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:47 smithi175 bash[21527]: audit 2024-03-26T04:02:46.760018+0000 mon.c (mon.2) 734 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-148"}]: dispatch 2024-03-26T04:02:47.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:47 smithi175 bash[21527]: audit 2024-03-26T04:02:46.760694+0000 mon.a (mon.0) 3480 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-148"}]: dispatch 2024-03-26T04:02:47.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:47 smithi175 bash[21527]: cluster 2024-03-26T04:02:47.096556+0000 mgr.y (mgr.24431) 702 : cluster [DBG] pgmap v1185: 268 pgs: 268 active+clean; 4.4 MiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-26T04:02:47.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:47 smithi190 bash[23392]: audit 2024-03-26T04:02:46.670020+0000 mon.a (mon.0) 3477 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-148"}]': finished 2024-03-26T04:02:47.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:47 smithi190 bash[23392]: cluster 2024-03-26T04:02:46.681369+0000 mon.a (mon.0) 3478 : cluster [DBG] osdmap e733: 8 total, 8 up, 8 in 2024-03-26T04:02:47.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:47 smithi190 bash[23392]: audit 2024-03-26T04:02:46.757737+0000 mon.c (mon.2) 733 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:47.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:47 smithi190 bash[23392]: audit 2024-03-26T04:02:46.758613+0000 mon.a (mon.0) 3479 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:47.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:47 smithi190 bash[23392]: audit 2024-03-26T04:02:46.760018+0000 mon.c (mon.2) 734 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-148"}]: dispatch 2024-03-26T04:02:47.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:47 smithi190 bash[23392]: audit 2024-03-26T04:02:46.760694+0000 mon.a (mon.0) 3480 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-148"}]: dispatch 2024-03-26T04:02:47.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:47 smithi190 bash[23392]: cluster 2024-03-26T04:02:47.096556+0000 mgr.y (mgr.24431) 702 : cluster [DBG] pgmap v1185: 268 pgs: 268 active+clean; 4.4 MiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-26T04:02:48.992 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:02:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:02:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:02:48.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:48 smithi175 bash[21527]: cluster 2024-03-26T04:02:47.670567+0000 mon.a (mon.0) 3481 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:02:48.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:48 smithi175 bash[21527]: cluster 2024-03-26T04:02:47.693542+0000 mon.a (mon.0) 3482 : cluster [DBG] osdmap e734: 8 total, 8 up, 8 in 2024-03-26T04:02:48.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:48 smithi175 bash[29284]: cluster 2024-03-26T04:02:47.670567+0000 mon.a (mon.0) 3481 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:02:48.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:48 smithi175 bash[29284]: cluster 2024-03-26T04:02:47.693542+0000 mon.a (mon.0) 3482 : cluster [DBG] osdmap e734: 8 total, 8 up, 8 in 2024-03-26T04:02:48.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:48 smithi190 bash[23392]: cluster 2024-03-26T04:02:47.670567+0000 mon.a (mon.0) 3481 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:02:48.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:48 smithi190 bash[23392]: cluster 2024-03-26T04:02:47.693542+0000 mon.a (mon.0) 3482 : cluster [DBG] osdmap e734: 8 total, 8 up, 8 in 2024-03-26T04:02:49.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:49 smithi175 bash[21527]: cluster 2024-03-26T04:02:48.700445+0000 mon.a (mon.0) 3483 : cluster [DBG] osdmap e735: 8 total, 8 up, 8 in 2024-03-26T04:02:49.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:49 smithi175 bash[21527]: audit 2024-03-26T04:02:48.708244+0000 mon.c (mon.2) 735 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:49.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:49 smithi175 bash[21527]: audit 2024-03-26T04:02:48.711040+0000 mon.a (mon.0) 3484 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:49.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:49 smithi175 bash[21527]: cluster 2024-03-26T04:02:49.097519+0000 mgr.y (mgr.24431) 703 : cluster [DBG] pgmap v1188: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 794 MiB used, 714 GiB / 715 GiB avail 2024-03-26T04:02:49.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:49 smithi175 bash[29284]: cluster 2024-03-26T04:02:48.700445+0000 mon.a (mon.0) 3483 : cluster [DBG] osdmap e735: 8 total, 8 up, 8 in 2024-03-26T04:02:49.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:49 smithi175 bash[29284]: audit 2024-03-26T04:02:48.708244+0000 mon.c (mon.2) 735 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:49.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:49 smithi175 bash[29284]: audit 2024-03-26T04:02:48.711040+0000 mon.a (mon.0) 3484 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:49.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:49 smithi175 bash[29284]: cluster 2024-03-26T04:02:49.097519+0000 mgr.y (mgr.24431) 703 : cluster [DBG] pgmap v1188: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 794 MiB used, 714 GiB / 715 GiB avail 2024-03-26T04:02:49.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:49 smithi190 bash[23392]: cluster 2024-03-26T04:02:48.700445+0000 mon.a (mon.0) 3483 : cluster [DBG] osdmap e735: 8 total, 8 up, 8 in 2024-03-26T04:02:49.993 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:49 smithi190 bash[23392]: audit 2024-03-26T04:02:48.708244+0000 mon.c (mon.2) 735 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:49.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:49 smithi190 bash[23392]: audit 2024-03-26T04:02:48.711040+0000 mon.a (mon.0) 3484 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:49.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:49 smithi190 bash[23392]: cluster 2024-03-26T04:02:49.097519+0000 mgr.y (mgr.24431) 703 : cluster [DBG] pgmap v1188: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 794 MiB used, 714 GiB / 715 GiB avail 2024-03-26T04:02:50.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:50 smithi175 bash[21527]: audit 2024-03-26T04:02:49.702936+0000 mon.a (mon.0) 3485 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:02:50.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:50 smithi175 bash[21527]: cluster 2024-03-26T04:02:49.719505+0000 mon.a (mon.0) 3486 : cluster [DBG] osdmap e736: 8 total, 8 up, 8 in 2024-03-26T04:02:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:50 smithi175 bash[21527]: audit 2024-03-26T04:02:49.804736+0000 mon.c (mon.2) 736 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:50 smithi175 bash[21527]: audit 2024-03-26T04:02:49.805208+0000 mon.a (mon.0) 3487 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:50 smithi175 bash[21527]: audit 2024-03-26T04:02:49.806115+0000 mon.c (mon.2) 737 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-150"}]: dispatch 2024-03-26T04:02:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:50 smithi175 bash[21527]: audit 2024-03-26T04:02:49.806595+0000 mon.a (mon.0) 3488 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-150"}]: dispatch 2024-03-26T04:02:50.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:50 smithi175 bash[21527]: audit 2024-03-26T04:02:50.515815+0000 mgr.y (mgr.24431) 704 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:02:50.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:50 smithi175 bash[29284]: audit 2024-03-26T04:02:49.702936+0000 mon.a (mon.0) 3485 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:02:50.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:50 smithi175 bash[29284]: cluster 2024-03-26T04:02:49.719505+0000 mon.a (mon.0) 3486 : cluster [DBG] osdmap e736: 8 total, 8 up, 8 in 2024-03-26T04:02:50.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:50 smithi175 bash[29284]: audit 2024-03-26T04:02:49.804736+0000 mon.c (mon.2) 736 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:50.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:50 smithi175 bash[29284]: audit 2024-03-26T04:02:49.805208+0000 mon.a (mon.0) 3487 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:50.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:50 smithi175 bash[29284]: audit 2024-03-26T04:02:49.806115+0000 mon.c (mon.2) 737 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-150"}]: dispatch 2024-03-26T04:02:50.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:50 smithi175 bash[29284]: audit 2024-03-26T04:02:49.806595+0000 mon.a (mon.0) 3488 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-150"}]: dispatch 2024-03-26T04:02:50.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:50 smithi175 bash[29284]: audit 2024-03-26T04:02:50.515815+0000 mgr.y (mgr.24431) 704 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:02:50.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:02:50 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:02:50.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:50 smithi190 bash[23392]: audit 2024-03-26T04:02:49.702936+0000 mon.a (mon.0) 3485 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:02:50.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:50 smithi190 bash[23392]: cluster 2024-03-26T04:02:49.719505+0000 mon.a (mon.0) 3486 : cluster [DBG] osdmap e736: 8 total, 8 up, 8 in 2024-03-26T04:02:50.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:50 smithi190 bash[23392]: audit 2024-03-26T04:02:49.804736+0000 mon.c (mon.2) 736 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:50.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:50 smithi190 bash[23392]: audit 2024-03-26T04:02:49.805208+0000 mon.a (mon.0) 3487 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:50.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:50 smithi190 bash[23392]: audit 2024-03-26T04:02:49.806115+0000 mon.c (mon.2) 737 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-150"}]: dispatch 2024-03-26T04:02:50.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:50 smithi190 bash[23392]: audit 2024-03-26T04:02:49.806595+0000 mon.a (mon.0) 3488 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-150"}]: dispatch 2024-03-26T04:02:50.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:50 smithi190 bash[23392]: audit 2024-03-26T04:02:50.515815+0000 mgr.y (mgr.24431) 704 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:02:51.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:51 smithi175 bash[21527]: cluster 2024-03-26T04:02:50.722839+0000 mon.a (mon.0) 3489 : cluster [DBG] osdmap e737: 8 total, 8 up, 8 in 2024-03-26T04:02:51.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:51 smithi175 bash[21527]: cluster 2024-03-26T04:02:51.098380+0000 mgr.y (mgr.24431) 705 : cluster [DBG] pgmap v1191: 236 pgs: 236 active+clean; 4.4 MiB data, 778 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T04:02:51.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:51 smithi175 bash[21527]: audit 2024-03-26T04:02:51.190398+0000 mon.a (mon.0) 3490 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:02:51.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:51 smithi175 bash[21527]: audit 2024-03-26T04:02:51.193282+0000 mon.c (mon.2) 738 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:02:51.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:51 smithi175 bash[29284]: cluster 2024-03-26T04:02:50.722839+0000 mon.a (mon.0) 3489 : cluster [DBG] osdmap e737: 8 total, 8 up, 8 in 2024-03-26T04:02:51.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:51 smithi175 bash[29284]: cluster 2024-03-26T04:02:51.098380+0000 mgr.y (mgr.24431) 705 : cluster [DBG] pgmap v1191: 236 pgs: 236 active+clean; 4.4 MiB data, 778 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T04:02:51.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:51 smithi175 bash[29284]: audit 2024-03-26T04:02:51.190398+0000 mon.a (mon.0) 3490 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:02:51.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:51 smithi175 bash[29284]: audit 2024-03-26T04:02:51.193282+0000 mon.c (mon.2) 738 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:02:51.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:51 smithi190 bash[23392]: cluster 2024-03-26T04:02:50.722839+0000 mon.a (mon.0) 3489 : cluster [DBG] osdmap e737: 8 total, 8 up, 8 in 2024-03-26T04:02:51.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:51 smithi190 bash[23392]: cluster 2024-03-26T04:02:51.098380+0000 mgr.y (mgr.24431) 705 : cluster [DBG] pgmap v1191: 236 pgs: 236 active+clean; 4.4 MiB data, 778 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T04:02:51.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:51 smithi190 bash[23392]: audit 2024-03-26T04:02:51.190398+0000 mon.a (mon.0) 3490 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:02:51.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:51 smithi190 bash[23392]: audit 2024-03-26T04:02:51.193282+0000 mon.c (mon.2) 738 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:02:53.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:52 smithi175 bash[29284]: cluster 2024-03-26T04:02:51.735809+0000 mon.a (mon.0) 3491 : cluster [DBG] osdmap e738: 8 total, 8 up, 8 in 2024-03-26T04:02:53.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:52 smithi175 bash[29284]: audit 2024-03-26T04:02:51.741366+0000 mon.c (mon.2) 739 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:53.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:52 smithi175 bash[29284]: audit 2024-03-26T04:02:51.743050+0000 mon.a (mon.0) 3492 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:53.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:52 smithi175 bash[21527]: cluster 2024-03-26T04:02:51.735809+0000 mon.a (mon.0) 3491 : cluster [DBG] osdmap e738: 8 total, 8 up, 8 in 2024-03-26T04:02:53.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:52 smithi175 bash[21527]: audit 2024-03-26T04:02:51.741366+0000 mon.c (mon.2) 739 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:53.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:52 smithi175 bash[21527]: audit 2024-03-26T04:02:51.743050+0000 mon.a (mon.0) 3492 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:53.243 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:52 smithi190 bash[23392]: cluster 2024-03-26T04:02:51.735809+0000 mon.a (mon.0) 3491 : cluster [DBG] osdmap e738: 8 total, 8 up, 8 in 2024-03-26T04:02:53.243 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:52 smithi190 bash[23392]: audit 2024-03-26T04:02:51.741366+0000 mon.c (mon.2) 739 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:53.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:52 smithi190 bash[23392]: audit 2024-03-26T04:02:51.743050+0000 mon.a (mon.0) 3492 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:54.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:53 smithi175 bash[21527]: audit 2024-03-26T04:02:52.729933+0000 mon.a (mon.0) 3493 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:02:54.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:53 smithi175 bash[21527]: cluster 2024-03-26T04:02:52.738336+0000 mon.a (mon.0) 3494 : cluster [DBG] osdmap e739: 8 total, 8 up, 8 in 2024-03-26T04:02:54.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:53 smithi175 bash[21527]: audit 2024-03-26T04:02:52.849536+0000 mon.c (mon.2) 740 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:54.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:53 smithi175 bash[21527]: audit 2024-03-26T04:02:52.850177+0000 mon.a (mon.0) 3495 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:54.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:53 smithi175 bash[21527]: audit 2024-03-26T04:02:52.850996+0000 mon.c (mon.2) 741 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-152"}]: dispatch 2024-03-26T04:02:54.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:53 smithi175 bash[21527]: audit 2024-03-26T04:02:52.851516+0000 mon.a (mon.0) 3496 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-152"}]: dispatch 2024-03-26T04:02:54.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:53 smithi175 bash[21527]: cluster 2024-03-26T04:02:53.099272+0000 mgr.y (mgr.24431) 706 : cluster [DBG] pgmap v1194: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 778 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-26T04:02:54.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:53 smithi175 bash[29284]: audit 2024-03-26T04:02:52.729933+0000 mon.a (mon.0) 3493 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:02:54.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:53 smithi175 bash[29284]: cluster 2024-03-26T04:02:52.738336+0000 mon.a (mon.0) 3494 : cluster [DBG] osdmap e739: 8 total, 8 up, 8 in 2024-03-26T04:02:54.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:53 smithi175 bash[29284]: audit 2024-03-26T04:02:52.849536+0000 mon.c (mon.2) 740 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:54.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:53 smithi175 bash[29284]: audit 2024-03-26T04:02:52.850177+0000 mon.a (mon.0) 3495 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:54.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:53 smithi175 bash[29284]: audit 2024-03-26T04:02:52.850996+0000 mon.c (mon.2) 741 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-152"}]: dispatch 2024-03-26T04:02:54.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:53 smithi175 bash[29284]: audit 2024-03-26T04:02:52.851516+0000 mon.a (mon.0) 3496 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-152"}]: dispatch 2024-03-26T04:02:54.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:53 smithi175 bash[29284]: cluster 2024-03-26T04:02:53.099272+0000 mgr.y (mgr.24431) 706 : cluster [DBG] pgmap v1194: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 778 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-26T04:02:54.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:53 smithi190 bash[23392]: audit 2024-03-26T04:02:52.729933+0000 mon.a (mon.0) 3493 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:02:54.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:53 smithi190 bash[23392]: cluster 2024-03-26T04:02:52.738336+0000 mon.a (mon.0) 3494 : cluster [DBG] osdmap e739: 8 total, 8 up, 8 in 2024-03-26T04:02:54.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:53 smithi190 bash[23392]: audit 2024-03-26T04:02:52.849536+0000 mon.c (mon.2) 740 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:54.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:53 smithi190 bash[23392]: audit 2024-03-26T04:02:52.850177+0000 mon.a (mon.0) 3495 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:54.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:53 smithi190 bash[23392]: audit 2024-03-26T04:02:52.850996+0000 mon.c (mon.2) 741 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-152"}]: dispatch 2024-03-26T04:02:54.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:53 smithi190 bash[23392]: audit 2024-03-26T04:02:52.851516+0000 mon.a (mon.0) 3496 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-152"}]: dispatch 2024-03-26T04:02:54.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:53 smithi190 bash[23392]: cluster 2024-03-26T04:02:53.099272+0000 mgr.y (mgr.24431) 706 : cluster [DBG] pgmap v1194: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 778 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-26T04:02:55.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:54 smithi175 bash[21527]: cluster 2024-03-26T04:02:53.771327+0000 mon.a (mon.0) 3497 : cluster [DBG] osdmap e740: 8 total, 8 up, 8 in 2024-03-26T04:02:55.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:54 smithi175 bash[29284]: cluster 2024-03-26T04:02:53.771327+0000 mon.a (mon.0) 3497 : cluster [DBG] osdmap e740: 8 total, 8 up, 8 in 2024-03-26T04:02:55.243 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:54 smithi190 bash[23392]: cluster 2024-03-26T04:02:53.771327+0000 mon.a (mon.0) 3497 : cluster [DBG] osdmap e740: 8 total, 8 up, 8 in 2024-03-26T04:02:56.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:55 smithi175 bash[29284]: cluster 2024-03-26T04:02:54.776818+0000 mon.a (mon.0) 3498 : cluster [DBG] osdmap e741: 8 total, 8 up, 8 in 2024-03-26T04:02:56.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:55 smithi175 bash[29284]: audit 2024-03-26T04:02:54.784545+0000 mon.c (mon.2) 742 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:56.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:55 smithi175 bash[29284]: audit 2024-03-26T04:02:54.785998+0000 mon.a (mon.0) 3499 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:56.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:55 smithi175 bash[29284]: cluster 2024-03-26T04:02:55.101824+0000 mgr.y (mgr.24431) 707 : cluster [DBG] pgmap v1197: 268 pgs: 30 creating+peering, 2 unknown, 236 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-26T04:02:56.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:55 smithi175 bash[21527]: cluster 2024-03-26T04:02:54.776818+0000 mon.a (mon.0) 3498 : cluster [DBG] osdmap e741: 8 total, 8 up, 8 in 2024-03-26T04:02:56.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:55 smithi175 bash[21527]: audit 2024-03-26T04:02:54.784545+0000 mon.c (mon.2) 742 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:56.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:55 smithi175 bash[21527]: audit 2024-03-26T04:02:54.785998+0000 mon.a (mon.0) 3499 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:56.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:55 smithi175 bash[21527]: cluster 2024-03-26T04:02:55.101824+0000 mgr.y (mgr.24431) 707 : cluster [DBG] pgmap v1197: 268 pgs: 30 creating+peering, 2 unknown, 236 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-26T04:02:56.243 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:55 smithi190 bash[23392]: cluster 2024-03-26T04:02:54.776818+0000 mon.a (mon.0) 3498 : cluster [DBG] osdmap e741: 8 total, 8 up, 8 in 2024-03-26T04:02:56.243 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:55 smithi190 bash[23392]: audit 2024-03-26T04:02:54.784545+0000 mon.c (mon.2) 742 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:56.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:55 smithi190 bash[23392]: audit 2024-03-26T04:02:54.785998+0000 mon.a (mon.0) 3499 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-26T04:02:56.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:55 smithi190 bash[23392]: cluster 2024-03-26T04:02:55.101824+0000 mgr.y (mgr.24431) 707 : cluster [DBG] pgmap v1197: 268 pgs: 30 creating+peering, 2 unknown, 236 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-26T04:02:57.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:56 smithi175 bash[21527]: cluster 2024-03-26T04:02:55.769191+0000 mon.a (mon.0) 3500 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:02:57.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:56 smithi175 bash[21527]: audit 2024-03-26T04:02:55.780703+0000 mon.a (mon.0) 3501 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:02:57.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:56 smithi175 bash[21527]: cluster 2024-03-26T04:02:55.793298+0000 mon.a (mon.0) 3502 : cluster [DBG] osdmap e742: 8 total, 8 up, 8 in 2024-03-26T04:02:57.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:56 smithi175 bash[21527]: audit 2024-03-26T04:02:55.798888+0000 mon.c (mon.2) 743 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-111","var": "dedup_tier","val": "test-rados-api-smithi175-64594-154"}]: dispatch 2024-03-26T04:02:57.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:56 smithi175 bash[21527]: audit 2024-03-26T04:02:55.800013+0000 mon.a (mon.0) 3503 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-111","var": "dedup_tier","val": "test-rados-api-smithi175-64594-154"}]: dispatch 2024-03-26T04:02:57.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:56 smithi175 bash[21527]: audit 2024-03-26T04:02:55.867511+0000 mon.c (mon.2) 744 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:57.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:56 smithi175 bash[21527]: audit 2024-03-26T04:02:55.868064+0000 mon.a (mon.0) 3504 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:57.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:56 smithi175 bash[21527]: audit 2024-03-26T04:02:55.869103+0000 mon.c (mon.2) 745 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-154"}]: dispatch 2024-03-26T04:02:57.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:56 smithi175 bash[21527]: audit 2024-03-26T04:02:55.869620+0000 mon.a (mon.0) 3505 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-154"}]: dispatch 2024-03-26T04:02:57.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:56 smithi175 bash[29284]: cluster 2024-03-26T04:02:55.769191+0000 mon.a (mon.0) 3500 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:02:57.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:56 smithi175 bash[29284]: audit 2024-03-26T04:02:55.780703+0000 mon.a (mon.0) 3501 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:02:57.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:56 smithi175 bash[29284]: cluster 2024-03-26T04:02:55.793298+0000 mon.a (mon.0) 3502 : cluster [DBG] osdmap e742: 8 total, 8 up, 8 in 2024-03-26T04:02:57.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:56 smithi175 bash[29284]: audit 2024-03-26T04:02:55.798888+0000 mon.c (mon.2) 743 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-111","var": "dedup_tier","val": "test-rados-api-smithi175-64594-154"}]: dispatch 2024-03-26T04:02:57.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:56 smithi175 bash[29284]: audit 2024-03-26T04:02:55.800013+0000 mon.a (mon.0) 3503 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-111","var": "dedup_tier","val": "test-rados-api-smithi175-64594-154"}]: dispatch 2024-03-26T04:02:57.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:56 smithi175 bash[29284]: audit 2024-03-26T04:02:55.867511+0000 mon.c (mon.2) 744 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:57.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:56 smithi175 bash[29284]: audit 2024-03-26T04:02:55.868064+0000 mon.a (mon.0) 3504 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:57.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:56 smithi175 bash[29284]: audit 2024-03-26T04:02:55.869103+0000 mon.c (mon.2) 745 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-154"}]: dispatch 2024-03-26T04:02:57.244 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:56 smithi175 bash[29284]: audit 2024-03-26T04:02:55.869620+0000 mon.a (mon.0) 3505 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-154"}]: dispatch 2024-03-26T04:02:57.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:56 smithi190 bash[23392]: cluster 2024-03-26T04:02:55.769191+0000 mon.a (mon.0) 3500 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:02:57.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:56 smithi190 bash[23392]: audit 2024-03-26T04:02:55.780703+0000 mon.a (mon.0) 3501 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi175-64594-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-26T04:02:57.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:56 smithi190 bash[23392]: cluster 2024-03-26T04:02:55.793298+0000 mon.a (mon.0) 3502 : cluster [DBG] osdmap e742: 8 total, 8 up, 8 in 2024-03-26T04:02:57.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:56 smithi190 bash[23392]: audit 2024-03-26T04:02:55.798888+0000 mon.c (mon.2) 743 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-111","var": "dedup_tier","val": "test-rados-api-smithi175-64594-154"}]: dispatch 2024-03-26T04:02:57.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:56 smithi190 bash[23392]: audit 2024-03-26T04:02:55.800013+0000 mon.a (mon.0) 3503 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi175-64594-111","var": "dedup_tier","val": "test-rados-api-smithi175-64594-154"}]: dispatch 2024-03-26T04:02:57.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:56 smithi190 bash[23392]: audit 2024-03-26T04:02:55.867511+0000 mon.c (mon.2) 744 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:57.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:56 smithi190 bash[23392]: audit 2024-03-26T04:02:55.868064+0000 mon.a (mon.0) 3504 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:57.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:56 smithi190 bash[23392]: audit 2024-03-26T04:02:55.869103+0000 mon.c (mon.2) 745 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-154"}]: dispatch 2024-03-26T04:02:57.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:56 smithi190 bash[23392]: audit 2024-03-26T04:02:55.869620+0000 mon.a (mon.0) 3505 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi175-64594-111", "tierpool": "test-rados-api-smithi175-64594-154"}]: dispatch 2024-03-26T04:02:58.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:57 smithi175 bash[21527]: cluster 2024-03-26T04:02:56.803982+0000 mon.a (mon.0) 3506 : cluster [DBG] osdmap e743: 8 total, 8 up, 8 in 2024-03-26T04:02:58.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:57 smithi175 bash[21527]: cluster 2024-03-26T04:02:57.102863+0000 mgr.y (mgr.24431) 708 : cluster [DBG] pgmap v1200: 236 pgs: 236 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-26T04:02:58.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:57 smithi175 bash[29284]: cluster 2024-03-26T04:02:56.803982+0000 mon.a (mon.0) 3506 : cluster [DBG] osdmap e743: 8 total, 8 up, 8 in 2024-03-26T04:02:58.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:57 smithi175 bash[29284]: cluster 2024-03-26T04:02:57.102863+0000 mgr.y (mgr.24431) 708 : cluster [DBG] pgmap v1200: 236 pgs: 236 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-26T04:02:58.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:57 smithi190 bash[23392]: cluster 2024-03-26T04:02:56.803982+0000 mon.a (mon.0) 3506 : cluster [DBG] osdmap e743: 8 total, 8 up, 8 in 2024-03-26T04:02:58.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:57 smithi190 bash[23392]: cluster 2024-03-26T04:02:57.102863+0000 mgr.y (mgr.24431) 708 : cluster [DBG] pgmap v1200: 236 pgs: 236 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-26T04:02:59.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:02:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:02:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:02:59.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:58 smithi175 bash[21527]: cluster 2024-03-26T04:02:57.815317+0000 mon.a (mon.0) 3507 : cluster [DBG] osdmap e744: 8 total, 8 up, 8 in 2024-03-26T04:02:59.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:58 smithi175 bash[21527]: audit 2024-03-26T04:02:57.818622+0000 mon.c (mon.2) 746 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:59.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:58 smithi175 bash[21527]: audit 2024-03-26T04:02:57.819620+0000 mon.a (mon.0) 3508 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:59.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:58 smithi175 bash[29284]: cluster 2024-03-26T04:02:57.815317+0000 mon.a (mon.0) 3507 : cluster [DBG] osdmap e744: 8 total, 8 up, 8 in 2024-03-26T04:02:59.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:58 smithi175 bash[29284]: audit 2024-03-26T04:02:57.818622+0000 mon.c (mon.2) 746 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:59.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:58 smithi175 bash[29284]: audit 2024-03-26T04:02:57.819620+0000 mon.a (mon.0) 3508 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:59.243 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:58 smithi190 bash[23392]: cluster 2024-03-26T04:02:57.815317+0000 mon.a (mon.0) 3507 : cluster [DBG] osdmap e744: 8 total, 8 up, 8 in 2024-03-26T04:02:59.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:58 smithi190 bash[23392]: audit 2024-03-26T04:02:57.818622+0000 mon.c (mon.2) 746 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:59.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:58 smithi190 bash[23392]: audit 2024-03-26T04:02:57.819620+0000 mon.a (mon.0) 3508 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:02:59.842 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlush (7688 ms) 2024-03-26T04:02:59.843 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-03-26T04:02:59.843 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (12344 ms) 2024-03-26T04:02:59.843 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2024-03-26T04:02:59.843 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (8065 ms) 2024-03-26T04:02:59.843 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-03-26T04:02:59.843 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (12603 ms) 2024-03-26T04:02:59.843 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-03-26T04:02:59.843 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (8088 ms) 2024-03-26T04:02:59.843 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-03-26T04:02:59.843 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (8084 ms) 2024-03-26T04:02:59.843 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-03-26T04:02:59.843 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: hmm, no HitSet yet 2024-03-26T04:02:59.844 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: ok, hit_set contains 330:602f83fe:::foo:head 2024-03-26T04:02:59.844 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (8176 ms) 2024-03-26T04:02:59.844 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-03-26T04:02:59.844 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: got ls 1711425694,0 2024-03-26T04:02:59.844 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: first is 1711425694 2024-03-26T04:02:59.844 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: got ls 1711425694,0 2024-03-26T04:02:59.844 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: got ls 1711425694,0 2024-03-26T04:02:59.844 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: got ls 1711425694,1711425696,1711425697,0 2024-03-26T04:02:59.844 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: got ls 1711425694,1711425696,1711425697,0 2024-03-26T04:02:59.844 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: got ls 1711425694,1711425696,1711425697,0 2024-03-26T04:02:59.844 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: got ls 1711425694,1711425696,1711425697,1711425699,1711425700,0 2024-03-26T04:02:59.844 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: got ls 1711425694,1711425696,1711425697,1711425699,1711425700,0 2024-03-26T04:02:59.844 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: got ls 1711425694,1711425696,1711425697,1711425699,1711425700,0 2024-03-26T04:02:59.844 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: got ls 1711425694,1711425696,1711425697,1711425699,1711425700,1711425702,1711425703,0 2024-03-26T04:02:59.844 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: got ls 1711425694,1711425696,1711425697,1711425699,1711425700,1711425702,1711425703,0 2024-03-26T04:02:59.845 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: got ls 1711425694,1711425696,1711425697,1711425699,1711425700,1711425702,1711425703,0 2024-03-26T04:02:59.845 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: got ls 1711425697,1711425699,1711425700,1711425702,1711425703,1711425705,1711425706,0 2024-03-26T04:02:59.845 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: first now 1711425697, trimmed 2024-03-26T04:02:59.845 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (21131 ms) 2024-03-26T04:02:59.845 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-03-26T04:02:59.845 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: foo0 2024-03-26T04:02:59.845 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-03-26T04:02:59.845 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (14135 ms) 2024-03-26T04:02:59.845 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-03-26T04:02:59.845 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (18120 ms) 2024-03-26T04:02:59.845 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-03-26T04:02:59.845 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (23210 ms) 2024-03-26T04:02:59.845 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-03-26T04:02:59.845 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (5083 ms) 2024-03-26T04:02:59.845 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-03-26T04:02:59.846 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (3026 ms) 2024-03-26T04:02:59.846 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-03-26T04:02:59.846 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (3048 ms) 2024-03-26T04:02:59.846 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-03-26T04:02:59.846 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (3033 ms) 2024-03-26T04:02:59.846 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (232087 ms total) 2024-03-26T04:02:59.846 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: 2024-03-26T04:02:59.846 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [----------] Global test environment tear-down 2024-03-26T04:02:59.846 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [==========] 77 tests from 4 test suites ran. (847518 ms total) 2024-03-26T04:02:59.846 INFO:tasks.workunit.client.0.smithi175.stdout: api_tier_pp: [ PASSED ] 77 tests. 2024-03-26T04:02:59.849 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:02:59.849 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64425 2024-03-26T04:02:59.849 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 64425 2024-03-26T04:02:59.849 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:02:59.850 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64617 2024-03-26T04:02:59.850 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 64617 2024-03-26T04:02:59.850 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:02:59.850 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64964 2024-03-26T04:02:59.850 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 64964 2024-03-26T04:02:59.850 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:02:59.850 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64829 2024-03-26T04:02:59.850 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 64829 2024-03-26T04:02:59.850 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:02:59.850 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64661 2024-03-26T04:02:59.850 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 64661 2024-03-26T04:02:59.851 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:02:59.851 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64520 2024-03-26T04:02:59.851 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 64520 2024-03-26T04:02:59.851 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:02:59.851 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64767 2024-03-26T04:02:59.851 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 64767 2024-03-26T04:02:59.851 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:02:59.851 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=65162 2024-03-26T04:02:59.851 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 65162 2024-03-26T04:02:59.851 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:02:59.851 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=65213 2024-03-26T04:02:59.851 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 65213 2024-03-26T04:02:59.851 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:02:59.852 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64541 2024-03-26T04:02:59.852 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 64541 2024-03-26T04:02:59.852 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:02:59.852 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64976 2024-03-26T04:02:59.852 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 64976 2024-03-26T04:02:59.852 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:02:59.852 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64914 2024-03-26T04:02:59.852 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 64914 2024-03-26T04:02:59.852 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:02:59.852 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64409 2024-03-26T04:02:59.853 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 64409 2024-03-26T04:02:59.853 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:02:59.853 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64456 2024-03-26T04:02:59.853 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 64456 2024-03-26T04:02:59.853 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:02:59.853 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64943 2024-03-26T04:02:59.853 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 64943 2024-03-26T04:02:59.853 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:02:59.853 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64401 2024-03-26T04:02:59.853 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 64401 2024-03-26T04:02:59.853 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:02:59.853 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64417 2024-03-26T04:02:59.853 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 64417 2024-03-26T04:02:59.853 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:02:59.854 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=65320 2024-03-26T04:02:59.854 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 65320 2024-03-26T04:02:59.855 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:02:59.855 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64630 2024-03-26T04:02:59.855 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 64630 2024-03-26T04:02:59.855 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:02:59.855 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=65343 2024-03-26T04:02:59.855 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 65343 2024-03-26T04:02:59.856 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:02:59.856 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64713 2024-03-26T04:02:59.856 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 64713 2024-03-26T04:02:59.856 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:02:59.856 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=65087 2024-03-26T04:02:59.856 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 65087 2024-03-26T04:02:59.856 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:02:59.856 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=65427 2024-03-26T04:02:59.856 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 65427 2024-03-26T04:03:00.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:59 smithi175 bash[21527]: audit 2024-03-26T04:02:58.805819+0000 mon.a (mon.0) 3509 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:03:00.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:59 smithi175 bash[21527]: cluster 2024-03-26T04:02:58.815626+0000 mon.a (mon.0) 3510 : cluster [DBG] osdmap e745: 8 total, 8 up, 8 in 2024-03-26T04:03:00.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:59 smithi175 bash[21527]: audit 2024-03-26T04:02:58.817257+0000 mon.c (mon.2) 747 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:03:00.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:59 smithi175 bash[21527]: audit 2024-03-26T04:02:58.818690+0000 mon.a (mon.0) 3511 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:03:00.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:59 smithi175 bash[21527]: cluster 2024-03-26T04:02:59.103763+0000 mgr.y (mgr.24431) 709 : cluster [DBG] pgmap v1203: 228 pgs: 228 active+clean; 582 KiB data, 783 MiB used, 714 GiB / 715 GiB avail 2024-03-26T04:03:00.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:02:59 smithi175 bash[21527]: audit 2024-03-26T04:02:59.809353+0000 mon.a (mon.0) 3512 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:03:00.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:59 smithi175 bash[29284]: audit 2024-03-26T04:02:58.805819+0000 mon.a (mon.0) 3509 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:03:00.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:59 smithi175 bash[29284]: cluster 2024-03-26T04:02:58.815626+0000 mon.a (mon.0) 3510 : cluster [DBG] osdmap e745: 8 total, 8 up, 8 in 2024-03-26T04:03:00.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:59 smithi175 bash[29284]: audit 2024-03-26T04:02:58.817257+0000 mon.c (mon.2) 747 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:03:00.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:59 smithi175 bash[29284]: audit 2024-03-26T04:02:58.818690+0000 mon.a (mon.0) 3511 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:03:00.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:59 smithi175 bash[29284]: cluster 2024-03-26T04:02:59.103763+0000 mgr.y (mgr.24431) 709 : cluster [DBG] pgmap v1203: 228 pgs: 228 active+clean; 582 KiB data, 783 MiB used, 714 GiB / 715 GiB avail 2024-03-26T04:03:00.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:02:59 smithi175 bash[29284]: audit 2024-03-26T04:02:59.809353+0000 mon.a (mon.0) 3512 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:03:00.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:59 smithi190 bash[23392]: audit 2024-03-26T04:02:58.805819+0000 mon.a (mon.0) 3509 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:03:00.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:59 smithi190 bash[23392]: cluster 2024-03-26T04:02:58.815626+0000 mon.a (mon.0) 3510 : cluster [DBG] osdmap e745: 8 total, 8 up, 8 in 2024-03-26T04:03:00.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:59 smithi190 bash[23392]: audit 2024-03-26T04:02:58.817257+0000 mon.c (mon.2) 747 : audit [INF] from='client.? 172.21.15.175:0/2342610817' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:03:00.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:59 smithi190 bash[23392]: audit 2024-03-26T04:02:58.818690+0000 mon.a (mon.0) 3511 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64594-111"}]: dispatch 2024-03-26T04:03:00.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:59 smithi190 bash[23392]: cluster 2024-03-26T04:02:59.103763+0000 mgr.y (mgr.24431) 709 : cluster [DBG] pgmap v1203: 228 pgs: 228 active+clean; 582 KiB data, 783 MiB used, 714 GiB / 715 GiB avail 2024-03-26T04:03:00.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:02:59 smithi190 bash[23392]: audit 2024-03-26T04:02:59.809353+0000 mon.a (mon.0) 3512 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi175-64594-111"}]': finished 2024-03-26T04:03:00.829 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:03:00 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:03:01.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:00 smithi175 bash[21527]: cluster 2024-03-26T04:02:59.819056+0000 mon.a (mon.0) 3513 : cluster [DBG] osdmap e746: 8 total, 8 up, 8 in 2024-03-26T04:03:01.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:00 smithi175 bash[21527]: audit 2024-03-26T04:03:00.523932+0000 mgr.y (mgr.24431) 710 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:03:01.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:00 smithi175 bash[29284]: cluster 2024-03-26T04:02:59.819056+0000 mon.a (mon.0) 3513 : cluster [DBG] osdmap e746: 8 total, 8 up, 8 in 2024-03-26T04:03:01.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:00 smithi175 bash[29284]: audit 2024-03-26T04:03:00.523932+0000 mgr.y (mgr.24431) 710 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:03:01.243 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:00 smithi190 bash[23392]: cluster 2024-03-26T04:02:59.819056+0000 mon.a (mon.0) 3513 : cluster [DBG] osdmap e746: 8 total, 8 up, 8 in 2024-03-26T04:03:01.243 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:00 smithi190 bash[23392]: audit 2024-03-26T04:03:00.523932+0000 mgr.y (mgr.24431) 710 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:03:02.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:01 smithi175 bash[29284]: cluster 2024-03-26T04:03:01.105161+0000 mgr.y (mgr.24431) 711 : cluster [DBG] pgmap v1205: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail 2024-03-26T04:03:02.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:01 smithi175 bash[29284]: cluster 2024-03-26T04:03:01.696779+0000 mon.a (mon.0) 3514 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:03:02.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:01 smithi175 bash[21527]: cluster 2024-03-26T04:03:01.105161+0000 mgr.y (mgr.24431) 711 : cluster [DBG] pgmap v1205: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail 2024-03-26T04:03:02.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:01 smithi175 bash[21527]: cluster 2024-03-26T04:03:01.696779+0000 mon.a (mon.0) 3514 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:03:02.243 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:01 smithi190 bash[23392]: cluster 2024-03-26T04:03:01.105161+0000 mgr.y (mgr.24431) 711 : cluster [DBG] pgmap v1205: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail 2024-03-26T04:03:02.244 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:01 smithi190 bash[23392]: cluster 2024-03-26T04:03:01.696779+0000 mon.a (mon.0) 3514 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:03:04.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:04 smithi175 bash[29284]: cluster 2024-03-26T04:03:03.105977+0000 mgr.y (mgr.24431) 712 : cluster [DBG] pgmap v1206: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail 2024-03-26T04:03:04.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:04 smithi175 bash[21527]: cluster 2024-03-26T04:03:03.105977+0000 mgr.y (mgr.24431) 712 : cluster [DBG] pgmap v1206: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail 2024-03-26T04:03:04.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:04 smithi190 bash[23392]: cluster 2024-03-26T04:03:03.105977+0000 mgr.y (mgr.24431) 712 : cluster [DBG] pgmap v1206: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail 2024-03-26T04:03:06.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:06 smithi175 bash[21527]: cluster 2024-03-26T04:03:05.108238+0000 mgr.y (mgr.24431) 713 : cluster [DBG] pgmap v1207: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-26T04:03:06.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:06 smithi175 bash[29284]: cluster 2024-03-26T04:03:05.108238+0000 mgr.y (mgr.24431) 713 : cluster [DBG] pgmap v1207: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-26T04:03:06.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:06 smithi190 bash[23392]: cluster 2024-03-26T04:03:05.108238+0000 mgr.y (mgr.24431) 713 : cluster [DBG] pgmap v1207: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-26T04:03:07.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:07 smithi175 bash[29284]: audit 2024-03-26T04:03:06.227508+0000 mon.c (mon.2) 748 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:03:07.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:07 smithi175 bash[21527]: audit 2024-03-26T04:03:06.227508+0000 mon.c (mon.2) 748 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:03:07.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:07 smithi190 bash[23392]: audit 2024-03-26T04:03:06.227508+0000 mon.c (mon.2) 748 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:03:08.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:08 smithi175 bash[29284]: cluster 2024-03-26T04:03:07.109093+0000 mgr.y (mgr.24431) 714 : cluster [DBG] pgmap v1208: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:03:08.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:08 smithi175 bash[21527]: cluster 2024-03-26T04:03:07.109093+0000 mgr.y (mgr.24431) 714 : cluster [DBG] pgmap v1208: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:03:08.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:08 smithi190 bash[23392]: cluster 2024-03-26T04:03:07.109093+0000 mgr.y (mgr.24431) 714 : cluster [DBG] pgmap v1208: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:03:09.181 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:03:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:03:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:03:09.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:09 smithi175 bash[21527]: audit 2024-03-26T04:03:08.323347+0000 mon.c (mon.2) 749 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:03:09.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:09 smithi175 bash[21527]: audit 2024-03-26T04:03:09.066470+0000 mon.c (mon.2) 750 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:03:09.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:09 smithi175 bash[21527]: audit 2024-03-26T04:03:09.068211+0000 mon.c (mon.2) 751 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:03:09.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:09 smithi175 bash[21527]: audit 2024-03-26T04:03:09.078427+0000 mon.a (mon.0) 3515 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:03:09.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:09 smithi175 bash[29284]: audit 2024-03-26T04:03:08.323347+0000 mon.c (mon.2) 749 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:03:09.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:09 smithi175 bash[29284]: audit 2024-03-26T04:03:09.066470+0000 mon.c (mon.2) 750 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:03:09.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:09 smithi175 bash[29284]: audit 2024-03-26T04:03:09.068211+0000 mon.c (mon.2) 751 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:03:09.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:09 smithi175 bash[29284]: audit 2024-03-26T04:03:09.078427+0000 mon.a (mon.0) 3515 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:03:09.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:09 smithi190 bash[23392]: audit 2024-03-26T04:03:08.323347+0000 mon.c (mon.2) 749 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:03:09.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:09 smithi190 bash[23392]: audit 2024-03-26T04:03:09.066470+0000 mon.c (mon.2) 750 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:03:09.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:09 smithi190 bash[23392]: audit 2024-03-26T04:03:09.068211+0000 mon.c (mon.2) 751 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:03:09.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:09 smithi190 bash[23392]: audit 2024-03-26T04:03:09.078427+0000 mon.a (mon.0) 3515 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:03:10.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:10 smithi175 bash[29284]: cluster 2024-03-26T04:03:09.109704+0000 mgr.y (mgr.24431) 715 : cluster [DBG] pgmap v1209: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T04:03:10.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:10 smithi175 bash[21527]: cluster 2024-03-26T04:03:09.109704+0000 mgr.y (mgr.24431) 715 : cluster [DBG] pgmap v1209: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T04:03:10.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:10 smithi190 bash[23392]: cluster 2024-03-26T04:03:09.109704+0000 mgr.y (mgr.24431) 715 : cluster [DBG] pgmap v1209: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T04:03:10.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:03:10 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:03:11.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:11 smithi175 bash[29284]: audit 2024-03-26T04:03:10.535570+0000 mgr.y (mgr.24431) 716 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:03:11.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:11 smithi175 bash[21527]: audit 2024-03-26T04:03:10.535570+0000 mgr.y (mgr.24431) 716 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:03:11.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:11 smithi190 bash[23392]: audit 2024-03-26T04:03:10.535570+0000 mgr.y (mgr.24431) 716 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:03:12.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:12 smithi175 bash[21527]: cluster 2024-03-26T04:03:11.111937+0000 mgr.y (mgr.24431) 717 : cluster [DBG] pgmap v1210: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-26T04:03:12.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:12 smithi175 bash[29284]: cluster 2024-03-26T04:03:11.111937+0000 mgr.y (mgr.24431) 717 : cluster [DBG] pgmap v1210: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-26T04:03:12.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:12 smithi190 bash[23392]: cluster 2024-03-26T04:03:11.111937+0000 mgr.y (mgr.24431) 717 : cluster [DBG] pgmap v1210: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-26T04:03:14.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:14 smithi175 bash[29284]: cluster 2024-03-26T04:03:13.112825+0000 mgr.y (mgr.24431) 718 : cluster [DBG] pgmap v1211: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:14.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:14 smithi175 bash[21527]: cluster 2024-03-26T04:03:13.112825+0000 mgr.y (mgr.24431) 718 : cluster [DBG] pgmap v1211: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:14.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:14 smithi190 bash[23392]: cluster 2024-03-26T04:03:13.112825+0000 mgr.y (mgr.24431) 718 : cluster [DBG] pgmap v1211: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:16.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:16 smithi175 bash[29284]: cluster 2024-03-26T04:03:15.115069+0000 mgr.y (mgr.24431) 719 : cluster [DBG] pgmap v1212: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:03:16.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:16 smithi175 bash[21527]: cluster 2024-03-26T04:03:15.115069+0000 mgr.y (mgr.24431) 719 : cluster [DBG] pgmap v1212: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:03:16.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:16 smithi190 bash[23392]: cluster 2024-03-26T04:03:15.115069+0000 mgr.y (mgr.24431) 719 : cluster [DBG] pgmap v1212: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:03:18.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:18 smithi175 bash[29284]: cluster 2024-03-26T04:03:17.115889+0000 mgr.y (mgr.24431) 720 : cluster [DBG] pgmap v1213: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:18.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:18 smithi175 bash[21527]: cluster 2024-03-26T04:03:17.115889+0000 mgr.y (mgr.24431) 720 : cluster [DBG] pgmap v1213: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:18.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:18 smithi190 bash[23392]: cluster 2024-03-26T04:03:17.115889+0000 mgr.y (mgr.24431) 720 : cluster [DBG] pgmap v1213: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:19.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:03:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:03:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:03:20.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:20 smithi175 bash[29284]: cluster 2024-03-26T04:03:19.116768+0000 mgr.y (mgr.24431) 721 : cluster [DBG] pgmap v1214: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:20.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:20 smithi175 bash[21527]: cluster 2024-03-26T04:03:19.116768+0000 mgr.y (mgr.24431) 721 : cluster [DBG] pgmap v1214: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:20.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:20 smithi190 bash[23392]: cluster 2024-03-26T04:03:19.116768+0000 mgr.y (mgr.24431) 721 : cluster [DBG] pgmap v1214: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:20.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:03:20 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:03:21.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:21 smithi175 bash[29284]: audit 2024-03-26T04:03:20.544139+0000 mgr.y (mgr.24431) 722 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:03:21.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:21 smithi175 bash[21527]: audit 2024-03-26T04:03:20.544139+0000 mgr.y (mgr.24431) 722 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:03:21.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:21 smithi190 bash[23392]: audit 2024-03-26T04:03:20.544139+0000 mgr.y (mgr.24431) 722 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:03:22.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:22 smithi175 bash[29284]: cluster 2024-03-26T04:03:21.118999+0000 mgr.y (mgr.24431) 723 : cluster [DBG] pgmap v1215: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:03:22.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:22 smithi175 bash[29284]: audit 2024-03-26T04:03:21.261564+0000 mon.c (mon.2) 752 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:03:22.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:22 smithi175 bash[21527]: cluster 2024-03-26T04:03:21.118999+0000 mgr.y (mgr.24431) 723 : cluster [DBG] pgmap v1215: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:03:22.493 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:22 smithi175 bash[21527]: audit 2024-03-26T04:03:21.261564+0000 mon.c (mon.2) 752 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:03:22.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:22 smithi190 bash[23392]: cluster 2024-03-26T04:03:21.118999+0000 mgr.y (mgr.24431) 723 : cluster [DBG] pgmap v1215: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:03:22.494 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:22 smithi190 bash[23392]: audit 2024-03-26T04:03:21.261564+0000 mon.c (mon.2) 752 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:03:24.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:24 smithi175 bash[21527]: cluster 2024-03-26T04:03:23.119897+0000 mgr.y (mgr.24431) 724 : cluster [DBG] pgmap v1216: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:24.493 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:24 smithi175 bash[29284]: cluster 2024-03-26T04:03:23.119897+0000 mgr.y (mgr.24431) 724 : cluster [DBG] pgmap v1216: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:24.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:24 smithi190 bash[23392]: cluster 2024-03-26T04:03:23.119897+0000 mgr.y (mgr.24431) 724 : cluster [DBG] pgmap v1216: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:26.492 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:26 smithi175 bash[21527]: cluster 2024-03-26T04:03:25.122121+0000 mgr.y (mgr.24431) 725 : cluster [DBG] pgmap v1217: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:03:26.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:26 smithi175 bash[29284]: cluster 2024-03-26T04:03:25.122121+0000 mgr.y (mgr.24431) 725 : cluster [DBG] pgmap v1217: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:03:26.493 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:26 smithi190 bash[23392]: cluster 2024-03-26T04:03:25.122121+0000 mgr.y (mgr.24431) 725 : cluster [DBG] pgmap v1217: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:03:28.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:28 smithi175 bash[21527]: cluster 2024-03-26T04:03:27.122956+0000 mgr.y (mgr.24431) 726 : cluster [DBG] pgmap v1218: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:28.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:28 smithi175 bash[29284]: cluster 2024-03-26T04:03:27.122956+0000 mgr.y (mgr.24431) 726 : cluster [DBG] pgmap v1218: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:28.743 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:28 smithi190 bash[23392]: cluster 2024-03-26T04:03:27.122956+0000 mgr.y (mgr.24431) 726 : cluster [DBG] pgmap v1218: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:29.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:03:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:03:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:03:30.553 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:30 smithi190 bash[23392]: cluster 2024-03-26T04:03:29.123817+0000 mgr.y (mgr.24431) 727 : cluster [DBG] pgmap v1219: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:30.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:30 smithi175 bash[29284]: cluster 2024-03-26T04:03:29.123817+0000 mgr.y (mgr.24431) 727 : cluster [DBG] pgmap v1219: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:30.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:30 smithi175 bash[21527]: cluster 2024-03-26T04:03:29.123817+0000 mgr.y (mgr.24431) 727 : cluster [DBG] pgmap v1219: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:30.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:03:30 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:03:31.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:31 smithi175 bash[29284]: audit 2024-03-26T04:03:30.555720+0000 mgr.y (mgr.24431) 728 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:03:31.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:31 smithi175 bash[21527]: audit 2024-03-26T04:03:30.555720+0000 mgr.y (mgr.24431) 728 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:03:31.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:31 smithi190 bash[23392]: audit 2024-03-26T04:03:30.555720+0000 mgr.y (mgr.24431) 728 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:03:32.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:32 smithi175 bash[29284]: cluster 2024-03-26T04:03:31.126070+0000 mgr.y (mgr.24431) 729 : cluster [DBG] pgmap v1220: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:03:32.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:32 smithi175 bash[21527]: cluster 2024-03-26T04:03:31.126070+0000 mgr.y (mgr.24431) 729 : cluster [DBG] pgmap v1220: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:03:32.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:32 smithi190 bash[23392]: cluster 2024-03-26T04:03:31.126070+0000 mgr.y (mgr.24431) 729 : cluster [DBG] pgmap v1220: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:03:34.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:34 smithi175 bash[29284]: cluster 2024-03-26T04:03:33.126870+0000 mgr.y (mgr.24431) 730 : cluster [DBG] pgmap v1221: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:34.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:34 smithi175 bash[21527]: cluster 2024-03-26T04:03:33.126870+0000 mgr.y (mgr.24431) 730 : cluster [DBG] pgmap v1221: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:34.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:34 smithi190 bash[23392]: cluster 2024-03-26T04:03:33.126870+0000 mgr.y (mgr.24431) 730 : cluster [DBG] pgmap v1221: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:36.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:36 smithi175 bash[29284]: cluster 2024-03-26T04:03:35.129111+0000 mgr.y (mgr.24431) 731 : cluster [DBG] pgmap v1222: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:03:36.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:36 smithi175 bash[21527]: cluster 2024-03-26T04:03:35.129111+0000 mgr.y (mgr.24431) 731 : cluster [DBG] pgmap v1222: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:03:36.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:36 smithi190 bash[23392]: cluster 2024-03-26T04:03:35.129111+0000 mgr.y (mgr.24431) 731 : cluster [DBG] pgmap v1222: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:03:37.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:37 smithi175 bash[29284]: audit 2024-03-26T04:03:36.294803+0000 mon.c (mon.2) 753 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:03:37.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:37 smithi175 bash[21527]: audit 2024-03-26T04:03:36.294803+0000 mon.c (mon.2) 753 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:03:37.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:37 smithi190 bash[23392]: audit 2024-03-26T04:03:36.294803+0000 mon.c (mon.2) 753 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:03:38.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:38 smithi175 bash[29284]: cluster 2024-03-26T04:03:37.130156+0000 mgr.y (mgr.24431) 732 : cluster [DBG] pgmap v1223: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:38.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:38 smithi175 bash[21527]: cluster 2024-03-26T04:03:37.130156+0000 mgr.y (mgr.24431) 732 : cluster [DBG] pgmap v1223: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:38.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:38 smithi190 bash[23392]: cluster 2024-03-26T04:03:37.130156+0000 mgr.y (mgr.24431) 732 : cluster [DBG] pgmap v1223: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:39.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:03:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:03:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:03:40.559 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:40 smithi190 bash[23392]: cluster 2024-03-26T04:03:39.131051+0000 mgr.y (mgr.24431) 733 : cluster [DBG] pgmap v1224: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:40.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:40 smithi175 bash[29284]: cluster 2024-03-26T04:03:39.131051+0000 mgr.y (mgr.24431) 733 : cluster [DBG] pgmap v1224: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:40.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:40 smithi175 bash[21527]: cluster 2024-03-26T04:03:39.131051+0000 mgr.y (mgr.24431) 733 : cluster [DBG] pgmap v1224: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:40.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:03:40 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:03:41.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:41 smithi175 bash[29284]: audit 2024-03-26T04:03:40.560859+0000 mgr.y (mgr.24431) 734 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:03:41.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:41 smithi175 bash[21527]: audit 2024-03-26T04:03:40.560859+0000 mgr.y (mgr.24431) 734 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:03:41.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:41 smithi190 bash[23392]: audit 2024-03-26T04:03:40.560859+0000 mgr.y (mgr.24431) 734 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:03:42.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:42 smithi175 bash[29284]: cluster 2024-03-26T04:03:41.133269+0000 mgr.y (mgr.24431) 735 : cluster [DBG] pgmap v1225: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:03:42.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:42 smithi175 bash[21527]: cluster 2024-03-26T04:03:41.133269+0000 mgr.y (mgr.24431) 735 : cluster [DBG] pgmap v1225: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:03:42.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:42 smithi190 bash[23392]: cluster 2024-03-26T04:03:41.133269+0000 mgr.y (mgr.24431) 735 : cluster [DBG] pgmap v1225: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:03:44.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:44 smithi175 bash[21527]: cluster 2024-03-26T04:03:43.134097+0000 mgr.y (mgr.24431) 736 : cluster [DBG] pgmap v1226: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:44.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:44 smithi175 bash[29284]: cluster 2024-03-26T04:03:43.134097+0000 mgr.y (mgr.24431) 736 : cluster [DBG] pgmap v1226: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:44.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:44 smithi190 bash[23392]: cluster 2024-03-26T04:03:43.134097+0000 mgr.y (mgr.24431) 736 : cluster [DBG] pgmap v1226: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:46.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:46 smithi175 bash[21527]: cluster 2024-03-26T04:03:45.136739+0000 mgr.y (mgr.24431) 737 : cluster [DBG] pgmap v1227: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:03:46.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:46 smithi175 bash[29284]: cluster 2024-03-26T04:03:45.136739+0000 mgr.y (mgr.24431) 737 : cluster [DBG] pgmap v1227: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:03:46.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:46 smithi190 bash[23392]: cluster 2024-03-26T04:03:45.136739+0000 mgr.y (mgr.24431) 737 : cluster [DBG] pgmap v1227: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:03:48.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:48 smithi175 bash[21527]: cluster 2024-03-26T04:03:47.137578+0000 mgr.y (mgr.24431) 738 : cluster [DBG] pgmap v1228: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:48.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:48 smithi175 bash[29284]: cluster 2024-03-26T04:03:47.137578+0000 mgr.y (mgr.24431) 738 : cluster [DBG] pgmap v1228: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:48.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:48 smithi190 bash[23392]: cluster 2024-03-26T04:03:47.137578+0000 mgr.y (mgr.24431) 738 : cluster [DBG] pgmap v1228: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:49.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:03:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:03:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:03:50.570 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:50 smithi190 bash[23392]: cluster 2024-03-26T04:03:49.138398+0000 mgr.y (mgr.24431) 739 : cluster [DBG] pgmap v1229: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:50.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:50 smithi175 bash[21527]: cluster 2024-03-26T04:03:49.138398+0000 mgr.y (mgr.24431) 739 : cluster [DBG] pgmap v1229: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:50.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:50 smithi175 bash[29284]: cluster 2024-03-26T04:03:49.138398+0000 mgr.y (mgr.24431) 739 : cluster [DBG] pgmap v1229: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:50.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:03:50 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:03:51.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:51 smithi175 bash[29284]: audit 2024-03-26T04:03:50.572443+0000 mgr.y (mgr.24431) 740 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:03:51.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:51 smithi175 bash[21527]: audit 2024-03-26T04:03:50.572443+0000 mgr.y (mgr.24431) 740 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:03:51.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:51 smithi190 bash[23392]: audit 2024-03-26T04:03:50.572443+0000 mgr.y (mgr.24431) 740 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:03:52.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:52 smithi175 bash[21527]: cluster 2024-03-26T04:03:51.140709+0000 mgr.y (mgr.24431) 741 : cluster [DBG] pgmap v1230: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:03:52.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:52 smithi175 bash[21527]: audit 2024-03-26T04:03:51.331876+0000 mon.c (mon.2) 754 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:03:52.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:52 smithi175 bash[29284]: cluster 2024-03-26T04:03:51.140709+0000 mgr.y (mgr.24431) 741 : cluster [DBG] pgmap v1230: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:03:52.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:52 smithi175 bash[29284]: audit 2024-03-26T04:03:51.331876+0000 mon.c (mon.2) 754 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:03:52.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:52 smithi190 bash[23392]: cluster 2024-03-26T04:03:51.140709+0000 mgr.y (mgr.24431) 741 : cluster [DBG] pgmap v1230: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:03:52.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:52 smithi190 bash[23392]: audit 2024-03-26T04:03:51.331876+0000 mon.c (mon.2) 754 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:03:54.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:54 smithi175 bash[21527]: cluster 2024-03-26T04:03:53.141711+0000 mgr.y (mgr.24431) 742 : cluster [DBG] pgmap v1231: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:54.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:54 smithi175 bash[29284]: cluster 2024-03-26T04:03:53.141711+0000 mgr.y (mgr.24431) 742 : cluster [DBG] pgmap v1231: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:54.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:54 smithi190 bash[23392]: cluster 2024-03-26T04:03:53.141711+0000 mgr.y (mgr.24431) 742 : cluster [DBG] pgmap v1231: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:56.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:56 smithi175 bash[21527]: cluster 2024-03-26T04:03:55.143973+0000 mgr.y (mgr.24431) 743 : cluster [DBG] pgmap v1232: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:03:56.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:56 smithi175 bash[29284]: cluster 2024-03-26T04:03:55.143973+0000 mgr.y (mgr.24431) 743 : cluster [DBG] pgmap v1232: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:03:56.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:56 smithi190 bash[23392]: cluster 2024-03-26T04:03:55.143973+0000 mgr.y (mgr.24431) 743 : cluster [DBG] pgmap v1232: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:03:58.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:03:58 smithi175 bash[29284]: cluster 2024-03-26T04:03:57.144743+0000 mgr.y (mgr.24431) 744 : cluster [DBG] pgmap v1233: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:58.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:03:58 smithi175 bash[21527]: cluster 2024-03-26T04:03:57.144743+0000 mgr.y (mgr.24431) 744 : cluster [DBG] pgmap v1233: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:58.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:03:58 smithi190 bash[23392]: cluster 2024-03-26T04:03:57.144743+0000 mgr.y (mgr.24431) 744 : cluster [DBG] pgmap v1233: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:03:59.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:03:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:03:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:04:00.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:00 smithi175 bash[21527]: cluster 2024-03-26T04:03:59.145565+0000 mgr.y (mgr.24431) 745 : cluster [DBG] pgmap v1234: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:00.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:00 smithi175 bash[29284]: cluster 2024-03-26T04:03:59.145565+0000 mgr.y (mgr.24431) 745 : cluster [DBG] pgmap v1234: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:00.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:00 smithi190 bash[23392]: cluster 2024-03-26T04:03:59.145565+0000 mgr.y (mgr.24431) 745 : cluster [DBG] pgmap v1234: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:00.744 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:04:00 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:04:01.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:01 smithi175 bash[21527]: audit 2024-03-26T04:04:00.575726+0000 mgr.y (mgr.24431) 746 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:04:01.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:01 smithi175 bash[29284]: audit 2024-03-26T04:04:00.575726+0000 mgr.y (mgr.24431) 746 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:04:01.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:01 smithi190 bash[23392]: audit 2024-03-26T04:04:00.575726+0000 mgr.y (mgr.24431) 746 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:04:02.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:02 smithi175 bash[21527]: cluster 2024-03-26T04:04:01.147786+0000 mgr.y (mgr.24431) 747 : cluster [DBG] pgmap v1235: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:04:02.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:02 smithi175 bash[29284]: cluster 2024-03-26T04:04:01.147786+0000 mgr.y (mgr.24431) 747 : cluster [DBG] pgmap v1235: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:04:02.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:02 smithi190 bash[23392]: cluster 2024-03-26T04:04:01.147786+0000 mgr.y (mgr.24431) 747 : cluster [DBG] pgmap v1235: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:04:04.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:04 smithi175 bash[29284]: cluster 2024-03-26T04:04:03.148630+0000 mgr.y (mgr.24431) 748 : cluster [DBG] pgmap v1236: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:04.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:04 smithi175 bash[21527]: cluster 2024-03-26T04:04:03.148630+0000 mgr.y (mgr.24431) 748 : cluster [DBG] pgmap v1236: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:04.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:04 smithi190 bash[23392]: cluster 2024-03-26T04:04:03.148630+0000 mgr.y (mgr.24431) 748 : cluster [DBG] pgmap v1236: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:06.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:06 smithi175 bash[21527]: cluster 2024-03-26T04:04:05.150822+0000 mgr.y (mgr.24431) 749 : cluster [DBG] pgmap v1237: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:04:06.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:06 smithi175 bash[21527]: audit 2024-03-26T04:04:06.373275+0000 mon.c (mon.2) 755 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:04:06.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:06 smithi175 bash[29284]: cluster 2024-03-26T04:04:05.150822+0000 mgr.y (mgr.24431) 749 : cluster [DBG] pgmap v1237: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:04:06.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:06 smithi175 bash[29284]: audit 2024-03-26T04:04:06.373275+0000 mon.c (mon.2) 755 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:04:06.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:06 smithi190 bash[23392]: cluster 2024-03-26T04:04:05.150822+0000 mgr.y (mgr.24431) 749 : cluster [DBG] pgmap v1237: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:04:06.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:06 smithi190 bash[23392]: audit 2024-03-26T04:04:06.373275+0000 mon.c (mon.2) 755 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:04:08.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:08 smithi175 bash[21527]: cluster 2024-03-26T04:04:07.151715+0000 mgr.y (mgr.24431) 750 : cluster [DBG] pgmap v1238: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:08.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:08 smithi175 bash[29284]: cluster 2024-03-26T04:04:07.151715+0000 mgr.y (mgr.24431) 750 : cluster [DBG] pgmap v1238: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:08.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:08 smithi190 bash[23392]: cluster 2024-03-26T04:04:07.151715+0000 mgr.y (mgr.24431) 750 : cluster [DBG] pgmap v1238: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:09.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:04:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:04:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:04:09.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:09 smithi175 bash[21527]: audit 2024-03-26T04:04:09.171346+0000 mon.c (mon.2) 756 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:04:09.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:09 smithi175 bash[29284]: audit 2024-03-26T04:04:09.171346+0000 mon.c (mon.2) 756 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:04:09.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:09 smithi190 bash[23392]: audit 2024-03-26T04:04:09.171346+0000 mon.c (mon.2) 756 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:04:10.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:10 smithi175 bash[21527]: cluster 2024-03-26T04:04:09.152496+0000 mgr.y (mgr.24431) 751 : cluster [DBG] pgmap v1239: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:10.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:10 smithi175 bash[21527]: audit 2024-03-26T04:04:09.960426+0000 mon.c (mon.2) 757 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:04:10.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:10 smithi175 bash[21527]: audit 2024-03-26T04:04:09.962215+0000 mon.c (mon.2) 758 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:04:10.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:10 smithi175 bash[21527]: audit 2024-03-26T04:04:09.974931+0000 mon.a (mon.0) 3516 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:04:10.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:10 smithi175 bash[29284]: cluster 2024-03-26T04:04:09.152496+0000 mgr.y (mgr.24431) 751 : cluster [DBG] pgmap v1239: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:10.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:10 smithi175 bash[29284]: audit 2024-03-26T04:04:09.960426+0000 mon.c (mon.2) 757 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:04:10.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:10 smithi175 bash[29284]: audit 2024-03-26T04:04:09.962215+0000 mon.c (mon.2) 758 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:04:10.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:10 smithi175 bash[29284]: audit 2024-03-26T04:04:09.974931+0000 mon.a (mon.0) 3516 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:04:10.745 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:04:10 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:04:10.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:10 smithi190 bash[23392]: cluster 2024-03-26T04:04:09.152496+0000 mgr.y (mgr.24431) 751 : cluster [DBG] pgmap v1239: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:10.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:10 smithi190 bash[23392]: audit 2024-03-26T04:04:09.960426+0000 mon.c (mon.2) 757 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:04:10.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:10 smithi190 bash[23392]: audit 2024-03-26T04:04:09.962215+0000 mon.c (mon.2) 758 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:04:10.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:10 smithi190 bash[23392]: audit 2024-03-26T04:04:09.974931+0000 mon.a (mon.0) 3516 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:04:11.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:11 smithi175 bash[21527]: audit 2024-03-26T04:04:10.587164+0000 mgr.y (mgr.24431) 752 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:04:11.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:11 smithi175 bash[29284]: audit 2024-03-26T04:04:10.587164+0000 mgr.y (mgr.24431) 752 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:04:11.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:11 smithi190 bash[23392]: audit 2024-03-26T04:04:10.587164+0000 mgr.y (mgr.24431) 752 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:04:12.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:12 smithi175 bash[21527]: cluster 2024-03-26T04:04:11.154687+0000 mgr.y (mgr.24431) 753 : cluster [DBG] pgmap v1240: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:04:12.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:12 smithi175 bash[29284]: cluster 2024-03-26T04:04:11.154687+0000 mgr.y (mgr.24431) 753 : cluster [DBG] pgmap v1240: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:04:12.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:12 smithi190 bash[23392]: cluster 2024-03-26T04:04:11.154687+0000 mgr.y (mgr.24431) 753 : cluster [DBG] pgmap v1240: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:04:14.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:14 smithi175 bash[21527]: cluster 2024-03-26T04:04:13.155566+0000 mgr.y (mgr.24431) 754 : cluster [DBG] pgmap v1241: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:14.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:14 smithi175 bash[29284]: cluster 2024-03-26T04:04:13.155566+0000 mgr.y (mgr.24431) 754 : cluster [DBG] pgmap v1241: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:14.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:14 smithi190 bash[23392]: cluster 2024-03-26T04:04:13.155566+0000 mgr.y (mgr.24431) 754 : cluster [DBG] pgmap v1241: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:16.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:16 smithi175 bash[29284]: cluster 2024-03-26T04:04:15.157750+0000 mgr.y (mgr.24431) 755 : cluster [DBG] pgmap v1242: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:04:16.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:16 smithi175 bash[21527]: cluster 2024-03-26T04:04:15.157750+0000 mgr.y (mgr.24431) 755 : cluster [DBG] pgmap v1242: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:04:16.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:16 smithi190 bash[23392]: cluster 2024-03-26T04:04:15.157750+0000 mgr.y (mgr.24431) 755 : cluster [DBG] pgmap v1242: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:04:18.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:18 smithi175 bash[21527]: cluster 2024-03-26T04:04:17.158612+0000 mgr.y (mgr.24431) 756 : cluster [DBG] pgmap v1243: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:18.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:18 smithi175 bash[29284]: cluster 2024-03-26T04:04:17.158612+0000 mgr.y (mgr.24431) 756 : cluster [DBG] pgmap v1243: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:18.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:18 smithi190 bash[23392]: cluster 2024-03-26T04:04:17.158612+0000 mgr.y (mgr.24431) 756 : cluster [DBG] pgmap v1243: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:19.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:04:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:04:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:04:20.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:20 smithi175 bash[29284]: cluster 2024-03-26T04:04:19.159436+0000 mgr.y (mgr.24431) 757 : cluster [DBG] pgmap v1244: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:20.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:20 smithi175 bash[21527]: cluster 2024-03-26T04:04:19.159436+0000 mgr.y (mgr.24431) 757 : cluster [DBG] pgmap v1244: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:20.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:20 smithi190 bash[23392]: cluster 2024-03-26T04:04:19.159436+0000 mgr.y (mgr.24431) 757 : cluster [DBG] pgmap v1244: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:20.744 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:04:20 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:04:21.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:21 smithi175 bash[21527]: audit 2024-03-26T04:04:20.598715+0000 mgr.y (mgr.24431) 758 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:04:21.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:21 smithi175 bash[21527]: audit 2024-03-26T04:04:21.411107+0000 mon.c (mon.2) 759 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:04:21.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:21 smithi175 bash[29284]: audit 2024-03-26T04:04:20.598715+0000 mgr.y (mgr.24431) 758 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:04:21.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:21 smithi175 bash[29284]: audit 2024-03-26T04:04:21.411107+0000 mon.c (mon.2) 759 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:04:21.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:21 smithi190 bash[23392]: audit 2024-03-26T04:04:20.598715+0000 mgr.y (mgr.24431) 758 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:04:21.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:21 smithi190 bash[23392]: audit 2024-03-26T04:04:21.411107+0000 mon.c (mon.2) 759 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:04:22.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:22 smithi175 bash[21527]: cluster 2024-03-26T04:04:21.161654+0000 mgr.y (mgr.24431) 759 : cluster [DBG] pgmap v1245: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:04:22.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:22 smithi175 bash[29284]: cluster 2024-03-26T04:04:21.161654+0000 mgr.y (mgr.24431) 759 : cluster [DBG] pgmap v1245: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:04:22.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:22 smithi190 bash[23392]: cluster 2024-03-26T04:04:21.161654+0000 mgr.y (mgr.24431) 759 : cluster [DBG] pgmap v1245: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:04:24.745 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:24 smithi175 bash[29284]: cluster 2024-03-26T04:04:23.162441+0000 mgr.y (mgr.24431) 760 : cluster [DBG] pgmap v1246: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:24.745 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:24 smithi175 bash[21527]: cluster 2024-03-26T04:04:23.162441+0000 mgr.y (mgr.24431) 760 : cluster [DBG] pgmap v1246: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:24.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:24 smithi190 bash[23392]: cluster 2024-03-26T04:04:23.162441+0000 mgr.y (mgr.24431) 760 : cluster [DBG] pgmap v1246: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:26.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:26 smithi175 bash[21527]: cluster 2024-03-26T04:04:25.164947+0000 mgr.y (mgr.24431) 761 : cluster [DBG] pgmap v1247: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 13 KiB/s rd, 0 B/s wr, 21 op/s 2024-03-26T04:04:26.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:26 smithi175 bash[29284]: cluster 2024-03-26T04:04:25.164947+0000 mgr.y (mgr.24431) 761 : cluster [DBG] pgmap v1247: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 13 KiB/s rd, 0 B/s wr, 21 op/s 2024-03-26T04:04:26.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:26 smithi190 bash[23392]: cluster 2024-03-26T04:04:25.164947+0000 mgr.y (mgr.24431) 761 : cluster [DBG] pgmap v1247: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 13 KiB/s rd, 0 B/s wr, 21 op/s 2024-03-26T04:04:28.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:28 smithi175 bash[29284]: cluster 2024-03-26T04:04:27.165839+0000 mgr.y (mgr.24431) 762 : cluster [DBG] pgmap v1248: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 13 KiB/s rd, 0 B/s wr, 20 op/s 2024-03-26T04:04:28.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:28 smithi175 bash[21527]: cluster 2024-03-26T04:04:27.165839+0000 mgr.y (mgr.24431) 762 : cluster [DBG] pgmap v1248: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 13 KiB/s rd, 0 B/s wr, 20 op/s 2024-03-26T04:04:28.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:28 smithi190 bash[23392]: cluster 2024-03-26T04:04:27.165839+0000 mgr.y (mgr.24431) 762 : cluster [DBG] pgmap v1248: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 13 KiB/s rd, 0 B/s wr, 20 op/s 2024-03-26T04:04:29.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:04:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:04:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:04:30.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:30 smithi175 bash[29284]: cluster 2024-03-26T04:04:29.166699+0000 mgr.y (mgr.24431) 763 : cluster [DBG] pgmap v1249: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 13 KiB/s rd, 0 B/s wr, 20 op/s 2024-03-26T04:04:30.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:30 smithi175 bash[21527]: cluster 2024-03-26T04:04:29.166699+0000 mgr.y (mgr.24431) 763 : cluster [DBG] pgmap v1249: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 13 KiB/s rd, 0 B/s wr, 20 op/s 2024-03-26T04:04:30.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:30 smithi190 bash[23392]: cluster 2024-03-26T04:04:29.166699+0000 mgr.y (mgr.24431) 763 : cluster [DBG] pgmap v1249: 228 pgs: 228 active+clean; 582 KiB data, 782 MiB used, 714 GiB / 715 GiB avail; 13 KiB/s rd, 0 B/s wr, 20 op/s 2024-03-26T04:04:30.744 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:04:30 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:04:31.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:31 smithi175 bash[21527]: audit 2024-03-26T04:04:30.609137+0000 mgr.y (mgr.24431) 764 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:04:31.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:31 smithi175 bash[29284]: audit 2024-03-26T04:04:30.609137+0000 mgr.y (mgr.24431) 764 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:04:31.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:31 smithi190 bash[23392]: audit 2024-03-26T04:04:30.609137+0000 mgr.y (mgr.24431) 764 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:04:32.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:32 smithi175 bash[21527]: cluster 2024-03-26T04:04:31.168987+0000 mgr.y (mgr.24431) 765 : cluster [DBG] pgmap v1250: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-26T04:04:32.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:32 smithi175 bash[29284]: cluster 2024-03-26T04:04:31.168987+0000 mgr.y (mgr.24431) 765 : cluster [DBG] pgmap v1250: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-26T04:04:32.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:32 smithi190 bash[23392]: cluster 2024-03-26T04:04:31.168987+0000 mgr.y (mgr.24431) 765 : cluster [DBG] pgmap v1250: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-26T04:04:34.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:34 smithi175 bash[21527]: cluster 2024-03-26T04:04:33.169780+0000 mgr.y (mgr.24431) 766 : cluster [DBG] pgmap v1251: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-26T04:04:34.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:34 smithi175 bash[29284]: cluster 2024-03-26T04:04:33.169780+0000 mgr.y (mgr.24431) 766 : cluster [DBG] pgmap v1251: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-26T04:04:34.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:34 smithi190 bash[23392]: cluster 2024-03-26T04:04:33.169780+0000 mgr.y (mgr.24431) 766 : cluster [DBG] pgmap v1251: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-26T04:04:36.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:36 smithi175 bash[21527]: cluster 2024-03-26T04:04:35.172027+0000 mgr.y (mgr.24431) 767 : cluster [DBG] pgmap v1252: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-26T04:04:36.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:36 smithi175 bash[21527]: audit 2024-03-26T04:04:36.447044+0000 mon.c (mon.2) 760 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:04:36.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:36 smithi175 bash[29284]: cluster 2024-03-26T04:04:35.172027+0000 mgr.y (mgr.24431) 767 : cluster [DBG] pgmap v1252: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-26T04:04:36.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:36 smithi175 bash[29284]: audit 2024-03-26T04:04:36.447044+0000 mon.c (mon.2) 760 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:04:36.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:36 smithi190 bash[23392]: cluster 2024-03-26T04:04:35.172027+0000 mgr.y (mgr.24431) 767 : cluster [DBG] pgmap v1252: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-26T04:04:36.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:36 smithi190 bash[23392]: audit 2024-03-26T04:04:36.447044+0000 mon.c (mon.2) 760 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:04:38.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:38 smithi175 bash[21527]: cluster 2024-03-26T04:04:37.172879+0000 mgr.y (mgr.24431) 768 : cluster [DBG] pgmap v1253: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 25 KiB/s rd, 0 B/s wr, 40 op/s 2024-03-26T04:04:38.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:38 smithi175 bash[29284]: cluster 2024-03-26T04:04:37.172879+0000 mgr.y (mgr.24431) 768 : cluster [DBG] pgmap v1253: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 25 KiB/s rd, 0 B/s wr, 40 op/s 2024-03-26T04:04:38.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:38 smithi190 bash[23392]: cluster 2024-03-26T04:04:37.172879+0000 mgr.y (mgr.24431) 768 : cluster [DBG] pgmap v1253: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 25 KiB/s rd, 0 B/s wr, 40 op/s 2024-03-26T04:04:39.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:04:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:04:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:04:40.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:40 smithi175 bash[21527]: cluster 2024-03-26T04:04:39.173688+0000 mgr.y (mgr.24431) 769 : cluster [DBG] pgmap v1254: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 25 KiB/s rd, 0 B/s wr, 40 op/s 2024-03-26T04:04:40.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:40 smithi175 bash[29284]: cluster 2024-03-26T04:04:39.173688+0000 mgr.y (mgr.24431) 769 : cluster [DBG] pgmap v1254: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 25 KiB/s rd, 0 B/s wr, 40 op/s 2024-03-26T04:04:40.744 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:04:40 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:04:40.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:40 smithi190 bash[23392]: cluster 2024-03-26T04:04:39.173688+0000 mgr.y (mgr.24431) 769 : cluster [DBG] pgmap v1254: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 25 KiB/s rd, 0 B/s wr, 40 op/s 2024-03-26T04:04:41.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:41 smithi175 bash[21527]: audit 2024-03-26T04:04:40.620272+0000 mgr.y (mgr.24431) 770 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:04:41.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:41 smithi175 bash[29284]: audit 2024-03-26T04:04:40.620272+0000 mgr.y (mgr.24431) 770 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:04:41.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:41 smithi190 bash[23392]: audit 2024-03-26T04:04:40.620272+0000 mgr.y (mgr.24431) 770 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:04:42.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:42 smithi175 bash[21527]: cluster 2024-03-26T04:04:41.175966+0000 mgr.y (mgr.24431) 771 : cluster [DBG] pgmap v1255: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 25 KiB/s rd, 0 B/s wr, 40 op/s 2024-03-26T04:04:42.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:42 smithi175 bash[29284]: cluster 2024-03-26T04:04:41.175966+0000 mgr.y (mgr.24431) 771 : cluster [DBG] pgmap v1255: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 25 KiB/s rd, 0 B/s wr, 40 op/s 2024-03-26T04:04:42.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:42 smithi190 bash[23392]: cluster 2024-03-26T04:04:41.175966+0000 mgr.y (mgr.24431) 771 : cluster [DBG] pgmap v1255: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 25 KiB/s rd, 0 B/s wr, 40 op/s 2024-03-26T04:04:44.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:44 smithi175 bash[29284]: cluster 2024-03-26T04:04:43.176816+0000 mgr.y (mgr.24431) 772 : cluster [DBG] pgmap v1256: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:45.061 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:44 smithi175 bash[21527]: cluster 2024-03-26T04:04:43.176816+0000 mgr.y (mgr.24431) 772 : cluster [DBG] pgmap v1256: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:45.061 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:44 smithi190 bash[23392]: cluster 2024-03-26T04:04:43.176816+0000 mgr.y (mgr.24431) 772 : cluster [DBG] pgmap v1256: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:46.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:46 smithi175 bash[29284]: cluster 2024-03-26T04:04:45.179081+0000 mgr.y (mgr.24431) 773 : cluster [DBG] pgmap v1257: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:04:46.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:46 smithi175 bash[21527]: cluster 2024-03-26T04:04:45.179081+0000 mgr.y (mgr.24431) 773 : cluster [DBG] pgmap v1257: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:04:46.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:46 smithi190 bash[23392]: cluster 2024-03-26T04:04:45.179081+0000 mgr.y (mgr.24431) 773 : cluster [DBG] pgmap v1257: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:04:48.772 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:48 smithi175 bash[29284]: cluster 2024-03-26T04:04:47.179961+0000 mgr.y (mgr.24431) 774 : cluster [DBG] pgmap v1258: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:48.772 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:48 smithi175 bash[21527]: cluster 2024-03-26T04:04:47.179961+0000 mgr.y (mgr.24431) 774 : cluster [DBG] pgmap v1258: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:48.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:48 smithi190 bash[23392]: cluster 2024-03-26T04:04:47.179961+0000 mgr.y (mgr.24431) 774 : cluster [DBG] pgmap v1258: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:49.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:04:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:04:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:04:50.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:50 smithi175 bash[21527]: cluster 2024-03-26T04:04:49.180798+0000 mgr.y (mgr.24431) 775 : cluster [DBG] pgmap v1259: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:50.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:50 smithi175 bash[29284]: cluster 2024-03-26T04:04:49.180798+0000 mgr.y (mgr.24431) 775 : cluster [DBG] pgmap v1259: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:50.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:50 smithi190 bash[23392]: cluster 2024-03-26T04:04:49.180798+0000 mgr.y (mgr.24431) 775 : cluster [DBG] pgmap v1259: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:50.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:04:50 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:04:51.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:51 smithi175 bash[21527]: audit 2024-03-26T04:04:50.631565+0000 mgr.y (mgr.24431) 776 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:04:51.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:51 smithi175 bash[21527]: audit 2024-03-26T04:04:51.480414+0000 mon.c (mon.2) 761 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:04:51.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:51 smithi175 bash[29284]: audit 2024-03-26T04:04:50.631565+0000 mgr.y (mgr.24431) 776 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:04:51.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:51 smithi175 bash[29284]: audit 2024-03-26T04:04:51.480414+0000 mon.c (mon.2) 761 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:04:51.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:51 smithi190 bash[23392]: audit 2024-03-26T04:04:50.631565+0000 mgr.y (mgr.24431) 776 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:04:51.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:51 smithi190 bash[23392]: audit 2024-03-26T04:04:51.480414+0000 mon.c (mon.2) 761 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:04:52.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:52 smithi175 bash[21527]: cluster 2024-03-26T04:04:51.183039+0000 mgr.y (mgr.24431) 777 : cluster [DBG] pgmap v1260: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:04:52.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:52 smithi175 bash[29284]: cluster 2024-03-26T04:04:51.183039+0000 mgr.y (mgr.24431) 777 : cluster [DBG] pgmap v1260: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:04:52.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:52 smithi190 bash[23392]: cluster 2024-03-26T04:04:51.183039+0000 mgr.y (mgr.24431) 777 : cluster [DBG] pgmap v1260: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:04:54.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:54 smithi175 bash[21527]: cluster 2024-03-26T04:04:53.183888+0000 mgr.y (mgr.24431) 778 : cluster [DBG] pgmap v1261: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:54.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:54 smithi175 bash[29284]: cluster 2024-03-26T04:04:53.183888+0000 mgr.y (mgr.24431) 778 : cluster [DBG] pgmap v1261: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:54.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:54 smithi190 bash[23392]: cluster 2024-03-26T04:04:53.183888+0000 mgr.y (mgr.24431) 778 : cluster [DBG] pgmap v1261: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:56.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:56 smithi175 bash[21527]: cluster 2024-03-26T04:04:55.186007+0000 mgr.y (mgr.24431) 779 : cluster [DBG] pgmap v1262: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:04:56.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:56 smithi175 bash[29284]: cluster 2024-03-26T04:04:55.186007+0000 mgr.y (mgr.24431) 779 : cluster [DBG] pgmap v1262: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:04:56.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:56 smithi190 bash[23392]: cluster 2024-03-26T04:04:55.186007+0000 mgr.y (mgr.24431) 779 : cluster [DBG] pgmap v1262: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:04:58.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:04:58 smithi175 bash[21527]: cluster 2024-03-26T04:04:57.186783+0000 mgr.y (mgr.24431) 780 : cluster [DBG] pgmap v1263: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:58.993 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:04:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:04:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:04:58.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:04:58 smithi175 bash[29284]: cluster 2024-03-26T04:04:57.186783+0000 mgr.y (mgr.24431) 780 : cluster [DBG] pgmap v1263: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:04:58.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:04:58 smithi190 bash[23392]: cluster 2024-03-26T04:04:57.186783+0000 mgr.y (mgr.24431) 780 : cluster [DBG] pgmap v1263: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:00.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:00 smithi175 bash[29284]: cluster 2024-03-26T04:04:59.187635+0000 mgr.y (mgr.24431) 781 : cluster [DBG] pgmap v1264: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:00.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:00 smithi175 bash[21527]: cluster 2024-03-26T04:04:59.187635+0000 mgr.y (mgr.24431) 781 : cluster [DBG] pgmap v1264: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:00.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:00 smithi190 bash[23392]: cluster 2024-03-26T04:04:59.187635+0000 mgr.y (mgr.24431) 781 : cluster [DBG] pgmap v1264: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:00.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:05:00 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:05:01.744 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:01 smithi190 bash[23392]: audit 2024-03-26T04:05:00.643161+0000 mgr.y (mgr.24431) 782 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:05:01.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:01 smithi175 bash[29284]: audit 2024-03-26T04:05:00.643161+0000 mgr.y (mgr.24431) 782 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:05:01.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:01 smithi175 bash[21527]: audit 2024-03-26T04:05:00.643161+0000 mgr.y (mgr.24431) 782 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:05:02.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:02 smithi175 bash[21527]: cluster 2024-03-26T04:05:01.189148+0000 mgr.y (mgr.24431) 783 : cluster [DBG] pgmap v1265: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:02.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:02 smithi175 bash[29284]: cluster 2024-03-26T04:05:01.189148+0000 mgr.y (mgr.24431) 783 : cluster [DBG] pgmap v1265: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:02.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:02 smithi190 bash[23392]: cluster 2024-03-26T04:05:01.189148+0000 mgr.y (mgr.24431) 783 : cluster [DBG] pgmap v1265: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:04.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:04 smithi175 bash[21527]: cluster 2024-03-26T04:05:03.189940+0000 mgr.y (mgr.24431) 784 : cluster [DBG] pgmap v1266: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:04.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:04 smithi175 bash[29284]: cluster 2024-03-26T04:05:03.189940+0000 mgr.y (mgr.24431) 784 : cluster [DBG] pgmap v1266: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:04.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:04 smithi190 bash[23392]: cluster 2024-03-26T04:05:03.189940+0000 mgr.y (mgr.24431) 784 : cluster [DBG] pgmap v1266: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:06.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:06 smithi175 bash[21527]: cluster 2024-03-26T04:05:05.192220+0000 mgr.y (mgr.24431) 785 : cluster [DBG] pgmap v1267: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:06.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:06 smithi175 bash[21527]: audit 2024-03-26T04:05:06.513339+0000 mon.c (mon.2) 762 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:05:06.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:06 smithi175 bash[29284]: cluster 2024-03-26T04:05:05.192220+0000 mgr.y (mgr.24431) 785 : cluster [DBG] pgmap v1267: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:06.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:06 smithi175 bash[29284]: audit 2024-03-26T04:05:06.513339+0000 mon.c (mon.2) 762 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:05:06.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:06 smithi190 bash[23392]: cluster 2024-03-26T04:05:05.192220+0000 mgr.y (mgr.24431) 785 : cluster [DBG] pgmap v1267: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:06.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:06 smithi190 bash[23392]: audit 2024-03-26T04:05:06.513339+0000 mon.c (mon.2) 762 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:05:08.992 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:05:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:05:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:05:08.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:08 smithi175 bash[29284]: cluster 2024-03-26T04:05:07.193064+0000 mgr.y (mgr.24431) 786 : cluster [DBG] pgmap v1268: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:08.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:08 smithi175 bash[21527]: cluster 2024-03-26T04:05:07.193064+0000 mgr.y (mgr.24431) 786 : cluster [DBG] pgmap v1268: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:08.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:08 smithi190 bash[23392]: cluster 2024-03-26T04:05:07.193064+0000 mgr.y (mgr.24431) 786 : cluster [DBG] pgmap v1268: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:10.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:10 smithi175 bash[21527]: cluster 2024-03-26T04:05:09.193829+0000 mgr.y (mgr.24431) 787 : cluster [DBG] pgmap v1269: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:10.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:10 smithi175 bash[21527]: audit 2024-03-26T04:05:10.070519+0000 mon.c (mon.2) 763 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:05:10.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:10 smithi175 bash[29284]: cluster 2024-03-26T04:05:09.193829+0000 mgr.y (mgr.24431) 787 : cluster [DBG] pgmap v1269: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:10.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:10 smithi175 bash[29284]: audit 2024-03-26T04:05:10.070519+0000 mon.c (mon.2) 763 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:05:10.995 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:05:10 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:05:10.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:10 smithi190 bash[23392]: cluster 2024-03-26T04:05:09.193829+0000 mgr.y (mgr.24431) 787 : cluster [DBG] pgmap v1269: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:10.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:10 smithi190 bash[23392]: audit 2024-03-26T04:05:10.070519+0000 mon.c (mon.2) 763 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:05:11.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:11 smithi175 bash[21527]: audit 2024-03-26T04:05:10.598657+0000 mon.a (mon.0) 3517 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:05:11.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:11 smithi175 bash[21527]: audit 2024-03-26T04:05:10.610151+0000 mon.a (mon.0) 3518 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:05:11.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:11 smithi175 bash[21527]: audit 2024-03-26T04:05:10.654514+0000 mgr.y (mgr.24431) 788 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:05:11.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:11 smithi175 bash[21527]: audit 2024-03-26T04:05:11.291174+0000 mon.c (mon.2) 764 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:05:11.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:11 smithi175 bash[21527]: audit 2024-03-26T04:05:11.293060+0000 mon.c (mon.2) 765 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:05:11.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:11 smithi175 bash[21527]: audit 2024-03-26T04:05:11.303284+0000 mon.a (mon.0) 3519 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:05:11.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:11 smithi175 bash[29284]: audit 2024-03-26T04:05:10.598657+0000 mon.a (mon.0) 3517 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:05:11.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:11 smithi175 bash[29284]: audit 2024-03-26T04:05:10.610151+0000 mon.a (mon.0) 3518 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:05:11.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:11 smithi175 bash[29284]: audit 2024-03-26T04:05:10.654514+0000 mgr.y (mgr.24431) 788 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:05:11.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:11 smithi175 bash[29284]: audit 2024-03-26T04:05:11.291174+0000 mon.c (mon.2) 764 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:05:11.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:11 smithi175 bash[29284]: audit 2024-03-26T04:05:11.293060+0000 mon.c (mon.2) 765 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:05:11.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:11 smithi175 bash[29284]: audit 2024-03-26T04:05:11.303284+0000 mon.a (mon.0) 3519 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:05:11.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:11 smithi190 bash[23392]: audit 2024-03-26T04:05:10.598657+0000 mon.a (mon.0) 3517 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:05:11.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:11 smithi190 bash[23392]: audit 2024-03-26T04:05:10.610151+0000 mon.a (mon.0) 3518 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:05:11.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:11 smithi190 bash[23392]: audit 2024-03-26T04:05:10.654514+0000 mgr.y (mgr.24431) 788 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:05:11.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:11 smithi190 bash[23392]: audit 2024-03-26T04:05:11.291174+0000 mon.c (mon.2) 764 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:05:11.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:11 smithi190 bash[23392]: audit 2024-03-26T04:05:11.293060+0000 mon.c (mon.2) 765 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:05:11.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:11 smithi190 bash[23392]: audit 2024-03-26T04:05:11.303284+0000 mon.a (mon.0) 3519 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:05:12.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:12 smithi175 bash[29284]: cluster 2024-03-26T04:05:11.195088+0000 mgr.y (mgr.24431) 789 : cluster [DBG] pgmap v1270: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:12.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:12 smithi175 bash[21527]: cluster 2024-03-26T04:05:11.195088+0000 mgr.y (mgr.24431) 789 : cluster [DBG] pgmap v1270: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:12.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:12 smithi190 bash[23392]: cluster 2024-03-26T04:05:11.195088+0000 mgr.y (mgr.24431) 789 : cluster [DBG] pgmap v1270: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:14.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:14 smithi175 bash[21527]: cluster 2024-03-26T04:05:13.195953+0000 mgr.y (mgr.24431) 790 : cluster [DBG] pgmap v1271: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:14.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:14 smithi175 bash[29284]: cluster 2024-03-26T04:05:13.195953+0000 mgr.y (mgr.24431) 790 : cluster [DBG] pgmap v1271: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:14.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:14 smithi190 bash[23392]: cluster 2024-03-26T04:05:13.195953+0000 mgr.y (mgr.24431) 790 : cluster [DBG] pgmap v1271: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:16.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:16 smithi175 bash[21527]: cluster 2024-03-26T04:05:15.198140+0000 mgr.y (mgr.24431) 791 : cluster [DBG] pgmap v1272: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:16.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:16 smithi175 bash[29284]: cluster 2024-03-26T04:05:15.198140+0000 mgr.y (mgr.24431) 791 : cluster [DBG] pgmap v1272: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:16.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:16 smithi190 bash[23392]: cluster 2024-03-26T04:05:15.198140+0000 mgr.y (mgr.24431) 791 : cluster [DBG] pgmap v1272: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:18.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:18 smithi175 bash[21527]: cluster 2024-03-26T04:05:17.198978+0000 mgr.y (mgr.24431) 792 : cluster [DBG] pgmap v1273: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:18.993 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:05:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:05:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:05:18.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:18 smithi175 bash[29284]: cluster 2024-03-26T04:05:17.198978+0000 mgr.y (mgr.24431) 792 : cluster [DBG] pgmap v1273: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:18.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:18 smithi190 bash[23392]: cluster 2024-03-26T04:05:17.198978+0000 mgr.y (mgr.24431) 792 : cluster [DBG] pgmap v1273: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:20.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:20 smithi175 bash[21527]: cluster 2024-03-26T04:05:19.199810+0000 mgr.y (mgr.24431) 793 : cluster [DBG] pgmap v1274: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:20.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:20 smithi175 bash[29284]: cluster 2024-03-26T04:05:19.199810+0000 mgr.y (mgr.24431) 793 : cluster [DBG] pgmap v1274: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:20.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:20 smithi190 bash[23392]: cluster 2024-03-26T04:05:19.199810+0000 mgr.y (mgr.24431) 793 : cluster [DBG] pgmap v1274: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:20.995 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:05:20 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:05:21.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:21 smithi175 bash[21527]: audit 2024-03-26T04:05:20.665734+0000 mgr.y (mgr.24431) 794 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:05:21.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:21 smithi175 bash[21527]: audit 2024-03-26T04:05:21.546306+0000 mon.c (mon.2) 766 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:05:21.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:21 smithi175 bash[29284]: audit 2024-03-26T04:05:20.665734+0000 mgr.y (mgr.24431) 794 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:05:21.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:21 smithi175 bash[29284]: audit 2024-03-26T04:05:21.546306+0000 mon.c (mon.2) 766 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:05:21.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:21 smithi190 bash[23392]: audit 2024-03-26T04:05:20.665734+0000 mgr.y (mgr.24431) 794 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:05:21.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:21 smithi190 bash[23392]: audit 2024-03-26T04:05:21.546306+0000 mon.c (mon.2) 766 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:05:22.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:22 smithi175 bash[29284]: cluster 2024-03-26T04:05:21.201989+0000 mgr.y (mgr.24431) 795 : cluster [DBG] pgmap v1275: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:22.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:22 smithi175 bash[21527]: cluster 2024-03-26T04:05:21.201989+0000 mgr.y (mgr.24431) 795 : cluster [DBG] pgmap v1275: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:22.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:22 smithi190 bash[23392]: cluster 2024-03-26T04:05:21.201989+0000 mgr.y (mgr.24431) 795 : cluster [DBG] pgmap v1275: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:24.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:24 smithi175 bash[29284]: cluster 2024-03-26T04:05:23.202923+0000 mgr.y (mgr.24431) 796 : cluster [DBG] pgmap v1276: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:24.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:24 smithi175 bash[21527]: cluster 2024-03-26T04:05:23.202923+0000 mgr.y (mgr.24431) 796 : cluster [DBG] pgmap v1276: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:24.994 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:24 smithi190 bash[23392]: cluster 2024-03-26T04:05:23.202923+0000 mgr.y (mgr.24431) 796 : cluster [DBG] pgmap v1276: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:26.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:26 smithi175 bash[21527]: cluster 2024-03-26T04:05:25.205335+0000 mgr.y (mgr.24431) 797 : cluster [DBG] pgmap v1277: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:26.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:26 smithi175 bash[29284]: cluster 2024-03-26T04:05:25.205335+0000 mgr.y (mgr.24431) 797 : cluster [DBG] pgmap v1277: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:26.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:26 smithi190 bash[23392]: cluster 2024-03-26T04:05:25.205335+0000 mgr.y (mgr.24431) 797 : cluster [DBG] pgmap v1277: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:28.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:28 smithi175 bash[21527]: cluster 2024-03-26T04:05:27.206210+0000 mgr.y (mgr.24431) 798 : cluster [DBG] pgmap v1278: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:28.992 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:05:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:05:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:05:28.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:28 smithi175 bash[29284]: cluster 2024-03-26T04:05:27.206210+0000 mgr.y (mgr.24431) 798 : cluster [DBG] pgmap v1278: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:28.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:28 smithi190 bash[23392]: cluster 2024-03-26T04:05:27.206210+0000 mgr.y (mgr.24431) 798 : cluster [DBG] pgmap v1278: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:30.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:30 smithi175 bash[21527]: cluster 2024-03-26T04:05:29.207172+0000 mgr.y (mgr.24431) 799 : cluster [DBG] pgmap v1279: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:30.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:30 smithi175 bash[29284]: cluster 2024-03-26T04:05:29.207172+0000 mgr.y (mgr.24431) 799 : cluster [DBG] pgmap v1279: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:30.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:30 smithi190 bash[23392]: cluster 2024-03-26T04:05:29.207172+0000 mgr.y (mgr.24431) 799 : cluster [DBG] pgmap v1279: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:30.995 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:05:30 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:05:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:32 smithi175 bash[21527]: audit 2024-03-26T04:05:30.677444+0000 mgr.y (mgr.24431) 800 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:05:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:32 smithi175 bash[21527]: cluster 2024-03-26T04:05:31.209980+0000 mgr.y (mgr.24431) 801 : cluster [DBG] pgmap v1280: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:32 smithi175 bash[29284]: audit 2024-03-26T04:05:30.677444+0000 mgr.y (mgr.24431) 800 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:05:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:32 smithi175 bash[29284]: cluster 2024-03-26T04:05:31.209980+0000 mgr.y (mgr.24431) 801 : cluster [DBG] pgmap v1280: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:32.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:32 smithi190 bash[23392]: audit 2024-03-26T04:05:30.677444+0000 mgr.y (mgr.24431) 800 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:05:32.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:32 smithi190 bash[23392]: cluster 2024-03-26T04:05:31.209980+0000 mgr.y (mgr.24431) 801 : cluster [DBG] pgmap v1280: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:34.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:34 smithi175 bash[29284]: cluster 2024-03-26T04:05:33.210794+0000 mgr.y (mgr.24431) 802 : cluster [DBG] pgmap v1281: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:34.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:34 smithi175 bash[21527]: cluster 2024-03-26T04:05:33.210794+0000 mgr.y (mgr.24431) 802 : cluster [DBG] pgmap v1281: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:34.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:34 smithi190 bash[23392]: cluster 2024-03-26T04:05:33.210794+0000 mgr.y (mgr.24431) 802 : cluster [DBG] pgmap v1281: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:36.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:36 smithi175 bash[29284]: cluster 2024-03-26T04:05:35.213012+0000 mgr.y (mgr.24431) 803 : cluster [DBG] pgmap v1282: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:36.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:36 smithi175 bash[29284]: audit 2024-03-26T04:05:36.578800+0000 mon.c (mon.2) 767 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:05:36.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:36 smithi175 bash[21527]: cluster 2024-03-26T04:05:35.213012+0000 mgr.y (mgr.24431) 803 : cluster [DBG] pgmap v1282: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:36.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:36 smithi175 bash[21527]: audit 2024-03-26T04:05:36.578800+0000 mon.c (mon.2) 767 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:05:36.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:36 smithi190 bash[23392]: cluster 2024-03-26T04:05:35.213012+0000 mgr.y (mgr.24431) 803 : cluster [DBG] pgmap v1282: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:36.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:36 smithi190 bash[23392]: audit 2024-03-26T04:05:36.578800+0000 mon.c (mon.2) 767 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:05:37.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:37 smithi175 bash[21527]: cluster 2024-03-26T04:05:37.213771+0000 mgr.y (mgr.24431) 804 : cluster [DBG] pgmap v1283: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:37.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:37 smithi175 bash[29284]: cluster 2024-03-26T04:05:37.213771+0000 mgr.y (mgr.24431) 804 : cluster [DBG] pgmap v1283: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:37.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:37 smithi190 bash[23392]: cluster 2024-03-26T04:05:37.213771+0000 mgr.y (mgr.24431) 804 : cluster [DBG] pgmap v1283: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:39.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:05:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:05:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:05:40.681 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:40 smithi190 bash[23392]: cluster 2024-03-26T04:05:39.214575+0000 mgr.y (mgr.24431) 805 : cluster [DBG] pgmap v1284: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:40.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:40 smithi175 bash[21527]: cluster 2024-03-26T04:05:39.214575+0000 mgr.y (mgr.24431) 805 : cluster [DBG] pgmap v1284: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:40.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:40 smithi175 bash[29284]: cluster 2024-03-26T04:05:39.214575+0000 mgr.y (mgr.24431) 805 : cluster [DBG] pgmap v1284: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:40.995 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:05:40 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:05:42.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:42 smithi175 bash[21527]: audit 2024-03-26T04:05:40.682978+0000 mgr.y (mgr.24431) 806 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:05:42.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:42 smithi175 bash[21527]: cluster 2024-03-26T04:05:41.216811+0000 mgr.y (mgr.24431) 807 : cluster [DBG] pgmap v1285: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:42.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:42 smithi175 bash[29284]: audit 2024-03-26T04:05:40.682978+0000 mgr.y (mgr.24431) 806 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:05:42.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:42 smithi175 bash[29284]: cluster 2024-03-26T04:05:41.216811+0000 mgr.y (mgr.24431) 807 : cluster [DBG] pgmap v1285: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:42.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:42 smithi190 bash[23392]: audit 2024-03-26T04:05:40.682978+0000 mgr.y (mgr.24431) 806 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:05:42.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:42 smithi190 bash[23392]: cluster 2024-03-26T04:05:41.216811+0000 mgr.y (mgr.24431) 807 : cluster [DBG] pgmap v1285: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:44.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:44 smithi175 bash[21527]: cluster 2024-03-26T04:05:43.217640+0000 mgr.y (mgr.24431) 808 : cluster [DBG] pgmap v1286: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:44.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:44 smithi175 bash[29284]: cluster 2024-03-26T04:05:43.217640+0000 mgr.y (mgr.24431) 808 : cluster [DBG] pgmap v1286: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:44.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:44 smithi190 bash[23392]: cluster 2024-03-26T04:05:43.217640+0000 mgr.y (mgr.24431) 808 : cluster [DBG] pgmap v1286: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:46.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:46 smithi175 bash[21527]: cluster 2024-03-26T04:05:45.219988+0000 mgr.y (mgr.24431) 809 : cluster [DBG] pgmap v1287: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:46.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:46 smithi175 bash[29284]: cluster 2024-03-26T04:05:45.219988+0000 mgr.y (mgr.24431) 809 : cluster [DBG] pgmap v1287: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:46.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:46 smithi190 bash[23392]: cluster 2024-03-26T04:05:45.219988+0000 mgr.y (mgr.24431) 809 : cluster [DBG] pgmap v1287: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:48.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:47 smithi175 bash[29284]: cluster 2024-03-26T04:05:47.220789+0000 mgr.y (mgr.24431) 810 : cluster [DBG] pgmap v1288: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:48.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:47 smithi175 bash[21527]: cluster 2024-03-26T04:05:47.220789+0000 mgr.y (mgr.24431) 810 : cluster [DBG] pgmap v1288: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:48.245 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:47 smithi190 bash[23392]: cluster 2024-03-26T04:05:47.220789+0000 mgr.y (mgr.24431) 810 : cluster [DBG] pgmap v1288: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:49.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:05:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:05:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:05:50.692 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:50 smithi190 bash[23392]: cluster 2024-03-26T04:05:49.221590+0000 mgr.y (mgr.24431) 811 : cluster [DBG] pgmap v1289: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:50.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:50 smithi175 bash[29284]: cluster 2024-03-26T04:05:49.221590+0000 mgr.y (mgr.24431) 811 : cluster [DBG] pgmap v1289: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:50.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:50 smithi175 bash[21527]: cluster 2024-03-26T04:05:49.221590+0000 mgr.y (mgr.24431) 811 : cluster [DBG] pgmap v1289: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:50.998 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:05:50 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:05:52.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:52 smithi175 bash[21527]: audit 2024-03-26T04:05:50.693859+0000 mgr.y (mgr.24431) 812 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:05:52.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:52 smithi175 bash[21527]: cluster 2024-03-26T04:05:51.223826+0000 mgr.y (mgr.24431) 813 : cluster [DBG] pgmap v1290: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:52.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:52 smithi175 bash[21527]: audit 2024-03-26T04:05:51.611971+0000 mon.c (mon.2) 768 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:05:52.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:52 smithi175 bash[29284]: audit 2024-03-26T04:05:50.693859+0000 mgr.y (mgr.24431) 812 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:05:52.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:52 smithi175 bash[29284]: cluster 2024-03-26T04:05:51.223826+0000 mgr.y (mgr.24431) 813 : cluster [DBG] pgmap v1290: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:52.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:52 smithi175 bash[29284]: audit 2024-03-26T04:05:51.611971+0000 mon.c (mon.2) 768 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:05:52.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:52 smithi190 bash[23392]: audit 2024-03-26T04:05:50.693859+0000 mgr.y (mgr.24431) 812 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:05:52.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:52 smithi190 bash[23392]: cluster 2024-03-26T04:05:51.223826+0000 mgr.y (mgr.24431) 813 : cluster [DBG] pgmap v1290: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:52.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:52 smithi190 bash[23392]: audit 2024-03-26T04:05:51.611971+0000 mon.c (mon.2) 768 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:05:54.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:54 smithi175 bash[21527]: cluster 2024-03-26T04:05:53.224623+0000 mgr.y (mgr.24431) 814 : cluster [DBG] pgmap v1291: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:54.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:54 smithi175 bash[29284]: cluster 2024-03-26T04:05:53.224623+0000 mgr.y (mgr.24431) 814 : cluster [DBG] pgmap v1291: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:54.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:54 smithi190 bash[23392]: cluster 2024-03-26T04:05:53.224623+0000 mgr.y (mgr.24431) 814 : cluster [DBG] pgmap v1291: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:56.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:56 smithi175 bash[21527]: cluster 2024-03-26T04:05:55.226877+0000 mgr.y (mgr.24431) 815 : cluster [DBG] pgmap v1292: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:56.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:56 smithi175 bash[29284]: cluster 2024-03-26T04:05:55.226877+0000 mgr.y (mgr.24431) 815 : cluster [DBG] pgmap v1292: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:56.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:56 smithi190 bash[23392]: cluster 2024-03-26T04:05:55.226877+0000 mgr.y (mgr.24431) 815 : cluster [DBG] pgmap v1292: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:05:58.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:05:58 smithi175 bash[21527]: cluster 2024-03-26T04:05:57.227737+0000 mgr.y (mgr.24431) 816 : cluster [DBG] pgmap v1293: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:58.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:05:58 smithi175 bash[29284]: cluster 2024-03-26T04:05:57.227737+0000 mgr.y (mgr.24431) 816 : cluster [DBG] pgmap v1293: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:58.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:05:58 smithi190 bash[23392]: cluster 2024-03-26T04:05:57.227737+0000 mgr.y (mgr.24431) 816 : cluster [DBG] pgmap v1293: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:05:59.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:05:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:05:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:06:00.696 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:00 smithi190 bash[23392]: cluster 2024-03-26T04:05:59.228571+0000 mgr.y (mgr.24431) 817 : cluster [DBG] pgmap v1294: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:00.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:00 smithi175 bash[29284]: cluster 2024-03-26T04:05:59.228571+0000 mgr.y (mgr.24431) 817 : cluster [DBG] pgmap v1294: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:00.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:00 smithi175 bash[21527]: cluster 2024-03-26T04:05:59.228571+0000 mgr.y (mgr.24431) 817 : cluster [DBG] pgmap v1294: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:00.995 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:06:00 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:06:02.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:02 smithi175 bash[21527]: audit 2024-03-26T04:06:00.698744+0000 mgr.y (mgr.24431) 818 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:06:02.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:02 smithi175 bash[21527]: cluster 2024-03-26T04:06:01.230804+0000 mgr.y (mgr.24431) 819 : cluster [DBG] pgmap v1295: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:02.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:02 smithi175 bash[29284]: audit 2024-03-26T04:06:00.698744+0000 mgr.y (mgr.24431) 818 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:06:02.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:02 smithi175 bash[29284]: cluster 2024-03-26T04:06:01.230804+0000 mgr.y (mgr.24431) 819 : cluster [DBG] pgmap v1295: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:02.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:02 smithi190 bash[23392]: audit 2024-03-26T04:06:00.698744+0000 mgr.y (mgr.24431) 818 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:06:02.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:02 smithi190 bash[23392]: cluster 2024-03-26T04:06:01.230804+0000 mgr.y (mgr.24431) 819 : cluster [DBG] pgmap v1295: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:04.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:04 smithi175 bash[21527]: cluster 2024-03-26T04:06:03.231689+0000 mgr.y (mgr.24431) 820 : cluster [DBG] pgmap v1296: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:04.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:04 smithi175 bash[29284]: cluster 2024-03-26T04:06:03.231689+0000 mgr.y (mgr.24431) 820 : cluster [DBG] pgmap v1296: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:04.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:04 smithi190 bash[23392]: cluster 2024-03-26T04:06:03.231689+0000 mgr.y (mgr.24431) 820 : cluster [DBG] pgmap v1296: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:06.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:06 smithi175 bash[21527]: cluster 2024-03-26T04:06:05.233877+0000 mgr.y (mgr.24431) 821 : cluster [DBG] pgmap v1297: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:06.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:06 smithi175 bash[29284]: cluster 2024-03-26T04:06:05.233877+0000 mgr.y (mgr.24431) 821 : cluster [DBG] pgmap v1297: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:06.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:06 smithi190 bash[23392]: cluster 2024-03-26T04:06:05.233877+0000 mgr.y (mgr.24431) 821 : cluster [DBG] pgmap v1297: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:07.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:07 smithi175 bash[21527]: audit 2024-03-26T04:06:06.649872+0000 mon.c (mon.2) 769 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:06:07.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:07 smithi175 bash[29284]: audit 2024-03-26T04:06:06.649872+0000 mon.c (mon.2) 769 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:06:07.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:07 smithi190 bash[23392]: audit 2024-03-26T04:06:06.649872+0000 mon.c (mon.2) 769 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:06:08.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:08 smithi175 bash[21527]: cluster 2024-03-26T04:06:07.234714+0000 mgr.y (mgr.24431) 822 : cluster [DBG] pgmap v1298: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:08.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:08 smithi175 bash[29284]: cluster 2024-03-26T04:06:07.234714+0000 mgr.y (mgr.24431) 822 : cluster [DBG] pgmap v1298: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:08.745 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:08 smithi190 bash[23392]: cluster 2024-03-26T04:06:07.234714+0000 mgr.y (mgr.24431) 822 : cluster [DBG] pgmap v1298: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:09.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:06:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:06:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:06:10.708 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:10 smithi190 bash[23392]: cluster 2024-03-26T04:06:09.235585+0000 mgr.y (mgr.24431) 823 : cluster [DBG] pgmap v1299: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:10.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:10 smithi175 bash[21527]: cluster 2024-03-26T04:06:09.235585+0000 mgr.y (mgr.24431) 823 : cluster [DBG] pgmap v1299: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:10.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:10 smithi175 bash[29284]: cluster 2024-03-26T04:06:09.235585+0000 mgr.y (mgr.24431) 823 : cluster [DBG] pgmap v1299: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:10.995 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:06:10 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:06:12.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:12 smithi175 bash[21527]: audit 2024-03-26T04:06:10.710362+0000 mgr.y (mgr.24431) 824 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:06:12.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:12 smithi175 bash[21527]: cluster 2024-03-26T04:06:11.237798+0000 mgr.y (mgr.24431) 825 : cluster [DBG] pgmap v1300: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:12.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:12 smithi175 bash[21527]: audit 2024-03-26T04:06:11.391491+0000 mon.c (mon.2) 770 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:06:12.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:12 smithi175 bash[21527]: audit 2024-03-26T04:06:11.917366+0000 mon.a (mon.0) 3520 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:06:12.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:12 smithi175 bash[21527]: audit 2024-03-26T04:06:11.930079+0000 mon.a (mon.0) 3521 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:06:12.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:12 smithi175 bash[29284]: audit 2024-03-26T04:06:10.710362+0000 mgr.y (mgr.24431) 824 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:06:12.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:12 smithi175 bash[29284]: cluster 2024-03-26T04:06:11.237798+0000 mgr.y (mgr.24431) 825 : cluster [DBG] pgmap v1300: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:12.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:12 smithi175 bash[29284]: audit 2024-03-26T04:06:11.391491+0000 mon.c (mon.2) 770 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:06:12.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:12 smithi175 bash[29284]: audit 2024-03-26T04:06:11.917366+0000 mon.a (mon.0) 3520 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:06:12.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:12 smithi175 bash[29284]: audit 2024-03-26T04:06:11.930079+0000 mon.a (mon.0) 3521 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:06:12.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:12 smithi190 bash[23392]: audit 2024-03-26T04:06:10.710362+0000 mgr.y (mgr.24431) 824 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:06:12.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:12 smithi190 bash[23392]: cluster 2024-03-26T04:06:11.237798+0000 mgr.y (mgr.24431) 825 : cluster [DBG] pgmap v1300: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:12.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:12 smithi190 bash[23392]: audit 2024-03-26T04:06:11.391491+0000 mon.c (mon.2) 770 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:06:12.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:12 smithi190 bash[23392]: audit 2024-03-26T04:06:11.917366+0000 mon.a (mon.0) 3520 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:06:12.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:12 smithi190 bash[23392]: audit 2024-03-26T04:06:11.930079+0000 mon.a (mon.0) 3521 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:06:13.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:13 smithi175 bash[21527]: audit 2024-03-26T04:06:12.625670+0000 mon.c (mon.2) 771 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:06:13.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:13 smithi175 bash[21527]: audit 2024-03-26T04:06:12.627631+0000 mon.c (mon.2) 772 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:06:13.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:13 smithi175 bash[21527]: audit 2024-03-26T04:06:12.640742+0000 mon.a (mon.0) 3522 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:06:13.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:13 smithi175 bash[29284]: audit 2024-03-26T04:06:12.625670+0000 mon.c (mon.2) 771 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:06:13.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:13 smithi175 bash[29284]: audit 2024-03-26T04:06:12.627631+0000 mon.c (mon.2) 772 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:06:13.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:13 smithi175 bash[29284]: audit 2024-03-26T04:06:12.640742+0000 mon.a (mon.0) 3522 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:06:13.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:13 smithi190 bash[23392]: audit 2024-03-26T04:06:12.625670+0000 mon.c (mon.2) 771 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:06:13.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:13 smithi190 bash[23392]: audit 2024-03-26T04:06:12.627631+0000 mon.c (mon.2) 772 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:06:13.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:13 smithi190 bash[23392]: audit 2024-03-26T04:06:12.640742+0000 mon.a (mon.0) 3522 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:06:14.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:14 smithi175 bash[21527]: cluster 2024-03-26T04:06:13.238713+0000 mgr.y (mgr.24431) 826 : cluster [DBG] pgmap v1301: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:14.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:14 smithi175 bash[29284]: cluster 2024-03-26T04:06:13.238713+0000 mgr.y (mgr.24431) 826 : cluster [DBG] pgmap v1301: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:14.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:14 smithi190 bash[23392]: cluster 2024-03-26T04:06:13.238713+0000 mgr.y (mgr.24431) 826 : cluster [DBG] pgmap v1301: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:16.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:16 smithi175 bash[29284]: cluster 2024-03-26T04:06:15.240952+0000 mgr.y (mgr.24431) 827 : cluster [DBG] pgmap v1302: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:16.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:16 smithi175 bash[21527]: cluster 2024-03-26T04:06:15.240952+0000 mgr.y (mgr.24431) 827 : cluster [DBG] pgmap v1302: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:16.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:16 smithi190 bash[23392]: cluster 2024-03-26T04:06:15.240952+0000 mgr.y (mgr.24431) 827 : cluster [DBG] pgmap v1302: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:18.992 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:06:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:06:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:06:18.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:18 smithi175 bash[29284]: cluster 2024-03-26T04:06:17.241834+0000 mgr.y (mgr.24431) 828 : cluster [DBG] pgmap v1303: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:18.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:18 smithi175 bash[21527]: cluster 2024-03-26T04:06:17.241834+0000 mgr.y (mgr.24431) 828 : cluster [DBG] pgmap v1303: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:18.995 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:18 smithi190 bash[23392]: cluster 2024-03-26T04:06:17.241834+0000 mgr.y (mgr.24431) 828 : cluster [DBG] pgmap v1303: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:20.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:20 smithi175 bash[29284]: cluster 2024-03-26T04:06:19.242689+0000 mgr.y (mgr.24431) 829 : cluster [DBG] pgmap v1304: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:20.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:20 smithi175 bash[21527]: cluster 2024-03-26T04:06:19.242689+0000 mgr.y (mgr.24431) 829 : cluster [DBG] pgmap v1304: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:20.996 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:20 smithi190 bash[23392]: cluster 2024-03-26T04:06:19.242689+0000 mgr.y (mgr.24431) 829 : cluster [DBG] pgmap v1304: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:20.996 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:06:20 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:06:22.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:22 smithi175 bash[29284]: audit 2024-03-26T04:06:20.715353+0000 mgr.y (mgr.24431) 830 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:06:22.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:22 smithi175 bash[29284]: cluster 2024-03-26T04:06:21.244903+0000 mgr.y (mgr.24431) 831 : cluster [DBG] pgmap v1305: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:22.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:22 smithi175 bash[29284]: audit 2024-03-26T04:06:21.682684+0000 mon.c (mon.2) 773 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:06:22.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:22 smithi175 bash[21527]: audit 2024-03-26T04:06:20.715353+0000 mgr.y (mgr.24431) 830 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:06:22.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:22 smithi175 bash[21527]: cluster 2024-03-26T04:06:21.244903+0000 mgr.y (mgr.24431) 831 : cluster [DBG] pgmap v1305: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:22.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:22 smithi175 bash[21527]: audit 2024-03-26T04:06:21.682684+0000 mon.c (mon.2) 773 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:06:22.996 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:22 smithi190 bash[23392]: audit 2024-03-26T04:06:20.715353+0000 mgr.y (mgr.24431) 830 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:06:22.996 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:22 smithi190 bash[23392]: cluster 2024-03-26T04:06:21.244903+0000 mgr.y (mgr.24431) 831 : cluster [DBG] pgmap v1305: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:22.996 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:22 smithi190 bash[23392]: audit 2024-03-26T04:06:21.682684+0000 mon.c (mon.2) 773 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:06:24.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:24 smithi175 bash[29284]: cluster 2024-03-26T04:06:23.245708+0000 mgr.y (mgr.24431) 832 : cluster [DBG] pgmap v1306: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:24.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:24 smithi175 bash[21527]: cluster 2024-03-26T04:06:23.245708+0000 mgr.y (mgr.24431) 832 : cluster [DBG] pgmap v1306: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:24.996 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:24 smithi190 bash[23392]: cluster 2024-03-26T04:06:23.245708+0000 mgr.y (mgr.24431) 832 : cluster [DBG] pgmap v1306: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:26.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:26 smithi175 bash[21527]: cluster 2024-03-26T04:06:25.247938+0000 mgr.y (mgr.24431) 833 : cluster [DBG] pgmap v1307: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:26.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:26 smithi175 bash[29284]: cluster 2024-03-26T04:06:25.247938+0000 mgr.y (mgr.24431) 833 : cluster [DBG] pgmap v1307: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:26.996 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:26 smithi190 bash[23392]: cluster 2024-03-26T04:06:25.247938+0000 mgr.y (mgr.24431) 833 : cluster [DBG] pgmap v1307: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:28.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:28 smithi175 bash[21527]: cluster 2024-03-26T04:06:27.248763+0000 mgr.y (mgr.24431) 834 : cluster [DBG] pgmap v1308: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:28.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:28 smithi175 bash[29284]: cluster 2024-03-26T04:06:27.248763+0000 mgr.y (mgr.24431) 834 : cluster [DBG] pgmap v1308: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:28.993 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:06:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:06:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:06:28.996 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:28 smithi190 bash[23392]: cluster 2024-03-26T04:06:27.248763+0000 mgr.y (mgr.24431) 834 : cluster [DBG] pgmap v1308: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:30.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:30 smithi175 bash[21527]: cluster 2024-03-26T04:06:29.249598+0000 mgr.y (mgr.24431) 835 : cluster [DBG] pgmap v1309: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:30.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:30 smithi175 bash[29284]: cluster 2024-03-26T04:06:29.249598+0000 mgr.y (mgr.24431) 835 : cluster [DBG] pgmap v1309: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:30.996 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:30 smithi190 bash[23392]: cluster 2024-03-26T04:06:29.249598+0000 mgr.y (mgr.24431) 835 : cluster [DBG] pgmap v1309: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:30.996 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:06:30 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:06:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:32 smithi175 bash[21527]: audit 2024-03-26T04:06:30.727041+0000 mgr.y (mgr.24431) 836 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:06:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:32 smithi175 bash[21527]: cluster 2024-03-26T04:06:31.251905+0000 mgr.y (mgr.24431) 837 : cluster [DBG] pgmap v1310: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:32.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:32 smithi175 bash[29284]: audit 2024-03-26T04:06:30.727041+0000 mgr.y (mgr.24431) 836 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:06:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:32 smithi175 bash[29284]: cluster 2024-03-26T04:06:31.251905+0000 mgr.y (mgr.24431) 837 : cluster [DBG] pgmap v1310: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:32.996 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:32 smithi190 bash[23392]: audit 2024-03-26T04:06:30.727041+0000 mgr.y (mgr.24431) 836 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:06:32.996 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:32 smithi190 bash[23392]: cluster 2024-03-26T04:06:31.251905+0000 mgr.y (mgr.24431) 837 : cluster [DBG] pgmap v1310: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:33.996 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 04:06:33 smithi190 bash[50860]: logger=cleanup t=2024-03-26T04:06:33.561417063Z level=info msg="Completed cleanup jobs" duration=56.039276ms 2024-03-26T04:06:34.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:34 smithi175 bash[21527]: cluster 2024-03-26T04:06:33.252736+0000 mgr.y (mgr.24431) 838 : cluster [DBG] pgmap v1311: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:34.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:34 smithi175 bash[29284]: cluster 2024-03-26T04:06:33.252736+0000 mgr.y (mgr.24431) 838 : cluster [DBG] pgmap v1311: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:34.996 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:34 smithi190 bash[23392]: cluster 2024-03-26T04:06:33.252736+0000 mgr.y (mgr.24431) 838 : cluster [DBG] pgmap v1311: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:36.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:36 smithi175 bash[21527]: cluster 2024-03-26T04:06:35.255001+0000 mgr.y (mgr.24431) 839 : cluster [DBG] pgmap v1312: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:36.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:36 smithi175 bash[29284]: cluster 2024-03-26T04:06:35.255001+0000 mgr.y (mgr.24431) 839 : cluster [DBG] pgmap v1312: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:36.996 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:36 smithi190 bash[23392]: cluster 2024-03-26T04:06:35.255001+0000 mgr.y (mgr.24431) 839 : cluster [DBG] pgmap v1312: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:37.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:37 smithi175 bash[29284]: audit 2024-03-26T04:06:36.717209+0000 mon.c (mon.2) 774 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:06:37.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:37 smithi175 bash[21527]: audit 2024-03-26T04:06:36.717209+0000 mon.c (mon.2) 774 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:06:37.996 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:37 smithi190 bash[23392]: audit 2024-03-26T04:06:36.717209+0000 mon.c (mon.2) 774 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:06:38.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:38 smithi175 bash[21527]: cluster 2024-03-26T04:06:37.255873+0000 mgr.y (mgr.24431) 840 : cluster [DBG] pgmap v1313: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:38.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:38 smithi175 bash[29284]: cluster 2024-03-26T04:06:37.255873+0000 mgr.y (mgr.24431) 840 : cluster [DBG] pgmap v1313: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:38.992 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:06:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:06:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:06:38.996 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:38 smithi190 bash[23392]: cluster 2024-03-26T04:06:37.255873+0000 mgr.y (mgr.24431) 840 : cluster [DBG] pgmap v1313: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:39.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:39 smithi175 bash[21527]: cluster 2024-03-26T04:06:39.256717+0000 mgr.y (mgr.24431) 841 : cluster [DBG] pgmap v1314: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:39.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:39 smithi175 bash[29284]: cluster 2024-03-26T04:06:39.256717+0000 mgr.y (mgr.24431) 841 : cluster [DBG] pgmap v1314: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:39.996 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:39 smithi190 bash[23392]: cluster 2024-03-26T04:06:39.256717+0000 mgr.y (mgr.24431) 841 : cluster [DBG] pgmap v1314: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:40.996 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:06:40 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:06:42.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:42 smithi175 bash[29284]: audit 2024-03-26T04:06:40.738807+0000 mgr.y (mgr.24431) 842 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:06:42.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:42 smithi175 bash[29284]: cluster 2024-03-26T04:06:41.259061+0000 mgr.y (mgr.24431) 843 : cluster [DBG] pgmap v1315: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:42.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:42 smithi175 bash[21527]: audit 2024-03-26T04:06:40.738807+0000 mgr.y (mgr.24431) 842 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:06:42.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:42 smithi175 bash[21527]: cluster 2024-03-26T04:06:41.259061+0000 mgr.y (mgr.24431) 843 : cluster [DBG] pgmap v1315: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:42.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:42 smithi190 bash[23392]: audit 2024-03-26T04:06:40.738807+0000 mgr.y (mgr.24431) 842 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:06:42.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:42 smithi190 bash[23392]: cluster 2024-03-26T04:06:41.259061+0000 mgr.y (mgr.24431) 843 : cluster [DBG] pgmap v1315: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:44.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:44 smithi175 bash[29284]: cluster 2024-03-26T04:06:43.259925+0000 mgr.y (mgr.24431) 844 : cluster [DBG] pgmap v1316: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:44.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:44 smithi175 bash[21527]: cluster 2024-03-26T04:06:43.259925+0000 mgr.y (mgr.24431) 844 : cluster [DBG] pgmap v1316: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:44.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:44 smithi190 bash[23392]: cluster 2024-03-26T04:06:43.259925+0000 mgr.y (mgr.24431) 844 : cluster [DBG] pgmap v1316: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:46.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:46 smithi175 bash[29284]: cluster 2024-03-26T04:06:45.262137+0000 mgr.y (mgr.24431) 845 : cluster [DBG] pgmap v1317: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:46.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:46 smithi175 bash[21527]: cluster 2024-03-26T04:06:45.262137+0000 mgr.y (mgr.24431) 845 : cluster [DBG] pgmap v1317: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:46.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:46 smithi190 bash[23392]: cluster 2024-03-26T04:06:45.262137+0000 mgr.y (mgr.24431) 845 : cluster [DBG] pgmap v1317: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:48.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:47 smithi175 bash[29284]: cluster 2024-03-26T04:06:47.262965+0000 mgr.y (mgr.24431) 846 : cluster [DBG] pgmap v1318: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:48.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:47 smithi175 bash[21527]: cluster 2024-03-26T04:06:47.262965+0000 mgr.y (mgr.24431) 846 : cluster [DBG] pgmap v1318: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:48.246 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:47 smithi190 bash[23392]: cluster 2024-03-26T04:06:47.262965+0000 mgr.y (mgr.24431) 846 : cluster [DBG] pgmap v1318: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:49.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:06:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:06:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:06:50.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:50 smithi175 bash[21527]: cluster 2024-03-26T04:06:49.263856+0000 mgr.y (mgr.24431) 847 : cluster [DBG] pgmap v1319: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:50.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:50 smithi175 bash[29284]: cluster 2024-03-26T04:06:49.263856+0000 mgr.y (mgr.24431) 847 : cluster [DBG] pgmap v1319: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:50.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:50 smithi190 bash[23392]: cluster 2024-03-26T04:06:49.263856+0000 mgr.y (mgr.24431) 847 : cluster [DBG] pgmap v1319: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:51.246 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:06:50 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:06:52.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:52 smithi175 bash[21527]: audit 2024-03-26T04:06:50.750498+0000 mgr.y (mgr.24431) 848 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:06:52.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:52 smithi175 bash[21527]: cluster 2024-03-26T04:06:51.266174+0000 mgr.y (mgr.24431) 849 : cluster [DBG] pgmap v1320: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:52.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:52 smithi175 bash[21527]: audit 2024-03-26T04:06:51.751635+0000 mon.c (mon.2) 775 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:06:52.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:52 smithi175 bash[29284]: audit 2024-03-26T04:06:50.750498+0000 mgr.y (mgr.24431) 848 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:06:52.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:52 smithi175 bash[29284]: cluster 2024-03-26T04:06:51.266174+0000 mgr.y (mgr.24431) 849 : cluster [DBG] pgmap v1320: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:52.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:52 smithi175 bash[29284]: audit 2024-03-26T04:06:51.751635+0000 mon.c (mon.2) 775 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:06:52.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:52 smithi190 bash[23392]: audit 2024-03-26T04:06:50.750498+0000 mgr.y (mgr.24431) 848 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:06:52.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:52 smithi190 bash[23392]: cluster 2024-03-26T04:06:51.266174+0000 mgr.y (mgr.24431) 849 : cluster [DBG] pgmap v1320: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:52.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:52 smithi190 bash[23392]: audit 2024-03-26T04:06:51.751635+0000 mon.c (mon.2) 775 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:06:54.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:54 smithi175 bash[21527]: cluster 2024-03-26T04:06:53.267039+0000 mgr.y (mgr.24431) 850 : cluster [DBG] pgmap v1321: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:54.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:54 smithi175 bash[29284]: cluster 2024-03-26T04:06:53.267039+0000 mgr.y (mgr.24431) 850 : cluster [DBG] pgmap v1321: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:54.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:54 smithi190 bash[23392]: cluster 2024-03-26T04:06:53.267039+0000 mgr.y (mgr.24431) 850 : cluster [DBG] pgmap v1321: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:56.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:56 smithi175 bash[21527]: cluster 2024-03-26T04:06:55.269332+0000 mgr.y (mgr.24431) 851 : cluster [DBG] pgmap v1322: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:56.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:56 smithi175 bash[29284]: cluster 2024-03-26T04:06:55.269332+0000 mgr.y (mgr.24431) 851 : cluster [DBG] pgmap v1322: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:56.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:56 smithi190 bash[23392]: cluster 2024-03-26T04:06:55.269332+0000 mgr.y (mgr.24431) 851 : cluster [DBG] pgmap v1322: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:06:58.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:06:58 smithi175 bash[21527]: cluster 2024-03-26T04:06:57.270122+0000 mgr.y (mgr.24431) 852 : cluster [DBG] pgmap v1323: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:58.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:06:58 smithi175 bash[29284]: cluster 2024-03-26T04:06:57.270122+0000 mgr.y (mgr.24431) 852 : cluster [DBG] pgmap v1323: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:58.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:06:58 smithi190 bash[23392]: cluster 2024-03-26T04:06:57.270122+0000 mgr.y (mgr.24431) 852 : cluster [DBG] pgmap v1323: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:06:59.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:06:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:06:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:07:00.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:00 smithi175 bash[21527]: cluster 2024-03-26T04:06:59.270963+0000 mgr.y (mgr.24431) 853 : cluster [DBG] pgmap v1324: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:00.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:00 smithi175 bash[29284]: cluster 2024-03-26T04:06:59.270963+0000 mgr.y (mgr.24431) 853 : cluster [DBG] pgmap v1324: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:00.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:00 smithi190 bash[23392]: cluster 2024-03-26T04:06:59.270963+0000 mgr.y (mgr.24431) 853 : cluster [DBG] pgmap v1324: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:01.246 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:07:00 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:07:02.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:02 smithi175 bash[21527]: audit 2024-03-26T04:07:00.759276+0000 mgr.y (mgr.24431) 854 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:07:02.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:02 smithi175 bash[21527]: cluster 2024-03-26T04:07:01.273235+0000 mgr.y (mgr.24431) 855 : cluster [DBG] pgmap v1325: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:02.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:02 smithi175 bash[29284]: audit 2024-03-26T04:07:00.759276+0000 mgr.y (mgr.24431) 854 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:07:02.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:02 smithi175 bash[29284]: cluster 2024-03-26T04:07:01.273235+0000 mgr.y (mgr.24431) 855 : cluster [DBG] pgmap v1325: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:02.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:02 smithi190 bash[23392]: audit 2024-03-26T04:07:00.759276+0000 mgr.y (mgr.24431) 854 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:07:02.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:02 smithi190 bash[23392]: cluster 2024-03-26T04:07:01.273235+0000 mgr.y (mgr.24431) 855 : cluster [DBG] pgmap v1325: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:04.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:04 smithi175 bash[21527]: cluster 2024-03-26T04:07:03.274076+0000 mgr.y (mgr.24431) 856 : cluster [DBG] pgmap v1326: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:04.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:04 smithi175 bash[29284]: cluster 2024-03-26T04:07:03.274076+0000 mgr.y (mgr.24431) 856 : cluster [DBG] pgmap v1326: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:04.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:04 smithi190 bash[23392]: cluster 2024-03-26T04:07:03.274076+0000 mgr.y (mgr.24431) 856 : cluster [DBG] pgmap v1326: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:06.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:06 smithi175 bash[21527]: cluster 2024-03-26T04:07:05.276285+0000 mgr.y (mgr.24431) 857 : cluster [DBG] pgmap v1327: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:06.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:06 smithi175 bash[29284]: cluster 2024-03-26T04:07:05.276285+0000 mgr.y (mgr.24431) 857 : cluster [DBG] pgmap v1327: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:06.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:06 smithi190 bash[23392]: cluster 2024-03-26T04:07:05.276285+0000 mgr.y (mgr.24431) 857 : cluster [DBG] pgmap v1327: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:07.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:07 smithi175 bash[21527]: audit 2024-03-26T04:07:06.785074+0000 mon.c (mon.2) 776 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:07:07.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:07 smithi175 bash[29284]: audit 2024-03-26T04:07:06.785074+0000 mon.c (mon.2) 776 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:07:07.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:07 smithi190 bash[23392]: audit 2024-03-26T04:07:06.785074+0000 mon.c (mon.2) 776 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:07:08.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:08 smithi175 bash[21527]: cluster 2024-03-26T04:07:07.277130+0000 mgr.y (mgr.24431) 858 : cluster [DBG] pgmap v1328: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:08.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:08 smithi175 bash[29284]: cluster 2024-03-26T04:07:07.277130+0000 mgr.y (mgr.24431) 858 : cluster [DBG] pgmap v1328: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:08.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:08 smithi190 bash[23392]: cluster 2024-03-26T04:07:07.277130+0000 mgr.y (mgr.24431) 858 : cluster [DBG] pgmap v1328: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:09.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:07:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:07:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:07:10.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:10 smithi175 bash[21527]: cluster 2024-03-26T04:07:09.277903+0000 mgr.y (mgr.24431) 859 : cluster [DBG] pgmap v1329: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:10.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:10 smithi175 bash[29284]: cluster 2024-03-26T04:07:09.277903+0000 mgr.y (mgr.24431) 859 : cluster [DBG] pgmap v1329: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:10.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:10 smithi190 bash[23392]: cluster 2024-03-26T04:07:09.277903+0000 mgr.y (mgr.24431) 859 : cluster [DBG] pgmap v1329: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:11.247 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:07:10 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:07:12.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:12 smithi175 bash[21527]: audit 2024-03-26T04:07:10.767357+0000 mgr.y (mgr.24431) 860 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:07:12.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:12 smithi175 bash[21527]: cluster 2024-03-26T04:07:11.280110+0000 mgr.y (mgr.24431) 861 : cluster [DBG] pgmap v1330: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:12.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:12 smithi175 bash[29284]: audit 2024-03-26T04:07:10.767357+0000 mgr.y (mgr.24431) 860 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:07:12.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:12 smithi175 bash[29284]: cluster 2024-03-26T04:07:11.280110+0000 mgr.y (mgr.24431) 861 : cluster [DBG] pgmap v1330: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:12.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:12 smithi190 bash[23392]: audit 2024-03-26T04:07:10.767357+0000 mgr.y (mgr.24431) 860 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:07:12.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:12 smithi190 bash[23392]: cluster 2024-03-26T04:07:11.280110+0000 mgr.y (mgr.24431) 861 : cluster [DBG] pgmap v1330: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:13.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:13 smithi175 bash[21527]: audit 2024-03-26T04:07:12.734273+0000 mon.c (mon.2) 777 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:07:13.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:13 smithi175 bash[29284]: audit 2024-03-26T04:07:12.734273+0000 mon.c (mon.2) 777 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:07:13.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:13 smithi190 bash[23392]: audit 2024-03-26T04:07:12.734273+0000 mon.c (mon.2) 777 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:07:14.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:14 smithi175 bash[21527]: cluster 2024-03-26T04:07:13.280942+0000 mgr.y (mgr.24431) 862 : cluster [DBG] pgmap v1331: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:14.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:14 smithi175 bash[29284]: cluster 2024-03-26T04:07:13.280942+0000 mgr.y (mgr.24431) 862 : cluster [DBG] pgmap v1331: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:14.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:14 smithi190 bash[23392]: cluster 2024-03-26T04:07:13.280942+0000 mgr.y (mgr.24431) 862 : cluster [DBG] pgmap v1331: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:16.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:16 smithi175 bash[21527]: cluster 2024-03-26T04:07:15.282434+0000 mgr.y (mgr.24431) 863 : cluster [DBG] pgmap v1332: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:16.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:16 smithi175 bash[29284]: cluster 2024-03-26T04:07:15.282434+0000 mgr.y (mgr.24431) 863 : cluster [DBG] pgmap v1332: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:16.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:16 smithi190 bash[23392]: cluster 2024-03-26T04:07:15.282434+0000 mgr.y (mgr.24431) 863 : cluster [DBG] pgmap v1332: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:18.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:18 smithi175 bash[21527]: cluster 2024-03-26T04:07:17.282977+0000 mgr.y (mgr.24431) 864 : cluster [DBG] pgmap v1333: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:18.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:18 smithi175 bash[21527]: audit 2024-03-26T04:07:18.374985+0000 mon.a (mon.0) 3523 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:07:18.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:18 smithi175 bash[21527]: audit 2024-03-26T04:07:18.384936+0000 mon.a (mon.0) 3524 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:07:18.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:18 smithi175 bash[29284]: cluster 2024-03-26T04:07:17.282977+0000 mgr.y (mgr.24431) 864 : cluster [DBG] pgmap v1333: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:18.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:18 smithi175 bash[29284]: audit 2024-03-26T04:07:18.374985+0000 mon.a (mon.0) 3523 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:07:18.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:18 smithi175 bash[29284]: audit 2024-03-26T04:07:18.384936+0000 mon.a (mon.0) 3524 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:07:18.746 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:18 smithi190 bash[23392]: cluster 2024-03-26T04:07:17.282977+0000 mgr.y (mgr.24431) 864 : cluster [DBG] pgmap v1333: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:18.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:18 smithi190 bash[23392]: audit 2024-03-26T04:07:18.374985+0000 mon.a (mon.0) 3523 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:07:18.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:18 smithi190 bash[23392]: audit 2024-03-26T04:07:18.384936+0000 mon.a (mon.0) 3524 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:07:19.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:07:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:07:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:07:20.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:19 smithi175 bash[21527]: audit 2024-03-26T04:07:18.829059+0000 mon.a (mon.0) 3525 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:07:20.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:19 smithi175 bash[21527]: audit 2024-03-26T04:07:18.840396+0000 mon.a (mon.0) 3526 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:07:20.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:19 smithi175 bash[21527]: cluster 2024-03-26T04:07:19.283802+0000 mgr.y (mgr.24431) 865 : cluster [DBG] pgmap v1334: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:20.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:19 smithi175 bash[21527]: audit 2024-03-26T04:07:19.575239+0000 mon.c (mon.2) 778 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:07:20.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:19 smithi175 bash[21527]: audit 2024-03-26T04:07:19.577158+0000 mon.c (mon.2) 779 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:07:20.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:19 smithi175 bash[21527]: audit 2024-03-26T04:07:19.589031+0000 mon.a (mon.0) 3527 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:07:20.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:19 smithi175 bash[29284]: audit 2024-03-26T04:07:18.829059+0000 mon.a (mon.0) 3525 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:07:20.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:19 smithi175 bash[29284]: audit 2024-03-26T04:07:18.840396+0000 mon.a (mon.0) 3526 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:07:20.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:19 smithi175 bash[29284]: cluster 2024-03-26T04:07:19.283802+0000 mgr.y (mgr.24431) 865 : cluster [DBG] pgmap v1334: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:20.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:19 smithi175 bash[29284]: audit 2024-03-26T04:07:19.575239+0000 mon.c (mon.2) 778 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:07:20.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:19 smithi175 bash[29284]: audit 2024-03-26T04:07:19.577158+0000 mon.c (mon.2) 779 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:07:20.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:19 smithi175 bash[29284]: audit 2024-03-26T04:07:19.589031+0000 mon.a (mon.0) 3527 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:07:20.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:19 smithi190 bash[23392]: audit 2024-03-26T04:07:18.829059+0000 mon.a (mon.0) 3525 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:07:20.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:19 smithi190 bash[23392]: audit 2024-03-26T04:07:18.840396+0000 mon.a (mon.0) 3526 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:07:20.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:19 smithi190 bash[23392]: cluster 2024-03-26T04:07:19.283802+0000 mgr.y (mgr.24431) 865 : cluster [DBG] pgmap v1334: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:20.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:19 smithi190 bash[23392]: audit 2024-03-26T04:07:19.575239+0000 mon.c (mon.2) 778 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:07:20.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:19 smithi190 bash[23392]: audit 2024-03-26T04:07:19.577158+0000 mon.c (mon.2) 779 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:07:20.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:19 smithi190 bash[23392]: audit 2024-03-26T04:07:19.589031+0000 mon.a (mon.0) 3527 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:07:21.247 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:07:20 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:07:22.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:22 smithi175 bash[21527]: audit 2024-03-26T04:07:20.779064+0000 mgr.y (mgr.24431) 866 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:07:22.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:22 smithi175 bash[21527]: cluster 2024-03-26T04:07:21.286134+0000 mgr.y (mgr.24431) 867 : cluster [DBG] pgmap v1335: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:22.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:22 smithi175 bash[21527]: audit 2024-03-26T04:07:21.819033+0000 mon.c (mon.2) 780 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:07:22.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:22 smithi175 bash[29284]: audit 2024-03-26T04:07:20.779064+0000 mgr.y (mgr.24431) 866 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:07:22.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:22 smithi175 bash[29284]: cluster 2024-03-26T04:07:21.286134+0000 mgr.y (mgr.24431) 867 : cluster [DBG] pgmap v1335: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:22.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:22 smithi175 bash[29284]: audit 2024-03-26T04:07:21.819033+0000 mon.c (mon.2) 780 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:07:22.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:22 smithi190 bash[23392]: audit 2024-03-26T04:07:20.779064+0000 mgr.y (mgr.24431) 866 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:07:22.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:22 smithi190 bash[23392]: cluster 2024-03-26T04:07:21.286134+0000 mgr.y (mgr.24431) 867 : cluster [DBG] pgmap v1335: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:22.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:22 smithi190 bash[23392]: audit 2024-03-26T04:07:21.819033+0000 mon.c (mon.2) 780 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:07:24.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:24 smithi175 bash[21527]: cluster 2024-03-26T04:07:23.287136+0000 mgr.y (mgr.24431) 868 : cluster [DBG] pgmap v1336: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:24.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:24 smithi175 bash[29284]: cluster 2024-03-26T04:07:23.287136+0000 mgr.y (mgr.24431) 868 : cluster [DBG] pgmap v1336: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:24.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:24 smithi190 bash[23392]: cluster 2024-03-26T04:07:23.287136+0000 mgr.y (mgr.24431) 868 : cluster [DBG] pgmap v1336: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:26.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:26 smithi175 bash[21527]: cluster 2024-03-26T04:07:25.289327+0000 mgr.y (mgr.24431) 869 : cluster [DBG] pgmap v1337: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:26.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:26 smithi175 bash[29284]: cluster 2024-03-26T04:07:25.289327+0000 mgr.y (mgr.24431) 869 : cluster [DBG] pgmap v1337: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:26.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:26 smithi190 bash[23392]: cluster 2024-03-26T04:07:25.289327+0000 mgr.y (mgr.24431) 869 : cluster [DBG] pgmap v1337: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:28.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:28 smithi175 bash[21527]: cluster 2024-03-26T04:07:27.290268+0000 mgr.y (mgr.24431) 870 : cluster [DBG] pgmap v1338: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:28.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:28 smithi175 bash[29284]: cluster 2024-03-26T04:07:27.290268+0000 mgr.y (mgr.24431) 870 : cluster [DBG] pgmap v1338: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:28.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:28 smithi190 bash[23392]: cluster 2024-03-26T04:07:27.290268+0000 mgr.y (mgr.24431) 870 : cluster [DBG] pgmap v1338: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:29.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:07:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:07:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:07:30.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:30 smithi175 bash[21527]: cluster 2024-03-26T04:07:29.291172+0000 mgr.y (mgr.24431) 871 : cluster [DBG] pgmap v1339: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:30.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:30 smithi175 bash[29284]: cluster 2024-03-26T04:07:29.291172+0000 mgr.y (mgr.24431) 871 : cluster [DBG] pgmap v1339: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:30.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:30 smithi190 bash[23392]: cluster 2024-03-26T04:07:29.291172+0000 mgr.y (mgr.24431) 871 : cluster [DBG] pgmap v1339: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:31.247 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:07:30 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:07:32.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:32 smithi175 bash[21527]: audit 2024-03-26T04:07:30.789374+0000 mgr.y (mgr.24431) 872 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:07:32.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:32 smithi175 bash[21527]: cluster 2024-03-26T04:07:31.293532+0000 mgr.y (mgr.24431) 873 : cluster [DBG] pgmap v1340: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:32.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:32 smithi175 bash[29284]: audit 2024-03-26T04:07:30.789374+0000 mgr.y (mgr.24431) 872 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:07:32.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:32 smithi175 bash[29284]: cluster 2024-03-26T04:07:31.293532+0000 mgr.y (mgr.24431) 873 : cluster [DBG] pgmap v1340: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:32.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:32 smithi190 bash[23392]: audit 2024-03-26T04:07:30.789374+0000 mgr.y (mgr.24431) 872 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:07:32.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:32 smithi190 bash[23392]: cluster 2024-03-26T04:07:31.293532+0000 mgr.y (mgr.24431) 873 : cluster [DBG] pgmap v1340: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:34.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:34 smithi175 bash[21527]: cluster 2024-03-26T04:07:33.294511+0000 mgr.y (mgr.24431) 874 : cluster [DBG] pgmap v1341: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:34.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:34 smithi175 bash[29284]: cluster 2024-03-26T04:07:33.294511+0000 mgr.y (mgr.24431) 874 : cluster [DBG] pgmap v1341: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:34.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:34 smithi190 bash[23392]: cluster 2024-03-26T04:07:33.294511+0000 mgr.y (mgr.24431) 874 : cluster [DBG] pgmap v1341: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:36.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:36 smithi175 bash[21527]: cluster 2024-03-26T04:07:35.296720+0000 mgr.y (mgr.24431) 875 : cluster [DBG] pgmap v1342: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:36.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:36 smithi175 bash[29284]: cluster 2024-03-26T04:07:35.296720+0000 mgr.y (mgr.24431) 875 : cluster [DBG] pgmap v1342: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:36.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:36 smithi190 bash[23392]: cluster 2024-03-26T04:07:35.296720+0000 mgr.y (mgr.24431) 875 : cluster [DBG] pgmap v1342: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:37.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:37 smithi175 bash[21527]: audit 2024-03-26T04:07:36.852368+0000 mon.c (mon.2) 781 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:07:37.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:37 smithi175 bash[29284]: audit 2024-03-26T04:07:36.852368+0000 mon.c (mon.2) 781 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:07:37.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:37 smithi190 bash[23392]: audit 2024-03-26T04:07:36.852368+0000 mon.c (mon.2) 781 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:07:38.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:38 smithi175 bash[21527]: cluster 2024-03-26T04:07:37.297626+0000 mgr.y (mgr.24431) 876 : cluster [DBG] pgmap v1343: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:38.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:38 smithi175 bash[29284]: cluster 2024-03-26T04:07:37.297626+0000 mgr.y (mgr.24431) 876 : cluster [DBG] pgmap v1343: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:38.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:38 smithi190 bash[23392]: cluster 2024-03-26T04:07:37.297626+0000 mgr.y (mgr.24431) 876 : cluster [DBG] pgmap v1343: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:39.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:07:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:07:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:07:40.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:40 smithi175 bash[21527]: cluster 2024-03-26T04:07:39.298518+0000 mgr.y (mgr.24431) 877 : cluster [DBG] pgmap v1344: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:40.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:40 smithi175 bash[29284]: cluster 2024-03-26T04:07:39.298518+0000 mgr.y (mgr.24431) 877 : cluster [DBG] pgmap v1344: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:40.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:40 smithi190 bash[23392]: cluster 2024-03-26T04:07:39.298518+0000 mgr.y (mgr.24431) 877 : cluster [DBG] pgmap v1344: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:41.247 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:07:40 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:07:42.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:42 smithi175 bash[21527]: audit 2024-03-26T04:07:40.800913+0000 mgr.y (mgr.24431) 878 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:07:42.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:42 smithi175 bash[21527]: cluster 2024-03-26T04:07:41.300825+0000 mgr.y (mgr.24431) 879 : cluster [DBG] pgmap v1345: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:42.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:42 smithi175 bash[29284]: audit 2024-03-26T04:07:40.800913+0000 mgr.y (mgr.24431) 878 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:07:42.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:42 smithi175 bash[29284]: cluster 2024-03-26T04:07:41.300825+0000 mgr.y (mgr.24431) 879 : cluster [DBG] pgmap v1345: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:42.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:42 smithi190 bash[23392]: audit 2024-03-26T04:07:40.800913+0000 mgr.y (mgr.24431) 878 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:07:42.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:42 smithi190 bash[23392]: cluster 2024-03-26T04:07:41.300825+0000 mgr.y (mgr.24431) 879 : cluster [DBG] pgmap v1345: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:44.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:44 smithi175 bash[21527]: cluster 2024-03-26T04:07:43.301669+0000 mgr.y (mgr.24431) 880 : cluster [DBG] pgmap v1346: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:44.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:44 smithi175 bash[29284]: cluster 2024-03-26T04:07:43.301669+0000 mgr.y (mgr.24431) 880 : cluster [DBG] pgmap v1346: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:44.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:44 smithi190 bash[23392]: cluster 2024-03-26T04:07:43.301669+0000 mgr.y (mgr.24431) 880 : cluster [DBG] pgmap v1346: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:46.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:46 smithi175 bash[21527]: cluster 2024-03-26T04:07:45.303926+0000 mgr.y (mgr.24431) 881 : cluster [DBG] pgmap v1347: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:46.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:46 smithi175 bash[29284]: cluster 2024-03-26T04:07:45.303926+0000 mgr.y (mgr.24431) 881 : cluster [DBG] pgmap v1347: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:46.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:46 smithi190 bash[23392]: cluster 2024-03-26T04:07:45.303926+0000 mgr.y (mgr.24431) 881 : cluster [DBG] pgmap v1347: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:48.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:48 smithi175 bash[21527]: cluster 2024-03-26T04:07:47.304776+0000 mgr.y (mgr.24431) 882 : cluster [DBG] pgmap v1348: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:48.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:48 smithi175 bash[29284]: cluster 2024-03-26T04:07:47.304776+0000 mgr.y (mgr.24431) 882 : cluster [DBG] pgmap v1348: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:48.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:48 smithi190 bash[23392]: cluster 2024-03-26T04:07:47.304776+0000 mgr.y (mgr.24431) 882 : cluster [DBG] pgmap v1348: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:49.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:07:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:07:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:07:50.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:50 smithi175 bash[29284]: cluster 2024-03-26T04:07:49.305621+0000 mgr.y (mgr.24431) 883 : cluster [DBG] pgmap v1349: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:50.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:50 smithi175 bash[21527]: cluster 2024-03-26T04:07:49.305621+0000 mgr.y (mgr.24431) 883 : cluster [DBG] pgmap v1349: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:50.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:50 smithi190 bash[23392]: cluster 2024-03-26T04:07:49.305621+0000 mgr.y (mgr.24431) 883 : cluster [DBG] pgmap v1349: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:51.247 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:07:50 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:07:52.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:52 smithi175 bash[21527]: audit 2024-03-26T04:07:50.812564+0000 mgr.y (mgr.24431) 884 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:07:52.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:52 smithi175 bash[21527]: cluster 2024-03-26T04:07:51.307923+0000 mgr.y (mgr.24431) 885 : cluster [DBG] pgmap v1350: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:52.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:52 smithi175 bash[21527]: audit 2024-03-26T04:07:51.886191+0000 mon.c (mon.2) 782 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:07:52.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:52 smithi175 bash[29284]: audit 2024-03-26T04:07:50.812564+0000 mgr.y (mgr.24431) 884 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:07:52.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:52 smithi175 bash[29284]: cluster 2024-03-26T04:07:51.307923+0000 mgr.y (mgr.24431) 885 : cluster [DBG] pgmap v1350: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:52.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:52 smithi175 bash[29284]: audit 2024-03-26T04:07:51.886191+0000 mon.c (mon.2) 782 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:07:52.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:52 smithi190 bash[23392]: audit 2024-03-26T04:07:50.812564+0000 mgr.y (mgr.24431) 884 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:07:52.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:52 smithi190 bash[23392]: cluster 2024-03-26T04:07:51.307923+0000 mgr.y (mgr.24431) 885 : cluster [DBG] pgmap v1350: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:52.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:52 smithi190 bash[23392]: audit 2024-03-26T04:07:51.886191+0000 mon.c (mon.2) 782 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:07:54.748 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:54 smithi175 bash[21527]: cluster 2024-03-26T04:07:53.308758+0000 mgr.y (mgr.24431) 886 : cluster [DBG] pgmap v1351: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:54.748 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:54 smithi175 bash[29284]: cluster 2024-03-26T04:07:53.308758+0000 mgr.y (mgr.24431) 886 : cluster [DBG] pgmap v1351: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:54.751 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:54 smithi190 bash[23392]: cluster 2024-03-26T04:07:53.308758+0000 mgr.y (mgr.24431) 886 : cluster [DBG] pgmap v1351: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:56.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:56 smithi175 bash[21527]: cluster 2024-03-26T04:07:55.310934+0000 mgr.y (mgr.24431) 887 : cluster [DBG] pgmap v1352: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:56.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:56 smithi175 bash[29284]: cluster 2024-03-26T04:07:55.310934+0000 mgr.y (mgr.24431) 887 : cluster [DBG] pgmap v1352: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:56.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:56 smithi190 bash[23392]: cluster 2024-03-26T04:07:55.310934+0000 mgr.y (mgr.24431) 887 : cluster [DBG] pgmap v1352: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:07:58.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:07:58 smithi175 bash[29284]: cluster 2024-03-26T04:07:57.311812+0000 mgr.y (mgr.24431) 888 : cluster [DBG] pgmap v1353: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:58.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:07:58 smithi175 bash[21527]: cluster 2024-03-26T04:07:57.311812+0000 mgr.y (mgr.24431) 888 : cluster [DBG] pgmap v1353: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:58.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:07:58 smithi190 bash[23392]: cluster 2024-03-26T04:07:57.311812+0000 mgr.y (mgr.24431) 888 : cluster [DBG] pgmap v1353: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:07:59.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:07:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:07:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:08:00.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:00 smithi175 bash[21527]: cluster 2024-03-26T04:07:59.312659+0000 mgr.y (mgr.24431) 889 : cluster [DBG] pgmap v1354: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:00.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:00 smithi175 bash[29284]: cluster 2024-03-26T04:07:59.312659+0000 mgr.y (mgr.24431) 889 : cluster [DBG] pgmap v1354: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:00.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:00 smithi190 bash[23392]: cluster 2024-03-26T04:07:59.312659+0000 mgr.y (mgr.24431) 889 : cluster [DBG] pgmap v1354: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:01.247 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:08:00 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:08:02.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:02 smithi175 bash[29284]: audit 2024-03-26T04:08:00.822971+0000 mgr.y (mgr.24431) 890 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:08:02.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:02 smithi175 bash[29284]: cluster 2024-03-26T04:08:01.314903+0000 mgr.y (mgr.24431) 891 : cluster [DBG] pgmap v1355: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:02.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:02 smithi175 bash[21527]: audit 2024-03-26T04:08:00.822971+0000 mgr.y (mgr.24431) 890 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:08:02.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:02 smithi175 bash[21527]: cluster 2024-03-26T04:08:01.314903+0000 mgr.y (mgr.24431) 891 : cluster [DBG] pgmap v1355: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:02.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:02 smithi190 bash[23392]: audit 2024-03-26T04:08:00.822971+0000 mgr.y (mgr.24431) 890 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:08:02.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:02 smithi190 bash[23392]: cluster 2024-03-26T04:08:01.314903+0000 mgr.y (mgr.24431) 891 : cluster [DBG] pgmap v1355: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:04.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:04 smithi175 bash[29284]: cluster 2024-03-26T04:08:03.315781+0000 mgr.y (mgr.24431) 892 : cluster [DBG] pgmap v1356: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T04:08:04.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:04 smithi175 bash[21527]: cluster 2024-03-26T04:08:03.315781+0000 mgr.y (mgr.24431) 892 : cluster [DBG] pgmap v1356: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T04:08:04.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:04 smithi190 bash[23392]: cluster 2024-03-26T04:08:03.315781+0000 mgr.y (mgr.24431) 892 : cluster [DBG] pgmap v1356: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T04:08:06.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:06 smithi175 bash[29284]: cluster 2024-03-26T04:08:05.318024+0000 mgr.y (mgr.24431) 893 : cluster [DBG] pgmap v1357: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:06.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:06 smithi175 bash[21527]: cluster 2024-03-26T04:08:05.318024+0000 mgr.y (mgr.24431) 893 : cluster [DBG] pgmap v1357: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:06.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:06 smithi190 bash[23392]: cluster 2024-03-26T04:08:05.318024+0000 mgr.y (mgr.24431) 893 : cluster [DBG] pgmap v1357: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:07.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:07 smithi175 bash[21527]: audit 2024-03-26T04:08:06.919266+0000 mon.c (mon.2) 783 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:08:07.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:07 smithi175 bash[29284]: audit 2024-03-26T04:08:06.919266+0000 mon.c (mon.2) 783 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:08:07.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:07 smithi190 bash[23392]: audit 2024-03-26T04:08:06.919266+0000 mon.c (mon.2) 783 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:08:08.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:08 smithi175 bash[21527]: cluster 2024-03-26T04:08:07.318962+0000 mgr.y (mgr.24431) 894 : cluster [DBG] pgmap v1358: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T04:08:08.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:08 smithi175 bash[29284]: cluster 2024-03-26T04:08:07.318962+0000 mgr.y (mgr.24431) 894 : cluster [DBG] pgmap v1358: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T04:08:08.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:08 smithi190 bash[23392]: cluster 2024-03-26T04:08:07.318962+0000 mgr.y (mgr.24431) 894 : cluster [DBG] pgmap v1358: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T04:08:09.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:08:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:08:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:08:10.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:10 smithi175 bash[29284]: cluster 2024-03-26T04:08:09.319878+0000 mgr.y (mgr.24431) 895 : cluster [DBG] pgmap v1359: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T04:08:10.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:10 smithi175 bash[21527]: cluster 2024-03-26T04:08:09.319878+0000 mgr.y (mgr.24431) 895 : cluster [DBG] pgmap v1359: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T04:08:10.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:10 smithi190 bash[23392]: cluster 2024-03-26T04:08:09.319878+0000 mgr.y (mgr.24431) 895 : cluster [DBG] pgmap v1359: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T04:08:11.247 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:08:10 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:08:12.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:12 smithi175 bash[29284]: audit 2024-03-26T04:08:10.834589+0000 mgr.y (mgr.24431) 896 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:08:12.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:12 smithi175 bash[29284]: cluster 2024-03-26T04:08:11.322232+0000 mgr.y (mgr.24431) 897 : cluster [DBG] pgmap v1360: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:12.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:12 smithi175 bash[21527]: audit 2024-03-26T04:08:10.834589+0000 mgr.y (mgr.24431) 896 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:08:12.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:12 smithi175 bash[21527]: cluster 2024-03-26T04:08:11.322232+0000 mgr.y (mgr.24431) 897 : cluster [DBG] pgmap v1360: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:12.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:12 smithi190 bash[23392]: audit 2024-03-26T04:08:10.834589+0000 mgr.y (mgr.24431) 896 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:08:12.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:12 smithi190 bash[23392]: cluster 2024-03-26T04:08:11.322232+0000 mgr.y (mgr.24431) 897 : cluster [DBG] pgmap v1360: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:14.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:14 smithi175 bash[29284]: cluster 2024-03-26T04:08:13.323199+0000 mgr.y (mgr.24431) 898 : cluster [DBG] pgmap v1361: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:14.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:14 smithi175 bash[21527]: cluster 2024-03-26T04:08:13.323199+0000 mgr.y (mgr.24431) 898 : cluster [DBG] pgmap v1361: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:14.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:14 smithi190 bash[23392]: cluster 2024-03-26T04:08:13.323199+0000 mgr.y (mgr.24431) 898 : cluster [DBG] pgmap v1361: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:16.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:16 smithi175 bash[29284]: cluster 2024-03-26T04:08:15.325550+0000 mgr.y (mgr.24431) 899 : cluster [DBG] pgmap v1362: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:16.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:16 smithi175 bash[21527]: cluster 2024-03-26T04:08:15.325550+0000 mgr.y (mgr.24431) 899 : cluster [DBG] pgmap v1362: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:16.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:16 smithi190 bash[23392]: cluster 2024-03-26T04:08:15.325550+0000 mgr.y (mgr.24431) 899 : cluster [DBG] pgmap v1362: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:18.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:18 smithi175 bash[29284]: cluster 2024-03-26T04:08:17.326563+0000 mgr.y (mgr.24431) 900 : cluster [DBG] pgmap v1363: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:18.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:18 smithi175 bash[21527]: cluster 2024-03-26T04:08:17.326563+0000 mgr.y (mgr.24431) 900 : cluster [DBG] pgmap v1363: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:18.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:18 smithi190 bash[23392]: cluster 2024-03-26T04:08:17.326563+0000 mgr.y (mgr.24431) 900 : cluster [DBG] pgmap v1363: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:19.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:08:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:08:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:08:20.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:20 smithi175 bash[21527]: cluster 2024-03-26T04:08:19.327545+0000 mgr.y (mgr.24431) 901 : cluster [DBG] pgmap v1364: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:20.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:20 smithi175 bash[21527]: audit 2024-03-26T04:08:19.678957+0000 mon.c (mon.2) 784 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:08:20.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:20 smithi175 bash[21527]: audit 2024-03-26T04:08:20.471231+0000 mon.c (mon.2) 785 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:08:20.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:20 smithi175 bash[21527]: audit 2024-03-26T04:08:20.472960+0000 mon.c (mon.2) 786 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:08:20.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:20 smithi175 bash[29284]: cluster 2024-03-26T04:08:19.327545+0000 mgr.y (mgr.24431) 901 : cluster [DBG] pgmap v1364: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:20.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:20 smithi175 bash[29284]: audit 2024-03-26T04:08:19.678957+0000 mon.c (mon.2) 784 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:08:20.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:20 smithi175 bash[29284]: audit 2024-03-26T04:08:20.471231+0000 mon.c (mon.2) 785 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:08:20.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:20 smithi175 bash[29284]: audit 2024-03-26T04:08:20.472960+0000 mon.c (mon.2) 786 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:08:20.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:20 smithi190 bash[23392]: cluster 2024-03-26T04:08:19.327545+0000 mgr.y (mgr.24431) 901 : cluster [DBG] pgmap v1364: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:20.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:20 smithi190 bash[23392]: audit 2024-03-26T04:08:19.678957+0000 mon.c (mon.2) 784 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:08:20.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:20 smithi190 bash[23392]: audit 2024-03-26T04:08:20.471231+0000 mon.c (mon.2) 785 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:08:20.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:20 smithi190 bash[23392]: audit 2024-03-26T04:08:20.472960+0000 mon.c (mon.2) 786 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:08:21.248 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:08:20 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:08:21.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:21 smithi175 bash[21527]: audit 2024-03-26T04:08:20.485588+0000 mon.a (mon.0) 3528 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:08:21.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:21 smithi175 bash[29284]: audit 2024-03-26T04:08:20.485588+0000 mon.a (mon.0) 3528 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:08:21.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:21 smithi190 bash[23392]: audit 2024-03-26T04:08:20.485588+0000 mon.a (mon.0) 3528 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:08:22.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:22 smithi175 bash[21527]: audit 2024-03-26T04:08:20.846191+0000 mgr.y (mgr.24431) 902 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:08:22.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:22 smithi175 bash[21527]: cluster 2024-03-26T04:08:21.329817+0000 mgr.y (mgr.24431) 903 : cluster [DBG] pgmap v1365: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:22.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:22 smithi175 bash[21527]: audit 2024-03-26T04:08:21.952982+0000 mon.c (mon.2) 787 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:08:22.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:22 smithi175 bash[29284]: audit 2024-03-26T04:08:20.846191+0000 mgr.y (mgr.24431) 902 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:08:22.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:22 smithi175 bash[29284]: cluster 2024-03-26T04:08:21.329817+0000 mgr.y (mgr.24431) 903 : cluster [DBG] pgmap v1365: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:22.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:22 smithi175 bash[29284]: audit 2024-03-26T04:08:21.952982+0000 mon.c (mon.2) 787 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:08:22.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:22 smithi190 bash[23392]: audit 2024-03-26T04:08:20.846191+0000 mgr.y (mgr.24431) 902 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:08:22.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:22 smithi190 bash[23392]: cluster 2024-03-26T04:08:21.329817+0000 mgr.y (mgr.24431) 903 : cluster [DBG] pgmap v1365: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:22.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:22 smithi190 bash[23392]: audit 2024-03-26T04:08:21.952982+0000 mon.c (mon.2) 787 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:08:24.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:24 smithi175 bash[21527]: cluster 2024-03-26T04:08:23.330626+0000 mgr.y (mgr.24431) 904 : cluster [DBG] pgmap v1366: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:24.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:24 smithi175 bash[29284]: cluster 2024-03-26T04:08:23.330626+0000 mgr.y (mgr.24431) 904 : cluster [DBG] pgmap v1366: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:24.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:24 smithi190 bash[23392]: cluster 2024-03-26T04:08:23.330626+0000 mgr.y (mgr.24431) 904 : cluster [DBG] pgmap v1366: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:26.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:26 smithi175 bash[21527]: cluster 2024-03-26T04:08:25.332831+0000 mgr.y (mgr.24431) 905 : cluster [DBG] pgmap v1367: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:26.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:26 smithi175 bash[29284]: cluster 2024-03-26T04:08:25.332831+0000 mgr.y (mgr.24431) 905 : cluster [DBG] pgmap v1367: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:26.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:26 smithi190 bash[23392]: cluster 2024-03-26T04:08:25.332831+0000 mgr.y (mgr.24431) 905 : cluster [DBG] pgmap v1367: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:28.775 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:28 smithi175 bash[29284]: cluster 2024-03-26T04:08:27.333675+0000 mgr.y (mgr.24431) 906 : cluster [DBG] pgmap v1368: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:28.775 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:28 smithi175 bash[21527]: cluster 2024-03-26T04:08:27.333675+0000 mgr.y (mgr.24431) 906 : cluster [DBG] pgmap v1368: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:28.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:28 smithi190 bash[23392]: cluster 2024-03-26T04:08:27.333675+0000 mgr.y (mgr.24431) 906 : cluster [DBG] pgmap v1368: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:29.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:08:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:08:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:08:30.850 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:30 smithi190 bash[23392]: cluster 2024-03-26T04:08:29.334529+0000 mgr.y (mgr.24431) 907 : cluster [DBG] pgmap v1369: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:30.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:30 smithi175 bash[21527]: cluster 2024-03-26T04:08:29.334529+0000 mgr.y (mgr.24431) 907 : cluster [DBG] pgmap v1369: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:30.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:30 smithi175 bash[29284]: cluster 2024-03-26T04:08:29.334529+0000 mgr.y (mgr.24431) 907 : cluster [DBG] pgmap v1369: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:31.247 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:08:30 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:08:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:32 smithi175 bash[21527]: audit 2024-03-26T04:08:30.852450+0000 mgr.y (mgr.24431) 908 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:08:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:32 smithi175 bash[21527]: cluster 2024-03-26T04:08:31.336752+0000 mgr.y (mgr.24431) 909 : cluster [DBG] pgmap v1370: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:32.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:32 smithi175 bash[29284]: audit 2024-03-26T04:08:30.852450+0000 mgr.y (mgr.24431) 908 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:08:32.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:32 smithi175 bash[29284]: cluster 2024-03-26T04:08:31.336752+0000 mgr.y (mgr.24431) 909 : cluster [DBG] pgmap v1370: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:32.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:32 smithi190 bash[23392]: audit 2024-03-26T04:08:30.852450+0000 mgr.y (mgr.24431) 908 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:08:32.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:32 smithi190 bash[23392]: cluster 2024-03-26T04:08:31.336752+0000 mgr.y (mgr.24431) 909 : cluster [DBG] pgmap v1370: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:34.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:34 smithi175 bash[29284]: cluster 2024-03-26T04:08:33.337648+0000 mgr.y (mgr.24431) 910 : cluster [DBG] pgmap v1371: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:34.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:34 smithi175 bash[21527]: cluster 2024-03-26T04:08:33.337648+0000 mgr.y (mgr.24431) 910 : cluster [DBG] pgmap v1371: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:34.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:34 smithi190 bash[23392]: cluster 2024-03-26T04:08:33.337648+0000 mgr.y (mgr.24431) 910 : cluster [DBG] pgmap v1371: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:36.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:36 smithi175 bash[29284]: cluster 2024-03-26T04:08:35.339868+0000 mgr.y (mgr.24431) 911 : cluster [DBG] pgmap v1372: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:36.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:36 smithi175 bash[21527]: cluster 2024-03-26T04:08:35.339868+0000 mgr.y (mgr.24431) 911 : cluster [DBG] pgmap v1372: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:36.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:36 smithi190 bash[23392]: cluster 2024-03-26T04:08:35.339868+0000 mgr.y (mgr.24431) 911 : cluster [DBG] pgmap v1372: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:37.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:37 smithi175 bash[21527]: audit 2024-03-26T04:08:36.986624+0000 mon.c (mon.2) 788 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:08:37.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:37 smithi175 bash[29284]: audit 2024-03-26T04:08:36.986624+0000 mon.c (mon.2) 788 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:08:37.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:37 smithi190 bash[23392]: audit 2024-03-26T04:08:36.986624+0000 mon.c (mon.2) 788 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:08:38.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:38 smithi175 bash[21527]: cluster 2024-03-26T04:08:37.340673+0000 mgr.y (mgr.24431) 912 : cluster [DBG] pgmap v1373: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:38.992 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:08:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:08:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:08:38.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:38 smithi175 bash[29284]: cluster 2024-03-26T04:08:37.340673+0000 mgr.y (mgr.24431) 912 : cluster [DBG] pgmap v1373: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:38.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:38 smithi190 bash[23392]: cluster 2024-03-26T04:08:37.340673+0000 mgr.y (mgr.24431) 912 : cluster [DBG] pgmap v1373: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:40.861 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:40 smithi190 bash[23392]: cluster 2024-03-26T04:08:39.341487+0000 mgr.y (mgr.24431) 913 : cluster [DBG] pgmap v1374: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:40.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:40 smithi175 bash[21527]: cluster 2024-03-26T04:08:39.341487+0000 mgr.y (mgr.24431) 913 : cluster [DBG] pgmap v1374: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:40.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:40 smithi175 bash[29284]: cluster 2024-03-26T04:08:39.341487+0000 mgr.y (mgr.24431) 913 : cluster [DBG] pgmap v1374: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:41.248 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:08:40 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:08:42.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:42 smithi175 bash[21527]: audit 2024-03-26T04:08:40.863943+0000 mgr.y (mgr.24431) 914 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:08:42.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:42 smithi175 bash[21527]: cluster 2024-03-26T04:08:41.343761+0000 mgr.y (mgr.24431) 915 : cluster [DBG] pgmap v1375: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:42.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:42 smithi175 bash[29284]: audit 2024-03-26T04:08:40.863943+0000 mgr.y (mgr.24431) 914 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:08:42.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:42 smithi175 bash[29284]: cluster 2024-03-26T04:08:41.343761+0000 mgr.y (mgr.24431) 915 : cluster [DBG] pgmap v1375: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:42.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:42 smithi190 bash[23392]: audit 2024-03-26T04:08:40.863943+0000 mgr.y (mgr.24431) 914 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:08:42.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:42 smithi190 bash[23392]: cluster 2024-03-26T04:08:41.343761+0000 mgr.y (mgr.24431) 915 : cluster [DBG] pgmap v1375: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:44.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:44 smithi175 bash[21527]: cluster 2024-03-26T04:08:43.344542+0000 mgr.y (mgr.24431) 916 : cluster [DBG] pgmap v1376: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:44.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:44 smithi175 bash[29284]: cluster 2024-03-26T04:08:43.344542+0000 mgr.y (mgr.24431) 916 : cluster [DBG] pgmap v1376: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:44.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:44 smithi190 bash[23392]: cluster 2024-03-26T04:08:43.344542+0000 mgr.y (mgr.24431) 916 : cluster [DBG] pgmap v1376: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:46.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:46 smithi175 bash[21527]: cluster 2024-03-26T04:08:45.346814+0000 mgr.y (mgr.24431) 917 : cluster [DBG] pgmap v1377: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:46.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:46 smithi175 bash[29284]: cluster 2024-03-26T04:08:45.346814+0000 mgr.y (mgr.24431) 917 : cluster [DBG] pgmap v1377: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:46.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:46 smithi190 bash[23392]: cluster 2024-03-26T04:08:45.346814+0000 mgr.y (mgr.24431) 917 : cluster [DBG] pgmap v1377: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:48.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:48 smithi175 bash[29284]: cluster 2024-03-26T04:08:47.347678+0000 mgr.y (mgr.24431) 918 : cluster [DBG] pgmap v1378: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:48.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:48 smithi175 bash[21527]: cluster 2024-03-26T04:08:47.347678+0000 mgr.y (mgr.24431) 918 : cluster [DBG] pgmap v1378: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:48.993 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:08:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:08:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:08:48.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:48 smithi190 bash[23392]: cluster 2024-03-26T04:08:47.347678+0000 mgr.y (mgr.24431) 918 : cluster [DBG] pgmap v1378: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:50.872 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:50 smithi190 bash[23392]: cluster 2024-03-26T04:08:49.348511+0000 mgr.y (mgr.24431) 919 : cluster [DBG] pgmap v1379: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:50.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:50 smithi175 bash[21527]: cluster 2024-03-26T04:08:49.348511+0000 mgr.y (mgr.24431) 919 : cluster [DBG] pgmap v1379: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:50.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:50 smithi175 bash[29284]: cluster 2024-03-26T04:08:49.348511+0000 mgr.y (mgr.24431) 919 : cluster [DBG] pgmap v1379: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:51.248 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:08:50 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:08:52.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:52 smithi175 bash[29284]: audit 2024-03-26T04:08:50.874666+0000 mgr.y (mgr.24431) 920 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:08:52.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:52 smithi175 bash[29284]: cluster 2024-03-26T04:08:51.350759+0000 mgr.y (mgr.24431) 921 : cluster [DBG] pgmap v1380: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:52.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:52 smithi175 bash[29284]: audit 2024-03-26T04:08:52.019544+0000 mon.c (mon.2) 789 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:08:52.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:52 smithi175 bash[21527]: audit 2024-03-26T04:08:50.874666+0000 mgr.y (mgr.24431) 920 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:08:52.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:52 smithi175 bash[21527]: cluster 2024-03-26T04:08:51.350759+0000 mgr.y (mgr.24431) 921 : cluster [DBG] pgmap v1380: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:52.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:52 smithi175 bash[21527]: audit 2024-03-26T04:08:52.019544+0000 mon.c (mon.2) 789 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:08:52.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:52 smithi190 bash[23392]: audit 2024-03-26T04:08:50.874666+0000 mgr.y (mgr.24431) 920 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:08:52.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:52 smithi190 bash[23392]: cluster 2024-03-26T04:08:51.350759+0000 mgr.y (mgr.24431) 921 : cluster [DBG] pgmap v1380: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:52.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:52 smithi190 bash[23392]: audit 2024-03-26T04:08:52.019544+0000 mon.c (mon.2) 789 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:08:54.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:54 smithi175 bash[29284]: cluster 2024-03-26T04:08:53.351709+0000 mgr.y (mgr.24431) 922 : cluster [DBG] pgmap v1381: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:54.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:54 smithi175 bash[21527]: cluster 2024-03-26T04:08:53.351709+0000 mgr.y (mgr.24431) 922 : cluster [DBG] pgmap v1381: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:54.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:54 smithi190 bash[23392]: cluster 2024-03-26T04:08:53.351709+0000 mgr.y (mgr.24431) 922 : cluster [DBG] pgmap v1381: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:56.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:56 smithi175 bash[21527]: cluster 2024-03-26T04:08:55.353902+0000 mgr.y (mgr.24431) 923 : cluster [DBG] pgmap v1382: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:56.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:56 smithi175 bash[29284]: cluster 2024-03-26T04:08:55.353902+0000 mgr.y (mgr.24431) 923 : cluster [DBG] pgmap v1382: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:56.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:56 smithi190 bash[23392]: cluster 2024-03-26T04:08:55.353902+0000 mgr.y (mgr.24431) 923 : cluster [DBG] pgmap v1382: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:08:58.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:08:57 smithi175 bash[21527]: cluster 2024-03-26T04:08:57.354705+0000 mgr.y (mgr.24431) 924 : cluster [DBG] pgmap v1383: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:58.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:08:57 smithi175 bash[29284]: cluster 2024-03-26T04:08:57.354705+0000 mgr.y (mgr.24431) 924 : cluster [DBG] pgmap v1383: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:58.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:08:57 smithi190 bash[23392]: cluster 2024-03-26T04:08:57.354705+0000 mgr.y (mgr.24431) 924 : cluster [DBG] pgmap v1383: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:08:59.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:08:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:08:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:09:00.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:00 smithi175 bash[21527]: cluster 2024-03-26T04:08:59.355588+0000 mgr.y (mgr.24431) 925 : cluster [DBG] pgmap v1384: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:00.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:00 smithi175 bash[29284]: cluster 2024-03-26T04:08:59.355588+0000 mgr.y (mgr.24431) 925 : cluster [DBG] pgmap v1384: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:00.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:00 smithi190 bash[23392]: cluster 2024-03-26T04:08:59.355588+0000 mgr.y (mgr.24431) 925 : cluster [DBG] pgmap v1384: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:01.249 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:09:00 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:09:02.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:02 smithi175 bash[21527]: audit 2024-03-26T04:09:00.883828+0000 mgr.y (mgr.24431) 926 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:09:02.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:02 smithi175 bash[21527]: cluster 2024-03-26T04:09:01.357861+0000 mgr.y (mgr.24431) 927 : cluster [DBG] pgmap v1385: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:02.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:02 smithi175 bash[29284]: audit 2024-03-26T04:09:00.883828+0000 mgr.y (mgr.24431) 926 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:09:02.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:02 smithi175 bash[29284]: cluster 2024-03-26T04:09:01.357861+0000 mgr.y (mgr.24431) 927 : cluster [DBG] pgmap v1385: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:02.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:02 smithi190 bash[23392]: audit 2024-03-26T04:09:00.883828+0000 mgr.y (mgr.24431) 926 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:09:02.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:02 smithi190 bash[23392]: cluster 2024-03-26T04:09:01.357861+0000 mgr.y (mgr.24431) 927 : cluster [DBG] pgmap v1385: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:04.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:04 smithi175 bash[21527]: cluster 2024-03-26T04:09:03.358671+0000 mgr.y (mgr.24431) 928 : cluster [DBG] pgmap v1386: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:04.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:04 smithi175 bash[29284]: cluster 2024-03-26T04:09:03.358671+0000 mgr.y (mgr.24431) 928 : cluster [DBG] pgmap v1386: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:04.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:04 smithi190 bash[23392]: cluster 2024-03-26T04:09:03.358671+0000 mgr.y (mgr.24431) 928 : cluster [DBG] pgmap v1386: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:06.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:06 smithi175 bash[21527]: cluster 2024-03-26T04:09:05.360873+0000 mgr.y (mgr.24431) 929 : cluster [DBG] pgmap v1387: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:06.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:06 smithi175 bash[29284]: cluster 2024-03-26T04:09:05.360873+0000 mgr.y (mgr.24431) 929 : cluster [DBG] pgmap v1387: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:06.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:06 smithi190 bash[23392]: cluster 2024-03-26T04:09:05.360873+0000 mgr.y (mgr.24431) 929 : cluster [DBG] pgmap v1387: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:08.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:07 smithi175 bash[21527]: audit 2024-03-26T04:09:07.053619+0000 mon.c (mon.2) 790 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:09:08.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:07 smithi175 bash[21527]: cluster 2024-03-26T04:09:07.361685+0000 mgr.y (mgr.24431) 930 : cluster [DBG] pgmap v1388: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:08.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:07 smithi175 bash[29284]: audit 2024-03-26T04:09:07.053619+0000 mon.c (mon.2) 790 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:09:08.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:07 smithi175 bash[29284]: cluster 2024-03-26T04:09:07.361685+0000 mgr.y (mgr.24431) 930 : cluster [DBG] pgmap v1388: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:08.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:07 smithi190 bash[23392]: audit 2024-03-26T04:09:07.053619+0000 mon.c (mon.2) 790 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:09:08.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:07 smithi190 bash[23392]: cluster 2024-03-26T04:09:07.361685+0000 mgr.y (mgr.24431) 930 : cluster [DBG] pgmap v1388: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:09.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:09:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:09:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:09:10.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:10 smithi175 bash[21527]: cluster 2024-03-26T04:09:09.362456+0000 mgr.y (mgr.24431) 931 : cluster [DBG] pgmap v1389: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:10.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:10 smithi175 bash[29284]: cluster 2024-03-26T04:09:09.362456+0000 mgr.y (mgr.24431) 931 : cluster [DBG] pgmap v1389: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:10.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:10 smithi190 bash[23392]: cluster 2024-03-26T04:09:09.362456+0000 mgr.y (mgr.24431) 931 : cluster [DBG] pgmap v1389: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:11.248 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:09:10 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:09:12.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:12 smithi175 bash[21527]: audit 2024-03-26T04:09:10.895332+0000 mgr.y (mgr.24431) 932 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:09:12.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:12 smithi175 bash[21527]: cluster 2024-03-26T04:09:11.364758+0000 mgr.y (mgr.24431) 933 : cluster [DBG] pgmap v1390: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:12.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:12 smithi175 bash[29284]: audit 2024-03-26T04:09:10.895332+0000 mgr.y (mgr.24431) 932 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:09:12.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:12 smithi175 bash[29284]: cluster 2024-03-26T04:09:11.364758+0000 mgr.y (mgr.24431) 933 : cluster [DBG] pgmap v1390: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:12.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:12 smithi190 bash[23392]: audit 2024-03-26T04:09:10.895332+0000 mgr.y (mgr.24431) 932 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:09:12.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:12 smithi190 bash[23392]: cluster 2024-03-26T04:09:11.364758+0000 mgr.y (mgr.24431) 933 : cluster [DBG] pgmap v1390: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:14.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:14 smithi175 bash[21527]: cluster 2024-03-26T04:09:13.365599+0000 mgr.y (mgr.24431) 934 : cluster [DBG] pgmap v1391: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:14.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:14 smithi175 bash[29284]: cluster 2024-03-26T04:09:13.365599+0000 mgr.y (mgr.24431) 934 : cluster [DBG] pgmap v1391: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:14.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:14 smithi190 bash[23392]: cluster 2024-03-26T04:09:13.365599+0000 mgr.y (mgr.24431) 934 : cluster [DBG] pgmap v1391: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:16.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:16 smithi175 bash[29284]: cluster 2024-03-26T04:09:15.367823+0000 mgr.y (mgr.24431) 935 : cluster [DBG] pgmap v1392: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:16.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:16 smithi175 bash[21527]: cluster 2024-03-26T04:09:15.367823+0000 mgr.y (mgr.24431) 935 : cluster [DBG] pgmap v1392: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:16.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:16 smithi190 bash[23392]: cluster 2024-03-26T04:09:15.367823+0000 mgr.y (mgr.24431) 935 : cluster [DBG] pgmap v1392: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:18.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:18 smithi175 bash[21527]: cluster 2024-03-26T04:09:17.368606+0000 mgr.y (mgr.24431) 936 : cluster [DBG] pgmap v1393: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:18.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:18 smithi175 bash[29284]: cluster 2024-03-26T04:09:17.368606+0000 mgr.y (mgr.24431) 936 : cluster [DBG] pgmap v1393: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:18.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:18 smithi190 bash[23392]: cluster 2024-03-26T04:09:17.368606+0000 mgr.y (mgr.24431) 936 : cluster [DBG] pgmap v1393: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:19.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:09:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:09:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:09:20.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:20 smithi175 bash[21527]: cluster 2024-03-26T04:09:19.369409+0000 mgr.y (mgr.24431) 937 : cluster [DBG] pgmap v1394: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:20.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:20 smithi175 bash[29284]: cluster 2024-03-26T04:09:19.369409+0000 mgr.y (mgr.24431) 937 : cluster [DBG] pgmap v1394: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:20.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:20 smithi190 bash[23392]: cluster 2024-03-26T04:09:19.369409+0000 mgr.y (mgr.24431) 937 : cluster [DBG] pgmap v1394: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:21.248 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:09:20 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:09:21.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:21 smithi175 bash[21527]: audit 2024-03-26T04:09:20.563859+0000 mon.c (mon.2) 791 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:09:21.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:21 smithi175 bash[21527]: audit 2024-03-26T04:09:21.291033+0000 mon.c (mon.2) 792 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:09:21.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:21 smithi175 bash[21527]: audit 2024-03-26T04:09:21.292876+0000 mon.c (mon.2) 793 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:09:21.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:21 smithi175 bash[21527]: audit 2024-03-26T04:09:21.303694+0000 mon.a (mon.0) 3529 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:09:21.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:21 smithi175 bash[29284]: audit 2024-03-26T04:09:20.563859+0000 mon.c (mon.2) 791 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:09:21.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:21 smithi175 bash[29284]: audit 2024-03-26T04:09:21.291033+0000 mon.c (mon.2) 792 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:09:21.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:21 smithi175 bash[29284]: audit 2024-03-26T04:09:21.292876+0000 mon.c (mon.2) 793 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:09:21.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:21 smithi175 bash[29284]: audit 2024-03-26T04:09:21.303694+0000 mon.a (mon.0) 3529 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:09:21.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:21 smithi190 bash[23392]: audit 2024-03-26T04:09:20.563859+0000 mon.c (mon.2) 791 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:09:21.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:21 smithi190 bash[23392]: audit 2024-03-26T04:09:21.291033+0000 mon.c (mon.2) 792 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:09:21.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:21 smithi190 bash[23392]: audit 2024-03-26T04:09:21.292876+0000 mon.c (mon.2) 793 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:09:21.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:21 smithi190 bash[23392]: audit 2024-03-26T04:09:21.303694+0000 mon.a (mon.0) 3529 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:09:22.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:22 smithi175 bash[29284]: audit 2024-03-26T04:09:20.906830+0000 mgr.y (mgr.24431) 938 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:09:22.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:22 smithi175 bash[29284]: cluster 2024-03-26T04:09:21.371636+0000 mgr.y (mgr.24431) 939 : cluster [DBG] pgmap v1395: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:22.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:22 smithi175 bash[29284]: audit 2024-03-26T04:09:22.087371+0000 mon.c (mon.2) 794 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:09:22.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:22 smithi175 bash[21527]: audit 2024-03-26T04:09:20.906830+0000 mgr.y (mgr.24431) 938 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:09:22.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:22 smithi175 bash[21527]: cluster 2024-03-26T04:09:21.371636+0000 mgr.y (mgr.24431) 939 : cluster [DBG] pgmap v1395: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:22.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:22 smithi175 bash[21527]: audit 2024-03-26T04:09:22.087371+0000 mon.c (mon.2) 794 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:09:22.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:22 smithi190 bash[23392]: audit 2024-03-26T04:09:20.906830+0000 mgr.y (mgr.24431) 938 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:09:22.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:22 smithi190 bash[23392]: cluster 2024-03-26T04:09:21.371636+0000 mgr.y (mgr.24431) 939 : cluster [DBG] pgmap v1395: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:22.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:22 smithi190 bash[23392]: audit 2024-03-26T04:09:22.087371+0000 mon.c (mon.2) 794 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:09:24.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:24 smithi175 bash[21527]: cluster 2024-03-26T04:09:23.372400+0000 mgr.y (mgr.24431) 940 : cluster [DBG] pgmap v1396: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:24.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:24 smithi175 bash[29284]: cluster 2024-03-26T04:09:23.372400+0000 mgr.y (mgr.24431) 940 : cluster [DBG] pgmap v1396: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:24.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:24 smithi190 bash[23392]: cluster 2024-03-26T04:09:23.372400+0000 mgr.y (mgr.24431) 940 : cluster [DBG] pgmap v1396: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:26.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:26 smithi175 bash[21527]: cluster 2024-03-26T04:09:25.374562+0000 mgr.y (mgr.24431) 941 : cluster [DBG] pgmap v1397: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:26.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:26 smithi175 bash[29284]: cluster 2024-03-26T04:09:25.374562+0000 mgr.y (mgr.24431) 941 : cluster [DBG] pgmap v1397: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:26.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:26 smithi190 bash[23392]: cluster 2024-03-26T04:09:25.374562+0000 mgr.y (mgr.24431) 941 : cluster [DBG] pgmap v1397: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:28.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:28 smithi175 bash[21527]: cluster 2024-03-26T04:09:27.375321+0000 mgr.y (mgr.24431) 942 : cluster [DBG] pgmap v1398: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:28.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:28 smithi175 bash[29284]: cluster 2024-03-26T04:09:27.375321+0000 mgr.y (mgr.24431) 942 : cluster [DBG] pgmap v1398: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:28.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:28 smithi190 bash[23392]: cluster 2024-03-26T04:09:27.375321+0000 mgr.y (mgr.24431) 942 : cluster [DBG] pgmap v1398: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:29.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:09:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:09:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:09:30.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:30 smithi175 bash[29284]: cluster 2024-03-26T04:09:29.376124+0000 mgr.y (mgr.24431) 943 : cluster [DBG] pgmap v1399: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:30.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:30 smithi175 bash[21527]: cluster 2024-03-26T04:09:29.376124+0000 mgr.y (mgr.24431) 943 : cluster [DBG] pgmap v1399: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:30.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:30 smithi190 bash[23392]: cluster 2024-03-26T04:09:29.376124+0000 mgr.y (mgr.24431) 943 : cluster [DBG] pgmap v1399: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:31.248 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:09:30 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:09:32.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:32 smithi175 bash[29284]: audit 2024-03-26T04:09:30.918401+0000 mgr.y (mgr.24431) 944 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:09:32.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:32 smithi175 bash[29284]: cluster 2024-03-26T04:09:31.378336+0000 mgr.y (mgr.24431) 945 : cluster [DBG] pgmap v1400: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:32.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:32 smithi175 bash[21527]: audit 2024-03-26T04:09:30.918401+0000 mgr.y (mgr.24431) 944 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:09:32.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:32 smithi175 bash[21527]: cluster 2024-03-26T04:09:31.378336+0000 mgr.y (mgr.24431) 945 : cluster [DBG] pgmap v1400: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:32.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:32 smithi190 bash[23392]: audit 2024-03-26T04:09:30.918401+0000 mgr.y (mgr.24431) 944 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:09:32.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:32 smithi190 bash[23392]: cluster 2024-03-26T04:09:31.378336+0000 mgr.y (mgr.24431) 945 : cluster [DBG] pgmap v1400: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:34.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:34 smithi175 bash[29284]: cluster 2024-03-26T04:09:33.379295+0000 mgr.y (mgr.24431) 946 : cluster [DBG] pgmap v1401: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:34.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:34 smithi175 bash[21527]: cluster 2024-03-26T04:09:33.379295+0000 mgr.y (mgr.24431) 946 : cluster [DBG] pgmap v1401: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:34.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:34 smithi190 bash[23392]: cluster 2024-03-26T04:09:33.379295+0000 mgr.y (mgr.24431) 946 : cluster [DBG] pgmap v1401: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:36.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:36 smithi175 bash[29284]: cluster 2024-03-26T04:09:35.381546+0000 mgr.y (mgr.24431) 947 : cluster [DBG] pgmap v1402: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:36.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:36 smithi175 bash[21527]: cluster 2024-03-26T04:09:35.381546+0000 mgr.y (mgr.24431) 947 : cluster [DBG] pgmap v1402: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:36.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:36 smithi190 bash[23392]: cluster 2024-03-26T04:09:35.381546+0000 mgr.y (mgr.24431) 947 : cluster [DBG] pgmap v1402: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:37.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:37 smithi175 bash[29284]: audit 2024-03-26T04:09:37.121313+0000 mon.c (mon.2) 795 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:09:37.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:37 smithi175 bash[21527]: audit 2024-03-26T04:09:37.121313+0000 mon.c (mon.2) 795 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:09:37.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:37 smithi190 bash[23392]: audit 2024-03-26T04:09:37.121313+0000 mon.c (mon.2) 795 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:09:38.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:38 smithi175 bash[29284]: cluster 2024-03-26T04:09:37.382375+0000 mgr.y (mgr.24431) 948 : cluster [DBG] pgmap v1403: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:38.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:38 smithi175 bash[21527]: cluster 2024-03-26T04:09:37.382375+0000 mgr.y (mgr.24431) 948 : cluster [DBG] pgmap v1403: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:38.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:38 smithi190 bash[23392]: cluster 2024-03-26T04:09:37.382375+0000 mgr.y (mgr.24431) 948 : cluster [DBG] pgmap v1403: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:39.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:09:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:09:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:09:40.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:40 smithi190 bash[23392]: cluster 2024-03-26T04:09:39.383300+0000 mgr.y (mgr.24431) 949 : cluster [DBG] pgmap v1404: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:40.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:40 smithi175 bash[21527]: cluster 2024-03-26T04:09:39.383300+0000 mgr.y (mgr.24431) 949 : cluster [DBG] pgmap v1404: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:40.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:40 smithi175 bash[29284]: cluster 2024-03-26T04:09:39.383300+0000 mgr.y (mgr.24431) 949 : cluster [DBG] pgmap v1404: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:41.248 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:09:40 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:09:42.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:42 smithi175 bash[21527]: audit 2024-03-26T04:09:40.929970+0000 mgr.y (mgr.24431) 950 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:09:42.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:42 smithi175 bash[21527]: cluster 2024-03-26T04:09:41.385517+0000 mgr.y (mgr.24431) 951 : cluster [DBG] pgmap v1405: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:42.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:42 smithi175 bash[29284]: audit 2024-03-26T04:09:40.929970+0000 mgr.y (mgr.24431) 950 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:09:42.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:42 smithi175 bash[29284]: cluster 2024-03-26T04:09:41.385517+0000 mgr.y (mgr.24431) 951 : cluster [DBG] pgmap v1405: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:42.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:42 smithi190 bash[23392]: audit 2024-03-26T04:09:40.929970+0000 mgr.y (mgr.24431) 950 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:09:42.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:42 smithi190 bash[23392]: cluster 2024-03-26T04:09:41.385517+0000 mgr.y (mgr.24431) 951 : cluster [DBG] pgmap v1405: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:44.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:44 smithi175 bash[21527]: cluster 2024-03-26T04:09:43.386326+0000 mgr.y (mgr.24431) 952 : cluster [DBG] pgmap v1406: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:44.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:44 smithi175 bash[29284]: cluster 2024-03-26T04:09:43.386326+0000 mgr.y (mgr.24431) 952 : cluster [DBG] pgmap v1406: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:44.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:44 smithi190 bash[23392]: cluster 2024-03-26T04:09:43.386326+0000 mgr.y (mgr.24431) 952 : cluster [DBG] pgmap v1406: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:46.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:46 smithi175 bash[21527]: cluster 2024-03-26T04:09:45.388547+0000 mgr.y (mgr.24431) 953 : cluster [DBG] pgmap v1407: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:46.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:46 smithi175 bash[29284]: cluster 2024-03-26T04:09:45.388547+0000 mgr.y (mgr.24431) 953 : cluster [DBG] pgmap v1407: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:46.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:46 smithi190 bash[23392]: cluster 2024-03-26T04:09:45.388547+0000 mgr.y (mgr.24431) 953 : cluster [DBG] pgmap v1407: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:48.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:47 smithi175 bash[21527]: cluster 2024-03-26T04:09:47.389297+0000 mgr.y (mgr.24431) 954 : cluster [DBG] pgmap v1408: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:48.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:47 smithi175 bash[29284]: cluster 2024-03-26T04:09:47.389297+0000 mgr.y (mgr.24431) 954 : cluster [DBG] pgmap v1408: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:48.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:47 smithi190 bash[23392]: cluster 2024-03-26T04:09:47.389297+0000 mgr.y (mgr.24431) 954 : cluster [DBG] pgmap v1408: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:49.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:09:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:09:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:09:50.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:50 smithi175 bash[21527]: cluster 2024-03-26T04:09:49.390254+0000 mgr.y (mgr.24431) 955 : cluster [DBG] pgmap v1409: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:50.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:50 smithi175 bash[29284]: cluster 2024-03-26T04:09:49.390254+0000 mgr.y (mgr.24431) 955 : cluster [DBG] pgmap v1409: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:50.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:50 smithi190 bash[23392]: cluster 2024-03-26T04:09:49.390254+0000 mgr.y (mgr.24431) 955 : cluster [DBG] pgmap v1409: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:51.248 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:09:50 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:09:52.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:52 smithi175 bash[29284]: audit 2024-03-26T04:09:50.941217+0000 mgr.y (mgr.24431) 956 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:09:52.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:52 smithi175 bash[29284]: cluster 2024-03-26T04:09:51.392549+0000 mgr.y (mgr.24431) 957 : cluster [DBG] pgmap v1410: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:52.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:52 smithi175 bash[29284]: audit 2024-03-26T04:09:52.154328+0000 mon.c (mon.2) 796 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:09:52.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:52 smithi175 bash[21527]: audit 2024-03-26T04:09:50.941217+0000 mgr.y (mgr.24431) 956 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:09:52.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:52 smithi175 bash[21527]: cluster 2024-03-26T04:09:51.392549+0000 mgr.y (mgr.24431) 957 : cluster [DBG] pgmap v1410: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:52.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:52 smithi175 bash[21527]: audit 2024-03-26T04:09:52.154328+0000 mon.c (mon.2) 796 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:09:52.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:52 smithi190 bash[23392]: audit 2024-03-26T04:09:50.941217+0000 mgr.y (mgr.24431) 956 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:09:52.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:52 smithi190 bash[23392]: cluster 2024-03-26T04:09:51.392549+0000 mgr.y (mgr.24431) 957 : cluster [DBG] pgmap v1410: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:52.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:52 smithi190 bash[23392]: audit 2024-03-26T04:09:52.154328+0000 mon.c (mon.2) 796 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:09:54.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:54 smithi175 bash[29284]: cluster 2024-03-26T04:09:53.393340+0000 mgr.y (mgr.24431) 958 : cluster [DBG] pgmap v1411: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:54.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:54 smithi175 bash[21527]: cluster 2024-03-26T04:09:53.393340+0000 mgr.y (mgr.24431) 958 : cluster [DBG] pgmap v1411: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:54.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:54 smithi190 bash[23392]: cluster 2024-03-26T04:09:53.393340+0000 mgr.y (mgr.24431) 958 : cluster [DBG] pgmap v1411: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:56.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:56 smithi175 bash[21527]: cluster 2024-03-26T04:09:55.395558+0000 mgr.y (mgr.24431) 959 : cluster [DBG] pgmap v1412: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:56.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:56 smithi175 bash[29284]: cluster 2024-03-26T04:09:55.395558+0000 mgr.y (mgr.24431) 959 : cluster [DBG] pgmap v1412: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:56.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:56 smithi190 bash[23392]: cluster 2024-03-26T04:09:55.395558+0000 mgr.y (mgr.24431) 959 : cluster [DBG] pgmap v1412: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:09:58.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:09:58 smithi175 bash[29284]: cluster 2024-03-26T04:09:57.396328+0000 mgr.y (mgr.24431) 960 : cluster [DBG] pgmap v1413: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:58.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:09:58 smithi175 bash[21527]: cluster 2024-03-26T04:09:57.396328+0000 mgr.y (mgr.24431) 960 : cluster [DBG] pgmap v1413: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:58.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:09:58 smithi190 bash[23392]: cluster 2024-03-26T04:09:57.396328+0000 mgr.y (mgr.24431) 960 : cluster [DBG] pgmap v1413: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:09:59.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:09:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:09:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:10:00.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:00 smithi175 bash[21527]: cluster 2024-03-26T04:09:59.397139+0000 mgr.y (mgr.24431) 961 : cluster [DBG] pgmap v1414: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:00.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:00 smithi175 bash[21527]: cluster 2024-03-26T04:10:00.000180+0000 mon.a (mon.0) 3530 : cluster [WRN] Health detail: HEALTH_WARN 3 pool(s) do not have an application enabled 2024-03-26T04:10:00.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:00 smithi175 bash[21527]: cluster 2024-03-26T04:10:00.000246+0000 mon.a (mon.0) 3531 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 3 pool(s) do not have an application enabled 2024-03-26T04:10:00.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:00 smithi175 bash[21527]: cluster 2024-03-26T04:10:00.000287+0000 mon.a (mon.0) 3532 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-26T04:10:00.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:00 smithi175 bash[21527]: cluster 2024-03-26T04:10:00.000337+0000 mon.a (mon.0) 3533 : cluster [WRN] application not enabled on pool 'WatchNotifysmithi175-65446-1' 2024-03-26T04:10:00.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:00 smithi175 bash[21527]: cluster 2024-03-26T04:10:00.000380+0000 mon.a (mon.0) 3534 : cluster [WRN] application not enabled on pool 'AssertExistssmithi175-65459-1' 2024-03-26T04:10:00.743 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:00 smithi175 bash[21527]: cluster 2024-03-26T04:10:00.000451+0000 mon.a (mon.0) 3535 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-26T04:10:00.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:00 smithi175 bash[29284]: cluster 2024-03-26T04:09:59.397139+0000 mgr.y (mgr.24431) 961 : cluster [DBG] pgmap v1414: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:00.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:00 smithi175 bash[29284]: cluster 2024-03-26T04:10:00.000180+0000 mon.a (mon.0) 3530 : cluster [WRN] Health detail: HEALTH_WARN 3 pool(s) do not have an application enabled 2024-03-26T04:10:00.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:00 smithi175 bash[29284]: cluster 2024-03-26T04:10:00.000246+0000 mon.a (mon.0) 3531 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 3 pool(s) do not have an application enabled 2024-03-26T04:10:00.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:00 smithi175 bash[29284]: cluster 2024-03-26T04:10:00.000287+0000 mon.a (mon.0) 3532 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-26T04:10:00.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:00 smithi175 bash[29284]: cluster 2024-03-26T04:10:00.000337+0000 mon.a (mon.0) 3533 : cluster [WRN] application not enabled on pool 'WatchNotifysmithi175-65446-1' 2024-03-26T04:10:00.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:00 smithi175 bash[29284]: cluster 2024-03-26T04:10:00.000380+0000 mon.a (mon.0) 3534 : cluster [WRN] application not enabled on pool 'AssertExistssmithi175-65459-1' 2024-03-26T04:10:00.743 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:00 smithi175 bash[29284]: cluster 2024-03-26T04:10:00.000451+0000 mon.a (mon.0) 3535 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-26T04:10:00.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:00 smithi190 bash[23392]: cluster 2024-03-26T04:09:59.397139+0000 mgr.y (mgr.24431) 961 : cluster [DBG] pgmap v1414: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:00.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:00 smithi190 bash[23392]: cluster 2024-03-26T04:10:00.000180+0000 mon.a (mon.0) 3530 : cluster [WRN] Health detail: HEALTH_WARN 3 pool(s) do not have an application enabled 2024-03-26T04:10:00.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:00 smithi190 bash[23392]: cluster 2024-03-26T04:10:00.000246+0000 mon.a (mon.0) 3531 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 3 pool(s) do not have an application enabled 2024-03-26T04:10:00.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:00 smithi190 bash[23392]: cluster 2024-03-26T04:10:00.000287+0000 mon.a (mon.0) 3532 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-26T04:10:00.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:00 smithi190 bash[23392]: cluster 2024-03-26T04:10:00.000337+0000 mon.a (mon.0) 3533 : cluster [WRN] application not enabled on pool 'WatchNotifysmithi175-65446-1' 2024-03-26T04:10:00.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:00 smithi190 bash[23392]: cluster 2024-03-26T04:10:00.000380+0000 mon.a (mon.0) 3534 : cluster [WRN] application not enabled on pool 'AssertExistssmithi175-65459-1' 2024-03-26T04:10:00.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:00 smithi190 bash[23392]: cluster 2024-03-26T04:10:00.000451+0000 mon.a (mon.0) 3535 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-26T04:10:01.248 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:10:00 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:10:02.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:02 smithi175 bash[21527]: audit 2024-03-26T04:10:00.949292+0000 mgr.y (mgr.24431) 962 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:10:02.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:02 smithi175 bash[21527]: cluster 2024-03-26T04:10:01.399462+0000 mgr.y (mgr.24431) 963 : cluster [DBG] pgmap v1415: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:02.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:02 smithi175 bash[29284]: audit 2024-03-26T04:10:00.949292+0000 mgr.y (mgr.24431) 962 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:10:02.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:02 smithi175 bash[29284]: cluster 2024-03-26T04:10:01.399462+0000 mgr.y (mgr.24431) 963 : cluster [DBG] pgmap v1415: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:02.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:02 smithi190 bash[23392]: audit 2024-03-26T04:10:00.949292+0000 mgr.y (mgr.24431) 962 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:10:02.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:02 smithi190 bash[23392]: cluster 2024-03-26T04:10:01.399462+0000 mgr.y (mgr.24431) 963 : cluster [DBG] pgmap v1415: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:04.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:04 smithi175 bash[29284]: cluster 2024-03-26T04:10:03.400375+0000 mgr.y (mgr.24431) 964 : cluster [DBG] pgmap v1416: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:04.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:04 smithi175 bash[21527]: cluster 2024-03-26T04:10:03.400375+0000 mgr.y (mgr.24431) 964 : cluster [DBG] pgmap v1416: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:04.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:04 smithi190 bash[23392]: cluster 2024-03-26T04:10:03.400375+0000 mgr.y (mgr.24431) 964 : cluster [DBG] pgmap v1416: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:06.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:06 smithi175 bash[29284]: cluster 2024-03-26T04:10:05.402746+0000 mgr.y (mgr.24431) 965 : cluster [DBG] pgmap v1417: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:06.742 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:06 smithi175 bash[21527]: cluster 2024-03-26T04:10:05.402746+0000 mgr.y (mgr.24431) 965 : cluster [DBG] pgmap v1417: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:06.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:06 smithi190 bash[23392]: cluster 2024-03-26T04:10:05.402746+0000 mgr.y (mgr.24431) 965 : cluster [DBG] pgmap v1417: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:07.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:07 smithi175 bash[21527]: audit 2024-03-26T04:10:07.187659+0000 mon.c (mon.2) 797 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:10:07.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:07 smithi175 bash[29284]: audit 2024-03-26T04:10:07.187659+0000 mon.c (mon.2) 797 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:10:07.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:07 smithi190 bash[23392]: audit 2024-03-26T04:10:07.187659+0000 mon.c (mon.2) 797 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:10:08.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:08 smithi175 bash[21527]: cluster 2024-03-26T04:10:07.403601+0000 mgr.y (mgr.24431) 966 : cluster [DBG] pgmap v1418: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:08.742 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:08 smithi175 bash[29284]: cluster 2024-03-26T04:10:07.403601+0000 mgr.y (mgr.24431) 966 : cluster [DBG] pgmap v1418: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:08.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:08 smithi190 bash[23392]: cluster 2024-03-26T04:10:07.403601+0000 mgr.y (mgr.24431) 966 : cluster [DBG] pgmap v1418: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:09.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:10:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:10:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:10:10.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:10 smithi190 bash[23392]: cluster 2024-03-26T04:10:09.404386+0000 mgr.y (mgr.24431) 967 : cluster [DBG] pgmap v1419: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:10.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:10 smithi175 bash[21527]: cluster 2024-03-26T04:10:09.404386+0000 mgr.y (mgr.24431) 967 : cluster [DBG] pgmap v1419: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:10.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:10 smithi175 bash[29284]: cluster 2024-03-26T04:10:09.404386+0000 mgr.y (mgr.24431) 967 : cluster [DBG] pgmap v1419: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:11.250 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:10:10 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:10:12.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:12 smithi175 bash[29284]: audit 2024-03-26T04:10:10.960915+0000 mgr.y (mgr.24431) 968 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:10:12.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:12 smithi175 bash[29284]: cluster 2024-03-26T04:10:11.406622+0000 mgr.y (mgr.24431) 969 : cluster [DBG] pgmap v1420: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:12.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:12 smithi175 bash[21527]: audit 2024-03-26T04:10:10.960915+0000 mgr.y (mgr.24431) 968 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:10:12.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:12 smithi175 bash[21527]: cluster 2024-03-26T04:10:11.406622+0000 mgr.y (mgr.24431) 969 : cluster [DBG] pgmap v1420: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:12.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:12 smithi190 bash[23392]: audit 2024-03-26T04:10:10.960915+0000 mgr.y (mgr.24431) 968 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:10:12.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:12 smithi190 bash[23392]: cluster 2024-03-26T04:10:11.406622+0000 mgr.y (mgr.24431) 969 : cluster [DBG] pgmap v1420: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:14.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:14 smithi175 bash[29284]: cluster 2024-03-26T04:10:13.407465+0000 mgr.y (mgr.24431) 970 : cluster [DBG] pgmap v1421: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:14.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:14 smithi175 bash[21527]: cluster 2024-03-26T04:10:13.407465+0000 mgr.y (mgr.24431) 970 : cluster [DBG] pgmap v1421: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:14.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:14 smithi190 bash[23392]: cluster 2024-03-26T04:10:13.407465+0000 mgr.y (mgr.24431) 970 : cluster [DBG] pgmap v1421: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:16.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:16 smithi175 bash[21527]: cluster 2024-03-26T04:10:15.409639+0000 mgr.y (mgr.24431) 971 : cluster [DBG] pgmap v1422: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:16.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:16 smithi175 bash[29284]: cluster 2024-03-26T04:10:15.409639+0000 mgr.y (mgr.24431) 971 : cluster [DBG] pgmap v1422: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:16.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:16 smithi190 bash[23392]: cluster 2024-03-26T04:10:15.409639+0000 mgr.y (mgr.24431) 971 : cluster [DBG] pgmap v1422: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:18.775 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:18 smithi175 bash[29284]: cluster 2024-03-26T04:10:17.410478+0000 mgr.y (mgr.24431) 972 : cluster [DBG] pgmap v1423: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:18.775 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:18 smithi175 bash[21527]: cluster 2024-03-26T04:10:17.410478+0000 mgr.y (mgr.24431) 972 : cluster [DBG] pgmap v1423: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:18.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:18 smithi190 bash[23392]: cluster 2024-03-26T04:10:17.410478+0000 mgr.y (mgr.24431) 972 : cluster [DBG] pgmap v1423: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:19.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:10:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:10:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:10:20.970 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:20 smithi190 bash[23392]: cluster 2024-03-26T04:10:19.411326+0000 mgr.y (mgr.24431) 973 : cluster [DBG] pgmap v1424: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:20.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:20 smithi175 bash[29284]: cluster 2024-03-26T04:10:19.411326+0000 mgr.y (mgr.24431) 973 : cluster [DBG] pgmap v1424: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:20.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:20 smithi175 bash[21527]: cluster 2024-03-26T04:10:19.411326+0000 mgr.y (mgr.24431) 973 : cluster [DBG] pgmap v1424: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:21.248 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:10:20 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:10:21.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:21 smithi175 bash[29284]: audit 2024-03-26T04:10:21.398795+0000 mon.c (mon.2) 798 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:10:21.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:21 smithi175 bash[21527]: audit 2024-03-26T04:10:21.398795+0000 mon.c (mon.2) 798 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:10:21.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:21 smithi190 bash[23392]: audit 2024-03-26T04:10:21.398795+0000 mon.c (mon.2) 798 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:10:22.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:22 smithi175 bash[21527]: audit 2024-03-26T04:10:20.972551+0000 mgr.y (mgr.24431) 974 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:10:22.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:22 smithi175 bash[21527]: cluster 2024-03-26T04:10:21.413736+0000 mgr.y (mgr.24431) 975 : cluster [DBG] pgmap v1425: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:22.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:22 smithi175 bash[21527]: audit 2024-03-26T04:10:22.221348+0000 mon.c (mon.2) 799 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:10:22.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:22 smithi175 bash[29284]: audit 2024-03-26T04:10:20.972551+0000 mgr.y (mgr.24431) 974 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:10:22.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:22 smithi175 bash[29284]: cluster 2024-03-26T04:10:21.413736+0000 mgr.y (mgr.24431) 975 : cluster [DBG] pgmap v1425: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:22.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:22 smithi175 bash[29284]: audit 2024-03-26T04:10:22.221348+0000 mon.c (mon.2) 799 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:10:22.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:22 smithi190 bash[23392]: audit 2024-03-26T04:10:20.972551+0000 mgr.y (mgr.24431) 974 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:10:22.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:22 smithi190 bash[23392]: cluster 2024-03-26T04:10:21.413736+0000 mgr.y (mgr.24431) 975 : cluster [DBG] pgmap v1425: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:22.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:22 smithi190 bash[23392]: audit 2024-03-26T04:10:22.221348+0000 mon.c (mon.2) 799 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:10:24.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:24 smithi175 bash[21527]: cluster 2024-03-26T04:10:23.414607+0000 mgr.y (mgr.24431) 976 : cluster [DBG] pgmap v1426: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:24.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:24 smithi175 bash[29284]: cluster 2024-03-26T04:10:23.414607+0000 mgr.y (mgr.24431) 976 : cluster [DBG] pgmap v1426: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:24.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:24 smithi190 bash[23392]: cluster 2024-03-26T04:10:23.414607+0000 mgr.y (mgr.24431) 976 : cluster [DBG] pgmap v1426: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:26.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:26 smithi175 bash[21527]: cluster 2024-03-26T04:10:25.416934+0000 mgr.y (mgr.24431) 977 : cluster [DBG] pgmap v1427: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:26.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:26 smithi175 bash[29284]: cluster 2024-03-26T04:10:25.416934+0000 mgr.y (mgr.24431) 977 : cluster [DBG] pgmap v1427: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:26.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:26 smithi190 bash[23392]: cluster 2024-03-26T04:10:25.416934+0000 mgr.y (mgr.24431) 977 : cluster [DBG] pgmap v1427: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:28.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:28 smithi175 bash[21527]: cluster 2024-03-26T04:10:27.417801+0000 mgr.y (mgr.24431) 978 : cluster [DBG] pgmap v1428: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:28.992 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:10:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:10:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:10:28.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:28 smithi175 bash[29284]: cluster 2024-03-26T04:10:27.417801+0000 mgr.y (mgr.24431) 978 : cluster [DBG] pgmap v1428: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:28.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:28 smithi190 bash[23392]: cluster 2024-03-26T04:10:27.417801+0000 mgr.y (mgr.24431) 978 : cluster [DBG] pgmap v1428: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:30.981 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:30 smithi190 bash[23392]: cluster 2024-03-26T04:10:29.418598+0000 mgr.y (mgr.24431) 979 : cluster [DBG] pgmap v1429: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:30.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:30 smithi175 bash[29284]: cluster 2024-03-26T04:10:29.418598+0000 mgr.y (mgr.24431) 979 : cluster [DBG] pgmap v1429: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:30.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:30 smithi175 bash[21527]: cluster 2024-03-26T04:10:29.418598+0000 mgr.y (mgr.24431) 979 : cluster [DBG] pgmap v1429: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:31.248 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:10:30 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:10:32.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:32 smithi175 bash[21527]: audit 2024-03-26T04:10:30.983582+0000 mgr.y (mgr.24431) 980 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:10:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:32 smithi175 bash[21527]: cluster 2024-03-26T04:10:31.420873+0000 mgr.y (mgr.24431) 981 : cluster [DBG] pgmap v1430: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:32 smithi175 bash[21527]: audit 2024-03-26T04:10:32.084987+0000 mon.a (mon.0) 3536 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:10:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:32 smithi175 bash[21527]: audit 2024-03-26T04:10:32.100508+0000 mon.a (mon.0) 3537 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:10:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:32 smithi175 bash[21527]: audit 2024-03-26T04:10:32.105190+0000 mon.c (mon.2) 800 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:10:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:32 smithi175 bash[21527]: audit 2024-03-26T04:10:32.107207+0000 mon.c (mon.2) 801 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:10:32.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:32 smithi175 bash[21527]: audit 2024-03-26T04:10:32.120647+0000 mon.a (mon.0) 3538 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:10:32.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:32 smithi175 bash[29284]: audit 2024-03-26T04:10:30.983582+0000 mgr.y (mgr.24431) 980 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:10:32.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:32 smithi175 bash[29284]: cluster 2024-03-26T04:10:31.420873+0000 mgr.y (mgr.24431) 981 : cluster [DBG] pgmap v1430: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:32.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:32 smithi175 bash[29284]: audit 2024-03-26T04:10:32.084987+0000 mon.a (mon.0) 3536 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:10:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:32 smithi175 bash[29284]: audit 2024-03-26T04:10:32.100508+0000 mon.a (mon.0) 3537 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:10:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:32 smithi175 bash[29284]: audit 2024-03-26T04:10:32.105190+0000 mon.c (mon.2) 800 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:10:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:32 smithi175 bash[29284]: audit 2024-03-26T04:10:32.107207+0000 mon.c (mon.2) 801 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:10:32.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:32 smithi175 bash[29284]: audit 2024-03-26T04:10:32.120647+0000 mon.a (mon.0) 3538 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:10:32.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:32 smithi190 bash[23392]: audit 2024-03-26T04:10:30.983582+0000 mgr.y (mgr.24431) 980 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:10:32.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:32 smithi190 bash[23392]: cluster 2024-03-26T04:10:31.420873+0000 mgr.y (mgr.24431) 981 : cluster [DBG] pgmap v1430: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:32.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:32 smithi190 bash[23392]: audit 2024-03-26T04:10:32.084987+0000 mon.a (mon.0) 3536 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:10:32.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:32 smithi190 bash[23392]: audit 2024-03-26T04:10:32.100508+0000 mon.a (mon.0) 3537 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:10:32.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:32 smithi190 bash[23392]: audit 2024-03-26T04:10:32.105190+0000 mon.c (mon.2) 800 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:10:32.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:32 smithi190 bash[23392]: audit 2024-03-26T04:10:32.107207+0000 mon.c (mon.2) 801 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:10:32.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:32 smithi190 bash[23392]: audit 2024-03-26T04:10:32.120647+0000 mon.a (mon.0) 3538 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:10:34.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:34 smithi175 bash[21527]: cluster 2024-03-26T04:10:33.421719+0000 mgr.y (mgr.24431) 982 : cluster [DBG] pgmap v1431: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:34.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:34 smithi175 bash[29284]: cluster 2024-03-26T04:10:33.421719+0000 mgr.y (mgr.24431) 982 : cluster [DBG] pgmap v1431: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:34.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:34 smithi190 bash[23392]: cluster 2024-03-26T04:10:33.421719+0000 mgr.y (mgr.24431) 982 : cluster [DBG] pgmap v1431: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:36.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:36 smithi175 bash[21527]: cluster 2024-03-26T04:10:35.423979+0000 mgr.y (mgr.24431) 983 : cluster [DBG] pgmap v1432: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:36.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:36 smithi175 bash[29284]: cluster 2024-03-26T04:10:35.423979+0000 mgr.y (mgr.24431) 983 : cluster [DBG] pgmap v1432: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:36.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:36 smithi190 bash[23392]: cluster 2024-03-26T04:10:35.423979+0000 mgr.y (mgr.24431) 983 : cluster [DBG] pgmap v1432: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:37.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:37 smithi175 bash[21527]: audit 2024-03-26T04:10:37.253867+0000 mon.c (mon.2) 802 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:10:37.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:37 smithi175 bash[29284]: audit 2024-03-26T04:10:37.253867+0000 mon.c (mon.2) 802 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:10:37.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:37 smithi190 bash[23392]: audit 2024-03-26T04:10:37.253867+0000 mon.c (mon.2) 802 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:10:38.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:38 smithi175 bash[21527]: cluster 2024-03-26T04:10:37.424765+0000 mgr.y (mgr.24431) 984 : cluster [DBG] pgmap v1433: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:38.992 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:10:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:10:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:10:38.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:38 smithi175 bash[29284]: cluster 2024-03-26T04:10:37.424765+0000 mgr.y (mgr.24431) 984 : cluster [DBG] pgmap v1433: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:38.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:38 smithi190 bash[23392]: cluster 2024-03-26T04:10:37.424765+0000 mgr.y (mgr.24431) 984 : cluster [DBG] pgmap v1433: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:40.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:40 smithi175 bash[21527]: cluster 2024-03-26T04:10:39.425608+0000 mgr.y (mgr.24431) 985 : cluster [DBG] pgmap v1434: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:40.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:40 smithi175 bash[29284]: cluster 2024-03-26T04:10:39.425608+0000 mgr.y (mgr.24431) 985 : cluster [DBG] pgmap v1434: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:40.992 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:40 smithi190 bash[23392]: cluster 2024-03-26T04:10:39.425608+0000 mgr.y (mgr.24431) 985 : cluster [DBG] pgmap v1434: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:41.248 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:10:40 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:10:42.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:42 smithi175 bash[21527]: audit 2024-03-26T04:10:40.995073+0000 mgr.y (mgr.24431) 986 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:10:42.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:42 smithi175 bash[21527]: cluster 2024-03-26T04:10:41.427821+0000 mgr.y (mgr.24431) 987 : cluster [DBG] pgmap v1435: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:42.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:42 smithi175 bash[29284]: audit 2024-03-26T04:10:40.995073+0000 mgr.y (mgr.24431) 986 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:10:42.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:42 smithi175 bash[29284]: cluster 2024-03-26T04:10:41.427821+0000 mgr.y (mgr.24431) 987 : cluster [DBG] pgmap v1435: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:42.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:42 smithi190 bash[23392]: audit 2024-03-26T04:10:40.995073+0000 mgr.y (mgr.24431) 986 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:10:42.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:42 smithi190 bash[23392]: cluster 2024-03-26T04:10:41.427821+0000 mgr.y (mgr.24431) 987 : cluster [DBG] pgmap v1435: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:44.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:44 smithi175 bash[21527]: cluster 2024-03-26T04:10:43.428612+0000 mgr.y (mgr.24431) 988 : cluster [DBG] pgmap v1436: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:44.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:44 smithi175 bash[29284]: cluster 2024-03-26T04:10:43.428612+0000 mgr.y (mgr.24431) 988 : cluster [DBG] pgmap v1436: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:44.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:44 smithi190 bash[23392]: cluster 2024-03-26T04:10:43.428612+0000 mgr.y (mgr.24431) 988 : cluster [DBG] pgmap v1436: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:46.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:46 smithi175 bash[21527]: cluster 2024-03-26T04:10:45.430826+0000 mgr.y (mgr.24431) 989 : cluster [DBG] pgmap v1437: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:46.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:46 smithi175 bash[29284]: cluster 2024-03-26T04:10:45.430826+0000 mgr.y (mgr.24431) 989 : cluster [DBG] pgmap v1437: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:46.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:46 smithi190 bash[23392]: cluster 2024-03-26T04:10:45.430826+0000 mgr.y (mgr.24431) 989 : cluster [DBG] pgmap v1437: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:48.992 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:10:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:10:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:10:48.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:48 smithi175 bash[29284]: cluster 2024-03-26T04:10:47.431693+0000 mgr.y (mgr.24431) 990 : cluster [DBG] pgmap v1438: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:48.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:48 smithi175 bash[21527]: cluster 2024-03-26T04:10:47.431693+0000 mgr.y (mgr.24431) 990 : cluster [DBG] pgmap v1438: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:48.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:48 smithi190 bash[23392]: cluster 2024-03-26T04:10:47.431693+0000 mgr.y (mgr.24431) 990 : cluster [DBG] pgmap v1438: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:50.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:50 smithi175 bash[21527]: cluster 2024-03-26T04:10:49.432554+0000 mgr.y (mgr.24431) 991 : cluster [DBG] pgmap v1439: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:50.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:50 smithi175 bash[29284]: cluster 2024-03-26T04:10:49.432554+0000 mgr.y (mgr.24431) 991 : cluster [DBG] pgmap v1439: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:50.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:50 smithi190 bash[23392]: cluster 2024-03-26T04:10:49.432554+0000 mgr.y (mgr.24431) 991 : cluster [DBG] pgmap v1439: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:51.498 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:10:51 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:10:52.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:52 smithi175 bash[21527]: audit 2024-03-26T04:10:51.002569+0000 mgr.y (mgr.24431) 992 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:10:52.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:52 smithi175 bash[21527]: cluster 2024-03-26T04:10:51.434870+0000 mgr.y (mgr.24431) 993 : cluster [DBG] pgmap v1440: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:52.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:52 smithi175 bash[21527]: audit 2024-03-26T04:10:52.287479+0000 mon.c (mon.2) 803 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:10:52.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:52 smithi175 bash[29284]: audit 2024-03-26T04:10:51.002569+0000 mgr.y (mgr.24431) 992 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:10:52.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:52 smithi175 bash[29284]: cluster 2024-03-26T04:10:51.434870+0000 mgr.y (mgr.24431) 993 : cluster [DBG] pgmap v1440: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:52.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:52 smithi175 bash[29284]: audit 2024-03-26T04:10:52.287479+0000 mon.c (mon.2) 803 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:10:52.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:52 smithi190 bash[23392]: audit 2024-03-26T04:10:51.002569+0000 mgr.y (mgr.24431) 992 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:10:52.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:52 smithi190 bash[23392]: cluster 2024-03-26T04:10:51.434870+0000 mgr.y (mgr.24431) 993 : cluster [DBG] pgmap v1440: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:52.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:52 smithi190 bash[23392]: audit 2024-03-26T04:10:52.287479+0000 mon.c (mon.2) 803 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:10:54.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:54 smithi175 bash[29284]: cluster 2024-03-26T04:10:53.435895+0000 mgr.y (mgr.24431) 994 : cluster [DBG] pgmap v1441: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:54.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:54 smithi175 bash[21527]: cluster 2024-03-26T04:10:53.435895+0000 mgr.y (mgr.24431) 994 : cluster [DBG] pgmap v1441: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:54.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:54 smithi190 bash[23392]: cluster 2024-03-26T04:10:53.435895+0000 mgr.y (mgr.24431) 994 : cluster [DBG] pgmap v1441: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:56.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:56 smithi175 bash[21527]: cluster 2024-03-26T04:10:55.438268+0000 mgr.y (mgr.24431) 995 : cluster [DBG] pgmap v1442: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:56.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:56 smithi175 bash[29284]: cluster 2024-03-26T04:10:55.438268+0000 mgr.y (mgr.24431) 995 : cluster [DBG] pgmap v1442: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:56.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:56 smithi190 bash[23392]: cluster 2024-03-26T04:10:55.438268+0000 mgr.y (mgr.24431) 995 : cluster [DBG] pgmap v1442: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:10:58.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:10:58 smithi175 bash[21527]: cluster 2024-03-26T04:10:57.439139+0000 mgr.y (mgr.24431) 996 : cluster [DBG] pgmap v1443: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:58.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:10:58 smithi175 bash[29284]: cluster 2024-03-26T04:10:57.439139+0000 mgr.y (mgr.24431) 996 : cluster [DBG] pgmap v1443: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:10:58.992 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:10:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:10:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:10:58.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:10:58 smithi190 bash[23392]: cluster 2024-03-26T04:10:57.439139+0000 mgr.y (mgr.24431) 996 : cluster [DBG] pgmap v1443: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:00.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:00 smithi175 bash[29284]: cluster 2024-03-26T04:10:59.440016+0000 mgr.y (mgr.24431) 997 : cluster [DBG] pgmap v1444: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:00.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:00 smithi175 bash[21527]: cluster 2024-03-26T04:10:59.440016+0000 mgr.y (mgr.24431) 997 : cluster [DBG] pgmap v1444: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:00.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:00 smithi190 bash[23392]: cluster 2024-03-26T04:10:59.440016+0000 mgr.y (mgr.24431) 997 : cluster [DBG] pgmap v1444: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:01.498 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:11:01 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:11:02.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:02 smithi175 bash[29284]: audit 2024-03-26T04:11:01.013601+0000 mgr.y (mgr.24431) 998 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:11:02.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:02 smithi175 bash[29284]: cluster 2024-03-26T04:11:01.442387+0000 mgr.y (mgr.24431) 999 : cluster [DBG] pgmap v1445: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:02.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:02 smithi175 bash[21527]: audit 2024-03-26T04:11:01.013601+0000 mgr.y (mgr.24431) 998 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:11:02.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:02 smithi175 bash[21527]: cluster 2024-03-26T04:11:01.442387+0000 mgr.y (mgr.24431) 999 : cluster [DBG] pgmap v1445: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:02.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:02 smithi190 bash[23392]: audit 2024-03-26T04:11:01.013601+0000 mgr.y (mgr.24431) 998 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:11:02.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:02 smithi190 bash[23392]: cluster 2024-03-26T04:11:01.442387+0000 mgr.y (mgr.24431) 999 : cluster [DBG] pgmap v1445: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:04.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:04 smithi175 bash[29284]: cluster 2024-03-26T04:11:03.443370+0000 mgr.y (mgr.24431) 1000 : cluster [DBG] pgmap v1446: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:04.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:04 smithi175 bash[21527]: cluster 2024-03-26T04:11:03.443370+0000 mgr.y (mgr.24431) 1000 : cluster [DBG] pgmap v1446: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:04.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:04 smithi190 bash[23392]: cluster 2024-03-26T04:11:03.443370+0000 mgr.y (mgr.24431) 1000 : cluster [DBG] pgmap v1446: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:06.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:06 smithi175 bash[29284]: cluster 2024-03-26T04:11:05.445601+0000 mgr.y (mgr.24431) 1001 : cluster [DBG] pgmap v1447: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:06.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:06 smithi175 bash[21527]: cluster 2024-03-26T04:11:05.445601+0000 mgr.y (mgr.24431) 1001 : cluster [DBG] pgmap v1447: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:06.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:06 smithi190 bash[23392]: cluster 2024-03-26T04:11:05.445601+0000 mgr.y (mgr.24431) 1001 : cluster [DBG] pgmap v1447: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:07.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:07 smithi175 bash[29284]: audit 2024-03-26T04:11:07.321887+0000 mon.c (mon.2) 804 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:11:07.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:07 smithi175 bash[21527]: audit 2024-03-26T04:11:07.321887+0000 mon.c (mon.2) 804 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:11:07.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:07 smithi190 bash[23392]: audit 2024-03-26T04:11:07.321887+0000 mon.c (mon.2) 804 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:11:08.991 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:11:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:11:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:11:08.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:08 smithi175 bash[29284]: cluster 2024-03-26T04:11:07.446521+0000 mgr.y (mgr.24431) 1002 : cluster [DBG] pgmap v1448: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:08.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:08 smithi175 bash[21527]: cluster 2024-03-26T04:11:07.446521+0000 mgr.y (mgr.24431) 1002 : cluster [DBG] pgmap v1448: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:08.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:08 smithi190 bash[23392]: cluster 2024-03-26T04:11:07.446521+0000 mgr.y (mgr.24431) 1002 : cluster [DBG] pgmap v1448: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:10.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:10 smithi175 bash[29284]: cluster 2024-03-26T04:11:09.447548+0000 mgr.y (mgr.24431) 1003 : cluster [DBG] pgmap v1449: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:10.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:10 smithi175 bash[21527]: cluster 2024-03-26T04:11:09.447548+0000 mgr.y (mgr.24431) 1003 : cluster [DBG] pgmap v1449: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:10.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:10 smithi190 bash[23392]: cluster 2024-03-26T04:11:09.447548+0000 mgr.y (mgr.24431) 1003 : cluster [DBG] pgmap v1449: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:11.498 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:11:11 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:11:12.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:12 smithi175 bash[29284]: audit 2024-03-26T04:11:11.021628+0000 mgr.y (mgr.24431) 1004 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:11:12.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:12 smithi175 bash[29284]: cluster 2024-03-26T04:11:11.449930+0000 mgr.y (mgr.24431) 1005 : cluster [DBG] pgmap v1450: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:12.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:12 smithi175 bash[21527]: audit 2024-03-26T04:11:11.021628+0000 mgr.y (mgr.24431) 1004 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:11:12.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:12 smithi175 bash[21527]: cluster 2024-03-26T04:11:11.449930+0000 mgr.y (mgr.24431) 1005 : cluster [DBG] pgmap v1450: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:12.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:12 smithi190 bash[23392]: audit 2024-03-26T04:11:11.021628+0000 mgr.y (mgr.24431) 1004 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:11:12.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:12 smithi190 bash[23392]: cluster 2024-03-26T04:11:11.449930+0000 mgr.y (mgr.24431) 1005 : cluster [DBG] pgmap v1450: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:14.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:14 smithi175 bash[21527]: cluster 2024-03-26T04:11:13.450917+0000 mgr.y (mgr.24431) 1006 : cluster [DBG] pgmap v1451: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:14.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:14 smithi175 bash[29284]: cluster 2024-03-26T04:11:13.450917+0000 mgr.y (mgr.24431) 1006 : cluster [DBG] pgmap v1451: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:14.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:14 smithi190 bash[23392]: cluster 2024-03-26T04:11:13.450917+0000 mgr.y (mgr.24431) 1006 : cluster [DBG] pgmap v1451: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:16.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:16 smithi175 bash[21527]: cluster 2024-03-26T04:11:15.453217+0000 mgr.y (mgr.24431) 1007 : cluster [DBG] pgmap v1452: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:16.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:16 smithi175 bash[29284]: cluster 2024-03-26T04:11:15.453217+0000 mgr.y (mgr.24431) 1007 : cluster [DBG] pgmap v1452: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:16.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:16 smithi190 bash[23392]: cluster 2024-03-26T04:11:15.453217+0000 mgr.y (mgr.24431) 1007 : cluster [DBG] pgmap v1452: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:18.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:18 smithi175 bash[21527]: cluster 2024-03-26T04:11:17.454037+0000 mgr.y (mgr.24431) 1008 : cluster [DBG] pgmap v1453: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:18.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:18 smithi175 bash[29284]: cluster 2024-03-26T04:11:17.454037+0000 mgr.y (mgr.24431) 1008 : cluster [DBG] pgmap v1453: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:18.992 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:11:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:11:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:11:18.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:18 smithi190 bash[23392]: cluster 2024-03-26T04:11:17.454037+0000 mgr.y (mgr.24431) 1008 : cluster [DBG] pgmap v1453: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:20.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:20 smithi175 bash[21527]: cluster 2024-03-26T04:11:19.454882+0000 mgr.y (mgr.24431) 1009 : cluster [DBG] pgmap v1454: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:20.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:20 smithi175 bash[29284]: cluster 2024-03-26T04:11:19.454882+0000 mgr.y (mgr.24431) 1009 : cluster [DBG] pgmap v1454: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:20.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:20 smithi190 bash[23392]: cluster 2024-03-26T04:11:19.454882+0000 mgr.y (mgr.24431) 1009 : cluster [DBG] pgmap v1454: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:21.498 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:11:21 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:11:22.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:22 smithi175 bash[21527]: audit 2024-03-26T04:11:21.033144+0000 mgr.y (mgr.24431) 1010 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:11:22.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:22 smithi175 bash[21527]: cluster 2024-03-26T04:11:21.457207+0000 mgr.y (mgr.24431) 1011 : cluster [DBG] pgmap v1455: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:22.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:22 smithi175 bash[21527]: audit 2024-03-26T04:11:22.355042+0000 mon.c (mon.2) 805 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:11:22.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:22 smithi175 bash[29284]: audit 2024-03-26T04:11:21.033144+0000 mgr.y (mgr.24431) 1010 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:11:22.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:22 smithi175 bash[29284]: cluster 2024-03-26T04:11:21.457207+0000 mgr.y (mgr.24431) 1011 : cluster [DBG] pgmap v1455: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:22.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:22 smithi175 bash[29284]: audit 2024-03-26T04:11:22.355042+0000 mon.c (mon.2) 805 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:11:22.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:22 smithi190 bash[23392]: audit 2024-03-26T04:11:21.033144+0000 mgr.y (mgr.24431) 1010 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:11:22.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:22 smithi190 bash[23392]: cluster 2024-03-26T04:11:21.457207+0000 mgr.y (mgr.24431) 1011 : cluster [DBG] pgmap v1455: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:22.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:22 smithi190 bash[23392]: audit 2024-03-26T04:11:22.355042+0000 mon.c (mon.2) 805 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:11:24.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:24 smithi175 bash[21527]: cluster 2024-03-26T04:11:23.458041+0000 mgr.y (mgr.24431) 1012 : cluster [DBG] pgmap v1456: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:24.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:24 smithi175 bash[29284]: cluster 2024-03-26T04:11:23.458041+0000 mgr.y (mgr.24431) 1012 : cluster [DBG] pgmap v1456: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:24.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:24 smithi190 bash[23392]: cluster 2024-03-26T04:11:23.458041+0000 mgr.y (mgr.24431) 1012 : cluster [DBG] pgmap v1456: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:26.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:26 smithi175 bash[21527]: cluster 2024-03-26T04:11:25.460265+0000 mgr.y (mgr.24431) 1013 : cluster [DBG] pgmap v1457: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:26.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:26 smithi175 bash[29284]: cluster 2024-03-26T04:11:25.460265+0000 mgr.y (mgr.24431) 1013 : cluster [DBG] pgmap v1457: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:26.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:26 smithi190 bash[23392]: cluster 2024-03-26T04:11:25.460265+0000 mgr.y (mgr.24431) 1013 : cluster [DBG] pgmap v1457: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:28.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:28 smithi175 bash[21527]: cluster 2024-03-26T04:11:27.461086+0000 mgr.y (mgr.24431) 1014 : cluster [DBG] pgmap v1458: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:28.992 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:11:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:11:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:11:28.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:28 smithi175 bash[29284]: cluster 2024-03-26T04:11:27.461086+0000 mgr.y (mgr.24431) 1014 : cluster [DBG] pgmap v1458: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:28.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:28 smithi190 bash[23392]: cluster 2024-03-26T04:11:27.461086+0000 mgr.y (mgr.24431) 1014 : cluster [DBG] pgmap v1458: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:30.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:30 smithi175 bash[21527]: cluster 2024-03-26T04:11:29.461952+0000 mgr.y (mgr.24431) 1015 : cluster [DBG] pgmap v1459: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:30.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:30 smithi175 bash[29284]: cluster 2024-03-26T04:11:29.461952+0000 mgr.y (mgr.24431) 1015 : cluster [DBG] pgmap v1459: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:30.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:30 smithi190 bash[23392]: cluster 2024-03-26T04:11:29.461952+0000 mgr.y (mgr.24431) 1015 : cluster [DBG] pgmap v1459: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:31.498 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:11:31 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:11:32.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:32 smithi175 bash[21527]: audit 2024-03-26T04:11:31.041650+0000 mgr.y (mgr.24431) 1016 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:11:32.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:32 smithi175 bash[21527]: cluster 2024-03-26T04:11:31.464215+0000 mgr.y (mgr.24431) 1017 : cluster [DBG] pgmap v1460: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:32.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:32 smithi175 bash[21527]: audit 2024-03-26T04:11:32.215522+0000 mon.c (mon.2) 806 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:11:32.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:32 smithi175 bash[29284]: audit 2024-03-26T04:11:31.041650+0000 mgr.y (mgr.24431) 1016 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:11:32.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:32 smithi175 bash[29284]: cluster 2024-03-26T04:11:31.464215+0000 mgr.y (mgr.24431) 1017 : cluster [DBG] pgmap v1460: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:32.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:32 smithi175 bash[29284]: audit 2024-03-26T04:11:32.215522+0000 mon.c (mon.2) 806 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:11:32.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:32 smithi190 bash[23392]: audit 2024-03-26T04:11:31.041650+0000 mgr.y (mgr.24431) 1016 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:11:32.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:32 smithi190 bash[23392]: cluster 2024-03-26T04:11:31.464215+0000 mgr.y (mgr.24431) 1017 : cluster [DBG] pgmap v1460: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:32.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:32 smithi190 bash[23392]: audit 2024-03-26T04:11:32.215522+0000 mon.c (mon.2) 806 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:11:33.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:33 smithi175 bash[21527]: audit 2024-03-26T04:11:32.945279+0000 mon.c (mon.2) 807 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:11:33.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:33 smithi175 bash[21527]: audit 2024-03-26T04:11:32.946820+0000 mon.c (mon.2) 808 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:11:33.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:33 smithi175 bash[21527]: audit 2024-03-26T04:11:32.957727+0000 mon.a (mon.0) 3539 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:11:33.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:33 smithi175 bash[29284]: audit 2024-03-26T04:11:32.945279+0000 mon.c (mon.2) 807 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:11:33.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:33 smithi175 bash[29284]: audit 2024-03-26T04:11:32.946820+0000 mon.c (mon.2) 808 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:11:33.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:33 smithi175 bash[29284]: audit 2024-03-26T04:11:32.957727+0000 mon.a (mon.0) 3539 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:11:33.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:33 smithi190 bash[23392]: audit 2024-03-26T04:11:32.945279+0000 mon.c (mon.2) 807 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:11:33.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:33 smithi190 bash[23392]: audit 2024-03-26T04:11:32.946820+0000 mon.c (mon.2) 808 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:11:33.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:33 smithi190 bash[23392]: audit 2024-03-26T04:11:32.957727+0000 mon.a (mon.0) 3539 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:11:34.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:34 smithi175 bash[21527]: cluster 2024-03-26T04:11:33.465061+0000 mgr.y (mgr.24431) 1018 : cluster [DBG] pgmap v1461: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:34.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:34 smithi175 bash[29284]: cluster 2024-03-26T04:11:33.465061+0000 mgr.y (mgr.24431) 1018 : cluster [DBG] pgmap v1461: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:34.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:34 smithi190 bash[23392]: cluster 2024-03-26T04:11:33.465061+0000 mgr.y (mgr.24431) 1018 : cluster [DBG] pgmap v1461: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:36.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:36 smithi175 bash[29284]: cluster 2024-03-26T04:11:35.467258+0000 mgr.y (mgr.24431) 1019 : cluster [DBG] pgmap v1462: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:36.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:36 smithi175 bash[21527]: cluster 2024-03-26T04:11:35.467258+0000 mgr.y (mgr.24431) 1019 : cluster [DBG] pgmap v1462: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:36.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:36 smithi190 bash[23392]: cluster 2024-03-26T04:11:35.467258+0000 mgr.y (mgr.24431) 1019 : cluster [DBG] pgmap v1462: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:37.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:37 smithi175 bash[29284]: audit 2024-03-26T04:11:37.389218+0000 mon.c (mon.2) 809 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:11:37.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:37 smithi175 bash[21527]: audit 2024-03-26T04:11:37.389218+0000 mon.c (mon.2) 809 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:11:37.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:37 smithi190 bash[23392]: audit 2024-03-26T04:11:37.389218+0000 mon.c (mon.2) 809 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:11:38.991 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:11:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:11:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:11:38.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:38 smithi175 bash[29284]: cluster 2024-03-26T04:11:37.468100+0000 mgr.y (mgr.24431) 1020 : cluster [DBG] pgmap v1463: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:38.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:38 smithi175 bash[21527]: cluster 2024-03-26T04:11:37.468100+0000 mgr.y (mgr.24431) 1020 : cluster [DBG] pgmap v1463: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:38.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:38 smithi190 bash[23392]: cluster 2024-03-26T04:11:37.468100+0000 mgr.y (mgr.24431) 1020 : cluster [DBG] pgmap v1463: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:40.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:40 smithi175 bash[29284]: cluster 2024-03-26T04:11:39.468944+0000 mgr.y (mgr.24431) 1021 : cluster [DBG] pgmap v1464: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:40.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:40 smithi175 bash[21527]: cluster 2024-03-26T04:11:39.468944+0000 mgr.y (mgr.24431) 1021 : cluster [DBG] pgmap v1464: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:40.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:40 smithi190 bash[23392]: cluster 2024-03-26T04:11:39.468944+0000 mgr.y (mgr.24431) 1021 : cluster [DBG] pgmap v1464: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:41.498 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:11:41 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:11:42.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:42 smithi175 bash[21527]: audit 2024-03-26T04:11:41.053144+0000 mgr.y (mgr.24431) 1022 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:11:42.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:42 smithi175 bash[21527]: cluster 2024-03-26T04:11:41.471251+0000 mgr.y (mgr.24431) 1023 : cluster [DBG] pgmap v1465: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:42.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:42 smithi175 bash[29284]: audit 2024-03-26T04:11:41.053144+0000 mgr.y (mgr.24431) 1022 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:11:42.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:42 smithi175 bash[29284]: cluster 2024-03-26T04:11:41.471251+0000 mgr.y (mgr.24431) 1023 : cluster [DBG] pgmap v1465: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:42.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:42 smithi190 bash[23392]: audit 2024-03-26T04:11:41.053144+0000 mgr.y (mgr.24431) 1022 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:11:42.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:42 smithi190 bash[23392]: cluster 2024-03-26T04:11:41.471251+0000 mgr.y (mgr.24431) 1023 : cluster [DBG] pgmap v1465: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:44.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:44 smithi175 bash[21527]: cluster 2024-03-26T04:11:43.472102+0000 mgr.y (mgr.24431) 1024 : cluster [DBG] pgmap v1466: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:44.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:44 smithi175 bash[29284]: cluster 2024-03-26T04:11:43.472102+0000 mgr.y (mgr.24431) 1024 : cluster [DBG] pgmap v1466: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:44.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:44 smithi190 bash[23392]: cluster 2024-03-26T04:11:43.472102+0000 mgr.y (mgr.24431) 1024 : cluster [DBG] pgmap v1466: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:46.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:46 smithi190 bash[23392]: cluster 2024-03-26T04:11:45.474332+0000 mgr.y (mgr.24431) 1025 : cluster [DBG] pgmap v1467: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:47.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:46 smithi175 bash[21527]: cluster 2024-03-26T04:11:45.474332+0000 mgr.y (mgr.24431) 1025 : cluster [DBG] pgmap v1467: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:47.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:46 smithi175 bash[29284]: cluster 2024-03-26T04:11:45.474332+0000 mgr.y (mgr.24431) 1025 : cluster [DBG] pgmap v1467: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:49.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:48 smithi175 bash[21527]: cluster 2024-03-26T04:11:47.475114+0000 mgr.y (mgr.24431) 1026 : cluster [DBG] pgmap v1468: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:49.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:48 smithi175 bash[29284]: cluster 2024-03-26T04:11:47.475114+0000 mgr.y (mgr.24431) 1026 : cluster [DBG] pgmap v1468: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:49.242 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:11:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:11:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:11:49.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:48 smithi190 bash[23392]: cluster 2024-03-26T04:11:47.475114+0000 mgr.y (mgr.24431) 1026 : cluster [DBG] pgmap v1468: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:50.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:49 smithi175 bash[21527]: cluster 2024-03-26T04:11:49.475984+0000 mgr.y (mgr.24431) 1027 : cluster [DBG] pgmap v1469: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:50.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:49 smithi175 bash[29284]: cluster 2024-03-26T04:11:49.475984+0000 mgr.y (mgr.24431) 1027 : cluster [DBG] pgmap v1469: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:50.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:49 smithi190 bash[23392]: cluster 2024-03-26T04:11:49.475984+0000 mgr.y (mgr.24431) 1027 : cluster [DBG] pgmap v1469: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:51.498 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:11:51 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:11:52.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:52 smithi175 bash[21527]: audit 2024-03-26T04:11:51.061758+0000 mgr.y (mgr.24431) 1028 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:11:52.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:52 smithi175 bash[21527]: cluster 2024-03-26T04:11:51.478285+0000 mgr.y (mgr.24431) 1029 : cluster [DBG] pgmap v1470: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:52.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:52 smithi175 bash[21527]: audit 2024-03-26T04:11:52.422855+0000 mon.c (mon.2) 810 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:11:52.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:52 smithi175 bash[29284]: audit 2024-03-26T04:11:51.061758+0000 mgr.y (mgr.24431) 1028 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:11:52.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:52 smithi175 bash[29284]: cluster 2024-03-26T04:11:51.478285+0000 mgr.y (mgr.24431) 1029 : cluster [DBG] pgmap v1470: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:52.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:52 smithi175 bash[29284]: audit 2024-03-26T04:11:52.422855+0000 mon.c (mon.2) 810 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:11:52.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:52 smithi190 bash[23392]: audit 2024-03-26T04:11:51.061758+0000 mgr.y (mgr.24431) 1028 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:11:52.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:52 smithi190 bash[23392]: cluster 2024-03-26T04:11:51.478285+0000 mgr.y (mgr.24431) 1029 : cluster [DBG] pgmap v1470: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:52.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:52 smithi190 bash[23392]: audit 2024-03-26T04:11:52.422855+0000 mon.c (mon.2) 810 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:11:54.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:54 smithi175 bash[21527]: cluster 2024-03-26T04:11:53.479338+0000 mgr.y (mgr.24431) 1030 : cluster [DBG] pgmap v1471: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:54.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:54 smithi175 bash[29284]: cluster 2024-03-26T04:11:53.479338+0000 mgr.y (mgr.24431) 1030 : cluster [DBG] pgmap v1471: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:54.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:54 smithi190 bash[23392]: cluster 2024-03-26T04:11:53.479338+0000 mgr.y (mgr.24431) 1030 : cluster [DBG] pgmap v1471: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:56.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:56 smithi175 bash[29284]: cluster 2024-03-26T04:11:55.481505+0000 mgr.y (mgr.24431) 1031 : cluster [DBG] pgmap v1472: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:56.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:56 smithi175 bash[21527]: cluster 2024-03-26T04:11:55.481505+0000 mgr.y (mgr.24431) 1031 : cluster [DBG] pgmap v1472: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:56.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:56 smithi190 bash[23392]: cluster 2024-03-26T04:11:55.481505+0000 mgr.y (mgr.24431) 1031 : cluster [DBG] pgmap v1472: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:11:58.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:11:58 smithi175 bash[21527]: cluster 2024-03-26T04:11:57.482333+0000 mgr.y (mgr.24431) 1032 : cluster [DBG] pgmap v1473: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:58.992 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:11:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:11:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:11:58.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:11:58 smithi175 bash[29284]: cluster 2024-03-26T04:11:57.482333+0000 mgr.y (mgr.24431) 1032 : cluster [DBG] pgmap v1473: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:11:58.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:11:58 smithi190 bash[23392]: cluster 2024-03-26T04:11:57.482333+0000 mgr.y (mgr.24431) 1032 : cluster [DBG] pgmap v1473: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:00.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:00 smithi175 bash[21527]: cluster 2024-03-26T04:11:59.483207+0000 mgr.y (mgr.24431) 1033 : cluster [DBG] pgmap v1474: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:00.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:00 smithi175 bash[29284]: cluster 2024-03-26T04:11:59.483207+0000 mgr.y (mgr.24431) 1033 : cluster [DBG] pgmap v1474: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:00.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:00 smithi190 bash[23392]: cluster 2024-03-26T04:11:59.483207+0000 mgr.y (mgr.24431) 1033 : cluster [DBG] pgmap v1474: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:01.498 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:12:01 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:12:02.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:02 smithi175 bash[21527]: audit 2024-03-26T04:12:01.073246+0000 mgr.y (mgr.24431) 1034 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:12:02.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:02 smithi175 bash[21527]: cluster 2024-03-26T04:12:01.485466+0000 mgr.y (mgr.24431) 1035 : cluster [DBG] pgmap v1475: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:02.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:02 smithi175 bash[29284]: audit 2024-03-26T04:12:01.073246+0000 mgr.y (mgr.24431) 1034 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:12:02.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:02 smithi175 bash[29284]: cluster 2024-03-26T04:12:01.485466+0000 mgr.y (mgr.24431) 1035 : cluster [DBG] pgmap v1475: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:02.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:02 smithi190 bash[23392]: audit 2024-03-26T04:12:01.073246+0000 mgr.y (mgr.24431) 1034 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:12:02.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:02 smithi190 bash[23392]: cluster 2024-03-26T04:12:01.485466+0000 mgr.y (mgr.24431) 1035 : cluster [DBG] pgmap v1475: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:04.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:04 smithi175 bash[21527]: cluster 2024-03-26T04:12:03.486304+0000 mgr.y (mgr.24431) 1036 : cluster [DBG] pgmap v1476: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:04.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:04 smithi175 bash[29284]: cluster 2024-03-26T04:12:03.486304+0000 mgr.y (mgr.24431) 1036 : cluster [DBG] pgmap v1476: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:04.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:04 smithi190 bash[23392]: cluster 2024-03-26T04:12:03.486304+0000 mgr.y (mgr.24431) 1036 : cluster [DBG] pgmap v1476: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:06.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:06 smithi175 bash[21527]: cluster 2024-03-26T04:12:05.488425+0000 mgr.y (mgr.24431) 1037 : cluster [DBG] pgmap v1477: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:06.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:06 smithi175 bash[29284]: cluster 2024-03-26T04:12:05.488425+0000 mgr.y (mgr.24431) 1037 : cluster [DBG] pgmap v1477: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:06.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:06 smithi190 bash[23392]: cluster 2024-03-26T04:12:05.488425+0000 mgr.y (mgr.24431) 1037 : cluster [DBG] pgmap v1477: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:07.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:07 smithi175 bash[29284]: audit 2024-03-26T04:12:07.456210+0000 mon.c (mon.2) 811 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:12:07.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:07 smithi175 bash[21527]: audit 2024-03-26T04:12:07.456210+0000 mon.c (mon.2) 811 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:12:07.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:07 smithi190 bash[23392]: audit 2024-03-26T04:12:07.456210+0000 mon.c (mon.2) 811 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:12:08.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:08 smithi175 bash[21527]: cluster 2024-03-26T04:12:07.489230+0000 mgr.y (mgr.24431) 1038 : cluster [DBG] pgmap v1478: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:08.992 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:12:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:12:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:12:08.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:08 smithi175 bash[29284]: cluster 2024-03-26T04:12:07.489230+0000 mgr.y (mgr.24431) 1038 : cluster [DBG] pgmap v1478: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:08.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:08 smithi190 bash[23392]: cluster 2024-03-26T04:12:07.489230+0000 mgr.y (mgr.24431) 1038 : cluster [DBG] pgmap v1478: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:10.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:10 smithi175 bash[21527]: cluster 2024-03-26T04:12:09.490089+0000 mgr.y (mgr.24431) 1039 : cluster [DBG] pgmap v1479: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:10.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:10 smithi175 bash[29284]: cluster 2024-03-26T04:12:09.490089+0000 mgr.y (mgr.24431) 1039 : cluster [DBG] pgmap v1479: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:10.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:10 smithi190 bash[23392]: cluster 2024-03-26T04:12:09.490089+0000 mgr.y (mgr.24431) 1039 : cluster [DBG] pgmap v1479: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:11.498 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:12:11 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:12:12.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:12 smithi175 bash[21527]: audit 2024-03-26T04:12:11.084704+0000 mgr.y (mgr.24431) 1040 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:12:12.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:12 smithi175 bash[21527]: cluster 2024-03-26T04:12:11.492385+0000 mgr.y (mgr.24431) 1041 : cluster [DBG] pgmap v1480: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:12.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:12 smithi175 bash[29284]: audit 2024-03-26T04:12:11.084704+0000 mgr.y (mgr.24431) 1040 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:12:12.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:12 smithi175 bash[29284]: cluster 2024-03-26T04:12:11.492385+0000 mgr.y (mgr.24431) 1041 : cluster [DBG] pgmap v1480: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:12.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:12 smithi190 bash[23392]: audit 2024-03-26T04:12:11.084704+0000 mgr.y (mgr.24431) 1040 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:12:12.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:12 smithi190 bash[23392]: cluster 2024-03-26T04:12:11.492385+0000 mgr.y (mgr.24431) 1041 : cluster [DBG] pgmap v1480: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:14.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:14 smithi175 bash[21527]: cluster 2024-03-26T04:12:13.493223+0000 mgr.y (mgr.24431) 1042 : cluster [DBG] pgmap v1481: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:14.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:14 smithi175 bash[29284]: cluster 2024-03-26T04:12:13.493223+0000 mgr.y (mgr.24431) 1042 : cluster [DBG] pgmap v1481: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:14.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:14 smithi190 bash[23392]: cluster 2024-03-26T04:12:13.493223+0000 mgr.y (mgr.24431) 1042 : cluster [DBG] pgmap v1481: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:16.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:16 smithi175 bash[29284]: cluster 2024-03-26T04:12:15.495447+0000 mgr.y (mgr.24431) 1043 : cluster [DBG] pgmap v1482: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:16.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:16 smithi175 bash[21527]: cluster 2024-03-26T04:12:15.495447+0000 mgr.y (mgr.24431) 1043 : cluster [DBG] pgmap v1482: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:16.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:16 smithi190 bash[23392]: cluster 2024-03-26T04:12:15.495447+0000 mgr.y (mgr.24431) 1043 : cluster [DBG] pgmap v1482: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:18.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:18 smithi175 bash[21527]: cluster 2024-03-26T04:12:17.496270+0000 mgr.y (mgr.24431) 1044 : cluster [DBG] pgmap v1483: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:18.991 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:12:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:12:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:12:18.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:18 smithi175 bash[29284]: cluster 2024-03-26T04:12:17.496270+0000 mgr.y (mgr.24431) 1044 : cluster [DBG] pgmap v1483: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:18.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:18 smithi190 bash[23392]: cluster 2024-03-26T04:12:17.496270+0000 mgr.y (mgr.24431) 1044 : cluster [DBG] pgmap v1483: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:20.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:20 smithi175 bash[21527]: cluster 2024-03-26T04:12:19.497057+0000 mgr.y (mgr.24431) 1045 : cluster [DBG] pgmap v1484: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:20.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:20 smithi175 bash[29284]: cluster 2024-03-26T04:12:19.497057+0000 mgr.y (mgr.24431) 1045 : cluster [DBG] pgmap v1484: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:20.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:20 smithi190 bash[23392]: cluster 2024-03-26T04:12:19.497057+0000 mgr.y (mgr.24431) 1045 : cluster [DBG] pgmap v1484: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:21.498 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:12:21 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:12:22.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:22 smithi175 bash[29284]: audit 2024-03-26T04:12:21.093139+0000 mgr.y (mgr.24431) 1046 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:12:22.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:22 smithi175 bash[29284]: cluster 2024-03-26T04:12:21.499301+0000 mgr.y (mgr.24431) 1047 : cluster [DBG] pgmap v1485: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:22.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:22 smithi175 bash[29284]: audit 2024-03-26T04:12:22.490507+0000 mon.c (mon.2) 812 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:12:22.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:22 smithi175 bash[21527]: audit 2024-03-26T04:12:21.093139+0000 mgr.y (mgr.24431) 1046 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:12:22.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:22 smithi175 bash[21527]: cluster 2024-03-26T04:12:21.499301+0000 mgr.y (mgr.24431) 1047 : cluster [DBG] pgmap v1485: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:22.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:22 smithi175 bash[21527]: audit 2024-03-26T04:12:22.490507+0000 mon.c (mon.2) 812 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:12:22.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:22 smithi190 bash[23392]: audit 2024-03-26T04:12:21.093139+0000 mgr.y (mgr.24431) 1046 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:12:22.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:22 smithi190 bash[23392]: cluster 2024-03-26T04:12:21.499301+0000 mgr.y (mgr.24431) 1047 : cluster [DBG] pgmap v1485: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:22.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:22 smithi190 bash[23392]: audit 2024-03-26T04:12:22.490507+0000 mon.c (mon.2) 812 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:12:24.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:24 smithi175 bash[21527]: cluster 2024-03-26T04:12:23.500204+0000 mgr.y (mgr.24431) 1048 : cluster [DBG] pgmap v1486: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:24.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:24 smithi175 bash[29284]: cluster 2024-03-26T04:12:23.500204+0000 mgr.y (mgr.24431) 1048 : cluster [DBG] pgmap v1486: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:24.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:24 smithi190 bash[23392]: cluster 2024-03-26T04:12:23.500204+0000 mgr.y (mgr.24431) 1048 : cluster [DBG] pgmap v1486: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:26.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:26 smithi175 bash[21527]: cluster 2024-03-26T04:12:25.502429+0000 mgr.y (mgr.24431) 1049 : cluster [DBG] pgmap v1487: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:26.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:26 smithi175 bash[29284]: cluster 2024-03-26T04:12:25.502429+0000 mgr.y (mgr.24431) 1049 : cluster [DBG] pgmap v1487: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:26.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:26 smithi190 bash[23392]: cluster 2024-03-26T04:12:25.502429+0000 mgr.y (mgr.24431) 1049 : cluster [DBG] pgmap v1487: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:28.991 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:12:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:12:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:12:28.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:28 smithi175 bash[21527]: cluster 2024-03-26T04:12:27.503360+0000 mgr.y (mgr.24431) 1050 : cluster [DBG] pgmap v1488: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:28.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:28 smithi175 bash[29284]: cluster 2024-03-26T04:12:27.503360+0000 mgr.y (mgr.24431) 1050 : cluster [DBG] pgmap v1488: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:28.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:28 smithi190 bash[23392]: cluster 2024-03-26T04:12:27.503360+0000 mgr.y (mgr.24431) 1050 : cluster [DBG] pgmap v1488: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:30.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:30 smithi175 bash[21527]: cluster 2024-03-26T04:12:29.504349+0000 mgr.y (mgr.24431) 1051 : cluster [DBG] pgmap v1489: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:30.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:30 smithi175 bash[29284]: cluster 2024-03-26T04:12:29.504349+0000 mgr.y (mgr.24431) 1051 : cluster [DBG] pgmap v1489: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:30.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:30 smithi190 bash[23392]: cluster 2024-03-26T04:12:29.504349+0000 mgr.y (mgr.24431) 1051 : cluster [DBG] pgmap v1489: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:31.498 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:12:31 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:12:32.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:32 smithi175 bash[21527]: audit 2024-03-26T04:12:31.101865+0000 mgr.y (mgr.24431) 1052 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:12:32.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:32 smithi175 bash[21527]: cluster 2024-03-26T04:12:31.506640+0000 mgr.y (mgr.24431) 1053 : cluster [DBG] pgmap v1490: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:32.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:32 smithi175 bash[29284]: audit 2024-03-26T04:12:31.101865+0000 mgr.y (mgr.24431) 1052 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:12:32.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:32 smithi175 bash[29284]: cluster 2024-03-26T04:12:31.506640+0000 mgr.y (mgr.24431) 1053 : cluster [DBG] pgmap v1490: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:32.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:32 smithi190 bash[23392]: audit 2024-03-26T04:12:31.101865+0000 mgr.y (mgr.24431) 1052 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:12:32.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:32 smithi190 bash[23392]: cluster 2024-03-26T04:12:31.506640+0000 mgr.y (mgr.24431) 1053 : cluster [DBG] pgmap v1490: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:33.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:33 smithi175 bash[21527]: audit 2024-03-26T04:12:33.018863+0000 mon.c (mon.2) 813 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:12:33.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:33 smithi175 bash[29284]: audit 2024-03-26T04:12:33.018863+0000 mon.c (mon.2) 813 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:12:33.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:33 smithi190 bash[23392]: audit 2024-03-26T04:12:33.018863+0000 mon.c (mon.2) 813 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:12:34.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:34 smithi175 bash[21527]: cluster 2024-03-26T04:12:33.507248+0000 mgr.y (mgr.24431) 1054 : cluster [DBG] pgmap v1491: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:34.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:34 smithi175 bash[21527]: audit 2024-03-26T04:12:33.809784+0000 mon.c (mon.2) 814 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:12:34.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:34 smithi175 bash[21527]: audit 2024-03-26T04:12:33.811536+0000 mon.c (mon.2) 815 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:12:34.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:34 smithi175 bash[21527]: audit 2024-03-26T04:12:33.820047+0000 mon.a (mon.0) 3540 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:12:34.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:34 smithi175 bash[29284]: cluster 2024-03-26T04:12:33.507248+0000 mgr.y (mgr.24431) 1054 : cluster [DBG] pgmap v1491: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:34.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:34 smithi175 bash[29284]: audit 2024-03-26T04:12:33.809784+0000 mon.c (mon.2) 814 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:12:34.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:34 smithi175 bash[29284]: audit 2024-03-26T04:12:33.811536+0000 mon.c (mon.2) 815 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:12:34.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:34 smithi175 bash[29284]: audit 2024-03-26T04:12:33.820047+0000 mon.a (mon.0) 3540 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:12:34.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:34 smithi190 bash[23392]: cluster 2024-03-26T04:12:33.507248+0000 mgr.y (mgr.24431) 1054 : cluster [DBG] pgmap v1491: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:34.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:34 smithi190 bash[23392]: audit 2024-03-26T04:12:33.809784+0000 mon.c (mon.2) 814 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:12:34.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:34 smithi190 bash[23392]: audit 2024-03-26T04:12:33.811536+0000 mon.c (mon.2) 815 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:12:34.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:34 smithi190 bash[23392]: audit 2024-03-26T04:12:33.820047+0000 mon.a (mon.0) 3540 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:12:36.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:36 smithi175 bash[21527]: cluster 2024-03-26T04:12:35.509447+0000 mgr.y (mgr.24431) 1055 : cluster [DBG] pgmap v1492: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:36.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:36 smithi175 bash[29284]: cluster 2024-03-26T04:12:35.509447+0000 mgr.y (mgr.24431) 1055 : cluster [DBG] pgmap v1492: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:36.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:36 smithi190 bash[23392]: cluster 2024-03-26T04:12:35.509447+0000 mgr.y (mgr.24431) 1055 : cluster [DBG] pgmap v1492: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:37.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:37 smithi175 bash[29284]: audit 2024-03-26T04:12:37.524380+0000 mon.c (mon.2) 816 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:12:37.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:37 smithi175 bash[21527]: audit 2024-03-26T04:12:37.524380+0000 mon.c (mon.2) 816 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:12:37.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:37 smithi190 bash[23392]: audit 2024-03-26T04:12:37.524380+0000 mon.c (mon.2) 816 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:12:38.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:38 smithi175 bash[29284]: cluster 2024-03-26T04:12:37.510281+0000 mgr.y (mgr.24431) 1056 : cluster [DBG] pgmap v1493: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:38.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:38 smithi175 bash[21527]: cluster 2024-03-26T04:12:37.510281+0000 mgr.y (mgr.24431) 1056 : cluster [DBG] pgmap v1493: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:38.992 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:12:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:12:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:12:38.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:38 smithi190 bash[23392]: cluster 2024-03-26T04:12:37.510281+0000 mgr.y (mgr.24431) 1056 : cluster [DBG] pgmap v1493: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:40.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:40 smithi175 bash[21527]: cluster 2024-03-26T04:12:39.511118+0000 mgr.y (mgr.24431) 1057 : cluster [DBG] pgmap v1494: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:40.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:40 smithi175 bash[29284]: cluster 2024-03-26T04:12:39.511118+0000 mgr.y (mgr.24431) 1057 : cluster [DBG] pgmap v1494: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:40.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:40 smithi190 bash[23392]: cluster 2024-03-26T04:12:39.511118+0000 mgr.y (mgr.24431) 1057 : cluster [DBG] pgmap v1494: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:41.498 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:12:41 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:12:42.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:42 smithi175 bash[21527]: audit 2024-03-26T04:12:41.113363+0000 mgr.y (mgr.24431) 1058 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:12:42.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:42 smithi175 bash[21527]: cluster 2024-03-26T04:12:41.513405+0000 mgr.y (mgr.24431) 1059 : cluster [DBG] pgmap v1495: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:42.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:42 smithi175 bash[29284]: audit 2024-03-26T04:12:41.113363+0000 mgr.y (mgr.24431) 1058 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:12:42.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:42 smithi175 bash[29284]: cluster 2024-03-26T04:12:41.513405+0000 mgr.y (mgr.24431) 1059 : cluster [DBG] pgmap v1495: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:42.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:42 smithi190 bash[23392]: audit 2024-03-26T04:12:41.113363+0000 mgr.y (mgr.24431) 1058 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:12:42.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:42 smithi190 bash[23392]: cluster 2024-03-26T04:12:41.513405+0000 mgr.y (mgr.24431) 1059 : cluster [DBG] pgmap v1495: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:44.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:44 smithi175 bash[21527]: cluster 2024-03-26T04:12:43.514191+0000 mgr.y (mgr.24431) 1060 : cluster [DBG] pgmap v1496: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:44.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:44 smithi175 bash[29284]: cluster 2024-03-26T04:12:43.514191+0000 mgr.y (mgr.24431) 1060 : cluster [DBG] pgmap v1496: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:44.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:44 smithi190 bash[23392]: cluster 2024-03-26T04:12:43.514191+0000 mgr.y (mgr.24431) 1060 : cluster [DBG] pgmap v1496: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:46.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:46 smithi175 bash[21527]: cluster 2024-03-26T04:12:45.516454+0000 mgr.y (mgr.24431) 1061 : cluster [DBG] pgmap v1497: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:46.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:46 smithi175 bash[29284]: cluster 2024-03-26T04:12:45.516454+0000 mgr.y (mgr.24431) 1061 : cluster [DBG] pgmap v1497: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:46.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:46 smithi190 bash[23392]: cluster 2024-03-26T04:12:45.516454+0000 mgr.y (mgr.24431) 1061 : cluster [DBG] pgmap v1497: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:48.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:48 smithi175 bash[21527]: cluster 2024-03-26T04:12:47.517275+0000 mgr.y (mgr.24431) 1062 : cluster [DBG] pgmap v1498: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:48.992 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:12:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:12:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:12:48.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:48 smithi175 bash[29284]: cluster 2024-03-26T04:12:47.517275+0000 mgr.y (mgr.24431) 1062 : cluster [DBG] pgmap v1498: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:48.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:48 smithi190 bash[23392]: cluster 2024-03-26T04:12:47.517275+0000 mgr.y (mgr.24431) 1062 : cluster [DBG] pgmap v1498: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:50.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:50 smithi175 bash[21527]: cluster 2024-03-26T04:12:49.518102+0000 mgr.y (mgr.24431) 1063 : cluster [DBG] pgmap v1499: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:50.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:50 smithi175 bash[29284]: cluster 2024-03-26T04:12:49.518102+0000 mgr.y (mgr.24431) 1063 : cluster [DBG] pgmap v1499: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:50.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:50 smithi190 bash[23392]: cluster 2024-03-26T04:12:49.518102+0000 mgr.y (mgr.24431) 1063 : cluster [DBG] pgmap v1499: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:51.498 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:12:51 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:12:52.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:52 smithi175 bash[21527]: audit 2024-03-26T04:12:51.124850+0000 mgr.y (mgr.24431) 1064 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:12:52.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:52 smithi175 bash[21527]: cluster 2024-03-26T04:12:51.520384+0000 mgr.y (mgr.24431) 1065 : cluster [DBG] pgmap v1500: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:52.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:52 smithi175 bash[21527]: audit 2024-03-26T04:12:52.560910+0000 mon.c (mon.2) 817 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:12:52.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:52 smithi175 bash[29284]: audit 2024-03-26T04:12:51.124850+0000 mgr.y (mgr.24431) 1064 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:12:52.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:52 smithi175 bash[29284]: cluster 2024-03-26T04:12:51.520384+0000 mgr.y (mgr.24431) 1065 : cluster [DBG] pgmap v1500: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:52.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:52 smithi175 bash[29284]: audit 2024-03-26T04:12:52.560910+0000 mon.c (mon.2) 817 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:12:52.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:52 smithi190 bash[23392]: audit 2024-03-26T04:12:51.124850+0000 mgr.y (mgr.24431) 1064 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:12:52.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:52 smithi190 bash[23392]: cluster 2024-03-26T04:12:51.520384+0000 mgr.y (mgr.24431) 1065 : cluster [DBG] pgmap v1500: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:52.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:52 smithi190 bash[23392]: audit 2024-03-26T04:12:52.560910+0000 mon.c (mon.2) 817 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:12:54.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:54 smithi175 bash[29284]: cluster 2024-03-26T04:12:53.521227+0000 mgr.y (mgr.24431) 1066 : cluster [DBG] pgmap v1501: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:54.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:54 smithi175 bash[21527]: cluster 2024-03-26T04:12:53.521227+0000 mgr.y (mgr.24431) 1066 : cluster [DBG] pgmap v1501: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:54.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:54 smithi190 bash[23392]: cluster 2024-03-26T04:12:53.521227+0000 mgr.y (mgr.24431) 1066 : cluster [DBG] pgmap v1501: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:56.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:56 smithi175 bash[29284]: cluster 2024-03-26T04:12:55.523451+0000 mgr.y (mgr.24431) 1067 : cluster [DBG] pgmap v1502: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:56.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:56 smithi175 bash[21527]: cluster 2024-03-26T04:12:55.523451+0000 mgr.y (mgr.24431) 1067 : cluster [DBG] pgmap v1502: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:56.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:56 smithi190 bash[23392]: cluster 2024-03-26T04:12:55.523451+0000 mgr.y (mgr.24431) 1067 : cluster [DBG] pgmap v1502: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:12:58.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:12:58 smithi175 bash[21527]: cluster 2024-03-26T04:12:57.524216+0000 mgr.y (mgr.24431) 1068 : cluster [DBG] pgmap v1503: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:58.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:12:58 smithi175 bash[29284]: cluster 2024-03-26T04:12:57.524216+0000 mgr.y (mgr.24431) 1068 : cluster [DBG] pgmap v1503: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:12:58.991 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:12:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:12:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:12:58.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:12:58 smithi190 bash[23392]: cluster 2024-03-26T04:12:57.524216+0000 mgr.y (mgr.24431) 1068 : cluster [DBG] pgmap v1503: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:00.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:00 smithi175 bash[21527]: cluster 2024-03-26T04:12:59.525062+0000 mgr.y (mgr.24431) 1069 : cluster [DBG] pgmap v1504: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:00.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:00 smithi175 bash[29284]: cluster 2024-03-26T04:12:59.525062+0000 mgr.y (mgr.24431) 1069 : cluster [DBG] pgmap v1504: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:00.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:00 smithi190 bash[23392]: cluster 2024-03-26T04:12:59.525062+0000 mgr.y (mgr.24431) 1069 : cluster [DBG] pgmap v1504: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:01.498 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:13:01 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:13:02.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:02 smithi175 bash[29284]: audit 2024-03-26T04:13:01.133997+0000 mgr.y (mgr.24431) 1070 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:13:02.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:02 smithi175 bash[29284]: cluster 2024-03-26T04:13:01.527357+0000 mgr.y (mgr.24431) 1071 : cluster [DBG] pgmap v1505: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:02.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:02 smithi175 bash[21527]: audit 2024-03-26T04:13:01.133997+0000 mgr.y (mgr.24431) 1070 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:13:02.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:02 smithi175 bash[21527]: cluster 2024-03-26T04:13:01.527357+0000 mgr.y (mgr.24431) 1071 : cluster [DBG] pgmap v1505: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:02.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:02 smithi190 bash[23392]: audit 2024-03-26T04:13:01.133997+0000 mgr.y (mgr.24431) 1070 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:13:02.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:02 smithi190 bash[23392]: cluster 2024-03-26T04:13:01.527357+0000 mgr.y (mgr.24431) 1071 : cluster [DBG] pgmap v1505: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:04.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:04 smithi175 bash[29284]: cluster 2024-03-26T04:13:03.528177+0000 mgr.y (mgr.24431) 1072 : cluster [DBG] pgmap v1506: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:04.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:04 smithi175 bash[21527]: cluster 2024-03-26T04:13:03.528177+0000 mgr.y (mgr.24431) 1072 : cluster [DBG] pgmap v1506: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:04.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:04 smithi190 bash[23392]: cluster 2024-03-26T04:13:03.528177+0000 mgr.y (mgr.24431) 1072 : cluster [DBG] pgmap v1506: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:06.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:06 smithi175 bash[21527]: cluster 2024-03-26T04:13:05.530308+0000 mgr.y (mgr.24431) 1073 : cluster [DBG] pgmap v1507: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:06.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:06 smithi175 bash[29284]: cluster 2024-03-26T04:13:05.530308+0000 mgr.y (mgr.24431) 1073 : cluster [DBG] pgmap v1507: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:06.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:06 smithi190 bash[23392]: cluster 2024-03-26T04:13:05.530308+0000 mgr.y (mgr.24431) 1073 : cluster [DBG] pgmap v1507: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:07.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:07 smithi175 bash[21527]: audit 2024-03-26T04:13:07.595598+0000 mon.c (mon.2) 818 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:13:07.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:07 smithi175 bash[29284]: audit 2024-03-26T04:13:07.595598+0000 mon.c (mon.2) 818 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:13:07.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:07 smithi190 bash[23392]: audit 2024-03-26T04:13:07.595598+0000 mon.c (mon.2) 818 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:13:08.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:08 smithi175 bash[21527]: cluster 2024-03-26T04:13:07.531082+0000 mgr.y (mgr.24431) 1074 : cluster [DBG] pgmap v1508: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:08.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:08 smithi175 bash[29284]: cluster 2024-03-26T04:13:07.531082+0000 mgr.y (mgr.24431) 1074 : cluster [DBG] pgmap v1508: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:08.992 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:13:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:13:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:13:08.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:08 smithi190 bash[23392]: cluster 2024-03-26T04:13:07.531082+0000 mgr.y (mgr.24431) 1074 : cluster [DBG] pgmap v1508: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:10.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:10 smithi175 bash[21527]: cluster 2024-03-26T04:13:09.531881+0000 mgr.y (mgr.24431) 1075 : cluster [DBG] pgmap v1509: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:10.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:10 smithi175 bash[29284]: cluster 2024-03-26T04:13:09.531881+0000 mgr.y (mgr.24431) 1075 : cluster [DBG] pgmap v1509: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:10.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:10 smithi190 bash[23392]: cluster 2024-03-26T04:13:09.531881+0000 mgr.y (mgr.24431) 1075 : cluster [DBG] pgmap v1509: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:11.498 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:13:11 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:13:12.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:12 smithi175 bash[29284]: audit 2024-03-26T04:13:11.145496+0000 mgr.y (mgr.24431) 1076 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:13:12.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:12 smithi175 bash[29284]: cluster 2024-03-26T04:13:11.534114+0000 mgr.y (mgr.24431) 1077 : cluster [DBG] pgmap v1510: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:12.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:12 smithi175 bash[21527]: audit 2024-03-26T04:13:11.145496+0000 mgr.y (mgr.24431) 1076 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:13:12.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:12 smithi175 bash[21527]: cluster 2024-03-26T04:13:11.534114+0000 mgr.y (mgr.24431) 1077 : cluster [DBG] pgmap v1510: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:12.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:12 smithi190 bash[23392]: audit 2024-03-26T04:13:11.145496+0000 mgr.y (mgr.24431) 1076 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:13:12.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:12 smithi190 bash[23392]: cluster 2024-03-26T04:13:11.534114+0000 mgr.y (mgr.24431) 1077 : cluster [DBG] pgmap v1510: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:14.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:14 smithi190 bash[23392]: cluster 2024-03-26T04:13:13.534959+0000 mgr.y (mgr.24431) 1078 : cluster [DBG] pgmap v1511: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:15.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:14 smithi175 bash[21527]: cluster 2024-03-26T04:13:13.534959+0000 mgr.y (mgr.24431) 1078 : cluster [DBG] pgmap v1511: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:15.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:14 smithi175 bash[29284]: cluster 2024-03-26T04:13:13.534959+0000 mgr.y (mgr.24431) 1078 : cluster [DBG] pgmap v1511: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:17.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:16 smithi175 bash[29284]: cluster 2024-03-26T04:13:15.537136+0000 mgr.y (mgr.24431) 1079 : cluster [DBG] pgmap v1512: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:17.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:16 smithi175 bash[21527]: cluster 2024-03-26T04:13:15.537136+0000 mgr.y (mgr.24431) 1079 : cluster [DBG] pgmap v1512: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:17.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:16 smithi190 bash[23392]: cluster 2024-03-26T04:13:15.537136+0000 mgr.y (mgr.24431) 1079 : cluster [DBG] pgmap v1512: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:19.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:13:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:13:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:13:19.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:18 smithi175 bash[21527]: cluster 2024-03-26T04:13:17.537947+0000 mgr.y (mgr.24431) 1080 : cluster [DBG] pgmap v1513: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:19.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:18 smithi175 bash[29284]: cluster 2024-03-26T04:13:17.537947+0000 mgr.y (mgr.24431) 1080 : cluster [DBG] pgmap v1513: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:19.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:18 smithi190 bash[23392]: cluster 2024-03-26T04:13:17.537947+0000 mgr.y (mgr.24431) 1080 : cluster [DBG] pgmap v1513: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:21.147 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:20 smithi190 bash[23392]: cluster 2024-03-26T04:13:19.538768+0000 mgr.y (mgr.24431) 1081 : cluster [DBG] pgmap v1514: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:21.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:20 smithi175 bash[21527]: cluster 2024-03-26T04:13:19.538768+0000 mgr.y (mgr.24431) 1081 : cluster [DBG] pgmap v1514: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:21.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:20 smithi175 bash[29284]: cluster 2024-03-26T04:13:19.538768+0000 mgr.y (mgr.24431) 1081 : cluster [DBG] pgmap v1514: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:21.498 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:13:21 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:13:22.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:21 smithi175 bash[29284]: audit 2024-03-26T04:13:21.150080+0000 mgr.y (mgr.24431) 1082 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:13:22.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:21 smithi175 bash[29284]: cluster 2024-03-26T04:13:21.541045+0000 mgr.y (mgr.24431) 1083 : cluster [DBG] pgmap v1515: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:22.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:21 smithi175 bash[21527]: audit 2024-03-26T04:13:21.150080+0000 mgr.y (mgr.24431) 1082 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:13:22.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:21 smithi175 bash[21527]: cluster 2024-03-26T04:13:21.541045+0000 mgr.y (mgr.24431) 1083 : cluster [DBG] pgmap v1515: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:22.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:21 smithi190 bash[23392]: audit 2024-03-26T04:13:21.150080+0000 mgr.y (mgr.24431) 1082 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:13:22.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:21 smithi190 bash[23392]: cluster 2024-03-26T04:13:21.541045+0000 mgr.y (mgr.24431) 1083 : cluster [DBG] pgmap v1515: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:23.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:22 smithi175 bash[21527]: audit 2024-03-26T04:13:22.629087+0000 mon.c (mon.2) 819 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:13:23.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:22 smithi175 bash[29284]: audit 2024-03-26T04:13:22.629087+0000 mon.c (mon.2) 819 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:13:23.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:22 smithi190 bash[23392]: audit 2024-03-26T04:13:22.629087+0000 mon.c (mon.2) 819 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:13:24.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:23 smithi175 bash[21527]: cluster 2024-03-26T04:13:23.541878+0000 mgr.y (mgr.24431) 1084 : cluster [DBG] pgmap v1516: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:24.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:23 smithi175 bash[29284]: cluster 2024-03-26T04:13:23.541878+0000 mgr.y (mgr.24431) 1084 : cluster [DBG] pgmap v1516: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:24.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:23 smithi190 bash[23392]: cluster 2024-03-26T04:13:23.541878+0000 mgr.y (mgr.24431) 1084 : cluster [DBG] pgmap v1516: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:26.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:26 smithi175 bash[21527]: cluster 2024-03-26T04:13:25.544131+0000 mgr.y (mgr.24431) 1085 : cluster [DBG] pgmap v1517: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:26.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:26 smithi175 bash[29284]: cluster 2024-03-26T04:13:25.544131+0000 mgr.y (mgr.24431) 1085 : cluster [DBG] pgmap v1517: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:26.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:26 smithi190 bash[23392]: cluster 2024-03-26T04:13:25.544131+0000 mgr.y (mgr.24431) 1085 : cluster [DBG] pgmap v1517: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:28.991 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:13:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:13:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:13:28.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:28 smithi175 bash[29284]: cluster 2024-03-26T04:13:27.544954+0000 mgr.y (mgr.24431) 1086 : cluster [DBG] pgmap v1518: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:28.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:28 smithi175 bash[21527]: cluster 2024-03-26T04:13:27.544954+0000 mgr.y (mgr.24431) 1086 : cluster [DBG] pgmap v1518: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:28.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:28 smithi190 bash[23392]: cluster 2024-03-26T04:13:27.544954+0000 mgr.y (mgr.24431) 1086 : cluster [DBG] pgmap v1518: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:30.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:30 smithi175 bash[29284]: cluster 2024-03-26T04:13:29.545791+0000 mgr.y (mgr.24431) 1087 : cluster [DBG] pgmap v1519: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:30.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:30 smithi175 bash[21527]: cluster 2024-03-26T04:13:29.545791+0000 mgr.y (mgr.24431) 1087 : cluster [DBG] pgmap v1519: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:30.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:30 smithi190 bash[23392]: cluster 2024-03-26T04:13:29.545791+0000 mgr.y (mgr.24431) 1087 : cluster [DBG] pgmap v1519: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:31.498 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:13:31 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:13:32.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:32 smithi175 bash[29284]: audit 2024-03-26T04:13:31.161422+0000 mgr.y (mgr.24431) 1088 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:13:32.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:32 smithi175 bash[29284]: cluster 2024-03-26T04:13:31.547983+0000 mgr.y (mgr.24431) 1089 : cluster [DBG] pgmap v1520: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:32.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:32 smithi175 bash[21527]: audit 2024-03-26T04:13:31.161422+0000 mgr.y (mgr.24431) 1088 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:13:32.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:32 smithi175 bash[21527]: cluster 2024-03-26T04:13:31.547983+0000 mgr.y (mgr.24431) 1089 : cluster [DBG] pgmap v1520: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:32.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:32 smithi190 bash[23392]: audit 2024-03-26T04:13:31.161422+0000 mgr.y (mgr.24431) 1088 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:13:32.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:32 smithi190 bash[23392]: cluster 2024-03-26T04:13:31.547983+0000 mgr.y (mgr.24431) 1089 : cluster [DBG] pgmap v1520: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:34.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:34 smithi175 bash[29284]: cluster 2024-03-26T04:13:33.549079+0000 mgr.y (mgr.24431) 1090 : cluster [DBG] pgmap v1521: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:34.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:34 smithi175 bash[29284]: audit 2024-03-26T04:13:33.913957+0000 mon.c (mon.2) 820 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:13:34.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:34 smithi175 bash[21527]: cluster 2024-03-26T04:13:33.549079+0000 mgr.y (mgr.24431) 1090 : cluster [DBG] pgmap v1521: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:34.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:34 smithi175 bash[21527]: audit 2024-03-26T04:13:33.913957+0000 mon.c (mon.2) 820 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:13:34.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:34 smithi190 bash[23392]: cluster 2024-03-26T04:13:33.549079+0000 mgr.y (mgr.24431) 1090 : cluster [DBG] pgmap v1521: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:34.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:34 smithi190 bash[23392]: audit 2024-03-26T04:13:33.913957+0000 mon.c (mon.2) 820 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:13:35.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:35 smithi175 bash[21527]: audit 2024-03-26T04:13:34.644586+0000 mon.c (mon.2) 821 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:13:35.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:35 smithi175 bash[21527]: audit 2024-03-26T04:13:34.646407+0000 mon.c (mon.2) 822 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:13:35.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:35 smithi175 bash[21527]: audit 2024-03-26T04:13:34.654175+0000 mon.a (mon.0) 3541 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:13:35.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:35 smithi175 bash[29284]: audit 2024-03-26T04:13:34.644586+0000 mon.c (mon.2) 821 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:13:35.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:35 smithi175 bash[29284]: audit 2024-03-26T04:13:34.646407+0000 mon.c (mon.2) 822 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:13:35.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:35 smithi175 bash[29284]: audit 2024-03-26T04:13:34.654175+0000 mon.a (mon.0) 3541 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:13:35.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:35 smithi190 bash[23392]: audit 2024-03-26T04:13:34.644586+0000 mon.c (mon.2) 821 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:13:35.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:35 smithi190 bash[23392]: audit 2024-03-26T04:13:34.646407+0000 mon.c (mon.2) 822 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:13:35.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:35 smithi190 bash[23392]: audit 2024-03-26T04:13:34.654175+0000 mon.a (mon.0) 3541 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:13:36.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:36 smithi175 bash[21527]: cluster 2024-03-26T04:13:35.551279+0000 mgr.y (mgr.24431) 1091 : cluster [DBG] pgmap v1522: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:36.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:36 smithi175 bash[29284]: cluster 2024-03-26T04:13:35.551279+0000 mgr.y (mgr.24431) 1091 : cluster [DBG] pgmap v1522: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:36.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:36 smithi190 bash[23392]: cluster 2024-03-26T04:13:35.551279+0000 mgr.y (mgr.24431) 1091 : cluster [DBG] pgmap v1522: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:38.991 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:13:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:13:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:13:38.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:38 smithi175 bash[29284]: cluster 2024-03-26T04:13:37.552053+0000 mgr.y (mgr.24431) 1092 : cluster [DBG] pgmap v1523: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:38.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:38 smithi175 bash[29284]: audit 2024-03-26T04:13:37.667945+0000 mon.c (mon.2) 823 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:13:38.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:38 smithi175 bash[21527]: cluster 2024-03-26T04:13:37.552053+0000 mgr.y (mgr.24431) 1092 : cluster [DBG] pgmap v1523: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:38.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:38 smithi175 bash[21527]: audit 2024-03-26T04:13:37.667945+0000 mon.c (mon.2) 823 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:13:38.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:38 smithi190 bash[23392]: cluster 2024-03-26T04:13:37.552053+0000 mgr.y (mgr.24431) 1092 : cluster [DBG] pgmap v1523: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:38.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:38 smithi190 bash[23392]: audit 2024-03-26T04:13:37.667945+0000 mon.c (mon.2) 823 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:13:40.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:40 smithi175 bash[21527]: cluster 2024-03-26T04:13:39.552878+0000 mgr.y (mgr.24431) 1093 : cluster [DBG] pgmap v1524: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:40.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:40 smithi175 bash[29284]: cluster 2024-03-26T04:13:39.552878+0000 mgr.y (mgr.24431) 1093 : cluster [DBG] pgmap v1524: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:40.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:40 smithi190 bash[23392]: cluster 2024-03-26T04:13:39.552878+0000 mgr.y (mgr.24431) 1093 : cluster [DBG] pgmap v1524: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:41.498 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:13:41 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:13:42.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:42 smithi175 bash[29284]: audit 2024-03-26T04:13:41.172948+0000 mgr.y (mgr.24431) 1094 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:13:42.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:42 smithi175 bash[29284]: cluster 2024-03-26T04:13:41.555100+0000 mgr.y (mgr.24431) 1095 : cluster [DBG] pgmap v1525: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:42.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:42 smithi175 bash[21527]: audit 2024-03-26T04:13:41.172948+0000 mgr.y (mgr.24431) 1094 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:13:42.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:42 smithi175 bash[21527]: cluster 2024-03-26T04:13:41.555100+0000 mgr.y (mgr.24431) 1095 : cluster [DBG] pgmap v1525: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:42.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:42 smithi190 bash[23392]: audit 2024-03-26T04:13:41.172948+0000 mgr.y (mgr.24431) 1094 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:13:42.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:42 smithi190 bash[23392]: cluster 2024-03-26T04:13:41.555100+0000 mgr.y (mgr.24431) 1095 : cluster [DBG] pgmap v1525: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:44.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:44 smithi175 bash[29284]: cluster 2024-03-26T04:13:43.556014+0000 mgr.y (mgr.24431) 1096 : cluster [DBG] pgmap v1526: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:44.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:44 smithi175 bash[21527]: cluster 2024-03-26T04:13:43.556014+0000 mgr.y (mgr.24431) 1096 : cluster [DBG] pgmap v1526: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:44.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:44 smithi190 bash[23392]: cluster 2024-03-26T04:13:43.556014+0000 mgr.y (mgr.24431) 1096 : cluster [DBG] pgmap v1526: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:46.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:46 smithi175 bash[29284]: cluster 2024-03-26T04:13:45.558225+0000 mgr.y (mgr.24431) 1097 : cluster [DBG] pgmap v1527: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:46.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:46 smithi175 bash[21527]: cluster 2024-03-26T04:13:45.558225+0000 mgr.y (mgr.24431) 1097 : cluster [DBG] pgmap v1527: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:46.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:46 smithi190 bash[23392]: cluster 2024-03-26T04:13:45.558225+0000 mgr.y (mgr.24431) 1097 : cluster [DBG] pgmap v1527: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:48.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:47 smithi175 bash[29284]: cluster 2024-03-26T04:13:47.559031+0000 mgr.y (mgr.24431) 1098 : cluster [DBG] pgmap v1528: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:48.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:47 smithi175 bash[21527]: cluster 2024-03-26T04:13:47.559031+0000 mgr.y (mgr.24431) 1098 : cluster [DBG] pgmap v1528: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:48.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:47 smithi190 bash[23392]: cluster 2024-03-26T04:13:47.559031+0000 mgr.y (mgr.24431) 1098 : cluster [DBG] pgmap v1528: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:49.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:13:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:13:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:13:50.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:50 smithi175 bash[21527]: cluster 2024-03-26T04:13:49.559951+0000 mgr.y (mgr.24431) 1099 : cluster [DBG] pgmap v1529: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:50.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:50 smithi175 bash[29284]: cluster 2024-03-26T04:13:49.559951+0000 mgr.y (mgr.24431) 1099 : cluster [DBG] pgmap v1529: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:50.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:50 smithi190 bash[23392]: cluster 2024-03-26T04:13:49.559951+0000 mgr.y (mgr.24431) 1099 : cluster [DBG] pgmap v1529: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:51.498 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:13:51 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:13:52.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:52 smithi175 bash[21527]: audit 2024-03-26T04:13:51.179824+0000 mgr.y (mgr.24431) 1100 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:13:52.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:52 smithi175 bash[21527]: cluster 2024-03-26T04:13:51.562248+0000 mgr.y (mgr.24431) 1101 : cluster [DBG] pgmap v1530: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:52.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:52 smithi175 bash[29284]: audit 2024-03-26T04:13:51.179824+0000 mgr.y (mgr.24431) 1100 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:13:52.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:52 smithi175 bash[29284]: cluster 2024-03-26T04:13:51.562248+0000 mgr.y (mgr.24431) 1101 : cluster [DBG] pgmap v1530: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:52.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:52 smithi190 bash[23392]: audit 2024-03-26T04:13:51.179824+0000 mgr.y (mgr.24431) 1100 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:13:52.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:52 smithi190 bash[23392]: cluster 2024-03-26T04:13:51.562248+0000 mgr.y (mgr.24431) 1101 : cluster [DBG] pgmap v1530: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:53.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:53 smithi175 bash[21527]: audit 2024-03-26T04:13:52.701935+0000 mon.c (mon.2) 824 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:13:53.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:53 smithi175 bash[29284]: audit 2024-03-26T04:13:52.701935+0000 mon.c (mon.2) 824 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:13:53.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:53 smithi190 bash[23392]: audit 2024-03-26T04:13:52.701935+0000 mon.c (mon.2) 824 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:13:54.993 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:54 smithi175 bash[29284]: cluster 2024-03-26T04:13:53.563067+0000 mgr.y (mgr.24431) 1102 : cluster [DBG] pgmap v1531: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:54.994 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:54 smithi175 bash[21527]: cluster 2024-03-26T04:13:53.563067+0000 mgr.y (mgr.24431) 1102 : cluster [DBG] pgmap v1531: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:54.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:54 smithi190 bash[23392]: cluster 2024-03-26T04:13:53.563067+0000 mgr.y (mgr.24431) 1102 : cluster [DBG] pgmap v1531: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:56.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:56 smithi175 bash[29284]: cluster 2024-03-26T04:13:55.565277+0000 mgr.y (mgr.24431) 1103 : cluster [DBG] pgmap v1532: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:56.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:56 smithi175 bash[21527]: cluster 2024-03-26T04:13:55.565277+0000 mgr.y (mgr.24431) 1103 : cluster [DBG] pgmap v1532: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:56.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:56 smithi190 bash[23392]: cluster 2024-03-26T04:13:55.565277+0000 mgr.y (mgr.24431) 1103 : cluster [DBG] pgmap v1532: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:13:58.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:13:58 smithi175 bash[21527]: cluster 2024-03-26T04:13:57.566086+0000 mgr.y (mgr.24431) 1104 : cluster [DBG] pgmap v1533: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:58.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:13:58 smithi175 bash[29284]: cluster 2024-03-26T04:13:57.566086+0000 mgr.y (mgr.24431) 1104 : cluster [DBG] pgmap v1533: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:13:58.991 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:13:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:13:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:13:58.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:13:58 smithi190 bash[23392]: cluster 2024-03-26T04:13:57.566086+0000 mgr.y (mgr.24431) 1104 : cluster [DBG] pgmap v1533: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:00.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:00 smithi175 bash[21527]: cluster 2024-03-26T04:13:59.566948+0000 mgr.y (mgr.24431) 1105 : cluster [DBG] pgmap v1534: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:00.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:00 smithi175 bash[29284]: cluster 2024-03-26T04:13:59.566948+0000 mgr.y (mgr.24431) 1105 : cluster [DBG] pgmap v1534: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:00.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:00 smithi190 bash[23392]: cluster 2024-03-26T04:13:59.566948+0000 mgr.y (mgr.24431) 1105 : cluster [DBG] pgmap v1534: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:01.498 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:14:01 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:14:02.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:02 smithi175 bash[29284]: audit 2024-03-26T04:14:01.188933+0000 mgr.y (mgr.24431) 1106 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:14:02.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:02 smithi175 bash[29284]: cluster 2024-03-26T04:14:01.569253+0000 mgr.y (mgr.24431) 1107 : cluster [DBG] pgmap v1535: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:14:02.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:02 smithi175 bash[21527]: audit 2024-03-26T04:14:01.188933+0000 mgr.y (mgr.24431) 1106 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:14:02.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:02 smithi175 bash[21527]: cluster 2024-03-26T04:14:01.569253+0000 mgr.y (mgr.24431) 1107 : cluster [DBG] pgmap v1535: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:14:02.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:02 smithi190 bash[23392]: audit 2024-03-26T04:14:01.188933+0000 mgr.y (mgr.24431) 1106 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:14:02.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:02 smithi190 bash[23392]: cluster 2024-03-26T04:14:01.569253+0000 mgr.y (mgr.24431) 1107 : cluster [DBG] pgmap v1535: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:14:04.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:04 smithi175 bash[21527]: cluster 2024-03-26T04:14:03.570138+0000 mgr.y (mgr.24431) 1108 : cluster [DBG] pgmap v1536: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:04.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:04 smithi175 bash[29284]: cluster 2024-03-26T04:14:03.570138+0000 mgr.y (mgr.24431) 1108 : cluster [DBG] pgmap v1536: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:04.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:04 smithi190 bash[23392]: cluster 2024-03-26T04:14:03.570138+0000 mgr.y (mgr.24431) 1108 : cluster [DBG] pgmap v1536: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:06.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:06 smithi175 bash[21527]: cluster 2024-03-26T04:14:05.572554+0000 mgr.y (mgr.24431) 1109 : cluster [DBG] pgmap v1537: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:14:06.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:06 smithi175 bash[29284]: cluster 2024-03-26T04:14:05.572554+0000 mgr.y (mgr.24431) 1109 : cluster [DBG] pgmap v1537: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:14:06.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:06 smithi190 bash[23392]: cluster 2024-03-26T04:14:05.572554+0000 mgr.y (mgr.24431) 1109 : cluster [DBG] pgmap v1537: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:14:08.991 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:14:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:14:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:14:08.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:08 smithi175 bash[21527]: cluster 2024-03-26T04:14:07.573437+0000 mgr.y (mgr.24431) 1110 : cluster [DBG] pgmap v1538: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:08.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:08 smithi175 bash[21527]: audit 2024-03-26T04:14:07.735524+0000 mon.c (mon.2) 825 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:14:08.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:08 smithi175 bash[29284]: cluster 2024-03-26T04:14:07.573437+0000 mgr.y (mgr.24431) 1110 : cluster [DBG] pgmap v1538: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:08.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:08 smithi175 bash[29284]: audit 2024-03-26T04:14:07.735524+0000 mon.c (mon.2) 825 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:14:08.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:08 smithi190 bash[23392]: cluster 2024-03-26T04:14:07.573437+0000 mgr.y (mgr.24431) 1110 : cluster [DBG] pgmap v1538: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:08.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:08 smithi190 bash[23392]: audit 2024-03-26T04:14:07.735524+0000 mon.c (mon.2) 825 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:14:10.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:10 smithi175 bash[21527]: cluster 2024-03-26T04:14:09.574237+0000 mgr.y (mgr.24431) 1111 : cluster [DBG] pgmap v1539: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:10.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:10 smithi175 bash[29284]: cluster 2024-03-26T04:14:09.574237+0000 mgr.y (mgr.24431) 1111 : cluster [DBG] pgmap v1539: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:10.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:10 smithi190 bash[23392]: cluster 2024-03-26T04:14:09.574237+0000 mgr.y (mgr.24431) 1111 : cluster [DBG] pgmap v1539: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:11.499 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:14:11 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:14:12.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:12 smithi175 bash[29284]: audit 2024-03-26T04:14:11.198079+0000 mgr.y (mgr.24431) 1112 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:14:12.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:12 smithi175 bash[29284]: cluster 2024-03-26T04:14:11.576520+0000 mgr.y (mgr.24431) 1113 : cluster [DBG] pgmap v1540: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:14:12.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:12 smithi175 bash[21527]: audit 2024-03-26T04:14:11.198079+0000 mgr.y (mgr.24431) 1112 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:14:12.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:12 smithi175 bash[21527]: cluster 2024-03-26T04:14:11.576520+0000 mgr.y (mgr.24431) 1113 : cluster [DBG] pgmap v1540: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:14:12.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:12 smithi190 bash[23392]: audit 2024-03-26T04:14:11.198079+0000 mgr.y (mgr.24431) 1112 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:14:12.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:12 smithi190 bash[23392]: cluster 2024-03-26T04:14:11.576520+0000 mgr.y (mgr.24431) 1113 : cluster [DBG] pgmap v1540: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:14:14.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:14 smithi175 bash[29284]: cluster 2024-03-26T04:14:13.577335+0000 mgr.y (mgr.24431) 1114 : cluster [DBG] pgmap v1541: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:14.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:14 smithi175 bash[21527]: cluster 2024-03-26T04:14:13.577335+0000 mgr.y (mgr.24431) 1114 : cluster [DBG] pgmap v1541: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:14.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:14 smithi190 bash[23392]: cluster 2024-03-26T04:14:13.577335+0000 mgr.y (mgr.24431) 1114 : cluster [DBG] pgmap v1541: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:16.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:16 smithi175 bash[29284]: cluster 2024-03-26T04:14:15.579540+0000 mgr.y (mgr.24431) 1115 : cluster [DBG] pgmap v1542: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:14:16.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:16 smithi175 bash[21527]: cluster 2024-03-26T04:14:15.579540+0000 mgr.y (mgr.24431) 1115 : cluster [DBG] pgmap v1542: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:14:16.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:16 smithi190 bash[23392]: cluster 2024-03-26T04:14:15.579540+0000 mgr.y (mgr.24431) 1115 : cluster [DBG] pgmap v1542: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:14:18.991 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:14:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:14:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:14:18.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:18 smithi175 bash[29284]: cluster 2024-03-26T04:14:17.580334+0000 mgr.y (mgr.24431) 1116 : cluster [DBG] pgmap v1543: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:18.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:18 smithi175 bash[21527]: cluster 2024-03-26T04:14:17.580334+0000 mgr.y (mgr.24431) 1116 : cluster [DBG] pgmap v1543: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:18.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:18 smithi190 bash[23392]: cluster 2024-03-26T04:14:17.580334+0000 mgr.y (mgr.24431) 1116 : cluster [DBG] pgmap v1543: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:20.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:20 smithi175 bash[21527]: cluster 2024-03-26T04:14:19.581187+0000 mgr.y (mgr.24431) 1117 : cluster [DBG] pgmap v1544: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:20.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:20 smithi175 bash[29284]: cluster 2024-03-26T04:14:19.581187+0000 mgr.y (mgr.24431) 1117 : cluster [DBG] pgmap v1544: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:20.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:20 smithi190 bash[23392]: cluster 2024-03-26T04:14:19.581187+0000 mgr.y (mgr.24431) 1117 : cluster [DBG] pgmap v1544: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:21.498 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:14:21 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:14:22.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:22 smithi175 bash[21527]: audit 2024-03-26T04:14:21.209383+0000 mgr.y (mgr.24431) 1118 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:14:22.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:22 smithi175 bash[21527]: cluster 2024-03-26T04:14:21.583524+0000 mgr.y (mgr.24431) 1119 : cluster [DBG] pgmap v1545: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:14:22.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:22 smithi175 bash[29284]: audit 2024-03-26T04:14:21.209383+0000 mgr.y (mgr.24431) 1118 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:14:22.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:22 smithi175 bash[29284]: cluster 2024-03-26T04:14:21.583524+0000 mgr.y (mgr.24431) 1119 : cluster [DBG] pgmap v1545: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:14:22.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:22 smithi190 bash[23392]: audit 2024-03-26T04:14:21.209383+0000 mgr.y (mgr.24431) 1118 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:14:22.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:22 smithi190 bash[23392]: cluster 2024-03-26T04:14:21.583524+0000 mgr.y (mgr.24431) 1119 : cluster [DBG] pgmap v1545: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:14:23.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:23 smithi175 bash[29284]: audit 2024-03-26T04:14:22.770437+0000 mon.c (mon.2) 826 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:14:23.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:23 smithi175 bash[21527]: audit 2024-03-26T04:14:22.770437+0000 mon.c (mon.2) 826 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:14:23.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:23 smithi190 bash[23392]: audit 2024-03-26T04:14:22.770437+0000 mon.c (mon.2) 826 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:14:24.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:24 smithi175 bash[21527]: cluster 2024-03-26T04:14:23.584308+0000 mgr.y (mgr.24431) 1120 : cluster [DBG] pgmap v1546: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:24.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:24 smithi175 bash[29284]: cluster 2024-03-26T04:14:23.584308+0000 mgr.y (mgr.24431) 1120 : cluster [DBG] pgmap v1546: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:24.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:24 smithi190 bash[23392]: cluster 2024-03-26T04:14:23.584308+0000 mgr.y (mgr.24431) 1120 : cluster [DBG] pgmap v1546: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:26.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:26 smithi175 bash[21527]: cluster 2024-03-26T04:14:25.586599+0000 mgr.y (mgr.24431) 1121 : cluster [DBG] pgmap v1547: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-26T04:14:26.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:26 smithi175 bash[29284]: cluster 2024-03-26T04:14:25.586599+0000 mgr.y (mgr.24431) 1121 : cluster [DBG] pgmap v1547: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-26T04:14:26.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:26 smithi190 bash[23392]: cluster 2024-03-26T04:14:25.586599+0000 mgr.y (mgr.24431) 1121 : cluster [DBG] pgmap v1547: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-26T04:14:28.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:28 smithi175 bash[21527]: cluster 2024-03-26T04:14:27.587511+0000 mgr.y (mgr.24431) 1122 : cluster [DBG] pgmap v1548: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-26T04:14:28.991 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:14:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:14:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:14:28.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:28 smithi175 bash[29284]: cluster 2024-03-26T04:14:27.587511+0000 mgr.y (mgr.24431) 1122 : cluster [DBG] pgmap v1548: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-26T04:14:28.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:28 smithi190 bash[23392]: cluster 2024-03-26T04:14:27.587511+0000 mgr.y (mgr.24431) 1122 : cluster [DBG] pgmap v1548: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-26T04:14:30.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:30 smithi175 bash[21527]: cluster 2024-03-26T04:14:29.588559+0000 mgr.y (mgr.24431) 1123 : cluster [DBG] pgmap v1549: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-26T04:14:30.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:30 smithi175 bash[29284]: cluster 2024-03-26T04:14:29.588559+0000 mgr.y (mgr.24431) 1123 : cluster [DBG] pgmap v1549: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-26T04:14:30.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:30 smithi190 bash[23392]: cluster 2024-03-26T04:14:29.588559+0000 mgr.y (mgr.24431) 1123 : cluster [DBG] pgmap v1549: 228 pgs: 228 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-26T04:14:31.498 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:14:31 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:14:32.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:32 smithi175 bash[21527]: audit 2024-03-26T04:14:31.212434+0000 mgr.y (mgr.24431) 1124 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:14:32.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:32 smithi175 bash[21527]: cluster 2024-03-26T04:14:31.591503+0000 mgr.y (mgr.24431) 1125 : cluster [DBG] pgmap v1550: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-26T04:14:32.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:32 smithi175 bash[29284]: audit 2024-03-26T04:14:31.212434+0000 mgr.y (mgr.24431) 1124 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:14:32.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:32 smithi175 bash[29284]: cluster 2024-03-26T04:14:31.591503+0000 mgr.y (mgr.24431) 1125 : cluster [DBG] pgmap v1550: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-26T04:14:32.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:32 smithi190 bash[23392]: audit 2024-03-26T04:14:31.212434+0000 mgr.y (mgr.24431) 1124 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:14:32.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:32 smithi190 bash[23392]: cluster 2024-03-26T04:14:31.591503+0000 mgr.y (mgr.24431) 1125 : cluster [DBG] pgmap v1550: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-26T04:14:34.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:34 smithi175 bash[21527]: cluster 2024-03-26T04:14:33.592310+0000 mgr.y (mgr.24431) 1126 : cluster [DBG] pgmap v1551: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-26T04:14:34.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:34 smithi175 bash[29284]: cluster 2024-03-26T04:14:33.592310+0000 mgr.y (mgr.24431) 1126 : cluster [DBG] pgmap v1551: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-26T04:14:34.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:34 smithi190 bash[23392]: cluster 2024-03-26T04:14:33.592310+0000 mgr.y (mgr.24431) 1126 : cluster [DBG] pgmap v1551: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-26T04:14:35.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:35 smithi175 bash[21527]: audit 2024-03-26T04:14:34.738025+0000 mon.c (mon.2) 827 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:14:35.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:35 smithi175 bash[29284]: audit 2024-03-26T04:14:34.738025+0000 mon.c (mon.2) 827 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:14:35.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:35 smithi190 bash[23392]: audit 2024-03-26T04:14:34.738025+0000 mon.c (mon.2) 827 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:14:36.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:36 smithi175 bash[21527]: cluster 2024-03-26T04:14:35.594564+0000 mgr.y (mgr.24431) 1127 : cluster [DBG] pgmap v1552: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-26T04:14:36.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:36 smithi175 bash[29284]: cluster 2024-03-26T04:14:35.594564+0000 mgr.y (mgr.24431) 1127 : cluster [DBG] pgmap v1552: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-26T04:14:36.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:36 smithi190 bash[23392]: cluster 2024-03-26T04:14:35.594564+0000 mgr.y (mgr.24431) 1127 : cluster [DBG] pgmap v1552: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-26T04:14:38.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:38 smithi190 bash[23392]: cluster 2024-03-26T04:14:37.595466+0000 mgr.y (mgr.24431) 1128 : cluster [DBG] pgmap v1553: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-03-26T04:14:38.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:38 smithi190 bash[23392]: audit 2024-03-26T04:14:37.804659+0000 mon.c (mon.2) 828 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:14:39.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:38 smithi175 bash[21527]: cluster 2024-03-26T04:14:37.595466+0000 mgr.y (mgr.24431) 1128 : cluster [DBG] pgmap v1553: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-03-26T04:14:39.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:38 smithi175 bash[21527]: audit 2024-03-26T04:14:37.804659+0000 mon.c (mon.2) 828 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:14:39.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:14:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:14:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:14:39.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:38 smithi175 bash[29284]: cluster 2024-03-26T04:14:37.595466+0000 mgr.y (mgr.24431) 1128 : cluster [DBG] pgmap v1553: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-03-26T04:14:39.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:38 smithi175 bash[29284]: audit 2024-03-26T04:14:37.804659+0000 mon.c (mon.2) 828 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:14:40.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:40 smithi190 bash[23392]: cluster 2024-03-26T04:14:39.596272+0000 mgr.y (mgr.24431) 1129 : cluster [DBG] pgmap v1554: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-03-26T04:14:41.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:40 smithi175 bash[29284]: cluster 2024-03-26T04:14:39.596272+0000 mgr.y (mgr.24431) 1129 : cluster [DBG] pgmap v1554: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-03-26T04:14:41.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:40 smithi175 bash[21527]: cluster 2024-03-26T04:14:39.596272+0000 mgr.y (mgr.24431) 1129 : cluster [DBG] pgmap v1554: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-03-26T04:14:41.498 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:14:41 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:14:42.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:42 smithi190 bash[23392]: audit 2024-03-26T04:14:41.222129+0000 mgr.y (mgr.24431) 1130 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:14:42.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:42 smithi190 bash[23392]: cluster 2024-03-26T04:14:41.598634+0000 mgr.y (mgr.24431) 1131 : cluster [DBG] pgmap v1555: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 54 op/s 2024-03-26T04:14:43.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:42 smithi175 bash[29284]: audit 2024-03-26T04:14:41.222129+0000 mgr.y (mgr.24431) 1130 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:14:43.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:42 smithi175 bash[29284]: cluster 2024-03-26T04:14:41.598634+0000 mgr.y (mgr.24431) 1131 : cluster [DBG] pgmap v1555: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 54 op/s 2024-03-26T04:14:43.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:42 smithi175 bash[21527]: audit 2024-03-26T04:14:41.222129+0000 mgr.y (mgr.24431) 1130 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:14:43.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:42 smithi175 bash[21527]: cluster 2024-03-26T04:14:41.598634+0000 mgr.y (mgr.24431) 1131 : cluster [DBG] pgmap v1555: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 54 op/s 2024-03-26T04:14:44.906 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:44 smithi190 bash[23392]: cluster 2024-03-26T04:14:43.599565+0000 mgr.y (mgr.24431) 1132 : cluster [DBG] pgmap v1556: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:45.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:44 smithi175 bash[21527]: cluster 2024-03-26T04:14:43.599565+0000 mgr.y (mgr.24431) 1132 : cluster [DBG] pgmap v1556: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:45.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:44 smithi175 bash[29284]: cluster 2024-03-26T04:14:43.599565+0000 mgr.y (mgr.24431) 1132 : cluster [DBG] pgmap v1556: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:46.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:46 smithi175 bash[21527]: cluster 2024-03-26T04:14:45.601894+0000 mgr.y (mgr.24431) 1133 : cluster [DBG] pgmap v1557: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:14:46.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:46 smithi175 bash[21527]: audit 2024-03-26T04:14:45.616233+0000 mon.a (mon.0) 3542 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:14:46.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:46 smithi175 bash[21527]: audit 2024-03-26T04:14:45.633188+0000 mon.a (mon.0) 3543 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:14:46.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:46 smithi175 bash[21527]: audit 2024-03-26T04:14:45.637833+0000 mon.c (mon.2) 829 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:14:46.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:46 smithi175 bash[21527]: audit 2024-03-26T04:14:45.639810+0000 mon.c (mon.2) 830 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:14:46.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:46 smithi175 bash[21527]: audit 2024-03-26T04:14:45.651153+0000 mon.a (mon.0) 3544 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:14:46.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:46 smithi175 bash[29284]: cluster 2024-03-26T04:14:45.601894+0000 mgr.y (mgr.24431) 1133 : cluster [DBG] pgmap v1557: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:14:46.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:46 smithi175 bash[29284]: audit 2024-03-26T04:14:45.616233+0000 mon.a (mon.0) 3542 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:14:46.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:46 smithi175 bash[29284]: audit 2024-03-26T04:14:45.633188+0000 mon.a (mon.0) 3543 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:14:46.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:46 smithi175 bash[29284]: audit 2024-03-26T04:14:45.637833+0000 mon.c (mon.2) 829 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:14:46.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:46 smithi175 bash[29284]: audit 2024-03-26T04:14:45.639810+0000 mon.c (mon.2) 830 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:14:46.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:46 smithi175 bash[29284]: audit 2024-03-26T04:14:45.651153+0000 mon.a (mon.0) 3544 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:14:46.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:46 smithi190 bash[23392]: cluster 2024-03-26T04:14:45.601894+0000 mgr.y (mgr.24431) 1133 : cluster [DBG] pgmap v1557: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:14:46.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:46 smithi190 bash[23392]: audit 2024-03-26T04:14:45.616233+0000 mon.a (mon.0) 3542 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:14:46.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:46 smithi190 bash[23392]: audit 2024-03-26T04:14:45.633188+0000 mon.a (mon.0) 3543 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:14:46.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:46 smithi190 bash[23392]: audit 2024-03-26T04:14:45.637833+0000 mon.c (mon.2) 829 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:14:46.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:46 smithi190 bash[23392]: audit 2024-03-26T04:14:45.639810+0000 mon.c (mon.2) 830 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:14:46.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:46 smithi190 bash[23392]: audit 2024-03-26T04:14:45.651153+0000 mon.a (mon.0) 3544 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:14:48.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:48 smithi175 bash[21527]: cluster 2024-03-26T04:14:47.602823+0000 mgr.y (mgr.24431) 1134 : cluster [DBG] pgmap v1558: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:48.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:48 smithi175 bash[29284]: cluster 2024-03-26T04:14:47.602823+0000 mgr.y (mgr.24431) 1134 : cluster [DBG] pgmap v1558: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:48.991 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:14:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:14:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:14:48.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:48 smithi190 bash[23392]: cluster 2024-03-26T04:14:47.602823+0000 mgr.y (mgr.24431) 1134 : cluster [DBG] pgmap v1558: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:50.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:50 smithi175 bash[21527]: cluster 2024-03-26T04:14:49.603618+0000 mgr.y (mgr.24431) 1135 : cluster [DBG] pgmap v1559: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:50.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:50 smithi175 bash[29284]: cluster 2024-03-26T04:14:49.603618+0000 mgr.y (mgr.24431) 1135 : cluster [DBG] pgmap v1559: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:50.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:50 smithi190 bash[23392]: cluster 2024-03-26T04:14:49.603618+0000 mgr.y (mgr.24431) 1135 : cluster [DBG] pgmap v1559: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:51.498 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:14:51 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:14:52.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:52 smithi175 bash[21527]: audit 2024-03-26T04:14:51.233548+0000 mgr.y (mgr.24431) 1136 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:14:52.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:52 smithi175 bash[21527]: cluster 2024-03-26T04:14:51.605777+0000 mgr.y (mgr.24431) 1137 : cluster [DBG] pgmap v1560: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:14:52.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:52 smithi175 bash[29284]: audit 2024-03-26T04:14:51.233548+0000 mgr.y (mgr.24431) 1136 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:14:52.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:52 smithi175 bash[29284]: cluster 2024-03-26T04:14:51.605777+0000 mgr.y (mgr.24431) 1137 : cluster [DBG] pgmap v1560: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:14:52.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:52 smithi190 bash[23392]: audit 2024-03-26T04:14:51.233548+0000 mgr.y (mgr.24431) 1136 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:14:52.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:52 smithi190 bash[23392]: cluster 2024-03-26T04:14:51.605777+0000 mgr.y (mgr.24431) 1137 : cluster [DBG] pgmap v1560: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:14:53.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:53 smithi175 bash[29284]: audit 2024-03-26T04:14:52.838424+0000 mon.c (mon.2) 831 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:14:53.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:53 smithi175 bash[21527]: audit 2024-03-26T04:14:52.838424+0000 mon.c (mon.2) 831 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:14:53.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:53 smithi190 bash[23392]: audit 2024-03-26T04:14:52.838424+0000 mon.c (mon.2) 831 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:14:54.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:54 smithi175 bash[21527]: cluster 2024-03-26T04:14:53.606575+0000 mgr.y (mgr.24431) 1138 : cluster [DBG] pgmap v1561: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:54.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:54 smithi175 bash[29284]: cluster 2024-03-26T04:14:53.606575+0000 mgr.y (mgr.24431) 1138 : cluster [DBG] pgmap v1561: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:54.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:54 smithi190 bash[23392]: cluster 2024-03-26T04:14:53.606575+0000 mgr.y (mgr.24431) 1138 : cluster [DBG] pgmap v1561: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:56.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:56 smithi175 bash[21527]: cluster 2024-03-26T04:14:55.608776+0000 mgr.y (mgr.24431) 1139 : cluster [DBG] pgmap v1562: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:14:56.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:56 smithi175 bash[29284]: cluster 2024-03-26T04:14:55.608776+0000 mgr.y (mgr.24431) 1139 : cluster [DBG] pgmap v1562: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:14:56.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:56 smithi190 bash[23392]: cluster 2024-03-26T04:14:55.608776+0000 mgr.y (mgr.24431) 1139 : cluster [DBG] pgmap v1562: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:14:58.990 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:14:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:14:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:14:58.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:14:58 smithi175 bash[29284]: cluster 2024-03-26T04:14:57.609612+0000 mgr.y (mgr.24431) 1140 : cluster [DBG] pgmap v1563: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:58.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:14:58 smithi175 bash[21527]: cluster 2024-03-26T04:14:57.609612+0000 mgr.y (mgr.24431) 1140 : cluster [DBG] pgmap v1563: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:14:58.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:14:58 smithi190 bash[23392]: cluster 2024-03-26T04:14:57.609612+0000 mgr.y (mgr.24431) 1140 : cluster [DBG] pgmap v1563: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:00.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:00 smithi175 bash[21527]: cluster 2024-03-26T04:14:59.610441+0000 mgr.y (mgr.24431) 1141 : cluster [DBG] pgmap v1564: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:00.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:00 smithi175 bash[29284]: cluster 2024-03-26T04:14:59.610441+0000 mgr.y (mgr.24431) 1141 : cluster [DBG] pgmap v1564: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:00.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:00 smithi190 bash[23392]: cluster 2024-03-26T04:14:59.610441+0000 mgr.y (mgr.24431) 1141 : cluster [DBG] pgmap v1564: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:01.496 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:15:01 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:15:02.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:02 smithi175 bash[21527]: audit 2024-03-26T04:15:01.245055+0000 mgr.y (mgr.24431) 1142 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:15:02.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:02 smithi175 bash[21527]: cluster 2024-03-26T04:15:01.612704+0000 mgr.y (mgr.24431) 1143 : cluster [DBG] pgmap v1565: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:02.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:02 smithi175 bash[29284]: audit 2024-03-26T04:15:01.245055+0000 mgr.y (mgr.24431) 1142 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:15:02.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:02 smithi175 bash[29284]: cluster 2024-03-26T04:15:01.612704+0000 mgr.y (mgr.24431) 1143 : cluster [DBG] pgmap v1565: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:02.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:02 smithi190 bash[23392]: audit 2024-03-26T04:15:01.245055+0000 mgr.y (mgr.24431) 1142 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:15:02.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:02 smithi190 bash[23392]: cluster 2024-03-26T04:15:01.612704+0000 mgr.y (mgr.24431) 1143 : cluster [DBG] pgmap v1565: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:04.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:04 smithi175 bash[21527]: cluster 2024-03-26T04:15:03.613498+0000 mgr.y (mgr.24431) 1144 : cluster [DBG] pgmap v1566: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:04.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:04 smithi175 bash[29284]: cluster 2024-03-26T04:15:03.613498+0000 mgr.y (mgr.24431) 1144 : cluster [DBG] pgmap v1566: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:04.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:04 smithi190 bash[23392]: cluster 2024-03-26T04:15:03.613498+0000 mgr.y (mgr.24431) 1144 : cluster [DBG] pgmap v1566: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:06.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:06 smithi175 bash[21527]: cluster 2024-03-26T04:15:05.615746+0000 mgr.y (mgr.24431) 1145 : cluster [DBG] pgmap v1567: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:06.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:06 smithi175 bash[29284]: cluster 2024-03-26T04:15:05.615746+0000 mgr.y (mgr.24431) 1145 : cluster [DBG] pgmap v1567: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:06.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:06 smithi190 bash[23392]: cluster 2024-03-26T04:15:05.615746+0000 mgr.y (mgr.24431) 1145 : cluster [DBG] pgmap v1567: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:08.991 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:15:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:15:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:15:08.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:08 smithi175 bash[21527]: cluster 2024-03-26T04:15:07.616530+0000 mgr.y (mgr.24431) 1146 : cluster [DBG] pgmap v1568: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:08.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:08 smithi175 bash[21527]: audit 2024-03-26T04:15:07.872309+0000 mon.c (mon.2) 832 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:15:08.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:08 smithi175 bash[29284]: cluster 2024-03-26T04:15:07.616530+0000 mgr.y (mgr.24431) 1146 : cluster [DBG] pgmap v1568: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:08.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:08 smithi175 bash[29284]: audit 2024-03-26T04:15:07.872309+0000 mon.c (mon.2) 832 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:15:08.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:08 smithi190 bash[23392]: cluster 2024-03-26T04:15:07.616530+0000 mgr.y (mgr.24431) 1146 : cluster [DBG] pgmap v1568: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:08.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:08 smithi190 bash[23392]: audit 2024-03-26T04:15:07.872309+0000 mon.c (mon.2) 832 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:15:10.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:10 smithi175 bash[21527]: cluster 2024-03-26T04:15:09.617530+0000 mgr.y (mgr.24431) 1147 : cluster [DBG] pgmap v1569: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:10.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:10 smithi175 bash[29284]: cluster 2024-03-26T04:15:09.617530+0000 mgr.y (mgr.24431) 1147 : cluster [DBG] pgmap v1569: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:10.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:10 smithi190 bash[23392]: cluster 2024-03-26T04:15:09.617530+0000 mgr.y (mgr.24431) 1147 : cluster [DBG] pgmap v1569: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:11.748 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:15:11 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:15:12.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:12 smithi175 bash[29284]: audit 2024-03-26T04:15:11.254249+0000 mgr.y (mgr.24431) 1148 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:15:12.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:12 smithi175 bash[29284]: cluster 2024-03-26T04:15:11.620011+0000 mgr.y (mgr.24431) 1149 : cluster [DBG] pgmap v1570: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:12.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:12 smithi175 bash[21527]: audit 2024-03-26T04:15:11.254249+0000 mgr.y (mgr.24431) 1148 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:15:12.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:12 smithi175 bash[21527]: cluster 2024-03-26T04:15:11.620011+0000 mgr.y (mgr.24431) 1149 : cluster [DBG] pgmap v1570: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:12.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:12 smithi190 bash[23392]: audit 2024-03-26T04:15:11.254249+0000 mgr.y (mgr.24431) 1148 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:15:12.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:12 smithi190 bash[23392]: cluster 2024-03-26T04:15:11.620011+0000 mgr.y (mgr.24431) 1149 : cluster [DBG] pgmap v1570: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:14.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:14 smithi175 bash[21527]: cluster 2024-03-26T04:15:13.621015+0000 mgr.y (mgr.24431) 1150 : cluster [DBG] pgmap v1571: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:14.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:14 smithi175 bash[29284]: cluster 2024-03-26T04:15:13.621015+0000 mgr.y (mgr.24431) 1150 : cluster [DBG] pgmap v1571: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:14.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:14 smithi190 bash[23392]: cluster 2024-03-26T04:15:13.621015+0000 mgr.y (mgr.24431) 1150 : cluster [DBG] pgmap v1571: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:16.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:16 smithi175 bash[29284]: cluster 2024-03-26T04:15:15.623374+0000 mgr.y (mgr.24431) 1151 : cluster [DBG] pgmap v1572: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:16.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:16 smithi175 bash[21527]: cluster 2024-03-26T04:15:15.623374+0000 mgr.y (mgr.24431) 1151 : cluster [DBG] pgmap v1572: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:16.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:16 smithi190 bash[23392]: cluster 2024-03-26T04:15:15.623374+0000 mgr.y (mgr.24431) 1151 : cluster [DBG] pgmap v1572: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:18.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:18 smithi175 bash[21527]: cluster 2024-03-26T04:15:17.624342+0000 mgr.y (mgr.24431) 1152 : cluster [DBG] pgmap v1573: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:18.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:18 smithi175 bash[29284]: cluster 2024-03-26T04:15:17.624342+0000 mgr.y (mgr.24431) 1152 : cluster [DBG] pgmap v1573: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:18.991 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:15:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:15:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:15:18.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:18 smithi190 bash[23392]: cluster 2024-03-26T04:15:17.624342+0000 mgr.y (mgr.24431) 1152 : cluster [DBG] pgmap v1573: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:20.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:20 smithi175 bash[29284]: cluster 2024-03-26T04:15:19.626561+0000 mgr.y (mgr.24431) 1153 : cluster [DBG] pgmap v1574: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:20.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:20 smithi175 bash[21527]: cluster 2024-03-26T04:15:19.626561+0000 mgr.y (mgr.24431) 1153 : cluster [DBG] pgmap v1574: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:20.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:20 smithi190 bash[23392]: cluster 2024-03-26T04:15:19.626561+0000 mgr.y (mgr.24431) 1153 : cluster [DBG] pgmap v1574: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:21.748 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:15:21 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:15:22.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:22 smithi175 bash[29284]: audit 2024-03-26T04:15:21.265774+0000 mgr.y (mgr.24431) 1154 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:15:22.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:22 smithi175 bash[29284]: cluster 2024-03-26T04:15:21.627376+0000 mgr.y (mgr.24431) 1155 : cluster [DBG] pgmap v1575: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:22.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:22 smithi175 bash[21527]: audit 2024-03-26T04:15:21.265774+0000 mgr.y (mgr.24431) 1154 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:15:22.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:22 smithi175 bash[21527]: cluster 2024-03-26T04:15:21.627376+0000 mgr.y (mgr.24431) 1155 : cluster [DBG] pgmap v1575: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:22.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:22 smithi190 bash[23392]: audit 2024-03-26T04:15:21.265774+0000 mgr.y (mgr.24431) 1154 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:15:22.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:22 smithi190 bash[23392]: cluster 2024-03-26T04:15:21.627376+0000 mgr.y (mgr.24431) 1155 : cluster [DBG] pgmap v1575: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:23.993 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:23 smithi175 bash[21527]: audit 2024-03-26T04:15:22.905719+0000 mon.c (mon.2) 833 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:15:23.994 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:23 smithi175 bash[29284]: audit 2024-03-26T04:15:22.905719+0000 mon.c (mon.2) 833 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:15:23.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:23 smithi190 bash[23392]: audit 2024-03-26T04:15:22.905719+0000 mon.c (mon.2) 833 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:15:24.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:24 smithi175 bash[21527]: cluster 2024-03-26T04:15:23.628225+0000 mgr.y (mgr.24431) 1156 : cluster [DBG] pgmap v1576: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T04:15:24.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:24 smithi175 bash[29284]: cluster 2024-03-26T04:15:23.628225+0000 mgr.y (mgr.24431) 1156 : cluster [DBG] pgmap v1576: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T04:15:24.999 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:24 smithi190 bash[23392]: cluster 2024-03-26T04:15:23.628225+0000 mgr.y (mgr.24431) 1156 : cluster [DBG] pgmap v1576: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T04:15:26.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:26 smithi175 bash[21527]: cluster 2024-03-26T04:15:25.630428+0000 mgr.y (mgr.24431) 1157 : cluster [DBG] pgmap v1577: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:26.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:26 smithi175 bash[29284]: cluster 2024-03-26T04:15:25.630428+0000 mgr.y (mgr.24431) 1157 : cluster [DBG] pgmap v1577: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:26.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:26 smithi190 bash[23392]: cluster 2024-03-26T04:15:25.630428+0000 mgr.y (mgr.24431) 1157 : cluster [DBG] pgmap v1577: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:28.990 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:15:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:15:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:15:28.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:28 smithi175 bash[29284]: cluster 2024-03-26T04:15:27.631258+0000 mgr.y (mgr.24431) 1158 : cluster [DBG] pgmap v1578: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T04:15:28.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:28 smithi175 bash[21527]: cluster 2024-03-26T04:15:27.631258+0000 mgr.y (mgr.24431) 1158 : cluster [DBG] pgmap v1578: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T04:15:28.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:28 smithi190 bash[23392]: cluster 2024-03-26T04:15:27.631258+0000 mgr.y (mgr.24431) 1158 : cluster [DBG] pgmap v1578: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T04:15:30.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:30 smithi175 bash[21527]: cluster 2024-03-26T04:15:29.633452+0000 mgr.y (mgr.24431) 1159 : cluster [DBG] pgmap v1579: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:30.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:30 smithi175 bash[29284]: cluster 2024-03-26T04:15:29.633452+0000 mgr.y (mgr.24431) 1159 : cluster [DBG] pgmap v1579: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:30.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:30 smithi190 bash[23392]: cluster 2024-03-26T04:15:29.633452+0000 mgr.y (mgr.24431) 1159 : cluster [DBG] pgmap v1579: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:31.748 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:15:31 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:15:32.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:32 smithi175 bash[29284]: audit 2024-03-26T04:15:31.277203+0000 mgr.y (mgr.24431) 1160 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:15:32.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:32 smithi175 bash[29284]: cluster 2024-03-26T04:15:31.634237+0000 mgr.y (mgr.24431) 1161 : cluster [DBG] pgmap v1580: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:32.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:32 smithi175 bash[21527]: audit 2024-03-26T04:15:31.277203+0000 mgr.y (mgr.24431) 1160 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:15:32.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:32 smithi175 bash[21527]: cluster 2024-03-26T04:15:31.634237+0000 mgr.y (mgr.24431) 1161 : cluster [DBG] pgmap v1580: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:32.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:32 smithi190 bash[23392]: audit 2024-03-26T04:15:31.277203+0000 mgr.y (mgr.24431) 1160 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:15:32.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:32 smithi190 bash[23392]: cluster 2024-03-26T04:15:31.634237+0000 mgr.y (mgr.24431) 1161 : cluster [DBG] pgmap v1580: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:34.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:34 smithi175 bash[21527]: cluster 2024-03-26T04:15:33.635459+0000 mgr.y (mgr.24431) 1162 : cluster [DBG] pgmap v1581: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:34.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:34 smithi175 bash[29284]: cluster 2024-03-26T04:15:33.635459+0000 mgr.y (mgr.24431) 1162 : cluster [DBG] pgmap v1581: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:34.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:34 smithi190 bash[23392]: cluster 2024-03-26T04:15:33.635459+0000 mgr.y (mgr.24431) 1162 : cluster [DBG] pgmap v1581: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:36.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:36 smithi175 bash[21527]: cluster 2024-03-26T04:15:35.638419+0000 mgr.y (mgr.24431) 1163 : cluster [DBG] pgmap v1582: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:36.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:36 smithi175 bash[29284]: cluster 2024-03-26T04:15:35.638419+0000 mgr.y (mgr.24431) 1163 : cluster [DBG] pgmap v1582: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:36.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:36 smithi190 bash[23392]: cluster 2024-03-26T04:15:35.638419+0000 mgr.y (mgr.24431) 1163 : cluster [DBG] pgmap v1582: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:38.990 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:15:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:15:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:15:38.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:38 smithi175 bash[21527]: cluster 2024-03-26T04:15:37.639291+0000 mgr.y (mgr.24431) 1164 : cluster [DBG] pgmap v1583: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:38.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:38 smithi175 bash[21527]: audit 2024-03-26T04:15:37.938719+0000 mon.c (mon.2) 834 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:15:38.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:38 smithi175 bash[29284]: cluster 2024-03-26T04:15:37.639291+0000 mgr.y (mgr.24431) 1164 : cluster [DBG] pgmap v1583: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:38.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:38 smithi175 bash[29284]: audit 2024-03-26T04:15:37.938719+0000 mon.c (mon.2) 834 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:15:38.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:38 smithi190 bash[23392]: cluster 2024-03-26T04:15:37.639291+0000 mgr.y (mgr.24431) 1164 : cluster [DBG] pgmap v1583: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:38.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:38 smithi190 bash[23392]: audit 2024-03-26T04:15:37.938719+0000 mon.c (mon.2) 834 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:15:40.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:40 smithi175 bash[29284]: cluster 2024-03-26T04:15:39.641457+0000 mgr.y (mgr.24431) 1165 : cluster [DBG] pgmap v1584: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:40.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:40 smithi175 bash[21527]: cluster 2024-03-26T04:15:39.641457+0000 mgr.y (mgr.24431) 1165 : cluster [DBG] pgmap v1584: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:40.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:40 smithi190 bash[23392]: cluster 2024-03-26T04:15:39.641457+0000 mgr.y (mgr.24431) 1165 : cluster [DBG] pgmap v1584: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:41.748 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:15:41 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:15:42.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:42 smithi190 bash[23392]: audit 2024-03-26T04:15:41.286185+0000 mgr.y (mgr.24431) 1166 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:15:42.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:42 smithi190 bash[23392]: cluster 2024-03-26T04:15:41.642367+0000 mgr.y (mgr.24431) 1167 : cluster [DBG] pgmap v1585: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:43.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:42 smithi175 bash[29284]: audit 2024-03-26T04:15:41.286185+0000 mgr.y (mgr.24431) 1166 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:15:43.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:42 smithi175 bash[29284]: cluster 2024-03-26T04:15:41.642367+0000 mgr.y (mgr.24431) 1167 : cluster [DBG] pgmap v1585: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:43.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:42 smithi175 bash[21527]: audit 2024-03-26T04:15:41.286185+0000 mgr.y (mgr.24431) 1166 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:15:43.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:42 smithi175 bash[21527]: cluster 2024-03-26T04:15:41.642367+0000 mgr.y (mgr.24431) 1167 : cluster [DBG] pgmap v1585: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:45.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:44 smithi175 bash[21527]: cluster 2024-03-26T04:15:43.643190+0000 mgr.y (mgr.24431) 1168 : cluster [DBG] pgmap v1586: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:45.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:44 smithi175 bash[29284]: cluster 2024-03-26T04:15:43.643190+0000 mgr.y (mgr.24431) 1168 : cluster [DBG] pgmap v1586: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:45.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:44 smithi190 bash[23392]: cluster 2024-03-26T04:15:43.643190+0000 mgr.y (mgr.24431) 1168 : cluster [DBG] pgmap v1586: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:46.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:45 smithi175 bash[21527]: audit 2024-03-26T04:15:45.746960+0000 mon.c (mon.2) 835 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:15:46.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:45 smithi175 bash[29284]: audit 2024-03-26T04:15:45.746960+0000 mon.c (mon.2) 835 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:15:46.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:45 smithi190 bash[23392]: audit 2024-03-26T04:15:45.746960+0000 mon.c (mon.2) 835 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:15:47.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:46 smithi175 bash[21527]: cluster 2024-03-26T04:15:45.645471+0000 mgr.y (mgr.24431) 1169 : cluster [DBG] pgmap v1587: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:47.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:46 smithi175 bash[21527]: audit 2024-03-26T04:15:46.272906+0000 mon.a (mon.0) 3545 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:15:47.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:46 smithi175 bash[21527]: audit 2024-03-26T04:15:46.280343+0000 mon.a (mon.0) 3546 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:15:47.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:46 smithi175 bash[29284]: cluster 2024-03-26T04:15:45.645471+0000 mgr.y (mgr.24431) 1169 : cluster [DBG] pgmap v1587: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:47.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:46 smithi175 bash[29284]: audit 2024-03-26T04:15:46.272906+0000 mon.a (mon.0) 3545 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:15:47.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:46 smithi175 bash[29284]: audit 2024-03-26T04:15:46.280343+0000 mon.a (mon.0) 3546 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:15:47.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:46 smithi190 bash[23392]: cluster 2024-03-26T04:15:45.645471+0000 mgr.y (mgr.24431) 1169 : cluster [DBG] pgmap v1587: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:47.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:46 smithi190 bash[23392]: audit 2024-03-26T04:15:46.272906+0000 mon.a (mon.0) 3545 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:15:47.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:46 smithi190 bash[23392]: audit 2024-03-26T04:15:46.280343+0000 mon.a (mon.0) 3546 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:15:48.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:47 smithi175 bash[21527]: audit 2024-03-26T04:15:46.901240+0000 mon.a (mon.0) 3547 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:15:48.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:47 smithi175 bash[21527]: audit 2024-03-26T04:15:46.911647+0000 mon.a (mon.0) 3548 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:15:48.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:47 smithi175 bash[21527]: audit 2024-03-26T04:15:47.344253+0000 mon.a (mon.0) 3549 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:15:48.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:47 smithi175 bash[21527]: audit 2024-03-26T04:15:47.362332+0000 mon.a (mon.0) 3550 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:15:48.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:47 smithi175 bash[21527]: audit 2024-03-26T04:15:47.368151+0000 mon.c (mon.2) 836 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:15:48.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:47 smithi175 bash[21527]: audit 2024-03-26T04:15:47.370048+0000 mon.c (mon.2) 837 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:15:48.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:47 smithi175 bash[21527]: audit 2024-03-26T04:15:47.381936+0000 mon.a (mon.0) 3551 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:15:48.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:47 smithi175 bash[21527]: cluster 2024-03-26T04:15:47.646314+0000 mgr.y (mgr.24431) 1170 : cluster [DBG] pgmap v1588: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:48.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:47 smithi175 bash[29284]: audit 2024-03-26T04:15:46.901240+0000 mon.a (mon.0) 3547 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:15:48.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:47 smithi175 bash[29284]: audit 2024-03-26T04:15:46.911647+0000 mon.a (mon.0) 3548 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:15:48.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:47 smithi175 bash[29284]: audit 2024-03-26T04:15:47.344253+0000 mon.a (mon.0) 3549 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:15:48.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:47 smithi175 bash[29284]: audit 2024-03-26T04:15:47.362332+0000 mon.a (mon.0) 3550 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:15:48.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:47 smithi175 bash[29284]: audit 2024-03-26T04:15:47.368151+0000 mon.c (mon.2) 836 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:15:48.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:47 smithi175 bash[29284]: audit 2024-03-26T04:15:47.370048+0000 mon.c (mon.2) 837 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:15:48.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:47 smithi175 bash[29284]: audit 2024-03-26T04:15:47.381936+0000 mon.a (mon.0) 3551 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:15:48.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:47 smithi175 bash[29284]: cluster 2024-03-26T04:15:47.646314+0000 mgr.y (mgr.24431) 1170 : cluster [DBG] pgmap v1588: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:48.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:47 smithi190 bash[23392]: audit 2024-03-26T04:15:46.901240+0000 mon.a (mon.0) 3547 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:15:48.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:47 smithi190 bash[23392]: audit 2024-03-26T04:15:46.911647+0000 mon.a (mon.0) 3548 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:15:48.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:47 smithi190 bash[23392]: audit 2024-03-26T04:15:47.344253+0000 mon.a (mon.0) 3549 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:15:48.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:47 smithi190 bash[23392]: audit 2024-03-26T04:15:47.362332+0000 mon.a (mon.0) 3550 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:15:48.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:47 smithi190 bash[23392]: audit 2024-03-26T04:15:47.368151+0000 mon.c (mon.2) 836 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:15:48.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:47 smithi190 bash[23392]: audit 2024-03-26T04:15:47.370048+0000 mon.c (mon.2) 837 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:15:48.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:47 smithi190 bash[23392]: audit 2024-03-26T04:15:47.381936+0000 mon.a (mon.0) 3551 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:15:48.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:47 smithi190 bash[23392]: cluster 2024-03-26T04:15:47.646314+0000 mgr.y (mgr.24431) 1170 : cluster [DBG] pgmap v1588: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:49.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:15:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:15:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:15:50.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:50 smithi175 bash[21527]: cluster 2024-03-26T04:15:49.648529+0000 mgr.y (mgr.24431) 1171 : cluster [DBG] pgmap v1589: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:50.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:50 smithi175 bash[29284]: cluster 2024-03-26T04:15:49.648529+0000 mgr.y (mgr.24431) 1171 : cluster [DBG] pgmap v1589: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:50.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:50 smithi190 bash[23392]: cluster 2024-03-26T04:15:49.648529+0000 mgr.y (mgr.24431) 1171 : cluster [DBG] pgmap v1589: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:51.748 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:15:51 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:15:52.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:52 smithi175 bash[29284]: audit 2024-03-26T04:15:51.297715+0000 mgr.y (mgr.24431) 1172 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:15:52.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:52 smithi175 bash[29284]: cluster 2024-03-26T04:15:51.649412+0000 mgr.y (mgr.24431) 1173 : cluster [DBG] pgmap v1590: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:52.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:52 smithi175 bash[21527]: audit 2024-03-26T04:15:51.297715+0000 mgr.y (mgr.24431) 1172 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:15:52.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:52 smithi175 bash[21527]: cluster 2024-03-26T04:15:51.649412+0000 mgr.y (mgr.24431) 1173 : cluster [DBG] pgmap v1590: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:52.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:52 smithi190 bash[23392]: audit 2024-03-26T04:15:51.297715+0000 mgr.y (mgr.24431) 1172 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:15:52.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:52 smithi190 bash[23392]: cluster 2024-03-26T04:15:51.649412+0000 mgr.y (mgr.24431) 1173 : cluster [DBG] pgmap v1590: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:53.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:53 smithi175 bash[21527]: audit 2024-03-26T04:15:52.971986+0000 mon.c (mon.2) 838 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:15:53.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:53 smithi175 bash[29284]: audit 2024-03-26T04:15:52.971986+0000 mon.c (mon.2) 838 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:15:53.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:53 smithi190 bash[23392]: audit 2024-03-26T04:15:52.971986+0000 mon.c (mon.2) 838 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:15:54.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:54 smithi175 bash[29284]: cluster 2024-03-26T04:15:53.650383+0000 mgr.y (mgr.24431) 1174 : cluster [DBG] pgmap v1591: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:54.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:54 smithi175 bash[21527]: cluster 2024-03-26T04:15:53.650383+0000 mgr.y (mgr.24431) 1174 : cluster [DBG] pgmap v1591: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:54.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:54 smithi190 bash[23392]: cluster 2024-03-26T04:15:53.650383+0000 mgr.y (mgr.24431) 1174 : cluster [DBG] pgmap v1591: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:56.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:56 smithi175 bash[21527]: cluster 2024-03-26T04:15:55.652713+0000 mgr.y (mgr.24431) 1175 : cluster [DBG] pgmap v1592: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:56.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:56 smithi175 bash[29284]: cluster 2024-03-26T04:15:55.652713+0000 mgr.y (mgr.24431) 1175 : cluster [DBG] pgmap v1592: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:56.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:56 smithi190 bash[23392]: cluster 2024-03-26T04:15:55.652713+0000 mgr.y (mgr.24431) 1175 : cluster [DBG] pgmap v1592: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:15:58.990 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:15:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:15:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:15:58.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:15:58 smithi175 bash[29284]: cluster 2024-03-26T04:15:57.653604+0000 mgr.y (mgr.24431) 1176 : cluster [DBG] pgmap v1593: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:58.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:15:58 smithi175 bash[21527]: cluster 2024-03-26T04:15:57.653604+0000 mgr.y (mgr.24431) 1176 : cluster [DBG] pgmap v1593: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:15:58.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:15:58 smithi190 bash[23392]: cluster 2024-03-26T04:15:57.653604+0000 mgr.y (mgr.24431) 1176 : cluster [DBG] pgmap v1593: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:00.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:00 smithi175 bash[29284]: cluster 2024-03-26T04:15:59.655933+0000 mgr.y (mgr.24431) 1177 : cluster [DBG] pgmap v1594: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:00.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:00 smithi175 bash[21527]: cluster 2024-03-26T04:15:59.655933+0000 mgr.y (mgr.24431) 1177 : cluster [DBG] pgmap v1594: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:00.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:00 smithi190 bash[23392]: cluster 2024-03-26T04:15:59.655933+0000 mgr.y (mgr.24431) 1177 : cluster [DBG] pgmap v1594: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:01.748 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:16:01 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:16:02.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:02 smithi175 bash[29284]: audit 2024-03-26T04:16:01.306298+0000 mgr.y (mgr.24431) 1178 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:16:02.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:02 smithi175 bash[29284]: cluster 2024-03-26T04:16:01.656723+0000 mgr.y (mgr.24431) 1179 : cluster [DBG] pgmap v1595: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:02.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:02 smithi175 bash[21527]: audit 2024-03-26T04:16:01.306298+0000 mgr.y (mgr.24431) 1178 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:16:02.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:02 smithi175 bash[21527]: cluster 2024-03-26T04:16:01.656723+0000 mgr.y (mgr.24431) 1179 : cluster [DBG] pgmap v1595: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:02.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:02 smithi190 bash[23392]: audit 2024-03-26T04:16:01.306298+0000 mgr.y (mgr.24431) 1178 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:16:02.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:02 smithi190 bash[23392]: cluster 2024-03-26T04:16:01.656723+0000 mgr.y (mgr.24431) 1179 : cluster [DBG] pgmap v1595: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:04.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:04 smithi190 bash[23392]: cluster 2024-03-26T04:16:03.657566+0000 mgr.y (mgr.24431) 1180 : cluster [DBG] pgmap v1596: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:05.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:04 smithi175 bash[29284]: cluster 2024-03-26T04:16:03.657566+0000 mgr.y (mgr.24431) 1180 : cluster [DBG] pgmap v1596: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:05.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:04 smithi175 bash[21527]: cluster 2024-03-26T04:16:03.657566+0000 mgr.y (mgr.24431) 1180 : cluster [DBG] pgmap v1596: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:06.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:06 smithi190 bash[23392]: cluster 2024-03-26T04:16:05.659885+0000 mgr.y (mgr.24431) 1181 : cluster [DBG] pgmap v1597: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:07.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:06 smithi175 bash[21527]: cluster 2024-03-26T04:16:05.659885+0000 mgr.y (mgr.24431) 1181 : cluster [DBG] pgmap v1597: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:07.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:06 smithi175 bash[29284]: cluster 2024-03-26T04:16:05.659885+0000 mgr.y (mgr.24431) 1181 : cluster [DBG] pgmap v1597: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:09.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:16:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:16:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:16:09.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:08 smithi175 bash[29284]: cluster 2024-03-26T04:16:07.660695+0000 mgr.y (mgr.24431) 1182 : cluster [DBG] pgmap v1598: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:09.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:08 smithi175 bash[29284]: audit 2024-03-26T04:16:08.006169+0000 mon.c (mon.2) 839 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:16:09.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:08 smithi175 bash[21527]: cluster 2024-03-26T04:16:07.660695+0000 mgr.y (mgr.24431) 1182 : cluster [DBG] pgmap v1598: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:09.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:08 smithi175 bash[21527]: audit 2024-03-26T04:16:08.006169+0000 mon.c (mon.2) 839 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:16:09.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:08 smithi190 bash[23392]: cluster 2024-03-26T04:16:07.660695+0000 mgr.y (mgr.24431) 1182 : cluster [DBG] pgmap v1598: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:09.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:08 smithi190 bash[23392]: audit 2024-03-26T04:16:08.006169+0000 mon.c (mon.2) 839 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:16:11.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:10 smithi175 bash[29284]: cluster 2024-03-26T04:16:09.662849+0000 mgr.y (mgr.24431) 1183 : cluster [DBG] pgmap v1599: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:11.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:10 smithi175 bash[21527]: cluster 2024-03-26T04:16:09.662849+0000 mgr.y (mgr.24431) 1183 : cluster [DBG] pgmap v1599: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:11.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:10 smithi190 bash[23392]: cluster 2024-03-26T04:16:09.662849+0000 mgr.y (mgr.24431) 1183 : cluster [DBG] pgmap v1599: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:11.748 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:16:11 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:16:13.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:12 smithi175 bash[21527]: audit 2024-03-26T04:16:11.317820+0000 mgr.y (mgr.24431) 1184 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:16:13.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:12 smithi175 bash[21527]: cluster 2024-03-26T04:16:11.663709+0000 mgr.y (mgr.24431) 1185 : cluster [DBG] pgmap v1600: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:13.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:12 smithi175 bash[29284]: audit 2024-03-26T04:16:11.317820+0000 mgr.y (mgr.24431) 1184 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:16:13.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:12 smithi175 bash[29284]: cluster 2024-03-26T04:16:11.663709+0000 mgr.y (mgr.24431) 1185 : cluster [DBG] pgmap v1600: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:13.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:12 smithi190 bash[23392]: audit 2024-03-26T04:16:11.317820+0000 mgr.y (mgr.24431) 1184 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:16:13.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:12 smithi190 bash[23392]: cluster 2024-03-26T04:16:11.663709+0000 mgr.y (mgr.24431) 1185 : cluster [DBG] pgmap v1600: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:15.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:14 smithi175 bash[21527]: cluster 2024-03-26T04:16:13.664568+0000 mgr.y (mgr.24431) 1186 : cluster [DBG] pgmap v1601: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:15.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:14 smithi175 bash[29284]: cluster 2024-03-26T04:16:13.664568+0000 mgr.y (mgr.24431) 1186 : cluster [DBG] pgmap v1601: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:15.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:14 smithi190 bash[23392]: cluster 2024-03-26T04:16:13.664568+0000 mgr.y (mgr.24431) 1186 : cluster [DBG] pgmap v1601: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:17.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:16 smithi175 bash[29284]: cluster 2024-03-26T04:16:15.666893+0000 mgr.y (mgr.24431) 1187 : cluster [DBG] pgmap v1602: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:17.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:16 smithi175 bash[21527]: cluster 2024-03-26T04:16:15.666893+0000 mgr.y (mgr.24431) 1187 : cluster [DBG] pgmap v1602: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:17.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:16 smithi190 bash[23392]: cluster 2024-03-26T04:16:15.666893+0000 mgr.y (mgr.24431) 1187 : cluster [DBG] pgmap v1602: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:19.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:18 smithi175 bash[21527]: cluster 2024-03-26T04:16:17.667877+0000 mgr.y (mgr.24431) 1188 : cluster [DBG] pgmap v1603: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:19.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:18 smithi175 bash[29284]: cluster 2024-03-26T04:16:17.667877+0000 mgr.y (mgr.24431) 1188 : cluster [DBG] pgmap v1603: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:19.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:16:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:16:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:16:19.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:18 smithi190 bash[23392]: cluster 2024-03-26T04:16:17.667877+0000 mgr.y (mgr.24431) 1188 : cluster [DBG] pgmap v1603: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:21.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:20 smithi175 bash[21527]: cluster 2024-03-26T04:16:19.670157+0000 mgr.y (mgr.24431) 1189 : cluster [DBG] pgmap v1604: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:21.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:20 smithi175 bash[29284]: cluster 2024-03-26T04:16:19.670157+0000 mgr.y (mgr.24431) 1189 : cluster [DBG] pgmap v1604: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:21.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:20 smithi190 bash[23392]: cluster 2024-03-26T04:16:19.670157+0000 mgr.y (mgr.24431) 1189 : cluster [DBG] pgmap v1604: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:21.748 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:16:21 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:16:23.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:22 smithi175 bash[29284]: audit 2024-03-26T04:16:21.326294+0000 mgr.y (mgr.24431) 1190 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:16:23.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:22 smithi175 bash[29284]: cluster 2024-03-26T04:16:21.670970+0000 mgr.y (mgr.24431) 1191 : cluster [DBG] pgmap v1605: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:23.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:22 smithi175 bash[21527]: audit 2024-03-26T04:16:21.326294+0000 mgr.y (mgr.24431) 1190 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:16:23.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:22 smithi175 bash[21527]: cluster 2024-03-26T04:16:21.670970+0000 mgr.y (mgr.24431) 1191 : cluster [DBG] pgmap v1605: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:23.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:22 smithi190 bash[23392]: audit 2024-03-26T04:16:21.326294+0000 mgr.y (mgr.24431) 1190 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:16:23.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:22 smithi190 bash[23392]: cluster 2024-03-26T04:16:21.670970+0000 mgr.y (mgr.24431) 1191 : cluster [DBG] pgmap v1605: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:24.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:23 smithi175 bash[21527]: audit 2024-03-26T04:16:23.040209+0000 mon.c (mon.2) 840 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:16:24.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:23 smithi175 bash[29284]: audit 2024-03-26T04:16:23.040209+0000 mon.c (mon.2) 840 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:16:24.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:23 smithi190 bash[23392]: audit 2024-03-26T04:16:23.040209+0000 mon.c (mon.2) 840 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:16:25.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:24 smithi175 bash[29284]: cluster 2024-03-26T04:16:23.671831+0000 mgr.y (mgr.24431) 1192 : cluster [DBG] pgmap v1606: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:25.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:24 smithi175 bash[21527]: cluster 2024-03-26T04:16:23.671831+0000 mgr.y (mgr.24431) 1192 : cluster [DBG] pgmap v1606: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:25.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:24 smithi190 bash[23392]: cluster 2024-03-26T04:16:23.671831+0000 mgr.y (mgr.24431) 1192 : cluster [DBG] pgmap v1606: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:27.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:26 smithi175 bash[21527]: cluster 2024-03-26T04:16:25.674083+0000 mgr.y (mgr.24431) 1193 : cluster [DBG] pgmap v1607: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:27.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:26 smithi175 bash[29284]: cluster 2024-03-26T04:16:25.674083+0000 mgr.y (mgr.24431) 1193 : cluster [DBG] pgmap v1607: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:27.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:26 smithi190 bash[23392]: cluster 2024-03-26T04:16:25.674083+0000 mgr.y (mgr.24431) 1193 : cluster [DBG] pgmap v1607: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:28.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:27 smithi175 bash[21527]: cluster 2024-03-26T04:16:27.674922+0000 mgr.y (mgr.24431) 1194 : cluster [DBG] pgmap v1608: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:28.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:27 smithi175 bash[29284]: cluster 2024-03-26T04:16:27.674922+0000 mgr.y (mgr.24431) 1194 : cluster [DBG] pgmap v1608: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:28.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:27 smithi190 bash[23392]: cluster 2024-03-26T04:16:27.674922+0000 mgr.y (mgr.24431) 1194 : cluster [DBG] pgmap v1608: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:29.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:16:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:16:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:16:30.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:30 smithi175 bash[21527]: cluster 2024-03-26T04:16:29.677077+0000 mgr.y (mgr.24431) 1195 : cluster [DBG] pgmap v1609: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:30.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:30 smithi175 bash[29284]: cluster 2024-03-26T04:16:29.677077+0000 mgr.y (mgr.24431) 1195 : cluster [DBG] pgmap v1609: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:30.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:30 smithi190 bash[23392]: cluster 2024-03-26T04:16:29.677077+0000 mgr.y (mgr.24431) 1195 : cluster [DBG] pgmap v1609: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:31.748 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:16:31 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:16:32.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:32 smithi175 bash[21527]: audit 2024-03-26T04:16:31.337767+0000 mgr.y (mgr.24431) 1196 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:16:32.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:32 smithi175 bash[21527]: cluster 2024-03-26T04:16:31.678123+0000 mgr.y (mgr.24431) 1197 : cluster [DBG] pgmap v1610: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:32.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:32 smithi175 bash[29284]: audit 2024-03-26T04:16:31.337767+0000 mgr.y (mgr.24431) 1196 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:16:32.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:32 smithi175 bash[29284]: cluster 2024-03-26T04:16:31.678123+0000 mgr.y (mgr.24431) 1197 : cluster [DBG] pgmap v1610: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:32.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:32 smithi190 bash[23392]: audit 2024-03-26T04:16:31.337767+0000 mgr.y (mgr.24431) 1196 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:16:32.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:32 smithi190 bash[23392]: cluster 2024-03-26T04:16:31.678123+0000 mgr.y (mgr.24431) 1197 : cluster [DBG] pgmap v1610: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:33.998 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 04:16:33 smithi190 bash[50860]: logger=cleanup t=2024-03-26T04:16:33.563907005Z level=info msg="Completed cleanup jobs" duration=58.84078ms 2024-03-26T04:16:34.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:34 smithi175 bash[29284]: cluster 2024-03-26T04:16:33.678976+0000 mgr.y (mgr.24431) 1198 : cluster [DBG] pgmap v1611: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:34.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:34 smithi175 bash[21527]: cluster 2024-03-26T04:16:33.678976+0000 mgr.y (mgr.24431) 1198 : cluster [DBG] pgmap v1611: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:34.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:34 smithi190 bash[23392]: cluster 2024-03-26T04:16:33.678976+0000 mgr.y (mgr.24431) 1198 : cluster [DBG] pgmap v1611: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:36.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:36 smithi190 bash[23392]: cluster 2024-03-26T04:16:35.681421+0000 mgr.y (mgr.24431) 1199 : cluster [DBG] pgmap v1612: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:37.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:36 smithi175 bash[21527]: cluster 2024-03-26T04:16:35.681421+0000 mgr.y (mgr.24431) 1199 : cluster [DBG] pgmap v1612: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:37.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:36 smithi175 bash[29284]: cluster 2024-03-26T04:16:35.681421+0000 mgr.y (mgr.24431) 1199 : cluster [DBG] pgmap v1612: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:38.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:38 smithi190 bash[23392]: cluster 2024-03-26T04:16:37.682246+0000 mgr.y (mgr.24431) 1200 : cluster [DBG] pgmap v1613: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:38.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:38 smithi190 bash[23392]: audit 2024-03-26T04:16:38.074274+0000 mon.c (mon.2) 841 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:16:39.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:16:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:16:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:16:39.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:38 smithi175 bash[29284]: cluster 2024-03-26T04:16:37.682246+0000 mgr.y (mgr.24431) 1200 : cluster [DBG] pgmap v1613: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:39.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:38 smithi175 bash[29284]: audit 2024-03-26T04:16:38.074274+0000 mon.c (mon.2) 841 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:16:39.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:38 smithi175 bash[21527]: cluster 2024-03-26T04:16:37.682246+0000 mgr.y (mgr.24431) 1200 : cluster [DBG] pgmap v1613: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:39.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:38 smithi175 bash[21527]: audit 2024-03-26T04:16:38.074274+0000 mon.c (mon.2) 841 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:16:41.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:40 smithi175 bash[21527]: cluster 2024-03-26T04:16:39.684958+0000 mgr.y (mgr.24431) 1201 : cluster [DBG] pgmap v1614: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:41.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:40 smithi175 bash[29284]: cluster 2024-03-26T04:16:39.684958+0000 mgr.y (mgr.24431) 1201 : cluster [DBG] pgmap v1614: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:41.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:40 smithi190 bash[23392]: cluster 2024-03-26T04:16:39.684958+0000 mgr.y (mgr.24431) 1201 : cluster [DBG] pgmap v1614: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:41.748 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:16:41 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:16:43.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:42 smithi175 bash[21527]: audit 2024-03-26T04:16:41.339985+0000 mgr.y (mgr.24431) 1202 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:16:43.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:42 smithi175 bash[21527]: cluster 2024-03-26T04:16:41.685774+0000 mgr.y (mgr.24431) 1203 : cluster [DBG] pgmap v1615: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:43.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:42 smithi175 bash[29284]: audit 2024-03-26T04:16:41.339985+0000 mgr.y (mgr.24431) 1202 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:16:43.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:42 smithi175 bash[29284]: cluster 2024-03-26T04:16:41.685774+0000 mgr.y (mgr.24431) 1203 : cluster [DBG] pgmap v1615: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:43.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:42 smithi190 bash[23392]: audit 2024-03-26T04:16:41.339985+0000 mgr.y (mgr.24431) 1202 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:16:43.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:42 smithi190 bash[23392]: cluster 2024-03-26T04:16:41.685774+0000 mgr.y (mgr.24431) 1203 : cluster [DBG] pgmap v1615: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:45.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:44 smithi175 bash[21527]: cluster 2024-03-26T04:16:43.686631+0000 mgr.y (mgr.24431) 1204 : cluster [DBG] pgmap v1616: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:45.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:44 smithi175 bash[29284]: cluster 2024-03-26T04:16:43.686631+0000 mgr.y (mgr.24431) 1204 : cluster [DBG] pgmap v1616: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:45.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:44 smithi190 bash[23392]: cluster 2024-03-26T04:16:43.686631+0000 mgr.y (mgr.24431) 1204 : cluster [DBG] pgmap v1616: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:47.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:46 smithi175 bash[21527]: cluster 2024-03-26T04:16:45.688930+0000 mgr.y (mgr.24431) 1205 : cluster [DBG] pgmap v1617: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:47.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:46 smithi175 bash[29284]: cluster 2024-03-26T04:16:45.688930+0000 mgr.y (mgr.24431) 1205 : cluster [DBG] pgmap v1617: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:47.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:46 smithi190 bash[23392]: cluster 2024-03-26T04:16:45.688930+0000 mgr.y (mgr.24431) 1205 : cluster [DBG] pgmap v1617: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:48.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:47 smithi175 bash[29284]: audit 2024-03-26T04:16:47.477190+0000 mon.c (mon.2) 842 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:16:48.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:47 smithi175 bash[29284]: cluster 2024-03-26T04:16:47.689818+0000 mgr.y (mgr.24431) 1206 : cluster [DBG] pgmap v1618: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:48.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:47 smithi175 bash[21527]: audit 2024-03-26T04:16:47.477190+0000 mon.c (mon.2) 842 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:16:48.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:47 smithi175 bash[21527]: cluster 2024-03-26T04:16:47.689818+0000 mgr.y (mgr.24431) 1206 : cluster [DBG] pgmap v1618: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:48.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:47 smithi190 bash[23392]: audit 2024-03-26T04:16:47.477190+0000 mon.c (mon.2) 842 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:16:48.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:47 smithi190 bash[23392]: cluster 2024-03-26T04:16:47.689818+0000 mgr.y (mgr.24431) 1206 : cluster [DBG] pgmap v1618: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:49.053 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:16:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:16:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:16:49.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:49 smithi175 bash[21527]: audit 2024-03-26T04:16:48.057364+0000 mon.a (mon.0) 3552 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:16:49.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:49 smithi175 bash[21527]: audit 2024-03-26T04:16:48.073888+0000 mon.a (mon.0) 3553 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:16:49.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:49 smithi175 bash[21527]: audit 2024-03-26T04:16:48.772018+0000 mon.c (mon.2) 843 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:16:49.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:49 smithi175 bash[21527]: audit 2024-03-26T04:16:48.773730+0000 mon.c (mon.2) 844 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:16:49.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:49 smithi175 bash[21527]: audit 2024-03-26T04:16:48.783276+0000 mon.a (mon.0) 3554 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:16:49.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:49 smithi175 bash[29284]: audit 2024-03-26T04:16:48.057364+0000 mon.a (mon.0) 3552 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:16:49.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:49 smithi175 bash[29284]: audit 2024-03-26T04:16:48.073888+0000 mon.a (mon.0) 3553 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:16:49.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:49 smithi175 bash[29284]: audit 2024-03-26T04:16:48.772018+0000 mon.c (mon.2) 843 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:16:49.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:49 smithi175 bash[29284]: audit 2024-03-26T04:16:48.773730+0000 mon.c (mon.2) 844 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:16:49.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:49 smithi175 bash[29284]: audit 2024-03-26T04:16:48.783276+0000 mon.a (mon.0) 3554 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:16:49.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:49 smithi190 bash[23392]: audit 2024-03-26T04:16:48.057364+0000 mon.a (mon.0) 3552 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:16:49.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:49 smithi190 bash[23392]: audit 2024-03-26T04:16:48.073888+0000 mon.a (mon.0) 3553 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:16:49.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:49 smithi190 bash[23392]: audit 2024-03-26T04:16:48.772018+0000 mon.c (mon.2) 843 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:16:49.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:49 smithi190 bash[23392]: audit 2024-03-26T04:16:48.773730+0000 mon.c (mon.2) 844 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:16:49.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:49 smithi190 bash[23392]: audit 2024-03-26T04:16:48.783276+0000 mon.a (mon.0) 3554 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:16:50.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:50 smithi175 bash[21527]: cluster 2024-03-26T04:16:49.692160+0000 mgr.y (mgr.24431) 1207 : cluster [DBG] pgmap v1619: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:50.490 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:50 smithi175 bash[29284]: cluster 2024-03-26T04:16:49.692160+0000 mgr.y (mgr.24431) 1207 : cluster [DBG] pgmap v1619: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:50.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:50 smithi190 bash[23392]: cluster 2024-03-26T04:16:49.692160+0000 mgr.y (mgr.24431) 1207 : cluster [DBG] pgmap v1619: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:51.748 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:16:51 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:16:52.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:52 smithi190 bash[23392]: audit 2024-03-26T04:16:51.348738+0000 mgr.y (mgr.24431) 1208 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:16:52.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:52 smithi190 bash[23392]: cluster 2024-03-26T04:16:51.692966+0000 mgr.y (mgr.24431) 1209 : cluster [DBG] pgmap v1620: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:53.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:52 smithi175 bash[29284]: audit 2024-03-26T04:16:51.348738+0000 mgr.y (mgr.24431) 1208 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:16:53.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:52 smithi175 bash[29284]: cluster 2024-03-26T04:16:51.692966+0000 mgr.y (mgr.24431) 1209 : cluster [DBG] pgmap v1620: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:53.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:52 smithi175 bash[21527]: audit 2024-03-26T04:16:51.348738+0000 mgr.y (mgr.24431) 1208 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:16:53.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:52 smithi175 bash[21527]: cluster 2024-03-26T04:16:51.692966+0000 mgr.y (mgr.24431) 1209 : cluster [DBG] pgmap v1620: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:54.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:53 smithi175 bash[29284]: audit 2024-03-26T04:16:53.109405+0000 mon.c (mon.2) 845 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:16:54.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:53 smithi175 bash[21527]: audit 2024-03-26T04:16:53.109405+0000 mon.c (mon.2) 845 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:16:54.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:53 smithi190 bash[23392]: audit 2024-03-26T04:16:53.109405+0000 mon.c (mon.2) 845 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:16:55.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:54 smithi175 bash[29284]: cluster 2024-03-26T04:16:53.693812+0000 mgr.y (mgr.24431) 1210 : cluster [DBG] pgmap v1621: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:55.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:54 smithi175 bash[21527]: cluster 2024-03-26T04:16:53.693812+0000 mgr.y (mgr.24431) 1210 : cluster [DBG] pgmap v1621: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:55.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:54 smithi190 bash[23392]: cluster 2024-03-26T04:16:53.693812+0000 mgr.y (mgr.24431) 1210 : cluster [DBG] pgmap v1621: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:57.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:56 smithi175 bash[29284]: cluster 2024-03-26T04:16:55.696087+0000 mgr.y (mgr.24431) 1211 : cluster [DBG] pgmap v1622: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:57.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:56 smithi175 bash[21527]: cluster 2024-03-26T04:16:55.696087+0000 mgr.y (mgr.24431) 1211 : cluster [DBG] pgmap v1622: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:57.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:56 smithi190 bash[23392]: cluster 2024-03-26T04:16:55.696087+0000 mgr.y (mgr.24431) 1211 : cluster [DBG] pgmap v1622: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:16:59.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:16:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:16:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:16:59.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:16:58 smithi175 bash[29284]: cluster 2024-03-26T04:16:57.696938+0000 mgr.y (mgr.24431) 1212 : cluster [DBG] pgmap v1623: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:59.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:16:58 smithi175 bash[21527]: cluster 2024-03-26T04:16:57.696938+0000 mgr.y (mgr.24431) 1212 : cluster [DBG] pgmap v1623: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:16:59.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:16:58 smithi190 bash[23392]: cluster 2024-03-26T04:16:57.696938+0000 mgr.y (mgr.24431) 1212 : cluster [DBG] pgmap v1623: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:01.158 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:00 smithi175 bash[29284]: cluster 2024-03-26T04:16:59.699068+0000 mgr.y (mgr.24431) 1213 : cluster [DBG] pgmap v1624: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:01.158 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:00 smithi175 bash[21527]: cluster 2024-03-26T04:16:59.699068+0000 mgr.y (mgr.24431) 1213 : cluster [DBG] pgmap v1624: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:01.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:00 smithi190 bash[23392]: cluster 2024-03-26T04:16:59.699068+0000 mgr.y (mgr.24431) 1213 : cluster [DBG] pgmap v1624: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:01.748 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:17:01 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:17:03.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:02 smithi175 bash[21527]: audit 2024-03-26T04:17:01.358402+0000 mgr.y (mgr.24431) 1214 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:17:03.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:02 smithi175 bash[21527]: cluster 2024-03-26T04:17:01.699985+0000 mgr.y (mgr.24431) 1215 : cluster [DBG] pgmap v1625: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:03.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:02 smithi175 bash[29284]: audit 2024-03-26T04:17:01.358402+0000 mgr.y (mgr.24431) 1214 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:17:03.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:02 smithi175 bash[29284]: cluster 2024-03-26T04:17:01.699985+0000 mgr.y (mgr.24431) 1215 : cluster [DBG] pgmap v1625: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:03.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:02 smithi190 bash[23392]: audit 2024-03-26T04:17:01.358402+0000 mgr.y (mgr.24431) 1214 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:17:03.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:02 smithi190 bash[23392]: cluster 2024-03-26T04:17:01.699985+0000 mgr.y (mgr.24431) 1215 : cluster [DBG] pgmap v1625: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:05.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:04 smithi175 bash[21527]: cluster 2024-03-26T04:17:03.700860+0000 mgr.y (mgr.24431) 1216 : cluster [DBG] pgmap v1626: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:05.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:04 smithi175 bash[29284]: cluster 2024-03-26T04:17:03.700860+0000 mgr.y (mgr.24431) 1216 : cluster [DBG] pgmap v1626: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:05.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:04 smithi190 bash[23392]: cluster 2024-03-26T04:17:03.700860+0000 mgr.y (mgr.24431) 1216 : cluster [DBG] pgmap v1626: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:07.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:06 smithi175 bash[21527]: cluster 2024-03-26T04:17:05.703057+0000 mgr.y (mgr.24431) 1217 : cluster [DBG] pgmap v1627: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:07.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:06 smithi175 bash[29284]: cluster 2024-03-26T04:17:05.703057+0000 mgr.y (mgr.24431) 1217 : cluster [DBG] pgmap v1627: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:07.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:06 smithi190 bash[23392]: cluster 2024-03-26T04:17:05.703057+0000 mgr.y (mgr.24431) 1217 : cluster [DBG] pgmap v1627: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:09.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:17:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:17:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:17:09.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:08 smithi175 bash[29284]: cluster 2024-03-26T04:17:07.703922+0000 mgr.y (mgr.24431) 1218 : cluster [DBG] pgmap v1628: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:09.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:08 smithi175 bash[29284]: audit 2024-03-26T04:17:08.142911+0000 mon.c (mon.2) 846 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:17:09.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:08 smithi175 bash[21527]: cluster 2024-03-26T04:17:07.703922+0000 mgr.y (mgr.24431) 1218 : cluster [DBG] pgmap v1628: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:09.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:08 smithi175 bash[21527]: audit 2024-03-26T04:17:08.142911+0000 mon.c (mon.2) 846 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:17:09.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:08 smithi190 bash[23392]: cluster 2024-03-26T04:17:07.703922+0000 mgr.y (mgr.24431) 1218 : cluster [DBG] pgmap v1628: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:09.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:08 smithi190 bash[23392]: audit 2024-03-26T04:17:08.142911+0000 mon.c (mon.2) 846 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:17:11.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:10 smithi175 bash[21527]: cluster 2024-03-26T04:17:09.706081+0000 mgr.y (mgr.24431) 1219 : cluster [DBG] pgmap v1629: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:11.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:10 smithi175 bash[29284]: cluster 2024-03-26T04:17:09.706081+0000 mgr.y (mgr.24431) 1219 : cluster [DBG] pgmap v1629: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:11.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:10 smithi190 bash[23392]: cluster 2024-03-26T04:17:09.706081+0000 mgr.y (mgr.24431) 1219 : cluster [DBG] pgmap v1629: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:11.748 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:17:11 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:17:13.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:12 smithi175 bash[21527]: audit 2024-03-26T04:17:11.369709+0000 mgr.y (mgr.24431) 1220 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:17:13.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:12 smithi175 bash[21527]: cluster 2024-03-26T04:17:11.706941+0000 mgr.y (mgr.24431) 1221 : cluster [DBG] pgmap v1630: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:13.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:12 smithi175 bash[29284]: audit 2024-03-26T04:17:11.369709+0000 mgr.y (mgr.24431) 1220 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:17:13.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:12 smithi175 bash[29284]: cluster 2024-03-26T04:17:11.706941+0000 mgr.y (mgr.24431) 1221 : cluster [DBG] pgmap v1630: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:13.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:12 smithi190 bash[23392]: audit 2024-03-26T04:17:11.369709+0000 mgr.y (mgr.24431) 1220 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:17:13.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:12 smithi190 bash[23392]: cluster 2024-03-26T04:17:11.706941+0000 mgr.y (mgr.24431) 1221 : cluster [DBG] pgmap v1630: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:15.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:14 smithi175 bash[21527]: cluster 2024-03-26T04:17:13.707787+0000 mgr.y (mgr.24431) 1222 : cluster [DBG] pgmap v1631: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:15.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:14 smithi175 bash[29284]: cluster 2024-03-26T04:17:13.707787+0000 mgr.y (mgr.24431) 1222 : cluster [DBG] pgmap v1631: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:15.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:14 smithi190 bash[23392]: cluster 2024-03-26T04:17:13.707787+0000 mgr.y (mgr.24431) 1222 : cluster [DBG] pgmap v1631: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:16.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:15 smithi175 bash[29284]: cluster 2024-03-26T04:17:15.709994+0000 mgr.y (mgr.24431) 1223 : cluster [DBG] pgmap v1632: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:16.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:15 smithi175 bash[21527]: cluster 2024-03-26T04:17:15.709994+0000 mgr.y (mgr.24431) 1223 : cluster [DBG] pgmap v1632: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:16.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:15 smithi190 bash[23392]: cluster 2024-03-26T04:17:15.709994+0000 mgr.y (mgr.24431) 1223 : cluster [DBG] pgmap v1632: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:18.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:17 smithi175 bash[29284]: cluster 2024-03-26T04:17:17.710826+0000 mgr.y (mgr.24431) 1224 : cluster [DBG] pgmap v1633: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:18.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:17 smithi175 bash[21527]: cluster 2024-03-26T04:17:17.710826+0000 mgr.y (mgr.24431) 1224 : cluster [DBG] pgmap v1633: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:18.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:17 smithi190 bash[23392]: cluster 2024-03-26T04:17:17.710826+0000 mgr.y (mgr.24431) 1224 : cluster [DBG] pgmap v1633: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:19.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:17:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:17:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:17:21.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:20 smithi175 bash[29284]: cluster 2024-03-26T04:17:19.713355+0000 mgr.y (mgr.24431) 1225 : cluster [DBG] pgmap v1634: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:21.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:20 smithi175 bash[21527]: cluster 2024-03-26T04:17:19.713355+0000 mgr.y (mgr.24431) 1225 : cluster [DBG] pgmap v1634: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:21.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:20 smithi190 bash[23392]: cluster 2024-03-26T04:17:19.713355+0000 mgr.y (mgr.24431) 1225 : cluster [DBG] pgmap v1634: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:21.748 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:17:21 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:17:23.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:22 smithi175 bash[21527]: audit 2024-03-26T04:17:21.378264+0000 mgr.y (mgr.24431) 1226 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:17:23.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:22 smithi175 bash[21527]: cluster 2024-03-26T04:17:21.714161+0000 mgr.y (mgr.24431) 1227 : cluster [DBG] pgmap v1635: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:23.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:22 smithi175 bash[29284]: audit 2024-03-26T04:17:21.378264+0000 mgr.y (mgr.24431) 1226 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:17:23.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:22 smithi175 bash[29284]: cluster 2024-03-26T04:17:21.714161+0000 mgr.y (mgr.24431) 1227 : cluster [DBG] pgmap v1635: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:23.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:22 smithi190 bash[23392]: audit 2024-03-26T04:17:21.378264+0000 mgr.y (mgr.24431) 1226 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:17:23.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:22 smithi190 bash[23392]: cluster 2024-03-26T04:17:21.714161+0000 mgr.y (mgr.24431) 1227 : cluster [DBG] pgmap v1635: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:24.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:23 smithi175 bash[21527]: audit 2024-03-26T04:17:23.176384+0000 mon.c (mon.2) 847 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:17:24.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:23 smithi175 bash[29284]: audit 2024-03-26T04:17:23.176384+0000 mon.c (mon.2) 847 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:17:24.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:23 smithi190 bash[23392]: audit 2024-03-26T04:17:23.176384+0000 mon.c (mon.2) 847 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:17:25.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:24 smithi175 bash[21527]: cluster 2024-03-26T04:17:23.714921+0000 mgr.y (mgr.24431) 1228 : cluster [DBG] pgmap v1636: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:25.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:24 smithi175 bash[29284]: cluster 2024-03-26T04:17:23.714921+0000 mgr.y (mgr.24431) 1228 : cluster [DBG] pgmap v1636: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:25.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:24 smithi190 bash[23392]: cluster 2024-03-26T04:17:23.714921+0000 mgr.y (mgr.24431) 1228 : cluster [DBG] pgmap v1636: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:27.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:26 smithi175 bash[21527]: cluster 2024-03-26T04:17:25.717249+0000 mgr.y (mgr.24431) 1229 : cluster [DBG] pgmap v1637: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:27.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:26 smithi175 bash[29284]: cluster 2024-03-26T04:17:25.717249+0000 mgr.y (mgr.24431) 1229 : cluster [DBG] pgmap v1637: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:27.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:26 smithi190 bash[23392]: cluster 2024-03-26T04:17:25.717249+0000 mgr.y (mgr.24431) 1229 : cluster [DBG] pgmap v1637: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:29.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:17:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:17:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:17:29.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:28 smithi175 bash[29284]: cluster 2024-03-26T04:17:27.718079+0000 mgr.y (mgr.24431) 1230 : cluster [DBG] pgmap v1638: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:29.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:28 smithi175 bash[21527]: cluster 2024-03-26T04:17:27.718079+0000 mgr.y (mgr.24431) 1230 : cluster [DBG] pgmap v1638: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:29.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:28 smithi190 bash[23392]: cluster 2024-03-26T04:17:27.718079+0000 mgr.y (mgr.24431) 1230 : cluster [DBG] pgmap v1638: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:31.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:30 smithi175 bash[21527]: cluster 2024-03-26T04:17:29.720238+0000 mgr.y (mgr.24431) 1231 : cluster [DBG] pgmap v1639: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:31.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:30 smithi175 bash[29284]: cluster 2024-03-26T04:17:29.720238+0000 mgr.y (mgr.24431) 1231 : cluster [DBG] pgmap v1639: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:31.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:30 smithi190 bash[23392]: cluster 2024-03-26T04:17:29.720238+0000 mgr.y (mgr.24431) 1231 : cluster [DBG] pgmap v1639: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:31.748 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:17:31 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:17:33.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:32 smithi175 bash[21527]: audit 2024-03-26T04:17:31.385042+0000 mgr.y (mgr.24431) 1232 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:17:33.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:32 smithi175 bash[21527]: cluster 2024-03-26T04:17:31.721043+0000 mgr.y (mgr.24431) 1233 : cluster [DBG] pgmap v1640: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:33.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:32 smithi175 bash[29284]: audit 2024-03-26T04:17:31.385042+0000 mgr.y (mgr.24431) 1232 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:17:33.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:32 smithi175 bash[29284]: cluster 2024-03-26T04:17:31.721043+0000 mgr.y (mgr.24431) 1233 : cluster [DBG] pgmap v1640: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:33.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:32 smithi190 bash[23392]: audit 2024-03-26T04:17:31.385042+0000 mgr.y (mgr.24431) 1232 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:17:33.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:32 smithi190 bash[23392]: cluster 2024-03-26T04:17:31.721043+0000 mgr.y (mgr.24431) 1233 : cluster [DBG] pgmap v1640: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:35.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:34 smithi175 bash[21527]: cluster 2024-03-26T04:17:33.721867+0000 mgr.y (mgr.24431) 1234 : cluster [DBG] pgmap v1641: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:35.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:34 smithi175 bash[29284]: cluster 2024-03-26T04:17:33.721867+0000 mgr.y (mgr.24431) 1234 : cluster [DBG] pgmap v1641: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:35.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:34 smithi190 bash[23392]: cluster 2024-03-26T04:17:33.721867+0000 mgr.y (mgr.24431) 1234 : cluster [DBG] pgmap v1641: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:37.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:36 smithi175 bash[29284]: cluster 2024-03-26T04:17:35.724123+0000 mgr.y (mgr.24431) 1235 : cluster [DBG] pgmap v1642: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:37.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:36 smithi175 bash[21527]: cluster 2024-03-26T04:17:35.724123+0000 mgr.y (mgr.24431) 1235 : cluster [DBG] pgmap v1642: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:37.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:36 smithi190 bash[23392]: cluster 2024-03-26T04:17:35.724123+0000 mgr.y (mgr.24431) 1235 : cluster [DBG] pgmap v1642: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:39.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:17:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:17:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:17:39.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:38 smithi175 bash[21527]: cluster 2024-03-26T04:17:37.724933+0000 mgr.y (mgr.24431) 1236 : cluster [DBG] pgmap v1643: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:39.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:38 smithi175 bash[21527]: audit 2024-03-26T04:17:38.210712+0000 mon.c (mon.2) 848 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:17:39.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:38 smithi175 bash[29284]: cluster 2024-03-26T04:17:37.724933+0000 mgr.y (mgr.24431) 1236 : cluster [DBG] pgmap v1643: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:39.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:38 smithi175 bash[29284]: audit 2024-03-26T04:17:38.210712+0000 mon.c (mon.2) 848 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:17:39.252 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:38 smithi190 bash[23392]: cluster 2024-03-26T04:17:37.724933+0000 mgr.y (mgr.24431) 1236 : cluster [DBG] pgmap v1643: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:39.253 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:38 smithi190 bash[23392]: audit 2024-03-26T04:17:38.210712+0000 mon.c (mon.2) 848 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:17:40.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:39 smithi175 bash[29284]: cluster 2024-03-26T04:17:39.727149+0000 mgr.y (mgr.24431) 1237 : cluster [DBG] pgmap v1644: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:40.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:39 smithi175 bash[21527]: cluster 2024-03-26T04:17:39.727149+0000 mgr.y (mgr.24431) 1237 : cluster [DBG] pgmap v1644: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:40.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:39 smithi190 bash[23392]: cluster 2024-03-26T04:17:39.727149+0000 mgr.y (mgr.24431) 1237 : cluster [DBG] pgmap v1644: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:41.748 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:17:41 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:17:43.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:42 smithi175 bash[29284]: audit 2024-03-26T04:17:41.394372+0000 mgr.y (mgr.24431) 1238 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:17:43.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:42 smithi175 bash[29284]: cluster 2024-03-26T04:17:41.728024+0000 mgr.y (mgr.24431) 1239 : cluster [DBG] pgmap v1645: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:43.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:42 smithi175 bash[21527]: audit 2024-03-26T04:17:41.394372+0000 mgr.y (mgr.24431) 1238 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:17:43.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:42 smithi175 bash[21527]: cluster 2024-03-26T04:17:41.728024+0000 mgr.y (mgr.24431) 1239 : cluster [DBG] pgmap v1645: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:43.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:42 smithi190 bash[23392]: audit 2024-03-26T04:17:41.394372+0000 mgr.y (mgr.24431) 1238 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:17:43.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:42 smithi190 bash[23392]: cluster 2024-03-26T04:17:41.728024+0000 mgr.y (mgr.24431) 1239 : cluster [DBG] pgmap v1645: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:45.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:44 smithi175 bash[29284]: cluster 2024-03-26T04:17:43.728914+0000 mgr.y (mgr.24431) 1240 : cluster [DBG] pgmap v1646: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:45.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:44 smithi175 bash[21527]: cluster 2024-03-26T04:17:43.728914+0000 mgr.y (mgr.24431) 1240 : cluster [DBG] pgmap v1646: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:45.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:44 smithi190 bash[23392]: cluster 2024-03-26T04:17:43.728914+0000 mgr.y (mgr.24431) 1240 : cluster [DBG] pgmap v1646: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:47.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:46 smithi175 bash[21527]: cluster 2024-03-26T04:17:45.731198+0000 mgr.y (mgr.24431) 1241 : cluster [DBG] pgmap v1647: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:47.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:46 smithi175 bash[29284]: cluster 2024-03-26T04:17:45.731198+0000 mgr.y (mgr.24431) 1241 : cluster [DBG] pgmap v1647: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:47.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:46 smithi190 bash[23392]: cluster 2024-03-26T04:17:45.731198+0000 mgr.y (mgr.24431) 1241 : cluster [DBG] pgmap v1647: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:48.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:47 smithi175 bash[21527]: cluster 2024-03-26T04:17:47.732024+0000 mgr.y (mgr.24431) 1242 : cluster [DBG] pgmap v1648: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:48.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:47 smithi175 bash[29284]: cluster 2024-03-26T04:17:47.732024+0000 mgr.y (mgr.24431) 1242 : cluster [DBG] pgmap v1648: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:48.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:47 smithi190 bash[23392]: cluster 2024-03-26T04:17:47.732024+0000 mgr.y (mgr.24431) 1242 : cluster [DBG] pgmap v1648: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:49.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:48 smithi175 bash[21527]: audit 2024-03-26T04:17:48.876691+0000 mon.c (mon.2) 849 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:17:49.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:17:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:17:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:17:49.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:48 smithi175 bash[29284]: audit 2024-03-26T04:17:48.876691+0000 mon.c (mon.2) 849 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:17:49.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:48 smithi190 bash[23392]: audit 2024-03-26T04:17:48.876691+0000 mon.c (mon.2) 849 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:17:50.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:49 smithi175 bash[21527]: cluster 2024-03-26T04:17:49.734335+0000 mgr.y (mgr.24431) 1243 : cluster [DBG] pgmap v1649: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:50.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:49 smithi175 bash[29284]: cluster 2024-03-26T04:17:49.734335+0000 mgr.y (mgr.24431) 1243 : cluster [DBG] pgmap v1649: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:50.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:49 smithi190 bash[23392]: cluster 2024-03-26T04:17:49.734335+0000 mgr.y (mgr.24431) 1243 : cluster [DBG] pgmap v1649: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:51.748 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:17:51 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:17:53.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:52 smithi175 bash[29284]: audit 2024-03-26T04:17:51.402198+0000 mgr.y (mgr.24431) 1244 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:17:53.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:52 smithi175 bash[29284]: cluster 2024-03-26T04:17:51.735192+0000 mgr.y (mgr.24431) 1245 : cluster [DBG] pgmap v1650: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:53.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:52 smithi175 bash[21527]: audit 2024-03-26T04:17:51.402198+0000 mgr.y (mgr.24431) 1244 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:17:53.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:52 smithi175 bash[21527]: cluster 2024-03-26T04:17:51.735192+0000 mgr.y (mgr.24431) 1245 : cluster [DBG] pgmap v1650: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:53.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:52 smithi190 bash[23392]: audit 2024-03-26T04:17:51.402198+0000 mgr.y (mgr.24431) 1244 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:17:53.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:52 smithi190 bash[23392]: cluster 2024-03-26T04:17:51.735192+0000 mgr.y (mgr.24431) 1245 : cluster [DBG] pgmap v1650: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:54.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:53 smithi175 bash[29284]: audit 2024-03-26T04:17:53.243498+0000 mon.c (mon.2) 850 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:17:54.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:53 smithi175 bash[21527]: audit 2024-03-26T04:17:53.243498+0000 mon.c (mon.2) 850 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:17:54.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:53 smithi190 bash[23392]: audit 2024-03-26T04:17:53.243498+0000 mon.c (mon.2) 850 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:17:55.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:54 smithi175 bash[21527]: cluster 2024-03-26T04:17:53.736037+0000 mgr.y (mgr.24431) 1246 : cluster [DBG] pgmap v1651: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:55.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:54 smithi175 bash[21527]: audit 2024-03-26T04:17:54.580978+0000 mon.a (mon.0) 3555 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:17:55.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:54 smithi175 bash[21527]: audit 2024-03-26T04:17:54.593647+0000 mon.a (mon.0) 3556 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:17:55.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:54 smithi175 bash[29284]: cluster 2024-03-26T04:17:53.736037+0000 mgr.y (mgr.24431) 1246 : cluster [DBG] pgmap v1651: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:55.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:54 smithi175 bash[29284]: audit 2024-03-26T04:17:54.580978+0000 mon.a (mon.0) 3555 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:17:55.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:54 smithi175 bash[29284]: audit 2024-03-26T04:17:54.593647+0000 mon.a (mon.0) 3556 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:17:55.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:54 smithi190 bash[23392]: cluster 2024-03-26T04:17:53.736037+0000 mgr.y (mgr.24431) 1246 : cluster [DBG] pgmap v1651: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:55.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:54 smithi190 bash[23392]: audit 2024-03-26T04:17:54.580978+0000 mon.a (mon.0) 3555 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:17:55.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:54 smithi190 bash[23392]: audit 2024-03-26T04:17:54.593647+0000 mon.a (mon.0) 3556 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:17:56.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:55 smithi175 bash[21527]: audit 2024-03-26T04:17:54.986887+0000 mon.a (mon.0) 3557 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:17:56.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:55 smithi175 bash[21527]: audit 2024-03-26T04:17:55.000947+0000 mon.a (mon.0) 3558 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:17:56.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:55 smithi175 bash[21527]: audit 2024-03-26T04:17:55.577837+0000 mon.c (mon.2) 851 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:17:56.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:55 smithi175 bash[21527]: audit 2024-03-26T04:17:55.579695+0000 mon.c (mon.2) 852 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:17:56.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:55 smithi175 bash[21527]: audit 2024-03-26T04:17:55.590734+0000 mon.a (mon.0) 3559 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:17:56.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:55 smithi175 bash[21527]: cluster 2024-03-26T04:17:55.738337+0000 mgr.y (mgr.24431) 1247 : cluster [DBG] pgmap v1652: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:56.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:55 smithi175 bash[29284]: audit 2024-03-26T04:17:54.986887+0000 mon.a (mon.0) 3557 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:17:56.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:55 smithi175 bash[29284]: audit 2024-03-26T04:17:55.000947+0000 mon.a (mon.0) 3558 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:17:56.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:55 smithi175 bash[29284]: audit 2024-03-26T04:17:55.577837+0000 mon.c (mon.2) 851 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:17:56.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:55 smithi175 bash[29284]: audit 2024-03-26T04:17:55.579695+0000 mon.c (mon.2) 852 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:17:56.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:55 smithi175 bash[29284]: audit 2024-03-26T04:17:55.590734+0000 mon.a (mon.0) 3559 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:17:56.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:55 smithi175 bash[29284]: cluster 2024-03-26T04:17:55.738337+0000 mgr.y (mgr.24431) 1247 : cluster [DBG] pgmap v1652: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:56.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:55 smithi190 bash[23392]: audit 2024-03-26T04:17:54.986887+0000 mon.a (mon.0) 3557 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:17:56.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:55 smithi190 bash[23392]: audit 2024-03-26T04:17:55.000947+0000 mon.a (mon.0) 3558 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:17:56.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:55 smithi190 bash[23392]: audit 2024-03-26T04:17:55.577837+0000 mon.c (mon.2) 851 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:17:56.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:55 smithi190 bash[23392]: audit 2024-03-26T04:17:55.579695+0000 mon.c (mon.2) 852 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:17:56.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:55 smithi190 bash[23392]: audit 2024-03-26T04:17:55.590734+0000 mon.a (mon.0) 3559 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:17:56.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:55 smithi190 bash[23392]: cluster 2024-03-26T04:17:55.738337+0000 mgr.y (mgr.24431) 1247 : cluster [DBG] pgmap v1652: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:17:59.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:17:58 smithi175 bash[21527]: cluster 2024-03-26T04:17:57.739217+0000 mgr.y (mgr.24431) 1248 : cluster [DBG] pgmap v1653: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:59.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:17:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:17:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:17:59.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:17:58 smithi175 bash[29284]: cluster 2024-03-26T04:17:57.739217+0000 mgr.y (mgr.24431) 1248 : cluster [DBG] pgmap v1653: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:17:59.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:17:58 smithi190 bash[23392]: cluster 2024-03-26T04:17:57.739217+0000 mgr.y (mgr.24431) 1248 : cluster [DBG] pgmap v1653: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:01.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:00 smithi175 bash[21527]: cluster 2024-03-26T04:17:59.741420+0000 mgr.y (mgr.24431) 1249 : cluster [DBG] pgmap v1654: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:01.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:00 smithi175 bash[29284]: cluster 2024-03-26T04:17:59.741420+0000 mgr.y (mgr.24431) 1249 : cluster [DBG] pgmap v1654: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:01.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:00 smithi190 bash[23392]: cluster 2024-03-26T04:17:59.741420+0000 mgr.y (mgr.24431) 1249 : cluster [DBG] pgmap v1654: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:01.748 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:18:01 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:18:03.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:02 smithi175 bash[21527]: audit 2024-03-26T04:18:01.406515+0000 mgr.y (mgr.24431) 1250 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:18:03.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:02 smithi175 bash[21527]: cluster 2024-03-26T04:18:01.742284+0000 mgr.y (mgr.24431) 1251 : cluster [DBG] pgmap v1655: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:03.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:02 smithi175 bash[29284]: audit 2024-03-26T04:18:01.406515+0000 mgr.y (mgr.24431) 1250 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:18:03.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:02 smithi175 bash[29284]: cluster 2024-03-26T04:18:01.742284+0000 mgr.y (mgr.24431) 1251 : cluster [DBG] pgmap v1655: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:03.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:02 smithi190 bash[23392]: audit 2024-03-26T04:18:01.406515+0000 mgr.y (mgr.24431) 1250 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:18:03.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:02 smithi190 bash[23392]: cluster 2024-03-26T04:18:01.742284+0000 mgr.y (mgr.24431) 1251 : cluster [DBG] pgmap v1655: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:05.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:04 smithi175 bash[29284]: cluster 2024-03-26T04:18:03.743094+0000 mgr.y (mgr.24431) 1252 : cluster [DBG] pgmap v1656: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:05.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:04 smithi175 bash[21527]: cluster 2024-03-26T04:18:03.743094+0000 mgr.y (mgr.24431) 1252 : cluster [DBG] pgmap v1656: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:05.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:04 smithi190 bash[23392]: cluster 2024-03-26T04:18:03.743094+0000 mgr.y (mgr.24431) 1252 : cluster [DBG] pgmap v1656: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:07.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:06 smithi175 bash[29284]: cluster 2024-03-26T04:18:05.745291+0000 mgr.y (mgr.24431) 1253 : cluster [DBG] pgmap v1657: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:07.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:06 smithi175 bash[21527]: cluster 2024-03-26T04:18:05.745291+0000 mgr.y (mgr.24431) 1253 : cluster [DBG] pgmap v1657: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:07.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:06 smithi190 bash[23392]: cluster 2024-03-26T04:18:05.745291+0000 mgr.y (mgr.24431) 1253 : cluster [DBG] pgmap v1657: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:09.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:18:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:18:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:18:09.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:08 smithi175 bash[21527]: cluster 2024-03-26T04:18:07.746102+0000 mgr.y (mgr.24431) 1254 : cluster [DBG] pgmap v1658: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:09.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:08 smithi175 bash[21527]: audit 2024-03-26T04:18:08.278115+0000 mon.c (mon.2) 853 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:18:09.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:08 smithi175 bash[29284]: cluster 2024-03-26T04:18:07.746102+0000 mgr.y (mgr.24431) 1254 : cluster [DBG] pgmap v1658: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:09.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:08 smithi175 bash[29284]: audit 2024-03-26T04:18:08.278115+0000 mon.c (mon.2) 853 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:18:09.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:08 smithi190 bash[23392]: cluster 2024-03-26T04:18:07.746102+0000 mgr.y (mgr.24431) 1254 : cluster [DBG] pgmap v1658: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:09.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:08 smithi190 bash[23392]: audit 2024-03-26T04:18:08.278115+0000 mon.c (mon.2) 853 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:18:11.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:10 smithi175 bash[21527]: cluster 2024-03-26T04:18:09.748259+0000 mgr.y (mgr.24431) 1255 : cluster [DBG] pgmap v1659: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:11.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:10 smithi175 bash[29284]: cluster 2024-03-26T04:18:09.748259+0000 mgr.y (mgr.24431) 1255 : cluster [DBG] pgmap v1659: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:11.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:10 smithi190 bash[23392]: cluster 2024-03-26T04:18:09.748259+0000 mgr.y (mgr.24431) 1255 : cluster [DBG] pgmap v1659: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:11.748 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:18:11 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:18:12.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:11 smithi175 bash[21527]: audit 2024-03-26T04:18:11.417862+0000 mgr.y (mgr.24431) 1256 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:18:12.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:11 smithi175 bash[21527]: cluster 2024-03-26T04:18:11.749052+0000 mgr.y (mgr.24431) 1257 : cluster [DBG] pgmap v1660: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:12.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:11 smithi175 bash[29284]: audit 2024-03-26T04:18:11.417862+0000 mgr.y (mgr.24431) 1256 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:18:12.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:11 smithi175 bash[29284]: cluster 2024-03-26T04:18:11.749052+0000 mgr.y (mgr.24431) 1257 : cluster [DBG] pgmap v1660: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:12.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:11 smithi190 bash[23392]: audit 2024-03-26T04:18:11.417862+0000 mgr.y (mgr.24431) 1256 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:18:12.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:11 smithi190 bash[23392]: cluster 2024-03-26T04:18:11.749052+0000 mgr.y (mgr.24431) 1257 : cluster [DBG] pgmap v1660: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:15.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:14 smithi175 bash[21527]: cluster 2024-03-26T04:18:13.749877+0000 mgr.y (mgr.24431) 1258 : cluster [DBG] pgmap v1661: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:15.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:14 smithi175 bash[29284]: cluster 2024-03-26T04:18:13.749877+0000 mgr.y (mgr.24431) 1258 : cluster [DBG] pgmap v1661: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:15.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:14 smithi190 bash[23392]: cluster 2024-03-26T04:18:13.749877+0000 mgr.y (mgr.24431) 1258 : cluster [DBG] pgmap v1661: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:17.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:16 smithi175 bash[21527]: cluster 2024-03-26T04:18:15.752109+0000 mgr.y (mgr.24431) 1259 : cluster [DBG] pgmap v1662: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:17.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:16 smithi175 bash[29284]: cluster 2024-03-26T04:18:15.752109+0000 mgr.y (mgr.24431) 1259 : cluster [DBG] pgmap v1662: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:17.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:16 smithi190 bash[23392]: cluster 2024-03-26T04:18:15.752109+0000 mgr.y (mgr.24431) 1259 : cluster [DBG] pgmap v1662: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:19.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:18:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:18:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:18:19.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:18 smithi175 bash[29284]: cluster 2024-03-26T04:18:17.752866+0000 mgr.y (mgr.24431) 1260 : cluster [DBG] pgmap v1663: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:19.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:18 smithi175 bash[21527]: cluster 2024-03-26T04:18:17.752866+0000 mgr.y (mgr.24431) 1260 : cluster [DBG] pgmap v1663: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:19.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:18 smithi190 bash[23392]: cluster 2024-03-26T04:18:17.752866+0000 mgr.y (mgr.24431) 1260 : cluster [DBG] pgmap v1663: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:21.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:20 smithi175 bash[29284]: cluster 2024-03-26T04:18:19.754932+0000 mgr.y (mgr.24431) 1261 : cluster [DBG] pgmap v1664: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:21.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:20 smithi175 bash[21527]: cluster 2024-03-26T04:18:19.754932+0000 mgr.y (mgr.24431) 1261 : cluster [DBG] pgmap v1664: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:21.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:20 smithi190 bash[23392]: cluster 2024-03-26T04:18:19.754932+0000 mgr.y (mgr.24431) 1261 : cluster [DBG] pgmap v1664: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:21.748 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:18:21 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:18:23.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:22 smithi175 bash[21527]: audit 2024-03-26T04:18:21.419263+0000 mgr.y (mgr.24431) 1262 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:18:23.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:22 smithi175 bash[21527]: cluster 2024-03-26T04:18:21.755813+0000 mgr.y (mgr.24431) 1263 : cluster [DBG] pgmap v1665: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:23.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:22 smithi175 bash[29284]: audit 2024-03-26T04:18:21.419263+0000 mgr.y (mgr.24431) 1262 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:18:23.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:22 smithi175 bash[29284]: cluster 2024-03-26T04:18:21.755813+0000 mgr.y (mgr.24431) 1263 : cluster [DBG] pgmap v1665: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:23.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:22 smithi190 bash[23392]: audit 2024-03-26T04:18:21.419263+0000 mgr.y (mgr.24431) 1262 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:18:23.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:22 smithi190 bash[23392]: cluster 2024-03-26T04:18:21.755813+0000 mgr.y (mgr.24431) 1263 : cluster [DBG] pgmap v1665: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:24.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:23 smithi175 bash[21527]: audit 2024-03-26T04:18:23.311468+0000 mon.c (mon.2) 854 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:18:24.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:23 smithi175 bash[29284]: audit 2024-03-26T04:18:23.311468+0000 mon.c (mon.2) 854 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:18:24.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:23 smithi190 bash[23392]: audit 2024-03-26T04:18:23.311468+0000 mon.c (mon.2) 854 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:18:25.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:24 smithi175 bash[29284]: cluster 2024-03-26T04:18:23.756608+0000 mgr.y (mgr.24431) 1264 : cluster [DBG] pgmap v1666: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:25.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:24 smithi175 bash[21527]: cluster 2024-03-26T04:18:23.756608+0000 mgr.y (mgr.24431) 1264 : cluster [DBG] pgmap v1666: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:25.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:24 smithi190 bash[23392]: cluster 2024-03-26T04:18:23.756608+0000 mgr.y (mgr.24431) 1264 : cluster [DBG] pgmap v1666: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:26.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:25 smithi175 bash[21527]: cluster 2024-03-26T04:18:25.758783+0000 mgr.y (mgr.24431) 1265 : cluster [DBG] pgmap v1667: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:26.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:25 smithi175 bash[29284]: cluster 2024-03-26T04:18:25.758783+0000 mgr.y (mgr.24431) 1265 : cluster [DBG] pgmap v1667: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:26.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:25 smithi190 bash[23392]: cluster 2024-03-26T04:18:25.758783+0000 mgr.y (mgr.24431) 1265 : cluster [DBG] pgmap v1667: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:29.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:28 smithi175 bash[21527]: cluster 2024-03-26T04:18:27.759738+0000 mgr.y (mgr.24431) 1266 : cluster [DBG] pgmap v1668: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T04:18:29.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:18:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:18:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:18:29.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:28 smithi175 bash[29284]: cluster 2024-03-26T04:18:27.759738+0000 mgr.y (mgr.24431) 1266 : cluster [DBG] pgmap v1668: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T04:18:29.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:28 smithi190 bash[23392]: cluster 2024-03-26T04:18:27.759738+0000 mgr.y (mgr.24431) 1266 : cluster [DBG] pgmap v1668: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T04:18:31.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:30 smithi175 bash[29284]: cluster 2024-03-26T04:18:29.761987+0000 mgr.y (mgr.24431) 1267 : cluster [DBG] pgmap v1669: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:31.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:30 smithi175 bash[21527]: cluster 2024-03-26T04:18:29.761987+0000 mgr.y (mgr.24431) 1267 : cluster [DBG] pgmap v1669: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:31.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:30 smithi190 bash[23392]: cluster 2024-03-26T04:18:29.761987+0000 mgr.y (mgr.24431) 1267 : cluster [DBG] pgmap v1669: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:31.748 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:18:31 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:18:33.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:32 smithi175 bash[21527]: audit 2024-03-26T04:18:31.430387+0000 mgr.y (mgr.24431) 1268 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:18:33.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:32 smithi175 bash[21527]: cluster 2024-03-26T04:18:31.762957+0000 mgr.y (mgr.24431) 1269 : cluster [DBG] pgmap v1670: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T04:18:33.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:32 smithi175 bash[29284]: audit 2024-03-26T04:18:31.430387+0000 mgr.y (mgr.24431) 1268 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:18:33.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:32 smithi175 bash[29284]: cluster 2024-03-26T04:18:31.762957+0000 mgr.y (mgr.24431) 1269 : cluster [DBG] pgmap v1670: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T04:18:33.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:32 smithi190 bash[23392]: audit 2024-03-26T04:18:31.430387+0000 mgr.y (mgr.24431) 1268 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:18:33.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:32 smithi190 bash[23392]: cluster 2024-03-26T04:18:31.762957+0000 mgr.y (mgr.24431) 1269 : cluster [DBG] pgmap v1670: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T04:18:34.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:33 smithi175 bash[21527]: cluster 2024-03-26T04:18:33.763889+0000 mgr.y (mgr.24431) 1270 : cluster [DBG] pgmap v1671: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T04:18:34.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:33 smithi175 bash[29284]: cluster 2024-03-26T04:18:33.763889+0000 mgr.y (mgr.24431) 1270 : cluster [DBG] pgmap v1671: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T04:18:34.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:33 smithi190 bash[23392]: cluster 2024-03-26T04:18:33.763889+0000 mgr.y (mgr.24431) 1270 : cluster [DBG] pgmap v1671: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-26T04:18:37.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:36 smithi175 bash[29284]: cluster 2024-03-26T04:18:35.766240+0000 mgr.y (mgr.24431) 1271 : cluster [DBG] pgmap v1672: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:37.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:36 smithi175 bash[21527]: cluster 2024-03-26T04:18:35.766240+0000 mgr.y (mgr.24431) 1271 : cluster [DBG] pgmap v1672: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:37.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:36 smithi190 bash[23392]: cluster 2024-03-26T04:18:35.766240+0000 mgr.y (mgr.24431) 1271 : cluster [DBG] pgmap v1672: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:39.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:18:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:18:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:18:39.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:38 smithi175 bash[21527]: cluster 2024-03-26T04:18:37.767219+0000 mgr.y (mgr.24431) 1272 : cluster [DBG] pgmap v1673: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:39.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:38 smithi175 bash[21527]: audit 2024-03-26T04:18:38.345119+0000 mon.c (mon.2) 855 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:18:39.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:38 smithi175 bash[29284]: cluster 2024-03-26T04:18:37.767219+0000 mgr.y (mgr.24431) 1272 : cluster [DBG] pgmap v1673: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:39.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:38 smithi175 bash[29284]: audit 2024-03-26T04:18:38.345119+0000 mon.c (mon.2) 855 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:18:39.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:38 smithi190 bash[23392]: cluster 2024-03-26T04:18:37.767219+0000 mgr.y (mgr.24431) 1272 : cluster [DBG] pgmap v1673: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:39.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:38 smithi190 bash[23392]: audit 2024-03-26T04:18:38.345119+0000 mon.c (mon.2) 855 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:18:41.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:40 smithi175 bash[29284]: cluster 2024-03-26T04:18:39.769471+0000 mgr.y (mgr.24431) 1273 : cluster [DBG] pgmap v1674: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:41.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:40 smithi175 bash[21527]: cluster 2024-03-26T04:18:39.769471+0000 mgr.y (mgr.24431) 1273 : cluster [DBG] pgmap v1674: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:41.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:40 smithi190 bash[23392]: cluster 2024-03-26T04:18:39.769471+0000 mgr.y (mgr.24431) 1273 : cluster [DBG] pgmap v1674: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:41.747 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:18:41 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:18:42.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:41 smithi175 bash[29284]: audit 2024-03-26T04:18:41.441897+0000 mgr.y (mgr.24431) 1274 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:18:42.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:41 smithi175 bash[29284]: cluster 2024-03-26T04:18:41.770298+0000 mgr.y (mgr.24431) 1275 : cluster [DBG] pgmap v1675: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:42.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:41 smithi175 bash[21527]: audit 2024-03-26T04:18:41.441897+0000 mgr.y (mgr.24431) 1274 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:18:42.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:41 smithi175 bash[21527]: cluster 2024-03-26T04:18:41.770298+0000 mgr.y (mgr.24431) 1275 : cluster [DBG] pgmap v1675: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:42.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:41 smithi190 bash[23392]: audit 2024-03-26T04:18:41.441897+0000 mgr.y (mgr.24431) 1274 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:18:42.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:41 smithi190 bash[23392]: cluster 2024-03-26T04:18:41.770298+0000 mgr.y (mgr.24431) 1275 : cluster [DBG] pgmap v1675: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:45.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:44 smithi175 bash[21527]: cluster 2024-03-26T04:18:43.771119+0000 mgr.y (mgr.24431) 1276 : cluster [DBG] pgmap v1676: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:45.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:44 smithi175 bash[29284]: cluster 2024-03-26T04:18:43.771119+0000 mgr.y (mgr.24431) 1276 : cluster [DBG] pgmap v1676: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:45.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:44 smithi190 bash[23392]: cluster 2024-03-26T04:18:43.771119+0000 mgr.y (mgr.24431) 1276 : cluster [DBG] pgmap v1676: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:47.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:46 smithi175 bash[29284]: cluster 2024-03-26T04:18:45.773357+0000 mgr.y (mgr.24431) 1277 : cluster [DBG] pgmap v1677: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:47.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:46 smithi175 bash[21527]: cluster 2024-03-26T04:18:45.773357+0000 mgr.y (mgr.24431) 1277 : cluster [DBG] pgmap v1677: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:47.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:46 smithi190 bash[23392]: cluster 2024-03-26T04:18:45.773357+0000 mgr.y (mgr.24431) 1277 : cluster [DBG] pgmap v1677: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:48.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:47 smithi175 bash[29284]: cluster 2024-03-26T04:18:47.774203+0000 mgr.y (mgr.24431) 1278 : cluster [DBG] pgmap v1678: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:48.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:47 smithi175 bash[21527]: cluster 2024-03-26T04:18:47.774203+0000 mgr.y (mgr.24431) 1278 : cluster [DBG] pgmap v1678: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:48.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:47 smithi190 bash[23392]: cluster 2024-03-26T04:18:47.774203+0000 mgr.y (mgr.24431) 1278 : cluster [DBG] pgmap v1678: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:49.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:18:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:18:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:18:51.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:50 smithi175 bash[21527]: cluster 2024-03-26T04:18:49.776363+0000 mgr.y (mgr.24431) 1279 : cluster [DBG] pgmap v1679: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:51.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:50 smithi175 bash[29284]: cluster 2024-03-26T04:18:49.776363+0000 mgr.y (mgr.24431) 1279 : cluster [DBG] pgmap v1679: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:51.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:50 smithi190 bash[23392]: cluster 2024-03-26T04:18:49.776363+0000 mgr.y (mgr.24431) 1279 : cluster [DBG] pgmap v1679: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:51.748 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:18:51 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:18:52.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:51 smithi175 bash[21527]: audit 2024-03-26T04:18:51.445910+0000 mgr.y (mgr.24431) 1280 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:18:52.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:51 smithi175 bash[21527]: cluster 2024-03-26T04:18:51.777186+0000 mgr.y (mgr.24431) 1281 : cluster [DBG] pgmap v1680: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:52.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:51 smithi175 bash[29284]: audit 2024-03-26T04:18:51.445910+0000 mgr.y (mgr.24431) 1280 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:18:52.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:51 smithi175 bash[29284]: cluster 2024-03-26T04:18:51.777186+0000 mgr.y (mgr.24431) 1281 : cluster [DBG] pgmap v1680: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:52.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:51 smithi190 bash[23392]: audit 2024-03-26T04:18:51.445910+0000 mgr.y (mgr.24431) 1280 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:18:52.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:51 smithi190 bash[23392]: cluster 2024-03-26T04:18:51.777186+0000 mgr.y (mgr.24431) 1281 : cluster [DBG] pgmap v1680: 228 pgs: 228 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:53.740 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:53 smithi175 bash[21527]: audit 2024-03-26T04:18:53.378865+0000 mon.c (mon.2) 856 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:18:53.740 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:53 smithi175 bash[29284]: audit 2024-03-26T04:18:53.378865+0000 mon.c (mon.2) 856 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:18:53.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:53 smithi190 bash[23392]: audit 2024-03-26T04:18:53.378865+0000 mon.c (mon.2) 856 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:18:54.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:54 smithi175 bash[21527]: cluster 2024-03-26T04:18:53.467514+0000 mon.a (mon.0) 3560 : cluster [DBG] osdmap e747: 8 total, 8 up, 8 in 2024-03-26T04:18:54.744 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:54 smithi175 bash[21527]: cluster 2024-03-26T04:18:53.778022+0000 mgr.y (mgr.24431) 1282 : cluster [DBG] pgmap v1682: 196 pgs: 196 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T04:18:54.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:54 smithi175 bash[29284]: cluster 2024-03-26T04:18:53.467514+0000 mon.a (mon.0) 3560 : cluster [DBG] osdmap e747: 8 total, 8 up, 8 in 2024-03-26T04:18:54.744 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:54 smithi175 bash[29284]: cluster 2024-03-26T04:18:53.778022+0000 mgr.y (mgr.24431) 1282 : cluster [DBG] pgmap v1682: 196 pgs: 196 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T04:18:54.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:54 smithi190 bash[23392]: cluster 2024-03-26T04:18:53.467514+0000 mon.a (mon.0) 3560 : cluster [DBG] osdmap e747: 8 total, 8 up, 8 in 2024-03-26T04:18:54.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:54 smithi190 bash[23392]: cluster 2024-03-26T04:18:53.778022+0000 mgr.y (mgr.24431) 1282 : cluster [DBG] pgmap v1682: 196 pgs: 196 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T04:18:55.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:55 smithi175 bash[21527]: cluster 2024-03-26T04:18:54.457059+0000 mon.a (mon.0) 3561 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:18:55.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:55 smithi175 bash[21527]: cluster 2024-03-26T04:18:54.478556+0000 mon.a (mon.0) 3562 : cluster [DBG] osdmap e748: 8 total, 8 up, 8 in 2024-03-26T04:18:55.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:55 smithi175 bash[29284]: cluster 2024-03-26T04:18:54.457059+0000 mon.a (mon.0) 3561 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:18:55.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:55 smithi175 bash[29284]: cluster 2024-03-26T04:18:54.478556+0000 mon.a (mon.0) 3562 : cluster [DBG] osdmap e748: 8 total, 8 up, 8 in 2024-03-26T04:18:55.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:55 smithi190 bash[23392]: cluster 2024-03-26T04:18:54.457059+0000 mon.a (mon.0) 3561 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:18:55.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:55 smithi190 bash[23392]: cluster 2024-03-26T04:18:54.478556+0000 mon.a (mon.0) 3562 : cluster [DBG] osdmap e748: 8 total, 8 up, 8 in 2024-03-26T04:18:56.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:56 smithi175 bash[21527]: cluster 2024-03-26T04:18:55.488994+0000 mon.a (mon.0) 3563 : cluster [DBG] osdmap e749: 8 total, 8 up, 8 in 2024-03-26T04:18:56.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:56 smithi175 bash[21527]: audit 2024-03-26T04:18:55.680171+0000 mon.c (mon.2) 857 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:18:56.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:56 smithi175 bash[21527]: cluster 2024-03-26T04:18:55.778742+0000 mgr.y (mgr.24431) 1283 : cluster [DBG] pgmap v1685: 228 pgs: 32 unknown, 32 creating+peering, 164 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:56.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:56 smithi175 bash[21527]: audit 2024-03-26T04:18:56.416209+0000 mon.c (mon.2) 858 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:18:56.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:56 smithi175 bash[21527]: audit 2024-03-26T04:18:56.418250+0000 mon.c (mon.2) 859 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:18:56.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:56 smithi175 bash[21527]: audit 2024-03-26T04:18:56.428982+0000 mon.a (mon.0) 3564 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:18:56.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:56 smithi175 bash[21527]: cluster 2024-03-26T04:18:56.484321+0000 mon.a (mon.0) 3565 : cluster [DBG] osdmap e750: 8 total, 8 up, 8 in 2024-03-26T04:18:56.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:56 smithi175 bash[29284]: cluster 2024-03-26T04:18:55.488994+0000 mon.a (mon.0) 3563 : cluster [DBG] osdmap e749: 8 total, 8 up, 8 in 2024-03-26T04:18:56.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:56 smithi175 bash[29284]: audit 2024-03-26T04:18:55.680171+0000 mon.c (mon.2) 857 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:18:56.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:56 smithi175 bash[29284]: cluster 2024-03-26T04:18:55.778742+0000 mgr.y (mgr.24431) 1283 : cluster [DBG] pgmap v1685: 228 pgs: 32 unknown, 32 creating+peering, 164 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:56.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:56 smithi175 bash[29284]: audit 2024-03-26T04:18:56.416209+0000 mon.c (mon.2) 858 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:18:56.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:56 smithi175 bash[29284]: audit 2024-03-26T04:18:56.418250+0000 mon.c (mon.2) 859 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:18:56.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:56 smithi175 bash[29284]: audit 2024-03-26T04:18:56.428982+0000 mon.a (mon.0) 3564 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:18:56.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:56 smithi175 bash[29284]: cluster 2024-03-26T04:18:56.484321+0000 mon.a (mon.0) 3565 : cluster [DBG] osdmap e750: 8 total, 8 up, 8 in 2024-03-26T04:18:56.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:56 smithi190 bash[23392]: cluster 2024-03-26T04:18:55.488994+0000 mon.a (mon.0) 3563 : cluster [DBG] osdmap e749: 8 total, 8 up, 8 in 2024-03-26T04:18:56.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:56 smithi190 bash[23392]: audit 2024-03-26T04:18:55.680171+0000 mon.c (mon.2) 857 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:18:56.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:56 smithi190 bash[23392]: cluster 2024-03-26T04:18:55.778742+0000 mgr.y (mgr.24431) 1283 : cluster [DBG] pgmap v1685: 228 pgs: 32 unknown, 32 creating+peering, 164 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:18:56.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:56 smithi190 bash[23392]: audit 2024-03-26T04:18:56.416209+0000 mon.c (mon.2) 858 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:18:56.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:56 smithi190 bash[23392]: audit 2024-03-26T04:18:56.418250+0000 mon.c (mon.2) 859 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:18:56.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:56 smithi190 bash[23392]: audit 2024-03-26T04:18:56.428982+0000 mon.a (mon.0) 3564 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:18:56.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:56 smithi190 bash[23392]: cluster 2024-03-26T04:18:56.484321+0000 mon.a (mon.0) 3565 : cluster [DBG] osdmap e750: 8 total, 8 up, 8 in 2024-03-26T04:18:58.740 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:58 smithi175 bash[29284]: cluster 2024-03-26T04:18:57.496801+0000 mon.a (mon.0) 3566 : cluster [DBG] osdmap e751: 8 total, 8 up, 8 in 2024-03-26T04:18:58.740 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:58 smithi175 bash[29284]: cluster 2024-03-26T04:18:57.779621+0000 mgr.y (mgr.24431) 1284 : cluster [DBG] pgmap v1688: 228 pgs: 64 unknown, 164 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:58.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:58 smithi175 bash[21527]: cluster 2024-03-26T04:18:57.496801+0000 mon.a (mon.0) 3566 : cluster [DBG] osdmap e751: 8 total, 8 up, 8 in 2024-03-26T04:18:58.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:58 smithi175 bash[21527]: cluster 2024-03-26T04:18:57.779621+0000 mgr.y (mgr.24431) 1284 : cluster [DBG] pgmap v1688: 228 pgs: 64 unknown, 164 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:58.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:58 smithi190 bash[23392]: cluster 2024-03-26T04:18:57.496801+0000 mon.a (mon.0) 3566 : cluster [DBG] osdmap e751: 8 total, 8 up, 8 in 2024-03-26T04:18:58.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:58 smithi190 bash[23392]: cluster 2024-03-26T04:18:57.779621+0000 mgr.y (mgr.24431) 1284 : cluster [DBG] pgmap v1688: 228 pgs: 64 unknown, 164 active+clean; 582 KiB data, 774 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:18:59.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:18:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:18:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:18:59.740 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:18:59 smithi175 bash[29284]: cluster 2024-03-26T04:18:58.499288+0000 mon.a (mon.0) 3567 : cluster [DBG] osdmap e752: 8 total, 8 up, 8 in 2024-03-26T04:18:59.740 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:18:59 smithi175 bash[21527]: cluster 2024-03-26T04:18:58.499288+0000 mon.a (mon.0) 3567 : cluster [DBG] osdmap e752: 8 total, 8 up, 8 in 2024-03-26T04:18:59.750 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:18:59 smithi190 bash[23392]: cluster 2024-03-26T04:18:58.499288+0000 mon.a (mon.0) 3567 : cluster [DBG] osdmap e752: 8 total, 8 up, 8 in 2024-03-26T04:19:00.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:00 smithi175 bash[29284]: cluster 2024-03-26T04:18:59.511916+0000 mon.a (mon.0) 3568 : cluster [DBG] osdmap e753: 8 total, 8 up, 8 in 2024-03-26T04:19:00.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:00 smithi175 bash[29284]: cluster 2024-03-26T04:18:59.781524+0000 mgr.y (mgr.24431) 1285 : cluster [DBG] pgmap v1691: 196 pgs: 196 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:00.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:00 smithi175 bash[21527]: cluster 2024-03-26T04:18:59.511916+0000 mon.a (mon.0) 3568 : cluster [DBG] osdmap e753: 8 total, 8 up, 8 in 2024-03-26T04:19:00.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:00 smithi175 bash[21527]: cluster 2024-03-26T04:18:59.781524+0000 mgr.y (mgr.24431) 1285 : cluster [DBG] pgmap v1691: 196 pgs: 196 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:00.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:00 smithi190 bash[23392]: cluster 2024-03-26T04:18:59.511916+0000 mon.a (mon.0) 3568 : cluster [DBG] osdmap e753: 8 total, 8 up, 8 in 2024-03-26T04:19:00.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:00 smithi190 bash[23392]: cluster 2024-03-26T04:18:59.781524+0000 mgr.y (mgr.24431) 1285 : cluster [DBG] pgmap v1691: 196 pgs: 196 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:01.544 INFO:tasks.workunit.client.0.smithi175.stdout: watch_notify: Running main() from gmock_main.cc 2024-03-26T04:19:01.544 INFO:tasks.workunit.client.0.smithi175.stdout: watch_notify: [==========] Running 2 tests from 1 test suite. 2024-03-26T04:19:01.544 INFO:tasks.workunit.client.0.smithi175.stdout: watch_notify: [----------] Global test environment set-up. 2024-03-26T04:19:01.544 INFO:tasks.workunit.client.0.smithi175.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify 2024-03-26T04:19:01.544 INFO:tasks.workunit.client.0.smithi175.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotify 2024-03-26T04:19:01.545 INFO:tasks.workunit.client.0.smithi175.stdout: watch_notify: handle_notify cookie 94738551388048 notify_id 3208340570115 notifier_gid 24854 2024-03-26T04:19:01.545 INFO:tasks.workunit.client.0.smithi175.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotify (1801851 ms) 2024-03-26T04:19:01.545 INFO:tasks.workunit.client.0.smithi175.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotifyTimeout 2024-03-26T04:19:01.545 INFO:tasks.workunit.client.0.smithi175.stdout: watch_notify: Trying... 2024-03-26T04:19:01.545 INFO:tasks.workunit.client.0.smithi175.stdout: watch_notify: handle_notify cookie 94738564082704 notify_id 3221225472008 notifier_gid 42434 2024-03-26T04:19:01.545 INFO:tasks.workunit.client.0.smithi175.stdout: watch_notify: Waiting for 3.000000000s 2024-03-26T04:19:01.545 INFO:tasks.workunit.client.0.smithi175.stdout: watch_notify: Timed out. 2024-03-26T04:19:01.545 INFO:tasks.workunit.client.0.smithi175.stdout: watch_notify: Flushing... 2024-03-26T04:19:01.545 INFO:tasks.workunit.client.0.smithi175.stdout: watch_notify: Flushed... 2024-03-26T04:19:01.545 INFO:tasks.workunit.client.0.smithi175.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotifyTimeout (7060 ms) 2024-03-26T04:19:01.545 INFO:tasks.workunit.client.0.smithi175.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify (1808911 ms total) 2024-03-26T04:19:01.545 INFO:tasks.workunit.client.0.smithi175.stdout: watch_notify: 2024-03-26T04:19:01.545 INFO:tasks.workunit.client.0.smithi175.stdout: watch_notify: [----------] Global test environment tear-down 2024-03-26T04:19:01.545 INFO:tasks.workunit.client.0.smithi175.stdout: watch_notify: [==========] 2 tests from 1 test suite ran. (1808911 ms total) 2024-03-26T04:19:01.546 INFO:tasks.workunit.client.0.smithi175.stdout: watch_notify: [ PASSED ] 2 tests. 2024-03-26T04:19:01.546 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:19:01.546 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64647 2024-03-26T04:19:01.546 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 64647 2024-03-26T04:19:01.546 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:19:01.546 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64953 2024-03-26T04:19:01.546 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 64953 2024-03-26T04:19:01.546 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:19:01.546 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=65291 2024-03-26T04:19:01.547 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 65291 2024-03-26T04:19:01.547 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:19:01.547 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=65030 2024-03-26T04:19:01.547 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 65030 2024-03-26T04:19:01.547 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:19:01.547 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=65336 2024-03-26T04:19:01.547 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 65336 2024-03-26T04:19:01.547 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:19:01.547 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64906 2024-03-26T04:19:01.547 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 64906 2024-03-26T04:19:01.547 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:19:01.547 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64588 2024-03-26T04:19:01.547 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 64588 2024-03-26T04:19:01.547 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:19:01.547 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=65367 2024-03-26T04:19:01.548 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 65367 2024-03-26T04:19:01.548 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:19:01.548 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64855 2024-03-26T04:19:01.548 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 64855 2024-03-26T04:19:01.548 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:19:01.548 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64393 2024-03-26T04:19:01.548 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 64393 2024-03-26T04:19:01.548 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:19:01.548 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64433 2024-03-26T04:19:01.548 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 64433 2024-03-26T04:19:01.548 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:19:01.548 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64779 2024-03-26T04:19:01.548 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 64779 2024-03-26T04:19:01.548 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:19:01.548 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64495 2024-03-26T04:19:01.548 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 64495 2024-03-26T04:19:01.549 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:19:01.549 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64865 2024-03-26T04:19:01.549 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 64865 2024-03-26T04:19:01.549 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:19:01.549 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=64878 2024-03-26T04:19:01.549 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 64878 2024-03-26T04:19:01.549 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:19:01.549 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=65268 2024-03-26T04:19:01.549 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 65268 2024-03-26T04:19:01.549 INFO:tasks.workunit.client.0.smithi175.stderr:+ for t in "${!pids[@]}" 2024-03-26T04:19:01.549 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=65456 2024-03-26T04:19:01.549 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 65456 2024-03-26T04:19:01.748 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:19:01 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:19:01.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:01 smithi190 bash[23392]: cluster 2024-03-26T04:19:00.503509+0000 mon.a (mon.0) 3569 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:19:01.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:01 smithi190 bash[23392]: cluster 2024-03-26T04:19:00.524937+0000 mon.a (mon.0) 3570 : cluster [DBG] osdmap e754: 8 total, 8 up, 8 in 2024-03-26T04:19:01.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:01 smithi175 bash[21527]: cluster 2024-03-26T04:19:00.503509+0000 mon.a (mon.0) 3569 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:19:01.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:01 smithi175 bash[21527]: cluster 2024-03-26T04:19:00.524937+0000 mon.a (mon.0) 3570 : cluster [DBG] osdmap e754: 8 total, 8 up, 8 in 2024-03-26T04:19:01.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:01 smithi175 bash[29284]: cluster 2024-03-26T04:19:00.503509+0000 mon.a (mon.0) 3569 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:19:01.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:01 smithi175 bash[29284]: cluster 2024-03-26T04:19:00.524937+0000 mon.a (mon.0) 3570 : cluster [DBG] osdmap e754: 8 total, 8 up, 8 in 2024-03-26T04:19:02.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:02 smithi175 bash[21527]: audit 2024-03-26T04:19:01.453992+0000 mgr.y (mgr.24431) 1286 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:19:02.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:02 smithi175 bash[21527]: cluster 2024-03-26T04:19:01.536585+0000 mon.a (mon.0) 3571 : cluster [DBG] osdmap e755: 8 total, 8 up, 8 in 2024-03-26T04:19:02.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:02 smithi175 bash[21527]: cluster 2024-03-26T04:19:01.782332+0000 mgr.y (mgr.24431) 1287 : cluster [DBG] pgmap v1694: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:02.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:02 smithi175 bash[29284]: audit 2024-03-26T04:19:01.453992+0000 mgr.y (mgr.24431) 1286 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:19:02.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:02 smithi175 bash[29284]: cluster 2024-03-26T04:19:01.536585+0000 mon.a (mon.0) 3571 : cluster [DBG] osdmap e755: 8 total, 8 up, 8 in 2024-03-26T04:19:02.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:02 smithi175 bash[29284]: cluster 2024-03-26T04:19:01.782332+0000 mgr.y (mgr.24431) 1287 : cluster [DBG] pgmap v1694: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:02.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:02 smithi190 bash[23392]: audit 2024-03-26T04:19:01.453992+0000 mgr.y (mgr.24431) 1286 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:19:02.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:02 smithi190 bash[23392]: cluster 2024-03-26T04:19:01.536585+0000 mon.a (mon.0) 3571 : cluster [DBG] osdmap e755: 8 total, 8 up, 8 in 2024-03-26T04:19:02.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:02 smithi190 bash[23392]: cluster 2024-03-26T04:19:01.782332+0000 mgr.y (mgr.24431) 1287 : cluster [DBG] pgmap v1694: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:03.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:03 smithi175 bash[21527]: cluster 2024-03-26T04:19:02.554274+0000 mon.a (mon.0) 3572 : cluster [DBG] osdmap e756: 8 total, 8 up, 8 in 2024-03-26T04:19:03.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:03 smithi175 bash[29284]: cluster 2024-03-26T04:19:02.554274+0000 mon.a (mon.0) 3572 : cluster [DBG] osdmap e756: 8 total, 8 up, 8 in 2024-03-26T04:19:03.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:03 smithi190 bash[23392]: cluster 2024-03-26T04:19:02.554274+0000 mon.a (mon.0) 3572 : cluster [DBG] osdmap e756: 8 total, 8 up, 8 in 2024-03-26T04:19:04.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:04 smithi175 bash[29284]: cluster 2024-03-26T04:19:03.562722+0000 mon.a (mon.0) 3573 : cluster [DBG] osdmap e757: 8 total, 8 up, 8 in 2024-03-26T04:19:04.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:04 smithi175 bash[29284]: cluster 2024-03-26T04:19:03.783119+0000 mgr.y (mgr.24431) 1288 : cluster [DBG] pgmap v1697: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail 2024-03-26T04:19:04.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:04 smithi175 bash[21527]: cluster 2024-03-26T04:19:03.562722+0000 mon.a (mon.0) 3573 : cluster [DBG] osdmap e757: 8 total, 8 up, 8 in 2024-03-26T04:19:04.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:04 smithi175 bash[21527]: cluster 2024-03-26T04:19:03.783119+0000 mgr.y (mgr.24431) 1288 : cluster [DBG] pgmap v1697: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail 2024-03-26T04:19:04.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:04 smithi190 bash[23392]: cluster 2024-03-26T04:19:03.562722+0000 mon.a (mon.0) 3573 : cluster [DBG] osdmap e757: 8 total, 8 up, 8 in 2024-03-26T04:19:04.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:04 smithi190 bash[23392]: cluster 2024-03-26T04:19:03.783119+0000 mgr.y (mgr.24431) 1288 : cluster [DBG] pgmap v1697: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail 2024-03-26T04:19:05.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:05 smithi175 bash[21527]: cluster 2024-03-26T04:19:04.578544+0000 mon.a (mon.0) 3574 : cluster [DBG] osdmap e758: 8 total, 8 up, 8 in 2024-03-26T04:19:05.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:05 smithi175 bash[29284]: cluster 2024-03-26T04:19:04.578544+0000 mon.a (mon.0) 3574 : cluster [DBG] osdmap e758: 8 total, 8 up, 8 in 2024-03-26T04:19:05.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:05 smithi190 bash[23392]: cluster 2024-03-26T04:19:04.578544+0000 mon.a (mon.0) 3574 : cluster [DBG] osdmap e758: 8 total, 8 up, 8 in 2024-03-26T04:19:06.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:06 smithi175 bash[21527]: cluster 2024-03-26T04:19:05.581167+0000 mon.a (mon.0) 3575 : cluster [DBG] osdmap e759: 8 total, 8 up, 8 in 2024-03-26T04:19:06.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:06 smithi175 bash[21527]: cluster 2024-03-26T04:19:05.783898+0000 mgr.y (mgr.24431) 1289 : cluster [DBG] pgmap v1700: 164 pgs: 164 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:06.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:06 smithi175 bash[29284]: cluster 2024-03-26T04:19:05.581167+0000 mon.a (mon.0) 3575 : cluster [DBG] osdmap e759: 8 total, 8 up, 8 in 2024-03-26T04:19:06.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:06 smithi175 bash[29284]: cluster 2024-03-26T04:19:05.783898+0000 mgr.y (mgr.24431) 1289 : cluster [DBG] pgmap v1700: 164 pgs: 164 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:06.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:06 smithi190 bash[23392]: cluster 2024-03-26T04:19:05.581167+0000 mon.a (mon.0) 3575 : cluster [DBG] osdmap e759: 8 total, 8 up, 8 in 2024-03-26T04:19:06.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:06 smithi190 bash[23392]: cluster 2024-03-26T04:19:05.783898+0000 mgr.y (mgr.24431) 1289 : cluster [DBG] pgmap v1700: 164 pgs: 164 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:07.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:07 smithi175 bash[21527]: cluster 2024-03-26T04:19:06.573704+0000 mon.a (mon.0) 3576 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:19:07.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:07 smithi175 bash[21527]: cluster 2024-03-26T04:19:06.597140+0000 mon.a (mon.0) 3577 : cluster [DBG] osdmap e760: 8 total, 8 up, 8 in 2024-03-26T04:19:07.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:07 smithi175 bash[29284]: cluster 2024-03-26T04:19:06.573704+0000 mon.a (mon.0) 3576 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:19:07.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:07 smithi175 bash[29284]: cluster 2024-03-26T04:19:06.597140+0000 mon.a (mon.0) 3577 : cluster [DBG] osdmap e760: 8 total, 8 up, 8 in 2024-03-26T04:19:07.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:07 smithi190 bash[23392]: cluster 2024-03-26T04:19:06.573704+0000 mon.a (mon.0) 3576 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:19:07.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:07 smithi190 bash[23392]: cluster 2024-03-26T04:19:06.597140+0000 mon.a (mon.0) 3577 : cluster [DBG] osdmap e760: 8 total, 8 up, 8 in 2024-03-26T04:19:08.990 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:19:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:19:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:19:08.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:08 smithi175 bash[21527]: cluster 2024-03-26T04:19:07.596693+0000 mon.a (mon.0) 3578 : cluster [DBG] osdmap e761: 8 total, 8 up, 8 in 2024-03-26T04:19:08.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:08 smithi175 bash[21527]: cluster 2024-03-26T04:19:07.784730+0000 mgr.y (mgr.24431) 1290 : cluster [DBG] pgmap v1703: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:08.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:08 smithi175 bash[21527]: audit 2024-03-26T04:19:08.414328+0000 mon.c (mon.2) 860 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:19:08.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:08 smithi175 bash[29284]: cluster 2024-03-26T04:19:07.596693+0000 mon.a (mon.0) 3578 : cluster [DBG] osdmap e761: 8 total, 8 up, 8 in 2024-03-26T04:19:08.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:08 smithi175 bash[29284]: cluster 2024-03-26T04:19:07.784730+0000 mgr.y (mgr.24431) 1290 : cluster [DBG] pgmap v1703: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:08.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:08 smithi175 bash[29284]: audit 2024-03-26T04:19:08.414328+0000 mon.c (mon.2) 860 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:19:08.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:08 smithi190 bash[23392]: cluster 2024-03-26T04:19:07.596693+0000 mon.a (mon.0) 3578 : cluster [DBG] osdmap e761: 8 total, 8 up, 8 in 2024-03-26T04:19:08.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:08 smithi190 bash[23392]: cluster 2024-03-26T04:19:07.784730+0000 mgr.y (mgr.24431) 1290 : cluster [DBG] pgmap v1703: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:08.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:08 smithi190 bash[23392]: audit 2024-03-26T04:19:08.414328+0000 mon.c (mon.2) 860 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:19:09.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:09 smithi175 bash[21527]: cluster 2024-03-26T04:19:08.626260+0000 mon.a (mon.0) 3579 : cluster [DBG] osdmap e762: 8 total, 8 up, 8 in 2024-03-26T04:19:09.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:09 smithi175 bash[29284]: cluster 2024-03-26T04:19:08.626260+0000 mon.a (mon.0) 3579 : cluster [DBG] osdmap e762: 8 total, 8 up, 8 in 2024-03-26T04:19:09.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:09 smithi190 bash[23392]: cluster 2024-03-26T04:19:08.626260+0000 mon.a (mon.0) 3579 : cluster [DBG] osdmap e762: 8 total, 8 up, 8 in 2024-03-26T04:19:10.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:10 smithi175 bash[21527]: cluster 2024-03-26T04:19:09.637572+0000 mon.a (mon.0) 3580 : cluster [DBG] osdmap e763: 8 total, 8 up, 8 in 2024-03-26T04:19:10.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:10 smithi175 bash[21527]: cluster 2024-03-26T04:19:09.785649+0000 mgr.y (mgr.24431) 1291 : cluster [DBG] pgmap v1706: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 856 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:10.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:10 smithi175 bash[29284]: cluster 2024-03-26T04:19:09.637572+0000 mon.a (mon.0) 3580 : cluster [DBG] osdmap e763: 8 total, 8 up, 8 in 2024-03-26T04:19:10.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:10 smithi175 bash[29284]: cluster 2024-03-26T04:19:09.785649+0000 mgr.y (mgr.24431) 1291 : cluster [DBG] pgmap v1706: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 856 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:11.001 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:10 smithi190 bash[23392]: cluster 2024-03-26T04:19:09.637572+0000 mon.a (mon.0) 3580 : cluster [DBG] osdmap e763: 8 total, 8 up, 8 in 2024-03-26T04:19:11.002 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:10 smithi190 bash[23392]: cluster 2024-03-26T04:19:09.785649+0000 mgr.y (mgr.24431) 1291 : cluster [DBG] pgmap v1706: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 856 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:11.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:11 smithi190 bash[23392]: cluster 2024-03-26T04:19:10.648634+0000 mon.a (mon.0) 3581 : cluster [DBG] osdmap e764: 8 total, 8 up, 8 in 2024-03-26T04:19:11.748 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:19:11 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:19:11.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:11 smithi175 bash[29284]: cluster 2024-03-26T04:19:10.648634+0000 mon.a (mon.0) 3581 : cluster [DBG] osdmap e764: 8 total, 8 up, 8 in 2024-03-26T04:19:11.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:11 smithi175 bash[21527]: cluster 2024-03-26T04:19:10.648634+0000 mon.a (mon.0) 3581 : cluster [DBG] osdmap e764: 8 total, 8 up, 8 in 2024-03-26T04:19:12.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:12 smithi175 bash[21527]: audit 2024-03-26T04:19:11.461852+0000 mgr.y (mgr.24431) 1292 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:19:12.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:12 smithi175 bash[21527]: cluster 2024-03-26T04:19:11.659874+0000 mon.a (mon.0) 3582 : cluster [DBG] osdmap e765: 8 total, 8 up, 8 in 2024-03-26T04:19:12.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:12 smithi175 bash[21527]: cluster 2024-03-26T04:19:11.786421+0000 mgr.y (mgr.24431) 1293 : cluster [DBG] pgmap v1709: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 856 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:12.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:12 smithi175 bash[21527]: cluster 2024-03-26T04:19:11.940577+0000 mon.a (mon.0) 3583 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:19:12.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:12 smithi175 bash[29284]: audit 2024-03-26T04:19:11.461852+0000 mgr.y (mgr.24431) 1292 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:19:12.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:12 smithi175 bash[29284]: cluster 2024-03-26T04:19:11.659874+0000 mon.a (mon.0) 3582 : cluster [DBG] osdmap e765: 8 total, 8 up, 8 in 2024-03-26T04:19:12.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:12 smithi175 bash[29284]: cluster 2024-03-26T04:19:11.786421+0000 mgr.y (mgr.24431) 1293 : cluster [DBG] pgmap v1709: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 856 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:12.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:12 smithi175 bash[29284]: cluster 2024-03-26T04:19:11.940577+0000 mon.a (mon.0) 3583 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:19:12.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:12 smithi190 bash[23392]: audit 2024-03-26T04:19:11.461852+0000 mgr.y (mgr.24431) 1292 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:19:12.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:12 smithi190 bash[23392]: cluster 2024-03-26T04:19:11.659874+0000 mon.a (mon.0) 3582 : cluster [DBG] osdmap e765: 8 total, 8 up, 8 in 2024-03-26T04:19:12.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:12 smithi190 bash[23392]: cluster 2024-03-26T04:19:11.786421+0000 mgr.y (mgr.24431) 1293 : cluster [DBG] pgmap v1709: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 856 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:12.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:12 smithi190 bash[23392]: cluster 2024-03-26T04:19:11.940577+0000 mon.a (mon.0) 3583 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:19:13.681 INFO:tasks.workunit.client.0.smithi175.stdout: write_operations: Running main() from gmock_main.cc 2024-03-26T04:19:13.681 INFO:tasks.workunit.client.0.smithi175.stdout: write_operations: [==========] Running 7 tests from 1 test suite. 2024-03-26T04:19:13.681 INFO:tasks.workunit.client.0.smithi175.stdout: write_operations: [----------] Global test environment set-up. 2024-03-26T04:19:13.681 INFO:tasks.workunit.client.0.smithi175.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps 2024-03-26T04:19:13.681 INFO:tasks.workunit.client.0.smithi175.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertExists 2024-03-26T04:19:13.681 INFO:tasks.workunit.client.0.smithi175.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertExists (1800832 ms) 2024-03-26T04:19:13.681 INFO:tasks.workunit.client.0.smithi175.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertVersion 2024-03-26T04:19:13.681 INFO:tasks.workunit.client.0.smithi175.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertVersion (3018 ms) 2024-03-26T04:19:13.681 INFO:tasks.workunit.client.0.smithi175.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Xattrs 2024-03-26T04:19:13.681 INFO:tasks.workunit.client.0.smithi175.stdout: write_operations: [ OK ] NeoRadosWriteOps.Xattrs (3043 ms) 2024-03-26T04:19:13.681 INFO:tasks.workunit.client.0.smithi175.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Write 2024-03-26T04:19:13.682 INFO:tasks.workunit.client.0.smithi175.stdout: write_operations: [ OK ] NeoRadosWriteOps.Write (3033 ms) 2024-03-26T04:19:13.682 INFO:tasks.workunit.client.0.smithi175.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Exec 2024-03-26T04:19:13.682 INFO:tasks.workunit.client.0.smithi175.stdout: write_operations: [ OK ] NeoRadosWriteOps.Exec (3024 ms) 2024-03-26T04:19:13.682 INFO:tasks.workunit.client.0.smithi175.stdout: write_operations: [ RUN ] NeoRadosWriteOps.WriteSame 2024-03-26T04:19:13.682 INFO:tasks.workunit.client.0.smithi175.stdout: write_operations: [ OK ] NeoRadosWriteOps.WriteSame (3045 ms) 2024-03-26T04:19:13.682 INFO:tasks.workunit.client.0.smithi175.stdout: write_operations: [ RUN ] NeoRadosWriteOps.CmpExt 2024-03-26T04:19:13.682 INFO:tasks.workunit.client.0.smithi175.stdout: write_operations: [ OK ] NeoRadosWriteOps.CmpExt (5046 ms) 2024-03-26T04:19:13.682 INFO:tasks.workunit.client.0.smithi175.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps (1821041 ms total) 2024-03-26T04:19:13.682 INFO:tasks.workunit.client.0.smithi175.stdout: write_operations: 2024-03-26T04:19:13.682 INFO:tasks.workunit.client.0.smithi175.stdout: write_operations: [----------] Global test environment tear-down 2024-03-26T04:19:13.682 INFO:tasks.workunit.client.0.smithi175.stdout: write_operations: [==========] 7 tests from 1 test suite ran. (1821041 ms total) 2024-03-26T04:19:13.682 INFO:tasks.workunit.client.0.smithi175.stdout: write_operations: [ PASSED ] 7 tests. 2024-03-26T04:19:13.683 INFO:tasks.workunit.client.0.smithi175.stderr:+ exit 0 2024-03-26T04:19:13.683 INFO:tasks.workunit.client.0.smithi175.stderr:+ cleanup 2024-03-26T04:19:13.683 INFO:tasks.workunit.client.0.smithi175.stderr:+ pkill -P 64387 2024-03-26T04:19:13.697 INFO:tasks.workunit.client.0.smithi175.stderr:+ true 2024-03-26T04:19:13.697 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-26T04:19:13.698 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-26T04:19:13.711 INFO:tasks.workunit:Running workunits matching rados/test_pool_quota.sh on client.0... 2024-03-26T04:19:13.712 INFO:tasks.workunit:Running workunit rados/test_pool_quota.sh... 2024-03-26T04:19:13.712 DEBUG:teuthology.orchestra.run.smithi175:workunit test rados/test_pool_quota.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e42b8ea316eec6c6f4fae026f6d6b62fa5fe7003 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_pool_quota.sh 2024-03-26T04:19:13.762 INFO:tasks.workunit.client.0.smithi175.stderr:+ uuidgen 2024-03-26T04:19:13.763 INFO:tasks.workunit.client.0.smithi175.stderr:+ p=0e4b321e-b83b-43c2-ade1-c76622f394b4 2024-03-26T04:19:13.763 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph osd pool create 0e4b321e-b83b-43c2-ade1-c76622f394b4 12 2024-03-26T04:19:13.889 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7fa6bc640 1 -- 172.21.15.175:0/2327962262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7f4101390 msgr2=0x7fe7f410d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:19:13.889 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7fa6bc640 1 --2- 172.21.15.175:0/2327962262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7f4101390 0x7fe7f410d8a0 secure :-1 s=READY pgs=3012 cs=0 l=1 rev1=1 crypto rx=0x7fe7e40098a0 tx=0x7fe7e401c700 comp rx=0 tx=0).stop 2024-03-26T04:19:13.889 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7fa6bc640 1 -- 172.21.15.175:0/2327962262 shutdown_connections 2024-03-26T04:19:13.889 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7fa6bc640 1 --2- 172.21.15.175:0/2327962262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7f4101390 0x7fe7f410d8a0 unknown :-1 s=CLOSED pgs=3012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:13.889 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7fa6bc640 1 --2- 172.21.15.175:0/2327962262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7f41009d0 0x7fe7f4100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:13.890 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7fa6bc640 1 --2- 172.21.15.175:0/2327962262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7f41069d0 0x7fe7f4106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:13.890 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7fa6bc640 1 -- 172.21.15.175:0/2327962262 >> 172.21.15.175:0/2327962262 conn(0x7fe7f40fc4c0 msgr2=0x7fe7f40fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T04:19:13.890 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7fa6bc640 1 -- 172.21.15.175:0/2327962262 shutdown_connections 2024-03-26T04:19:13.890 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7fa6bc640 1 -- 172.21.15.175:0/2327962262 wait complete. 2024-03-26T04:19:13.891 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7fa6bc640 1 Processor -- start 2024-03-26T04:19:13.891 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7fa6bc640 1 -- start start 2024-03-26T04:19:13.891 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7fa6bc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7f41009d0 0x7fe7f4198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:19:13.891 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7fa6bc640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7f4101390 0x7fe7f4199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:19:13.891 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7fa6bc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7f41069d0 0x7fe7f419d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:19:13.891 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7fa6bc640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f41129c0 con 0x7fe7f41069d0 2024-03-26T04:19:13.891 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7fa6bc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f4112840 con 0x7fe7f41009d0 2024-03-26T04:19:13.891 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7fa6bc640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe7f4112b40 con 0x7fe7f4101390 2024-03-26T04:19:13.891 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7f37fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7f4101390 0x7fe7f4199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:19:13.891 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7f8c59640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7f41069d0 0x7fe7f419d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:19:13.892 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7f37fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7f4101390 0x7fe7f4199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:44982/0 (socket says 172.21.15.175:44982) 2024-03-26T04:19:13.892 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7f8c59640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7f41069d0 0x7fe7f419d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:47888/0 (socket says 172.21.15.175:47888) 2024-03-26T04:19:13.892 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7f37fe640 1 -- 172.21.15.175:0/3227144857 learned_addr learned my addr 172.21.15.175:0/3227144857 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T04:19:13.892 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7f3fff640 1 --2- 172.21.15.175:0/3227144857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7f41009d0 0x7fe7f4198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:19:13.892 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7f37fe640 1 -- 172.21.15.175:0/3227144857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7f41009d0 msgr2=0x7fe7f4198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:19:13.892 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7f37fe640 1 --2- 172.21.15.175:0/3227144857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7f41009d0 0x7fe7f4198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:13.892 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7f37fe640 1 -- 172.21.15.175:0/3227144857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7f41069d0 msgr2=0x7fe7f419d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:19:13.892 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7f37fe640 1 --2- 172.21.15.175:0/3227144857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7f41069d0 0x7fe7f419d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:13.892 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7f37fe640 1 -- 172.21.15.175:0/3227144857 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f419dee0 con 0x7fe7f4101390 2024-03-26T04:19:13.892 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7f37fe640 1 --2- 172.21.15.175:0/3227144857 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7f4101390 0x7fe7f4199470 secure :-1 s=READY pgs=3013 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00e9d0 tx=0x7fe7dc00ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T04:19:13.892 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7f17fa640 1 -- 172.21.15.175:0/3227144857 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc00cde0 con 0x7fe7f4101390 2024-03-26T04:19:13.892 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7f17fa640 1 -- 172.21.15.175:0/3227144857 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fe7dc0044a0 con 0x7fe7f4101390 2024-03-26T04:19:13.892 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7fa6bc640 1 -- 172.21.15.175:0/3227144857 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f419e1d0 con 0x7fe7f4101390 2024-03-26T04:19:13.892 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7f17fa640 1 -- 172.21.15.175:0/3227144857 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc010430 con 0x7fe7f4101390 2024-03-26T04:19:13.893 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7fa6bc640 1 -- 172.21.15.175:0/3227144857 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f41a5ab0 con 0x7fe7f4101390 2024-03-26T04:19:13.893 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.890+0000 7fe7fa6bc640 1 -- 172.21.15.175:0/3227144857 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7f4068a40 con 0x7fe7f4101390 2024-03-26T04:19:13.893 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.894+0000 7fe7f17fa640 1 -- 172.21.15.175:0/3227144857 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7fe7dc003f60 con 0x7fe7f4101390 2024-03-26T04:19:13.894 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.894+0000 7fe7f17fa640 1 --2- 172.21.15.175:0/3227144857 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fe7e006e760 0x7fe7e0070c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:19:13.894 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.894+0000 7fe7f17fa640 1 -- 172.21.15.175:0/3227144857 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(766..766 src has 1..766) ==== 8692+0+0 (secure 0 0 0) 0x7fe7dc08f960 con 0x7fe7f4101390 2024-03-26T04:19:13.894 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.894+0000 7fe7f3fff640 1 --2- 172.21.15.175:0/3227144857 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fe7e006e760 0x7fe7e0070c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:19:13.894 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.894+0000 7fe7f3fff640 1 --2- 172.21.15.175:0/3227144857 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fe7e006e760 0x7fe7e0070c20 secure :-1 s=READY pgs=3754 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec009dd0 tx=0x7fe7ec0097b0 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T04:19:13.895 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:13.898+0000 7fe7f17fa640 1 -- 172.21.15.175:0/3227144857 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7fe7dc05c500 con 0x7fe7f4101390 2024-03-26T04:19:14.071 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.074+0000 7fe7fa6bc640 1 -- 172.21.15.175:0/3227144857 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "pg_num": 12} v 0) -- 0x7fe7f419a290 con 0x7fe7f4101390 2024-03-26T04:19:14.685 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.686+0000 7fe7f17fa640 1 -- 172.21.15.175:0/3227144857 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "pg_num": 12}]=0 pool '0e4b321e-b83b-43c2-ade1-c76622f394b4' created v767) ==== 176+0+0 (secure 0 0 0) 0x7fe7f419a290 con 0x7fe7f4101390 2024-03-26T04:19:14.797 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.798+0000 7fe7fa6bc640 1 -- 172.21.15.175:0/3227144857 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "pg_num": 12} v 0) -- 0x7fe7f4062160 con 0x7fe7f4101390 2024-03-26T04:19:14.798 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.802+0000 7fe7f17fa640 1 -- 172.21.15.175:0/3227144857 <== mon.2 v2:172.21.15.175:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "pg_num": 12}]=0 pool '0e4b321e-b83b-43c2-ade1-c76622f394b4' already exists v767) ==== 183+0+0 (secure 0 0 0) 0x7fe7dc0601b0 con 0x7fe7f4101390 2024-03-26T04:19:14.799 INFO:tasks.workunit.client.0.smithi175.stderr:pool '0e4b321e-b83b-43c2-ade1-c76622f394b4' already exists 2024-03-26T04:19:14.801 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.802+0000 7fe7fa6bc640 1 -- 172.21.15.175:0/3227144857 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fe7e006e760 msgr2=0x7fe7e0070c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:19:14.801 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.802+0000 7fe7fa6bc640 1 --2- 172.21.15.175:0/3227144857 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fe7e006e760 0x7fe7e0070c20 secure :-1 s=READY pgs=3754 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec009dd0 tx=0x7fe7ec0097b0 comp rx=0 tx=0).stop 2024-03-26T04:19:14.801 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.802+0000 7fe7fa6bc640 1 -- 172.21.15.175:0/3227144857 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7f4101390 msgr2=0x7fe7f4199470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:19:14.802 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.802+0000 7fe7fa6bc640 1 --2- 172.21.15.175:0/3227144857 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7f4101390 0x7fe7f4199470 secure :-1 s=READY pgs=3013 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00e9d0 tx=0x7fe7dc00ee90 comp rx=0 tx=0).stop 2024-03-26T04:19:14.802 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.802+0000 7fe7fa6bc640 1 -- 172.21.15.175:0/3227144857 shutdown_connections 2024-03-26T04:19:14.802 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.802+0000 7fe7fa6bc640 1 --2- 172.21.15.175:0/3227144857 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fe7e006e760 0x7fe7e0070c20 unknown :-1 s=CLOSED pgs=3754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:14.802 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.802+0000 7fe7fa6bc640 1 --2- 172.21.15.175:0/3227144857 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7f4101390 0x7fe7f4199470 unknown :-1 s=CLOSED pgs=3013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:14.802 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.802+0000 7fe7fa6bc640 1 --2- 172.21.15.175:0/3227144857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7f41069d0 0x7fe7f419d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:14.802 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.802+0000 7fe7fa6bc640 1 --2- 172.21.15.175:0/3227144857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7f41009d0 0x7fe7f4198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:14.802 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.802+0000 7fe7fa6bc640 1 -- 172.21.15.175:0/3227144857 >> 172.21.15.175:0/3227144857 conn(0x7fe7f40fc4c0 msgr2=0x7fe7f40fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T04:19:14.802 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.802+0000 7fe7fa6bc640 1 -- 172.21.15.175:0/3227144857 shutdown_connections 2024-03-26T04:19:14.802 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.802+0000 7fe7fa6bc640 1 -- 172.21.15.175:0/3227144857 wait complete. 2024-03-26T04:19:14.816 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph osd pool set-quota 0e4b321e-b83b-43c2-ade1-c76622f394b4 max_objects 10 2024-03-26T04:19:14.943 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.942+0000 7fc2afe4c640 1 -- 172.21.15.175:0/2718584396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2a81018f0 msgr2=0x7fc2a8105680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:19:14.943 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.942+0000 7fc2afe4c640 1 --2- 172.21.15.175:0/2718584396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2a81018f0 0x7fc2a8105680 secure :-1 s=READY pgs=3014 cs=0 l=1 rev1=1 crypto rx=0x7fc298009870 tx=0x7fc29801c720 comp rx=0 tx=0).stop 2024-03-26T04:19:14.943 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.942+0000 7fc2afe4c640 1 -- 172.21.15.175:0/2718584396 shutdown_connections 2024-03-26T04:19:14.943 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.942+0000 7fc2afe4c640 1 --2- 172.21.15.175:0/2718584396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2a81018f0 0x7fc2a8105680 unknown :-1 s=CLOSED pgs=3014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:14.943 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.942+0000 7fc2afe4c640 1 --2- 172.21.15.175:0/2718584396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2a8105d60 0x7fc2a810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:14.943 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.942+0000 7fc2afe4c640 1 --2- 172.21.15.175:0/2718584396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2a8100f40 0x7fc2a8101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:14.943 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.942+0000 7fc2afe4c640 1 -- 172.21.15.175:0/2718584396 >> 172.21.15.175:0/2718584396 conn(0x7fc2a80778e0 msgr2=0x7fc2a8077cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T04:19:14.943 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.942+0000 7fc2afe4c640 1 -- 172.21.15.175:0/2718584396 shutdown_connections 2024-03-26T04:19:14.943 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.942+0000 7fc2afe4c640 1 -- 172.21.15.175:0/2718584396 wait complete. 2024-03-26T04:19:14.943 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.942+0000 7fc2afe4c640 1 Processor -- start 2024-03-26T04:19:14.943 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.942+0000 7fc2afe4c640 1 -- start start 2024-03-26T04:19:14.943 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.942+0000 7fc2afe4c640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2a8100f40 0x7fc2a819a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:19:14.944 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.942+0000 7fc2afe4c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2a81018f0 0x7fc2a819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:19:14.944 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.942+0000 7fc2afe4c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2a8105d60 0x7fc2a819ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:19:14.944 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.942+0000 7fc2afe4c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a8112a20 con 0x7fc2a81018f0 2024-03-26T04:19:14.944 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.942+0000 7fc2afe4c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a81128a0 con 0x7fc2a8105d60 2024-03-26T04:19:14.944 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.942+0000 7fc2afe4c640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc2a8112ba0 con 0x7fc2a8100f40 2024-03-26T04:19:14.944 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.942+0000 7fc2adbe8640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2a8100f40 0x7fc2a819a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:19:14.944 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.942+0000 7fc2adbe8640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2a8100f40 0x7fc2a819a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:45002/0 (socket says 172.21.15.175:45002) 2024-03-26T04:19:14.944 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.942+0000 7fc2adbe8640 1 -- 172.21.15.175:0/1928261175 learned_addr learned my addr 172.21.15.175:0/1928261175 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T04:19:14.944 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.942+0000 7fc2ad3e7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2a81018f0 0x7fc2a819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:19:14.944 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.942+0000 7fc2ae3e9640 1 --2- 172.21.15.175:0/1928261175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2a8105d60 0x7fc2a819ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:19:14.944 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.942+0000 7fc2adbe8640 1 -- 172.21.15.175:0/1928261175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2a8105d60 msgr2=0x7fc2a819ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:19:14.944 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.942+0000 7fc2adbe8640 1 --2- 172.21.15.175:0/1928261175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2a8105d60 0x7fc2a819ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:14.944 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.942+0000 7fc2adbe8640 1 -- 172.21.15.175:0/1928261175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2a81018f0 msgr2=0x7fc2a819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:19:14.945 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.942+0000 7fc2adbe8640 1 --2- 172.21.15.175:0/1928261175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2a81018f0 0x7fc2a819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:14.945 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.946+0000 7fc2adbe8640 1 -- 172.21.15.175:0/1928261175 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a819f0e0 con 0x7fc2a8100f40 2024-03-26T04:19:14.945 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.946+0000 7fc2ad3e7640 1 --2- 172.21.15.175:0/1928261175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2a81018f0 0x7fc2a819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T04:19:14.945 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.946+0000 7fc2adbe8640 1 --2- 172.21.15.175:0/1928261175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2a8100f40 0x7fc2a819a130 secure :-1 s=READY pgs=3015 cs=0 l=1 rev1=1 crypto rx=0x7fc2a400e9d0 tx=0x7fc2a400ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T04:19:14.945 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.946+0000 7fc29effd640 1 -- 172.21.15.175:0/1928261175 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a400cde0 con 0x7fc2a8100f40 2024-03-26T04:19:14.945 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.946+0000 7fc29effd640 1 -- 172.21.15.175:0/1928261175 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fc2a40044a0 con 0x7fc2a8100f40 2024-03-26T04:19:14.945 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.946+0000 7fc2afe4c640 1 -- 172.21.15.175:0/1928261175 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2a819f3d0 con 0x7fc2a8100f40 2024-03-26T04:19:14.945 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.946+0000 7fc29effd640 1 -- 172.21.15.175:0/1928261175 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4010430 con 0x7fc2a8100f40 2024-03-26T04:19:14.945 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.946+0000 7fc2afe4c640 1 -- 172.21.15.175:0/1928261175 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2a81a6cb0 con 0x7fc2a8100f40 2024-03-26T04:19:14.946 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.946+0000 7fc2afe4c640 1 -- 172.21.15.175:0/1928261175 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2a8068e90 con 0x7fc2a8100f40 2024-03-26T04:19:14.946 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.946+0000 7fc29effd640 1 -- 172.21.15.175:0/1928261175 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7fc2a4003f80 con 0x7fc2a8100f40 2024-03-26T04:19:14.946 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.946+0000 7fc29effd640 1 --2- 172.21.15.175:0/1928261175 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fc29406e760 0x7fc294070c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:19:14.946 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.946+0000 7fc29effd640 1 -- 172.21.15.175:0/1928261175 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(767..767 src has 1..767) ==== 9067+0+0 (secure 0 0 0) 0x7fc2a408f980 con 0x7fc2a8100f40 2024-03-26T04:19:14.946 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.946+0000 7fc2ad3e7640 1 --2- 172.21.15.175:0/1928261175 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fc29406e760 0x7fc294070c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:19:14.946 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.946+0000 7fc2ad3e7640 1 --2- 172.21.15.175:0/1928261175 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fc29406e760 0x7fc294070c20 secure :-1 s=READY pgs=3755 cs=0 l=1 rev1=1 crypto rx=0x7fc298009790 tx=0x7fc298005790 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T04:19:14.949 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:14.950+0000 7fc29effd640 1 -- 172.21.15.175:0/1928261175 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7fc2a405c330 con 0x7fc2a8100f40 2024-03-26T04:19:14.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:14 smithi175 bash[21527]: cluster 2024-03-26T04:19:13.674155+0000 mon.a (mon.0) 3584 : cluster [DBG] osdmap e766: 8 total, 8 up, 8 in 2024-03-26T04:19:14.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:14 smithi175 bash[21527]: cluster 2024-03-26T04:19:13.786991+0000 mgr.y (mgr.24431) 1294 : cluster [DBG] pgmap v1711: 164 pgs: 164 active+clean; 582 KiB data, 856 MiB used, 714 GiB / 715 GiB avail 2024-03-26T04:19:14.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:14 smithi175 bash[21527]: audit 2024-03-26T04:19:14.076893+0000 mon.c (mon.2) 861 : audit [INF] from='client.? 172.21.15.175:0/3227144857' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "pg_num": 12}]: dispatch 2024-03-26T04:19:14.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:14 smithi175 bash[21527]: audit 2024-03-26T04:19:14.077488+0000 mon.a (mon.0) 3585 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "pg_num": 12}]: dispatch 2024-03-26T04:19:14.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:14 smithi175 bash[29284]: cluster 2024-03-26T04:19:13.674155+0000 mon.a (mon.0) 3584 : cluster [DBG] osdmap e766: 8 total, 8 up, 8 in 2024-03-26T04:19:14.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:14 smithi175 bash[29284]: cluster 2024-03-26T04:19:13.786991+0000 mgr.y (mgr.24431) 1294 : cluster [DBG] pgmap v1711: 164 pgs: 164 active+clean; 582 KiB data, 856 MiB used, 714 GiB / 715 GiB avail 2024-03-26T04:19:14.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:14 smithi175 bash[29284]: audit 2024-03-26T04:19:14.076893+0000 mon.c (mon.2) 861 : audit [INF] from='client.? 172.21.15.175:0/3227144857' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "pg_num": 12}]: dispatch 2024-03-26T04:19:14.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:14 smithi175 bash[29284]: audit 2024-03-26T04:19:14.077488+0000 mon.a (mon.0) 3585 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "pg_num": 12}]: dispatch 2024-03-26T04:19:14.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:14 smithi190 bash[23392]: cluster 2024-03-26T04:19:13.674155+0000 mon.a (mon.0) 3584 : cluster [DBG] osdmap e766: 8 total, 8 up, 8 in 2024-03-26T04:19:14.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:14 smithi190 bash[23392]: cluster 2024-03-26T04:19:13.786991+0000 mgr.y (mgr.24431) 1294 : cluster [DBG] pgmap v1711: 164 pgs: 164 active+clean; 582 KiB data, 856 MiB used, 714 GiB / 715 GiB avail 2024-03-26T04:19:14.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:14 smithi190 bash[23392]: audit 2024-03-26T04:19:14.076893+0000 mon.c (mon.2) 861 : audit [INF] from='client.? 172.21.15.175:0/3227144857' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "pg_num": 12}]: dispatch 2024-03-26T04:19:14.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:14 smithi190 bash[23392]: audit 2024-03-26T04:19:14.077488+0000 mon.a (mon.0) 3585 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "pg_num": 12}]: dispatch 2024-03-26T04:19:15.122 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:15.126+0000 7fc2afe4c640 1 -- 172.21.15.175:0/1928261175 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "10"} v 0) -- 0x7fc2a819b490 con 0x7fc2a8100f40 2024-03-26T04:19:15.686 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:15.686+0000 7fc29effd640 1 -- 172.21.15.175:0/1928261175 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 0e4b321e-b83b-43c2-ade1-c76622f394b4 v768) ==== 223+0+0 (secure 0 0 0) 0x7fc2a405ffe0 con 0x7fc2a8100f40 2024-03-26T04:19:15.795 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:15.798+0000 7fc2afe4c640 1 -- 172.21.15.175:0/1928261175 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "10"} v 0) -- 0x7fc2a80621e0 con 0x7fc2a8100f40 2024-03-26T04:19:15.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:15 smithi175 bash[21527]: audit 2024-03-26T04:19:14.677621+0000 mon.a (mon.0) 3586 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "pg_num": 12}]': finished 2024-03-26T04:19:15.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:15 smithi175 bash[21527]: cluster 2024-03-26T04:19:14.691069+0000 mon.a (mon.0) 3587 : cluster [DBG] osdmap e767: 8 total, 8 up, 8 in 2024-03-26T04:19:15.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:15 smithi175 bash[21527]: audit 2024-03-26T04:19:14.802641+0000 mon.c (mon.2) 862 : audit [INF] from='client.? 172.21.15.175:0/3227144857' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "pg_num": 12}]: dispatch 2024-03-26T04:19:15.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:15 smithi175 bash[21527]: audit 2024-03-26T04:19:14.803416+0000 mon.a (mon.0) 3588 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "pg_num": 12}]: dispatch 2024-03-26T04:19:15.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:15 smithi175 bash[21527]: audit 2024-03-26T04:19:15.128059+0000 mon.c (mon.2) 863 : audit [INF] from='client.? 172.21.15.175:0/1928261175' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "10"}]: dispatch 2024-03-26T04:19:15.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:15 smithi175 bash[21527]: audit 2024-03-26T04:19:15.128599+0000 mon.a (mon.0) 3589 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "10"}]: dispatch 2024-03-26T04:19:15.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:15 smithi175 bash[29284]: audit 2024-03-26T04:19:14.677621+0000 mon.a (mon.0) 3586 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "pg_num": 12}]': finished 2024-03-26T04:19:15.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:15 smithi175 bash[29284]: cluster 2024-03-26T04:19:14.691069+0000 mon.a (mon.0) 3587 : cluster [DBG] osdmap e767: 8 total, 8 up, 8 in 2024-03-26T04:19:15.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:15 smithi175 bash[29284]: audit 2024-03-26T04:19:14.802641+0000 mon.c (mon.2) 862 : audit [INF] from='client.? 172.21.15.175:0/3227144857' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "pg_num": 12}]: dispatch 2024-03-26T04:19:15.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:15 smithi175 bash[29284]: audit 2024-03-26T04:19:14.803416+0000 mon.a (mon.0) 3588 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "pg_num": 12}]: dispatch 2024-03-26T04:19:15.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:15 smithi175 bash[29284]: audit 2024-03-26T04:19:15.128059+0000 mon.c (mon.2) 863 : audit [INF] from='client.? 172.21.15.175:0/1928261175' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "10"}]: dispatch 2024-03-26T04:19:15.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:15 smithi175 bash[29284]: audit 2024-03-26T04:19:15.128599+0000 mon.a (mon.0) 3589 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "10"}]: dispatch 2024-03-26T04:19:15.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:15 smithi190 bash[23392]: audit 2024-03-26T04:19:14.677621+0000 mon.a (mon.0) 3586 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "pg_num": 12}]': finished 2024-03-26T04:19:15.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:15 smithi190 bash[23392]: cluster 2024-03-26T04:19:14.691069+0000 mon.a (mon.0) 3587 : cluster [DBG] osdmap e767: 8 total, 8 up, 8 in 2024-03-26T04:19:15.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:15 smithi190 bash[23392]: audit 2024-03-26T04:19:14.802641+0000 mon.c (mon.2) 862 : audit [INF] from='client.? 172.21.15.175:0/3227144857' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "pg_num": 12}]: dispatch 2024-03-26T04:19:15.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:15 smithi190 bash[23392]: audit 2024-03-26T04:19:14.803416+0000 mon.a (mon.0) 3588 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "pg_num": 12}]: dispatch 2024-03-26T04:19:15.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:15 smithi190 bash[23392]: audit 2024-03-26T04:19:15.128059+0000 mon.c (mon.2) 863 : audit [INF] from='client.? 172.21.15.175:0/1928261175' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "10"}]: dispatch 2024-03-26T04:19:15.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:15 smithi190 bash[23392]: audit 2024-03-26T04:19:15.128599+0000 mon.a (mon.0) 3589 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "10"}]: dispatch 2024-03-26T04:19:16.703 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.706+0000 7fc29effd640 1 -- 172.21.15.175:0/1928261175 <== mon.2 v2:172.21.15.175:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 0e4b321e-b83b-43c2-ade1-c76622f394b4 v769) ==== 223+0+0 (secure 0 0 0) 0x7fc2a4064810 con 0x7fc2a8100f40 2024-03-26T04:19:16.704 INFO:tasks.workunit.client.0.smithi175.stderr:set-quota max_objects = 10 for pool 0e4b321e-b83b-43c2-ade1-c76622f394b4 2024-03-26T04:19:16.707 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.710+0000 7fc2afe4c640 1 -- 172.21.15.175:0/1928261175 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fc29406e760 msgr2=0x7fc294070c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:19:16.708 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.710+0000 7fc2afe4c640 1 --2- 172.21.15.175:0/1928261175 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fc29406e760 0x7fc294070c20 secure :-1 s=READY pgs=3755 cs=0 l=1 rev1=1 crypto rx=0x7fc298009790 tx=0x7fc298005790 comp rx=0 tx=0).stop 2024-03-26T04:19:16.708 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.710+0000 7fc2afe4c640 1 -- 172.21.15.175:0/1928261175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2a8100f40 msgr2=0x7fc2a819a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:19:16.708 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.710+0000 7fc2afe4c640 1 --2- 172.21.15.175:0/1928261175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2a8100f40 0x7fc2a819a130 secure :-1 s=READY pgs=3015 cs=0 l=1 rev1=1 crypto rx=0x7fc2a400e9d0 tx=0x7fc2a400ee90 comp rx=0 tx=0).stop 2024-03-26T04:19:16.708 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.710+0000 7fc2afe4c640 1 -- 172.21.15.175:0/1928261175 shutdown_connections 2024-03-26T04:19:16.708 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.710+0000 7fc2afe4c640 1 --2- 172.21.15.175:0/1928261175 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fc29406e760 0x7fc294070c20 unknown :-1 s=CLOSED pgs=3755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:16.708 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.710+0000 7fc2afe4c640 1 --2- 172.21.15.175:0/1928261175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2a8105d60 0x7fc2a819ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:16.708 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.710+0000 7fc2afe4c640 1 --2- 172.21.15.175:0/1928261175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2a81018f0 0x7fc2a819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:16.708 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.710+0000 7fc2afe4c640 1 --2- 172.21.15.175:0/1928261175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2a8100f40 0x7fc2a819a130 unknown :-1 s=CLOSED pgs=3015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:16.708 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.710+0000 7fc2afe4c640 1 -- 172.21.15.175:0/1928261175 >> 172.21.15.175:0/1928261175 conn(0x7fc2a80778e0 msgr2=0x7fc2a806a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T04:19:16.708 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.710+0000 7fc2afe4c640 1 -- 172.21.15.175:0/1928261175 shutdown_connections 2024-03-26T04:19:16.708 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.710+0000 7fc2afe4c640 1 -- 172.21.15.175:0/1928261175 wait complete. 2024-03-26T04:19:16.731 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph osd pool application enable 0e4b321e-b83b-43c2-ade1-c76622f394b4 rados 2024-03-26T04:19:16.855 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3b86ec640 1 -- 172.21.15.175:0/1220704465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3b01069d0 msgr2=0x7fa3b0106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:19:16.855 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3b86ec640 1 --2- 172.21.15.175:0/1220704465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3b01069d0 0x7fa3b0106db0 secure :-1 s=READY pgs=2891 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac009960 tx=0x7fa3ac01c7c0 comp rx=0 tx=0).stop 2024-03-26T04:19:16.855 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3b86ec640 1 -- 172.21.15.175:0/1220704465 shutdown_connections 2024-03-26T04:19:16.855 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3b86ec640 1 --2- 172.21.15.175:0/1220704465 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3b0101390 0x7fa3b010d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:16.855 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3b86ec640 1 --2- 172.21.15.175:0/1220704465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3b01009d0 0x7fa3b0100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:16.855 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3b86ec640 1 --2- 172.21.15.175:0/1220704465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3b01069d0 0x7fa3b0106db0 unknown :-1 s=CLOSED pgs=2891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:16.855 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3b86ec640 1 -- 172.21.15.175:0/1220704465 >> 172.21.15.175:0/1220704465 conn(0x7fa3b00fc4c0 msgr2=0x7fa3b00fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T04:19:16.855 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3b86ec640 1 -- 172.21.15.175:0/1220704465 shutdown_connections 2024-03-26T04:19:16.856 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3b86ec640 1 -- 172.21.15.175:0/1220704465 wait complete. 2024-03-26T04:19:16.858 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3b86ec640 1 Processor -- start 2024-03-26T04:19:16.858 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3b86ec640 1 -- start start 2024-03-26T04:19:16.858 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3b86ec640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3b01009d0 0x7fa3b019b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:19:16.858 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3b86ec640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3b0101390 0x7fa3b019b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:19:16.858 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3b86ec640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3b01069d0 0x7fa3b01a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:19:16.858 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3b86ec640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b01129f0 con 0x7fa3b01069d0 2024-03-26T04:19:16.858 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3b86ec640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b0112870 con 0x7fa3b0101390 2024-03-26T04:19:16.858 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3b86ec640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa3b0112b70 con 0x7fa3b01009d0 2024-03-26T04:19:16.858 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3b6488640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3b01009d0 0x7fa3b019b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:19:16.858 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3b6c89640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3b01069d0 0x7fa3b01a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:19:16.858 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3b6488640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3b01009d0 0x7fa3b019b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:45022/0 (socket says 172.21.15.175:45022) 2024-03-26T04:19:16.858 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3b6488640 1 -- 172.21.15.175:0/1545855106 learned_addr learned my addr 172.21.15.175:0/1545855106 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T04:19:16.858 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3b5c87640 1 --2- 172.21.15.175:0/1545855106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3b0101390 0x7fa3b019b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:19:16.859 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3b6488640 1 -- 172.21.15.175:0/1545855106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3b0101390 msgr2=0x7fa3b019b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:19:16.859 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3b6488640 1 --2- 172.21.15.175:0/1545855106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3b0101390 0x7fa3b019b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:16.859 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3b6488640 1 -- 172.21.15.175:0/1545855106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3b01069d0 msgr2=0x7fa3b01a60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:19:16.859 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3b6488640 1 --2- 172.21.15.175:0/1545855106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3b01069d0 0x7fa3b01a60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:16.859 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3b6488640 1 -- 172.21.15.175:0/1545855106 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3b01a65e0 con 0x7fa3b01009d0 2024-03-26T04:19:16.859 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3b6c89640 1 --2- 172.21.15.175:0/1545855106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3b01069d0 0x7fa3b01a60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T04:19:16.859 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3b6488640 1 --2- 172.21.15.175:0/1545855106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3b01009d0 0x7fa3b019b020 secure :-1 s=READY pgs=3016 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac01b040 tx=0x7fa3ac0add40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T04:19:16.859 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3a77fe640 1 -- 172.21.15.175:0/1545855106 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac0bc020 con 0x7fa3b01009d0 2024-03-26T04:19:16.859 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3b86ec640 1 -- 172.21.15.175:0/1545855106 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3b01a67b0 con 0x7fa3b01009d0 2024-03-26T04:19:16.859 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3a77fe640 1 -- 172.21.15.175:0/1545855106 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fa3ac0044c0 con 0x7fa3b01009d0 2024-03-26T04:19:16.859 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3a77fe640 1 -- 172.21.15.175:0/1545855106 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac0b2040 con 0x7fa3b01009d0 2024-03-26T04:19:16.859 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3b86ec640 1 -- 172.21.15.175:0/1545855106 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3b01a6c00 con 0x7fa3b01009d0 2024-03-26T04:19:16.859 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.858+0000 7fa3b86ec640 1 -- 172.21.15.175:0/1545855106 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3b0068e90 con 0x7fa3b01009d0 2024-03-26T04:19:16.860 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.862+0000 7fa3a77fe640 1 -- 172.21.15.175:0/1545855106 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7fa3ac0a5810 con 0x7fa3b01009d0 2024-03-26T04:19:16.860 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.862+0000 7fa3a77fe640 1 --2- 172.21.15.175:0/1545855106 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fa39c06e760 0x7fa39c070c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:19:16.860 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.862+0000 7fa3a77fe640 1 -- 172.21.15.175:0/1545855106 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(769..769 src has 1..769) ==== 9067+0+0 (secure 0 0 0) 0x7fa3ac129710 con 0x7fa3b01009d0 2024-03-26T04:19:16.860 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.862+0000 7fa3b5c87640 1 --2- 172.21.15.175:0/1545855106 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fa39c06e760 0x7fa39c070c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:19:16.860 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.862+0000 7fa3b5c87640 1 --2- 172.21.15.175:0/1545855106 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fa39c06e760 0x7fa39c070c20 secure :-1 s=READY pgs=3756 cs=0 l=1 rev1=1 crypto rx=0x7fa3a00058d0 tx=0x7fa3a0005860 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T04:19:16.863 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:16.866+0000 7fa3a77fe640 1 -- 172.21.15.175:0/1545855106 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7fa3ac0c6060 con 0x7fa3b01009d0 2024-03-26T04:19:16.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:16 smithi175 bash[21527]: audit 2024-03-26T04:19:15.680381+0000 mon.a (mon.0) 3590 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "10"}]': finished 2024-03-26T04:19:16.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:16 smithi175 bash[21527]: cluster 2024-03-26T04:19:15.689700+0000 mon.a (mon.0) 3591 : cluster [DBG] osdmap e768: 8 total, 8 up, 8 in 2024-03-26T04:19:16.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:16 smithi175 bash[21527]: cluster 2024-03-26T04:19:15.787508+0000 mgr.y (mgr.24431) 1295 : cluster [DBG] pgmap v1714: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:16.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:16 smithi175 bash[21527]: audit 2024-03-26T04:19:15.801038+0000 mon.c (mon.2) 864 : audit [INF] from='client.? 172.21.15.175:0/1928261175' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "10"}]: dispatch 2024-03-26T04:19:16.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:16 smithi175 bash[21527]: audit 2024-03-26T04:19:15.801703+0000 mon.a (mon.0) 3592 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "10"}]: dispatch 2024-03-26T04:19:16.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:16 smithi175 bash[29284]: audit 2024-03-26T04:19:15.680381+0000 mon.a (mon.0) 3590 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "10"}]': finished 2024-03-26T04:19:16.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:16 smithi175 bash[29284]: cluster 2024-03-26T04:19:15.689700+0000 mon.a (mon.0) 3591 : cluster [DBG] osdmap e768: 8 total, 8 up, 8 in 2024-03-26T04:19:16.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:16 smithi175 bash[29284]: cluster 2024-03-26T04:19:15.787508+0000 mgr.y (mgr.24431) 1295 : cluster [DBG] pgmap v1714: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:16.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:16 smithi175 bash[29284]: audit 2024-03-26T04:19:15.801038+0000 mon.c (mon.2) 864 : audit [INF] from='client.? 172.21.15.175:0/1928261175' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "10"}]: dispatch 2024-03-26T04:19:16.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:16 smithi175 bash[29284]: audit 2024-03-26T04:19:15.801703+0000 mon.a (mon.0) 3592 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "10"}]: dispatch 2024-03-26T04:19:16.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:16 smithi190 bash[23392]: audit 2024-03-26T04:19:15.680381+0000 mon.a (mon.0) 3590 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "10"}]': finished 2024-03-26T04:19:16.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:16 smithi190 bash[23392]: cluster 2024-03-26T04:19:15.689700+0000 mon.a (mon.0) 3591 : cluster [DBG] osdmap e768: 8 total, 8 up, 8 in 2024-03-26T04:19:16.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:16 smithi190 bash[23392]: cluster 2024-03-26T04:19:15.787508+0000 mgr.y (mgr.24431) 1295 : cluster [DBG] pgmap v1714: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:16.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:16 smithi190 bash[23392]: audit 2024-03-26T04:19:15.801038+0000 mon.c (mon.2) 864 : audit [INF] from='client.? 172.21.15.175:0/1928261175' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "10"}]: dispatch 2024-03-26T04:19:16.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:16 smithi190 bash[23392]: audit 2024-03-26T04:19:15.801703+0000 mon.a (mon.0) 3592 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "10"}]: dispatch 2024-03-26T04:19:17.038 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:17.038+0000 7fa3b86ec640 1 -- 172.21.15.175:0/1545855106 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "app": "rados"} v 0) -- 0x7fa3b01a6e60 con 0x7fa3b01009d0 2024-03-26T04:19:17.728 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:17.730+0000 7fa3a77fe640 1 -- 172.21.15.175:0/1545855106 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "app": "rados"}]=0 enabled application 'rados' on pool '0e4b321e-b83b-43c2-ade1-c76622f394b4' v770) ==== 213+0+0 (secure 0 0 0) 0x7fa3ac0164d0 con 0x7fa3b01009d0 2024-03-26T04:19:17.836 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:17.838+0000 7fa3b86ec640 1 -- 172.21.15.175:0/1545855106 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "app": "rados"} v 0) -- 0x7fa3b00621e0 con 0x7fa3b01009d0 2024-03-26T04:19:17.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:17 smithi175 bash[21527]: audit 2024-03-26T04:19:16.699548+0000 mon.a (mon.0) 3593 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "10"}]': finished 2024-03-26T04:19:17.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:17 smithi175 bash[21527]: cluster 2024-03-26T04:19:16.707873+0000 mon.a (mon.0) 3594 : cluster [DBG] osdmap e769: 8 total, 8 up, 8 in 2024-03-26T04:19:17.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:17 smithi175 bash[21527]: audit 2024-03-26T04:19:17.043537+0000 mon.c (mon.2) 865 : audit [INF] from='client.? 172.21.15.175:0/1545855106' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "app": "rados"}]: dispatch 2024-03-26T04:19:17.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:17 smithi175 bash[21527]: audit 2024-03-26T04:19:17.044045+0000 mon.a (mon.0) 3595 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "app": "rados"}]: dispatch 2024-03-26T04:19:17.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:17 smithi175 bash[29284]: audit 2024-03-26T04:19:16.699548+0000 mon.a (mon.0) 3593 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "10"}]': finished 2024-03-26T04:19:17.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:17 smithi175 bash[29284]: cluster 2024-03-26T04:19:16.707873+0000 mon.a (mon.0) 3594 : cluster [DBG] osdmap e769: 8 total, 8 up, 8 in 2024-03-26T04:19:17.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:17 smithi175 bash[29284]: audit 2024-03-26T04:19:17.043537+0000 mon.c (mon.2) 865 : audit [INF] from='client.? 172.21.15.175:0/1545855106' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "app": "rados"}]: dispatch 2024-03-26T04:19:17.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:17 smithi175 bash[29284]: audit 2024-03-26T04:19:17.044045+0000 mon.a (mon.0) 3595 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "app": "rados"}]: dispatch 2024-03-26T04:19:17.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:17 smithi190 bash[23392]: audit 2024-03-26T04:19:16.699548+0000 mon.a (mon.0) 3593 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "10"}]': finished 2024-03-26T04:19:17.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:17 smithi190 bash[23392]: cluster 2024-03-26T04:19:16.707873+0000 mon.a (mon.0) 3594 : cluster [DBG] osdmap e769: 8 total, 8 up, 8 in 2024-03-26T04:19:17.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:17 smithi190 bash[23392]: audit 2024-03-26T04:19:17.043537+0000 mon.c (mon.2) 865 : audit [INF] from='client.? 172.21.15.175:0/1545855106' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "app": "rados"}]: dispatch 2024-03-26T04:19:17.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:17 smithi190 bash[23392]: audit 2024-03-26T04:19:17.044045+0000 mon.a (mon.0) 3595 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "app": "rados"}]: dispatch 2024-03-26T04:19:18.741 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:18.742+0000 7fa3a77fe640 1 -- 172.21.15.175:0/1545855106 <== mon.2 v2:172.21.15.175:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "app": "rados"}]=0 enabled application 'rados' on pool '0e4b321e-b83b-43c2-ade1-c76622f394b4' v771) ==== 213+0+0 (secure 0 0 0) 0x7fa3ac0f60c0 con 0x7fa3b01009d0 2024-03-26T04:19:18.741 INFO:tasks.workunit.client.0.smithi175.stderr:enabled application 'rados' on pool '0e4b321e-b83b-43c2-ade1-c76622f394b4' 2024-03-26T04:19:18.744 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:18.746+0000 7fa3b86ec640 1 -- 172.21.15.175:0/1545855106 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fa39c06e760 msgr2=0x7fa39c070c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:19:18.744 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:18.746+0000 7fa3b86ec640 1 --2- 172.21.15.175:0/1545855106 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fa39c06e760 0x7fa39c070c20 secure :-1 s=READY pgs=3756 cs=0 l=1 rev1=1 crypto rx=0x7fa3a00058d0 tx=0x7fa3a0005860 comp rx=0 tx=0).stop 2024-03-26T04:19:18.744 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:18.746+0000 7fa3b86ec640 1 -- 172.21.15.175:0/1545855106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3b01009d0 msgr2=0x7fa3b019b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:19:18.744 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:18.746+0000 7fa3b86ec640 1 --2- 172.21.15.175:0/1545855106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3b01009d0 0x7fa3b019b020 secure :-1 s=READY pgs=3016 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac01b040 tx=0x7fa3ac0add40 comp rx=0 tx=0).stop 2024-03-26T04:19:18.744 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:18.746+0000 7fa3b86ec640 1 -- 172.21.15.175:0/1545855106 shutdown_connections 2024-03-26T04:19:18.744 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:18.746+0000 7fa3b86ec640 1 --2- 172.21.15.175:0/1545855106 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fa39c06e760 0x7fa39c070c20 unknown :-1 s=CLOSED pgs=3756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:18.744 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:18.746+0000 7fa3b86ec640 1 --2- 172.21.15.175:0/1545855106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3b01069d0 0x7fa3b01a60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:18.745 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:18.746+0000 7fa3b86ec640 1 --2- 172.21.15.175:0/1545855106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3b0101390 0x7fa3b019b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:18.745 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:18.746+0000 7fa3b86ec640 1 --2- 172.21.15.175:0/1545855106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3b01009d0 0x7fa3b019b020 unknown :-1 s=CLOSED pgs=3016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:18.745 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:18.746+0000 7fa3b86ec640 1 -- 172.21.15.175:0/1545855106 >> 172.21.15.175:0/1545855106 conn(0x7fa3b00fc4c0 msgr2=0x7fa3b00fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T04:19:18.745 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:18.746+0000 7fa3b86ec640 1 -- 172.21.15.175:0/1545855106 shutdown_connections 2024-03-26T04:19:18.745 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:18.746+0000 7fa3b86ec640 1 -- 172.21.15.175:0/1545855106 wait complete. 2024-03-26T04:19:18.764 INFO:tasks.workunit.client.0.smithi175.stderr:+ seq 1 10 2024-03-26T04:19:18.768 INFO:tasks.workunit.client.0.smithi175.stderr:+ rados -p 0e4b321e-b83b-43c2-ade1-c76622f394b4 put obj1 /etc/passwd 2024-03-26T04:19:18.811 INFO:tasks.workunit.client.0.smithi175.stderr:+ rados -p 0e4b321e-b83b-43c2-ade1-c76622f394b4 put obj2 /etc/passwd 2024-03-26T04:19:18.850 INFO:tasks.workunit.client.0.smithi175.stderr:+ rados -p 0e4b321e-b83b-43c2-ade1-c76622f394b4 put obj3 /etc/passwd 2024-03-26T04:19:18.889 INFO:tasks.workunit.client.0.smithi175.stderr:+ rados -p 0e4b321e-b83b-43c2-ade1-c76622f394b4 put obj4 /etc/passwd 2024-03-26T04:19:18.927 INFO:tasks.workunit.client.0.smithi175.stderr:+ rados -p 0e4b321e-b83b-43c2-ade1-c76622f394b4 put obj5 /etc/passwd 2024-03-26T04:19:18.966 INFO:tasks.workunit.client.0.smithi175.stderr:+ rados -p 0e4b321e-b83b-43c2-ade1-c76622f394b4 put obj6 /etc/passwd 2024-03-26T04:19:18.990 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:19:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:19:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:19:18.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:18 smithi175 bash[29284]: audit 2024-03-26T04:19:17.724365+0000 mon.a (mon.0) 3596 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "app": "rados"}]': finished 2024-03-26T04:19:18.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:18 smithi175 bash[29284]: cluster 2024-03-26T04:19:17.731910+0000 mon.a (mon.0) 3597 : cluster [DBG] osdmap e770: 8 total, 8 up, 8 in 2024-03-26T04:19:18.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:18 smithi175 bash[29284]: cluster 2024-03-26T04:19:17.788031+0000 mgr.y (mgr.24431) 1296 : cluster [DBG] pgmap v1717: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:18.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:18 smithi175 bash[29284]: audit 2024-03-26T04:19:17.842136+0000 mon.c (mon.2) 866 : audit [INF] from='client.? 172.21.15.175:0/1545855106' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "app": "rados"}]: dispatch 2024-03-26T04:19:18.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:18 smithi175 bash[29284]: audit 2024-03-26T04:19:17.842853+0000 mon.a (mon.0) 3598 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "app": "rados"}]: dispatch 2024-03-26T04:19:18.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:18 smithi175 bash[21527]: audit 2024-03-26T04:19:17.724365+0000 mon.a (mon.0) 3596 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "app": "rados"}]': finished 2024-03-26T04:19:18.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:18 smithi175 bash[21527]: cluster 2024-03-26T04:19:17.731910+0000 mon.a (mon.0) 3597 : cluster [DBG] osdmap e770: 8 total, 8 up, 8 in 2024-03-26T04:19:18.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:18 smithi175 bash[21527]: cluster 2024-03-26T04:19:17.788031+0000 mgr.y (mgr.24431) 1296 : cluster [DBG] pgmap v1717: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:18.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:18 smithi175 bash[21527]: audit 2024-03-26T04:19:17.842136+0000 mon.c (mon.2) 866 : audit [INF] from='client.? 172.21.15.175:0/1545855106' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "app": "rados"}]: dispatch 2024-03-26T04:19:18.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:18 smithi175 bash[21527]: audit 2024-03-26T04:19:17.842853+0000 mon.a (mon.0) 3598 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "app": "rados"}]: dispatch 2024-03-26T04:19:18.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:18 smithi190 bash[23392]: audit 2024-03-26T04:19:17.724365+0000 mon.a (mon.0) 3596 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "app": "rados"}]': finished 2024-03-26T04:19:18.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:18 smithi190 bash[23392]: cluster 2024-03-26T04:19:17.731910+0000 mon.a (mon.0) 3597 : cluster [DBG] osdmap e770: 8 total, 8 up, 8 in 2024-03-26T04:19:18.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:18 smithi190 bash[23392]: cluster 2024-03-26T04:19:17.788031+0000 mgr.y (mgr.24431) 1296 : cluster [DBG] pgmap v1717: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:18.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:18 smithi190 bash[23392]: audit 2024-03-26T04:19:17.842136+0000 mon.c (mon.2) 866 : audit [INF] from='client.? 172.21.15.175:0/1545855106' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "app": "rados"}]: dispatch 2024-03-26T04:19:18.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:18 smithi190 bash[23392]: audit 2024-03-26T04:19:17.842853+0000 mon.a (mon.0) 3598 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "app": "rados"}]: dispatch 2024-03-26T04:19:19.005 INFO:tasks.workunit.client.0.smithi175.stderr:+ rados -p 0e4b321e-b83b-43c2-ade1-c76622f394b4 put obj7 /etc/passwd 2024-03-26T04:19:19.043 INFO:tasks.workunit.client.0.smithi175.stderr:+ rados -p 0e4b321e-b83b-43c2-ade1-c76622f394b4 put obj8 /etc/passwd 2024-03-26T04:19:19.080 INFO:tasks.workunit.client.0.smithi175.stderr:+ rados -p 0e4b321e-b83b-43c2-ade1-c76622f394b4 put obj9 /etc/passwd 2024-03-26T04:19:19.117 INFO:tasks.workunit.client.0.smithi175.stderr:+ rados -p 0e4b321e-b83b-43c2-ade1-c76622f394b4 put obj10 /etc/passwd 2024-03-26T04:19:19.154 INFO:tasks.workunit.client.0.smithi175.stderr:+ sleep 30 2024-03-26T04:19:19.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:19 smithi175 bash[29284]: cluster 2024-03-26T04:19:18.724413+0000 mon.a (mon.0) 3599 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:19:19.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:19 smithi175 bash[29284]: audit 2024-03-26T04:19:18.735845+0000 mon.a (mon.0) 3600 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "app": "rados"}]': finished 2024-03-26T04:19:19.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:19 smithi175 bash[29284]: cluster 2024-03-26T04:19:18.743969+0000 mon.a (mon.0) 3601 : cluster [DBG] osdmap e771: 8 total, 8 up, 8 in 2024-03-26T04:19:19.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:19 smithi175 bash[21527]: cluster 2024-03-26T04:19:18.724413+0000 mon.a (mon.0) 3599 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:19:19.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:19 smithi175 bash[21527]: audit 2024-03-26T04:19:18.735845+0000 mon.a (mon.0) 3600 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "app": "rados"}]': finished 2024-03-26T04:19:19.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:19 smithi175 bash[21527]: cluster 2024-03-26T04:19:18.743969+0000 mon.a (mon.0) 3601 : cluster [DBG] osdmap e771: 8 total, 8 up, 8 in 2024-03-26T04:19:19.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:19 smithi190 bash[23392]: cluster 2024-03-26T04:19:18.724413+0000 mon.a (mon.0) 3599 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T04:19:19.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:19 smithi190 bash[23392]: audit 2024-03-26T04:19:18.735845+0000 mon.a (mon.0) 3600 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "app": "rados"}]': finished 2024-03-26T04:19:19.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:19 smithi190 bash[23392]: cluster 2024-03-26T04:19:18.743969+0000 mon.a (mon.0) 3601 : cluster [DBG] osdmap e771: 8 total, 8 up, 8 in 2024-03-26T04:19:20.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:20 smithi175 bash[29284]: cluster 2024-03-26T04:19:19.789881+0000 mgr.y (mgr.24431) 1297 : cluster [DBG] pgmap v1719: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 29 KiB/s wr, 3 op/s 2024-03-26T04:19:20.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:20 smithi175 bash[21527]: cluster 2024-03-26T04:19:19.789881+0000 mgr.y (mgr.24431) 1297 : cluster [DBG] pgmap v1719: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 29 KiB/s wr, 3 op/s 2024-03-26T04:19:20.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:20 smithi190 bash[23392]: cluster 2024-03-26T04:19:19.789881+0000 mgr.y (mgr.24431) 1297 : cluster [DBG] pgmap v1719: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 29 KiB/s wr, 3 op/s 2024-03-26T04:19:21.747 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:19:21 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:19:22.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:21 smithi175 bash[21527]: audit 2024-03-26T04:19:21.469637+0000 mgr.y (mgr.24431) 1298 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:19:22.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:21 smithi175 bash[21527]: cluster 2024-03-26T04:19:21.790647+0000 mgr.y (mgr.24431) 1299 : cluster [DBG] pgmap v1720: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 20 KiB/s wr, 2 op/s 2024-03-26T04:19:22.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:21 smithi175 bash[29284]: audit 2024-03-26T04:19:21.469637+0000 mgr.y (mgr.24431) 1298 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:19:22.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:21 smithi175 bash[29284]: cluster 2024-03-26T04:19:21.790647+0000 mgr.y (mgr.24431) 1299 : cluster [DBG] pgmap v1720: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 20 KiB/s wr, 2 op/s 2024-03-26T04:19:22.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:21 smithi190 bash[23392]: audit 2024-03-26T04:19:21.469637+0000 mgr.y (mgr.24431) 1298 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:19:22.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:21 smithi190 bash[23392]: cluster 2024-03-26T04:19:21.790647+0000 mgr.y (mgr.24431) 1299 : cluster [DBG] pgmap v1720: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 20 KiB/s wr, 2 op/s 2024-03-26T04:19:23.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:22 smithi175 bash[21527]: cluster 2024-03-26T04:19:21.943139+0000 mon.a (mon.0) 3602 : cluster [WRN] pool '0e4b321e-b83b-43c2-ade1-c76622f394b4' is full (reached quota's max_objects: 10) 2024-03-26T04:19:23.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:22 smithi175 bash[21527]: cluster 2024-03-26T04:19:21.943706+0000 mon.a (mon.0) 3603 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-26T04:19:23.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:22 smithi175 bash[21527]: cluster 2024-03-26T04:19:21.954977+0000 mon.a (mon.0) 3604 : cluster [DBG] osdmap e772: 8 total, 8 up, 8 in 2024-03-26T04:19:23.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:22 smithi175 bash[29284]: cluster 2024-03-26T04:19:21.943139+0000 mon.a (mon.0) 3602 : cluster [WRN] pool '0e4b321e-b83b-43c2-ade1-c76622f394b4' is full (reached quota's max_objects: 10) 2024-03-26T04:19:23.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:22 smithi175 bash[29284]: cluster 2024-03-26T04:19:21.943706+0000 mon.a (mon.0) 3603 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-26T04:19:23.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:22 smithi175 bash[29284]: cluster 2024-03-26T04:19:21.954977+0000 mon.a (mon.0) 3604 : cluster [DBG] osdmap e772: 8 total, 8 up, 8 in 2024-03-26T04:19:23.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:22 smithi190 bash[23392]: cluster 2024-03-26T04:19:21.943139+0000 mon.a (mon.0) 3602 : cluster [WRN] pool '0e4b321e-b83b-43c2-ade1-c76622f394b4' is full (reached quota's max_objects: 10) 2024-03-26T04:19:23.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:22 smithi190 bash[23392]: cluster 2024-03-26T04:19:21.943706+0000 mon.a (mon.0) 3603 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-26T04:19:23.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:22 smithi190 bash[23392]: cluster 2024-03-26T04:19:21.954977+0000 mon.a (mon.0) 3604 : cluster [DBG] osdmap e772: 8 total, 8 up, 8 in 2024-03-26T04:19:24.740 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:24 smithi175 bash[21527]: audit 2024-03-26T04:19:23.458672+0000 mon.a (mon.0) 3605 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:19:24.740 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:24 smithi175 bash[21527]: audit 2024-03-26T04:19:23.461588+0000 mon.c (mon.2) 867 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:19:24.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:24 smithi175 bash[21527]: cluster 2024-03-26T04:19:23.791364+0000 mgr.y (mgr.24431) 1300 : cluster [DBG] pgmap v1722: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 844 B/s rd, 20 KiB/s wr, 2 op/s 2024-03-26T04:19:24.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:24 smithi175 bash[29284]: audit 2024-03-26T04:19:23.458672+0000 mon.a (mon.0) 3605 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:19:24.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:24 smithi175 bash[29284]: audit 2024-03-26T04:19:23.461588+0000 mon.c (mon.2) 867 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:19:24.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:24 smithi175 bash[29284]: cluster 2024-03-26T04:19:23.791364+0000 mgr.y (mgr.24431) 1300 : cluster [DBG] pgmap v1722: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 844 B/s rd, 20 KiB/s wr, 2 op/s 2024-03-26T04:19:24.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:24 smithi190 bash[23392]: audit 2024-03-26T04:19:23.458672+0000 mon.a (mon.0) 3605 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:19:24.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:24 smithi190 bash[23392]: audit 2024-03-26T04:19:23.461588+0000 mon.c (mon.2) 867 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:19:24.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:24 smithi190 bash[23392]: cluster 2024-03-26T04:19:23.791364+0000 mgr.y (mgr.24431) 1300 : cluster [DBG] pgmap v1722: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 844 B/s rd, 20 KiB/s wr, 2 op/s 2024-03-26T04:19:26.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:25 smithi175 bash[21527]: cluster 2024-03-26T04:19:25.793269+0000 mgr.y (mgr.24431) 1301 : cluster [DBG] pgmap v1723: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-03-26T04:19:26.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:25 smithi175 bash[29284]: cluster 2024-03-26T04:19:25.793269+0000 mgr.y (mgr.24431) 1301 : cluster [DBG] pgmap v1723: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-03-26T04:19:26.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:25 smithi190 bash[23392]: cluster 2024-03-26T04:19:25.793269+0000 mgr.y (mgr.24431) 1301 : cluster [DBG] pgmap v1723: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-03-26T04:19:28.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:27 smithi175 bash[21527]: cluster 2024-03-26T04:19:27.794020+0000 mgr.y (mgr.24431) 1302 : cluster [DBG] pgmap v1724: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 13 KiB/s wr, 2 op/s 2024-03-26T04:19:28.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:27 smithi175 bash[29284]: cluster 2024-03-26T04:19:27.794020+0000 mgr.y (mgr.24431) 1302 : cluster [DBG] pgmap v1724: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 13 KiB/s wr, 2 op/s 2024-03-26T04:19:28.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:27 smithi190 bash[23392]: cluster 2024-03-26T04:19:27.794020+0000 mgr.y (mgr.24431) 1302 : cluster [DBG] pgmap v1724: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 13 KiB/s wr, 2 op/s 2024-03-26T04:19:29.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:19:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:19:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:19:30.243 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:29 smithi175 bash[29284]: cluster 2024-03-26T04:19:29.795849+0000 mgr.y (mgr.24431) 1303 : cluster [DBG] pgmap v1725: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T04:19:30.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:29 smithi175 bash[21527]: cluster 2024-03-26T04:19:29.795849+0000 mgr.y (mgr.24431) 1303 : cluster [DBG] pgmap v1725: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T04:19:30.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:29 smithi190 bash[23392]: cluster 2024-03-26T04:19:29.795849+0000 mgr.y (mgr.24431) 1303 : cluster [DBG] pgmap v1725: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T04:19:31.748 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:19:31 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:19:32.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:31 smithi175 bash[29284]: audit 2024-03-26T04:19:31.475095+0000 mgr.y (mgr.24431) 1304 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:19:32.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:31 smithi175 bash[29284]: cluster 2024-03-26T04:19:31.796602+0000 mgr.y (mgr.24431) 1305 : cluster [DBG] pgmap v1726: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T04:19:32.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:31 smithi175 bash[21527]: audit 2024-03-26T04:19:31.475095+0000 mgr.y (mgr.24431) 1304 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:19:32.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:31 smithi175 bash[21527]: cluster 2024-03-26T04:19:31.796602+0000 mgr.y (mgr.24431) 1305 : cluster [DBG] pgmap v1726: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T04:19:32.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:31 smithi190 bash[23392]: audit 2024-03-26T04:19:31.475095+0000 mgr.y (mgr.24431) 1304 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:19:32.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:31 smithi190 bash[23392]: cluster 2024-03-26T04:19:31.796602+0000 mgr.y (mgr.24431) 1305 : cluster [DBG] pgmap v1726: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T04:19:34.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:33 smithi175 bash[21527]: cluster 2024-03-26T04:19:33.797388+0000 mgr.y (mgr.24431) 1306 : cluster [DBG] pgmap v1727: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 864 B/s rd, 0 op/s 2024-03-26T04:19:34.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:33 smithi175 bash[29284]: cluster 2024-03-26T04:19:33.797388+0000 mgr.y (mgr.24431) 1306 : cluster [DBG] pgmap v1727: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 864 B/s rd, 0 op/s 2024-03-26T04:19:34.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:33 smithi190 bash[23392]: cluster 2024-03-26T04:19:33.797388+0000 mgr.y (mgr.24431) 1306 : cluster [DBG] pgmap v1727: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 864 B/s rd, 0 op/s 2024-03-26T04:19:36.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:35 smithi175 bash[21527]: cluster 2024-03-26T04:19:35.799318+0000 mgr.y (mgr.24431) 1307 : cluster [DBG] pgmap v1728: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:36.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:35 smithi175 bash[29284]: cluster 2024-03-26T04:19:35.799318+0000 mgr.y (mgr.24431) 1307 : cluster [DBG] pgmap v1728: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:36.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:35 smithi190 bash[23392]: cluster 2024-03-26T04:19:35.799318+0000 mgr.y (mgr.24431) 1307 : cluster [DBG] pgmap v1728: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:38.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:37 smithi175 bash[21527]: cluster 2024-03-26T04:19:37.800135+0000 mgr.y (mgr.24431) 1308 : cluster [DBG] pgmap v1729: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:19:38.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:37 smithi175 bash[29284]: cluster 2024-03-26T04:19:37.800135+0000 mgr.y (mgr.24431) 1308 : cluster [DBG] pgmap v1729: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:19:38.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:37 smithi190 bash[23392]: cluster 2024-03-26T04:19:37.800135+0000 mgr.y (mgr.24431) 1308 : cluster [DBG] pgmap v1729: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:19:39.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:38 smithi175 bash[21527]: audit 2024-03-26T04:19:38.494245+0000 mon.c (mon.2) 868 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:19:39.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:19:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:19:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:19:39.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:38 smithi175 bash[29284]: audit 2024-03-26T04:19:38.494245+0000 mon.c (mon.2) 868 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:19:39.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:38 smithi190 bash[23392]: audit 2024-03-26T04:19:38.494245+0000 mon.c (mon.2) 868 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:19:40.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:39 smithi175 bash[29284]: cluster 2024-03-26T04:19:39.802100+0000 mgr.y (mgr.24431) 1309 : cluster [DBG] pgmap v1730: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:40.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:39 smithi175 bash[21527]: cluster 2024-03-26T04:19:39.802100+0000 mgr.y (mgr.24431) 1309 : cluster [DBG] pgmap v1730: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:40.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:39 smithi190 bash[23392]: cluster 2024-03-26T04:19:39.802100+0000 mgr.y (mgr.24431) 1309 : cluster [DBG] pgmap v1730: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:41.747 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:19:41 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:19:42.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:41 smithi175 bash[21527]: audit 2024-03-26T04:19:41.486583+0000 mgr.y (mgr.24431) 1310 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:19:42.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:41 smithi175 bash[21527]: cluster 2024-03-26T04:19:41.802918+0000 mgr.y (mgr.24431) 1311 : cluster [DBG] pgmap v1731: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:19:42.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:41 smithi175 bash[29284]: audit 2024-03-26T04:19:41.486583+0000 mgr.y (mgr.24431) 1310 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:19:42.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:41 smithi175 bash[29284]: cluster 2024-03-26T04:19:41.802918+0000 mgr.y (mgr.24431) 1311 : cluster [DBG] pgmap v1731: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:19:42.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:41 smithi190 bash[23392]: audit 2024-03-26T04:19:41.486583+0000 mgr.y (mgr.24431) 1310 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:19:42.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:41 smithi190 bash[23392]: cluster 2024-03-26T04:19:41.802918+0000 mgr.y (mgr.24431) 1311 : cluster [DBG] pgmap v1731: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:19:44.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:43 smithi175 bash[21527]: cluster 2024-03-26T04:19:43.803767+0000 mgr.y (mgr.24431) 1312 : cluster [DBG] pgmap v1732: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:19:44.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:43 smithi175 bash[29284]: cluster 2024-03-26T04:19:43.803767+0000 mgr.y (mgr.24431) 1312 : cluster [DBG] pgmap v1732: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:19:44.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:43 smithi190 bash[23392]: cluster 2024-03-26T04:19:43.803767+0000 mgr.y (mgr.24431) 1312 : cluster [DBG] pgmap v1732: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:19:46.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:45 smithi175 bash[21527]: cluster 2024-03-26T04:19:45.805766+0000 mgr.y (mgr.24431) 1313 : cluster [DBG] pgmap v1733: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:46.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:45 smithi175 bash[29284]: cluster 2024-03-26T04:19:45.805766+0000 mgr.y (mgr.24431) 1313 : cluster [DBG] pgmap v1733: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:46.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:45 smithi190 bash[23392]: cluster 2024-03-26T04:19:45.805766+0000 mgr.y (mgr.24431) 1313 : cluster [DBG] pgmap v1733: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:48.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:47 smithi175 bash[21527]: cluster 2024-03-26T04:19:47.806536+0000 mgr.y (mgr.24431) 1314 : cluster [DBG] pgmap v1734: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:19:48.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:47 smithi175 bash[29284]: cluster 2024-03-26T04:19:47.806536+0000 mgr.y (mgr.24431) 1314 : cluster [DBG] pgmap v1734: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:19:48.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:47 smithi190 bash[23392]: cluster 2024-03-26T04:19:47.806536+0000 mgr.y (mgr.24431) 1314 : cluster [DBG] pgmap v1734: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:19:49.156 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=136124 2024-03-26T04:19:49.156 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph osd pool set-quota 0e4b321e-b83b-43c2-ade1-c76622f394b4 max_objects 100 2024-03-26T04:19:49.156 INFO:tasks.workunit.client.0.smithi175.stderr:+ rados -p 0e4b321e-b83b-43c2-ade1-c76622f394b4 put onemore /etc/passwd 2024-03-26T04:19:49.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:19:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:19:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:19:49.283 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.281+0000 7fa14b6aa640 1 -- 172.21.15.175:0/511879716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1441024c0 msgr2=0x7fa14410e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:19:49.283 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.281+0000 7fa14b6aa640 1 --2- 172.21.15.175:0/511879716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1441024c0 0x7fa14410e9d0 secure :-1 s=READY pgs=2909 cs=0 l=1 rev1=1 crypto rx=0x7fa13c009ce0 tx=0x7fa13c01cc80 comp rx=0 tx=0).stop 2024-03-26T04:19:49.283 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.281+0000 7fa14b6aa640 1 -- 172.21.15.175:0/511879716 shutdown_connections 2024-03-26T04:19:49.283 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.281+0000 7fa14b6aa640 1 --2- 172.21.15.175:0/511879716 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa144101b00 0x7fa144101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:49.283 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.281+0000 7fa14b6aa640 1 --2- 172.21.15.175:0/511879716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1441024c0 0x7fa14410e9d0 unknown :-1 s=CLOSED pgs=2909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:49.283 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.281+0000 7fa14b6aa640 1 --2- 172.21.15.175:0/511879716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa144107b90 0x7fa144107f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:49.283 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.281+0000 7fa14b6aa640 1 -- 172.21.15.175:0/511879716 >> 172.21.15.175:0/511879716 conn(0x7fa1440fd610 msgr2=0x7fa1440ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T04:19:49.283 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.281+0000 7fa14b6aa640 1 -- 172.21.15.175:0/511879716 shutdown_connections 2024-03-26T04:19:49.284 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.281+0000 7fa14b6aa640 1 -- 172.21.15.175:0/511879716 wait complete. 2024-03-26T04:19:49.284 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.281+0000 7fa14b6aa640 1 Processor -- start 2024-03-26T04:19:49.284 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.281+0000 7fa14b6aa640 1 -- start start 2024-03-26T04:19:49.284 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.281+0000 7fa14b6aa640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa144101b00 0x7fa14419a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:19:49.284 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.281+0000 7fa14b6aa640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1441024c0 0x7fa14419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:19:49.284 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.281+0000 7fa14b6aa640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa144107b90 0x7fa14419e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:19:49.284 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.281+0000 7fa14b6aa640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa144113af0 con 0x7fa1441024c0 2024-03-26T04:19:49.284 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.281+0000 7fa14b6aa640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa144113970 con 0x7fa144101b00 2024-03-26T04:19:49.284 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.281+0000 7fa14b6aa640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa144113c70 con 0x7fa144107b90 2024-03-26T04:19:49.284 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.281+0000 7fa148c45640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1441024c0 0x7fa14419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:19:49.284 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.281+0000 7fa149c47640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa144107b90 0x7fa14419e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:19:49.284 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.281+0000 7fa148c45640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1441024c0 0x7fa14419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45376/0 (socket says 172.21.15.175:45376) 2024-03-26T04:19:49.284 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.281+0000 7fa148c45640 1 -- 172.21.15.175:0/4150729851 learned_addr learned my addr 172.21.15.175:0/4150729851 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T04:19:49.285 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.281+0000 7fa149446640 1 --2- 172.21.15.175:0/4150729851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa144101b00 0x7fa14419a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:19:49.285 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.281+0000 7fa148c45640 1 -- 172.21.15.175:0/4150729851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa144107b90 msgr2=0x7fa14419e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:19:49.285 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.281+0000 7fa148c45640 1 --2- 172.21.15.175:0/4150729851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa144107b90 0x7fa14419e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:49.285 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.281+0000 7fa148c45640 1 -- 172.21.15.175:0/4150729851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa144101b00 msgr2=0x7fa14419a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:19:49.285 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.285+0000 7fa148c45640 1 --2- 172.21.15.175:0/4150729851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa144101b00 0x7fa14419a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:49.285 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.285+0000 7fa148c45640 1 -- 172.21.15.175:0/4150729851 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14419f0a0 con 0x7fa1441024c0 2024-03-26T04:19:49.285 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.285+0000 7fa149c47640 1 --2- 172.21.15.175:0/4150729851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa144107b90 0x7fa14419e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T04:19:49.285 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.285+0000 7fa148c45640 1 --2- 172.21.15.175:0/4150729851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1441024c0 0x7fa14419a630 secure :-1 s=READY pgs=2910 cs=0 l=1 rev1=1 crypto rx=0x7fa13400ea10 tx=0x7fa13400eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T04:19:49.285 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.285+0000 7fa13a7fc640 1 -- 172.21.15.175:0/4150729851 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13400cde0 con 0x7fa1441024c0 2024-03-26T04:19:49.285 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.285+0000 7fa14b6aa640 1 -- 172.21.15.175:0/4150729851 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa14419f390 con 0x7fa1441024c0 2024-03-26T04:19:49.285 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.285+0000 7fa13a7fc640 1 -- 172.21.15.175:0/4150729851 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fa1340044a0 con 0x7fa1441024c0 2024-03-26T04:19:49.285 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.285+0000 7fa13a7fc640 1 -- 172.21.15.175:0/4150729851 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa134010430 con 0x7fa1441024c0 2024-03-26T04:19:49.286 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.285+0000 7fa14b6aa640 1 -- 172.21.15.175:0/4150729851 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1441a6c70 con 0x7fa1441024c0 2024-03-26T04:19:49.287 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.285+0000 7fa14b6aa640 1 -- 172.21.15.175:0/4150729851 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa144068e90 con 0x7fa1441024c0 2024-03-26T04:19:49.287 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.285+0000 7fa13a7fc640 1 -- 172.21.15.175:0/4150729851 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7fa134003fa0 con 0x7fa1441024c0 2024-03-26T04:19:49.287 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.285+0000 7fa13a7fc640 1 --2- 172.21.15.175:0/4150729851 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fa13006e760 0x7fa130070c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:19:49.287 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.285+0000 7fa149446640 1 --2- 172.21.15.175:0/4150729851 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fa13006e760 0x7fa130070c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:19:49.287 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.285+0000 7fa13a7fc640 1 -- 172.21.15.175:0/4150729851 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(772..772 src has 1..772) ==== 9080+0+0 (secure 0 0 0) 0x7fa13408f8e0 con 0x7fa1441024c0 2024-03-26T04:19:49.287 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.285+0000 7fa13a7fc640 1 -- 172.21.15.175:0/4150729851 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=773}) -- 0x7fa13007a1f0 con 0x7fa1441024c0 2024-03-26T04:19:49.287 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.285+0000 7fa149446640 1 --2- 172.21.15.175:0/4150729851 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fa13006e760 0x7fa130070c20 secure :-1 s=READY pgs=3768 cs=0 l=1 rev1=1 crypto rx=0x7fa14000b9e0 tx=0x7fa140005c60 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T04:19:49.290 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.289+0000 7fa13a7fc640 1 -- 172.21.15.175:0/4150729851 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7fa13405c280 con 0x7fa1441024c0 2024-03-26T04:19:49.459 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.461+0000 7fa14b6aa640 1 -- 172.21.15.175:0/4150729851 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "100"} v 0) -- 0x7fa1441a6e40 con 0x7fa1441024c0 2024-03-26T04:19:49.512 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.513+0000 7fa13a7fc640 1 -- 172.21.15.175:0/4150729851 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 0e4b321e-b83b-43c2-ade1-c76622f394b4 v773) ==== 225+0+0 (secure 0 0 0) 0x7fa13405ff30 con 0x7fa1441024c0 2024-03-26T04:19:49.523 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.521+0000 7fa13a7fc640 1 -- 172.21.15.175:0/4150729851 <== mon.0 v2:172.21.15.175:3300/0 8 ==== osd_map(773..773 src has 1..773) ==== 628+0+0 (secure 0 0 0) 0x7fa134004290 con 0x7fa1441024c0 2024-03-26T04:19:49.523 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.521+0000 7fa13a7fc640 1 -- 172.21.15.175:0/4150729851 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=774}) -- 0x7fa13007b0e0 con 0x7fa1441024c0 2024-03-26T04:19:49.626 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:49.625+0000 7fa14b6aa640 1 -- 172.21.15.175:0/4150729851 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "100"} v 0) -- 0x7fa1441a7010 con 0x7fa1441024c0 2024-03-26T04:19:49.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:49 smithi175 bash[21527]: audit 2024-03-26T04:19:49.464415+0000 mon.a (mon.0) 3606 : audit [INF] from='client.? 172.21.15.175:0/4150729851' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "100"}]: dispatch 2024-03-26T04:19:49.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:49 smithi175 bash[29284]: audit 2024-03-26T04:19:49.464415+0000 mon.a (mon.0) 3606 : audit [INF] from='client.? 172.21.15.175:0/4150729851' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "100"}]: dispatch 2024-03-26T04:19:49.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:49 smithi190 bash[23392]: audit 2024-03-26T04:19:49.464415+0000 mon.a (mon.0) 3606 : audit [INF] from='client.? 172.21.15.175:0/4150729851' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "100"}]: dispatch 2024-03-26T04:19:50.545 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:50.545+0000 7fa13a7fc640 1 -- 172.21.15.175:0/4150729851 <== mon.0 v2:172.21.15.175:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 0e4b321e-b83b-43c2-ade1-c76622f394b4 v774) ==== 225+0+0 (secure 0 0 0) 0x7fa134064760 con 0x7fa1441024c0 2024-03-26T04:19:50.545 INFO:tasks.workunit.client.0.smithi175.stderr:set-quota max_objects = 100 for pool 0e4b321e-b83b-43c2-ade1-c76622f394b4 2024-03-26T04:19:50.548 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:50.549+0000 7fa14b6aa640 1 -- 172.21.15.175:0/4150729851 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fa13006e760 msgr2=0x7fa130070c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:19:50.548 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:50.549+0000 7fa14b6aa640 1 --2- 172.21.15.175:0/4150729851 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fa13006e760 0x7fa130070c20 secure :-1 s=READY pgs=3768 cs=0 l=1 rev1=1 crypto rx=0x7fa14000b9e0 tx=0x7fa140005c60 comp rx=0 tx=0).stop 2024-03-26T04:19:50.548 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:50.549+0000 7fa14b6aa640 1 -- 172.21.15.175:0/4150729851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1441024c0 msgr2=0x7fa14419a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:19:50.548 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:50.549+0000 7fa14b6aa640 1 --2- 172.21.15.175:0/4150729851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1441024c0 0x7fa14419a630 secure :-1 s=READY pgs=2910 cs=0 l=1 rev1=1 crypto rx=0x7fa13400ea10 tx=0x7fa13400eed0 comp rx=0 tx=0).stop 2024-03-26T04:19:50.548 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:50.549+0000 7fa14b6aa640 1 -- 172.21.15.175:0/4150729851 shutdown_connections 2024-03-26T04:19:50.549 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:50.549+0000 7fa14b6aa640 1 --2- 172.21.15.175:0/4150729851 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fa13006e760 0x7fa130070c20 unknown :-1 s=CLOSED pgs=3768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:50.549 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:50.549+0000 7fa14b6aa640 1 --2- 172.21.15.175:0/4150729851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa144107b90 0x7fa14419e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:50.549 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:50.549+0000 7fa14b6aa640 1 --2- 172.21.15.175:0/4150729851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1441024c0 0x7fa14419a630 unknown :-1 s=CLOSED pgs=2910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:50.549 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:50.549+0000 7fa14b6aa640 1 --2- 172.21.15.175:0/4150729851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa144101b00 0x7fa14419a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:50.549 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:50.549+0000 7fa14b6aa640 1 -- 172.21.15.175:0/4150729851 >> 172.21.15.175:0/4150729851 conn(0x7fa1440fd610 msgr2=0x7fa1440fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T04:19:50.549 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:50.549+0000 7fa14b6aa640 1 -- 172.21.15.175:0/4150729851 shutdown_connections 2024-03-26T04:19:50.549 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:50.549+0000 7fa14b6aa640 1 -- 172.21.15.175:0/4150729851 wait complete. 2024-03-26T04:19:50.572 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 136124 2024-03-26T04:19:50.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:50 smithi175 bash[21527]: audit 2024-03-26T04:19:49.517220+0000 mon.a (mon.0) 3607 : audit [INF] from='client.? 172.21.15.175:0/4150729851' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "100"}]': finished 2024-03-26T04:19:50.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:50 smithi175 bash[21527]: cluster 2024-03-26T04:19:49.528348+0000 mon.a (mon.0) 3608 : cluster [DBG] osdmap e773: 8 total, 8 up, 8 in 2024-03-26T04:19:50.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:50 smithi175 bash[21527]: audit 2024-03-26T04:19:49.631494+0000 mon.a (mon.0) 3609 : audit [INF] from='client.? 172.21.15.175:0/4150729851' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "100"}]: dispatch 2024-03-26T04:19:50.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:50 smithi175 bash[21527]: cluster 2024-03-26T04:19:49.808405+0000 mgr.y (mgr.24431) 1315 : cluster [DBG] pgmap v1736: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T04:19:50.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:50 smithi175 bash[29284]: audit 2024-03-26T04:19:49.517220+0000 mon.a (mon.0) 3607 : audit [INF] from='client.? 172.21.15.175:0/4150729851' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "100"}]': finished 2024-03-26T04:19:50.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:50 smithi175 bash[29284]: cluster 2024-03-26T04:19:49.528348+0000 mon.a (mon.0) 3608 : cluster [DBG] osdmap e773: 8 total, 8 up, 8 in 2024-03-26T04:19:50.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:50 smithi175 bash[29284]: audit 2024-03-26T04:19:49.631494+0000 mon.a (mon.0) 3609 : audit [INF] from='client.? 172.21.15.175:0/4150729851' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "100"}]: dispatch 2024-03-26T04:19:50.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:50 smithi175 bash[29284]: cluster 2024-03-26T04:19:49.808405+0000 mgr.y (mgr.24431) 1315 : cluster [DBG] pgmap v1736: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T04:19:50.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:50 smithi190 bash[23392]: audit 2024-03-26T04:19:49.517220+0000 mon.a (mon.0) 3607 : audit [INF] from='client.? 172.21.15.175:0/4150729851' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "100"}]': finished 2024-03-26T04:19:50.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:50 smithi190 bash[23392]: cluster 2024-03-26T04:19:49.528348+0000 mon.a (mon.0) 3608 : cluster [DBG] osdmap e773: 8 total, 8 up, 8 in 2024-03-26T04:19:50.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:50 smithi190 bash[23392]: audit 2024-03-26T04:19:49.631494+0000 mon.a (mon.0) 3609 : audit [INF] from='client.? 172.21.15.175:0/4150729851' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "100"}]: dispatch 2024-03-26T04:19:50.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:50 smithi190 bash[23392]: cluster 2024-03-26T04:19:49.808405+0000 mgr.y (mgr.24431) 1315 : cluster [DBG] pgmap v1736: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T04:19:51.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:51 smithi190 bash[23392]: audit 2024-03-26T04:19:50.549483+0000 mon.a (mon.0) 3610 : audit [INF] from='client.? 172.21.15.175:0/4150729851' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "100"}]': finished 2024-03-26T04:19:51.748 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:51 smithi190 bash[23392]: cluster 2024-03-26T04:19:50.559383+0000 mon.a (mon.0) 3611 : cluster [DBG] osdmap e774: 8 total, 8 up, 8 in 2024-03-26T04:19:51.748 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:19:51 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:19:51.975 INFO:tasks.workunit.client.0.smithi175.stderr:+ [ 0 -ne 0 ] 2024-03-26T04:19:51.975 INFO:tasks.workunit.client.0.smithi175.stderr:+ true 2024-03-26T04:19:51.975 INFO:tasks.workunit.client.0.smithi175.stderr:+ rados -p 0e4b321e-b83b-43c2-ade1-c76622f394b4 put twomore /etc/passwd 2024-03-26T04:19:51.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:51 smithi175 bash[29284]: audit 2024-03-26T04:19:50.549483+0000 mon.a (mon.0) 3610 : audit [INF] from='client.? 172.21.15.175:0/4150729851' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "100"}]': finished 2024-03-26T04:19:51.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:51 smithi175 bash[29284]: cluster 2024-03-26T04:19:50.559383+0000 mon.a (mon.0) 3611 : cluster [DBG] osdmap e774: 8 total, 8 up, 8 in 2024-03-26T04:19:51.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:51 smithi175 bash[21527]: audit 2024-03-26T04:19:50.549483+0000 mon.a (mon.0) 3610 : audit [INF] from='client.? 172.21.15.175:0/4150729851' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "100"}]': finished 2024-03-26T04:19:51.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:51 smithi175 bash[21527]: cluster 2024-03-26T04:19:50.559383+0000 mon.a (mon.0) 3611 : cluster [DBG] osdmap e774: 8 total, 8 up, 8 in 2024-03-26T04:19:52.022 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph osd pool set-quota 0e4b321e-b83b-43c2-ade1-c76622f394b4 max_bytes 100 2024-03-26T04:19:52.145 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fb6297640 1 -- 172.21.15.175:0/1631516689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fb0109080 msgr2=0x7f3fb0110c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:19:52.145 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fb6297640 1 --2- 172.21.15.175:0/1631516689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fb0109080 0x7f3fb0110c00 secure :-1 s=READY pgs=2911 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8009cb0 tx=0x7f3fa801cca0 comp rx=0 tx=0).stop 2024-03-26T04:19:52.145 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fb6297640 1 -- 172.21.15.175:0/1631516689 shutdown_connections 2024-03-26T04:19:52.145 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fb6297640 1 --2- 172.21.15.175:0/1631516689 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fb0109080 0x7f3fb0110c00 unknown :-1 s=CLOSED pgs=2911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:52.145 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fb6297640 1 --2- 172.21.15.175:0/1631516689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fb0104900 0x7f3fb0108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:52.145 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fb6297640 1 --2- 172.21.15.175:0/1631516689 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fb0103f50 0x7f3fb0104330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:52.145 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fb6297640 1 -- 172.21.15.175:0/1631516689 >> 172.21.15.175:0/1631516689 conn(0x7f3fb00ff7a0 msgr2=0x7f3fb0101bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T04:19:52.145 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fb6297640 1 -- 172.21.15.175:0/1631516689 shutdown_connections 2024-03-26T04:19:52.145 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fb6297640 1 -- 172.21.15.175:0/1631516689 wait complete. 2024-03-26T04:19:52.145 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fb6297640 1 Processor -- start 2024-03-26T04:19:52.145 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fb6297640 1 -- start start 2024-03-26T04:19:52.145 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fb6297640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fb0103f50 0x7f3fb019c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:19:52.146 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fb6297640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fb0104900 0x7f3fb019c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:19:52.146 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fb6297640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fb0109080 0x7f3fb01a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:19:52.146 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fb6297640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb0115d20 con 0x7f3fb0104900 2024-03-26T04:19:52.146 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fb6297640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb0115ba0 con 0x7f3fb0103f50 2024-03-26T04:19:52.146 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fb6297640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3fb0115ea0 con 0x7f3fb0109080 2024-03-26T04:19:52.146 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fb4834640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fb0109080 0x7f3fb01a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:19:52.146 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fa7fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fb0104900 0x7f3fb019c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:19:52.146 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fb4834640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fb0109080 0x7f3fb01a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:60484/0 (socket says 172.21.15.175:60484) 2024-03-26T04:19:52.146 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fb4834640 1 -- 172.21.15.175:0/2462648960 learned_addr learned my addr 172.21.15.175:0/2462648960 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T04:19:52.146 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fa7fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fb0104900 0x7f3fb019c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:36908/0 (socket says 172.21.15.175:36908) 2024-03-26T04:19:52.146 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3faffff640 1 --2- 172.21.15.175:0/2462648960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fb0103f50 0x7f3fb019c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:19:52.146 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fb4834640 1 -- 172.21.15.175:0/2462648960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fb0103f50 msgr2=0x7f3fb019c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:19:52.146 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fb4834640 1 --2- 172.21.15.175:0/2462648960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fb0103f50 0x7f3fb019c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:52.147 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fb4834640 1 -- 172.21.15.175:0/2462648960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fb0104900 msgr2=0x7f3fb019c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:19:52.147 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fb4834640 1 --2- 172.21.15.175:0/2462648960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fb0104900 0x7f3fb019c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:52.147 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fb4834640 1 -- 172.21.15.175:0/2462648960 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb01a12b0 con 0x7f3fb0109080 2024-03-26T04:19:52.147 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fa7fff640 1 --2- 172.21.15.175:0/2462648960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fb0104900 0x7f3fb019c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T04:19:52.147 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fb4834640 1 --2- 172.21.15.175:0/2462648960 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fb0109080 0x7f3fb01a0bd0 secure :-1 s=READY pgs=3036 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8005cd0 tx=0x7f3fa8004660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T04:19:52.147 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fadffb640 1 -- 172.21.15.175:0/2462648960 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa80b2040 con 0x7f3fb0109080 2024-03-26T04:19:52.147 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fadffb640 1 -- 172.21.15.175:0/2462648960 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f3fa8004850 con 0x7f3fb0109080 2024-03-26T04:19:52.147 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fb6297640 1 -- 172.21.15.175:0/2462648960 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fb01a1540 con 0x7f3fb0109080 2024-03-26T04:19:52.147 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fadffb640 1 -- 172.21.15.175:0/2462648960 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa80a5e60 con 0x7f3fb0109080 2024-03-26T04:19:52.147 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fb6297640 1 -- 172.21.15.175:0/2462648960 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fb01a8e80 con 0x7f3fb0109080 2024-03-26T04:19:52.148 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.145+0000 7f3fb6297640 1 -- 172.21.15.175:0/2462648960 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fb0068a40 con 0x7f3fb0109080 2024-03-26T04:19:52.148 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.149+0000 7f3fadffb640 1 -- 172.21.15.175:0/2462648960 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7f3fa80ad440 con 0x7f3fb0109080 2024-03-26T04:19:52.148 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.149+0000 7f3fadffb640 1 --2- 172.21.15.175:0/2462648960 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f3f9006e760 0x7f3f90070c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:19:52.148 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.149+0000 7f3fadffb640 1 -- 172.21.15.175:0/2462648960 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(775..775 src has 1..775) ==== 9080+0+0 (secure 0 0 0) 0x7f3fa8129b10 con 0x7f3fb0109080 2024-03-26T04:19:52.148 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.149+0000 7f3faffff640 1 --2- 172.21.15.175:0/2462648960 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f3f9006e760 0x7f3f90070c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:19:52.148 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.149+0000 7f3faffff640 1 --2- 172.21.15.175:0/2462648960 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f3f9006e760 0x7f3f90070c20 secure :-1 s=READY pgs=3770 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009ab0 tx=0x7f3f980091e0 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T04:19:52.152 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.153+0000 7f3fadffb640 1 -- 172.21.15.175:0/2462648960 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f3fa80169b0 con 0x7f3fb0109080 2024-03-26T04:19:52.325 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.325+0000 7f3fb6297640 1 -- 172.21.15.175:0/2462648960 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "100"} v 0) -- 0x7f3fb01a9050 con 0x7f3fb0109080 2024-03-26T04:19:52.969 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:52.969+0000 7f3fadffb640 1 -- 172.21.15.175:0/2462648960 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 0e4b321e-b83b-43c2-ade1-c76622f394b4 v776) ==== 221+0+0 (secure 0 0 0) 0x7f3fa80f64b0 con 0x7f3fb0109080 2024-03-26T04:19:52.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:52 smithi175 bash[29284]: audit 2024-03-26T04:19:51.495655+0000 mgr.y (mgr.24431) 1316 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:19:52.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:52 smithi175 bash[29284]: cluster 2024-03-26T04:19:51.809196+0000 mgr.y (mgr.24431) 1317 : cluster [DBG] pgmap v1738: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:52.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:52 smithi175 bash[29284]: cluster 2024-03-26T04:19:51.950952+0000 mon.a (mon.0) 3612 : cluster [INF] pool '0e4b321e-b83b-43c2-ade1-c76622f394b4' no longer out of quota; removing NO_QUOTA flag 2024-03-26T04:19:52.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:52 smithi175 bash[29284]: cluster 2024-03-26T04:19:51.951519+0000 mon.a (mon.0) 3613 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-26T04:19:52.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:52 smithi175 bash[29284]: cluster 2024-03-26T04:19:51.961868+0000 mon.a (mon.0) 3614 : cluster [DBG] osdmap e775: 8 total, 8 up, 8 in 2024-03-26T04:19:52.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:52 smithi175 bash[29284]: audit 2024-03-26T04:19:52.330647+0000 mon.c (mon.2) 869 : audit [INF] from='client.? 172.21.15.175:0/2462648960' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-26T04:19:52.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:52 smithi175 bash[29284]: audit 2024-03-26T04:19:52.331249+0000 mon.a (mon.0) 3615 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-26T04:19:52.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:52 smithi175 bash[21527]: audit 2024-03-26T04:19:51.495655+0000 mgr.y (mgr.24431) 1316 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:19:52.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:52 smithi175 bash[21527]: cluster 2024-03-26T04:19:51.809196+0000 mgr.y (mgr.24431) 1317 : cluster [DBG] pgmap v1738: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:52.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:52 smithi175 bash[21527]: cluster 2024-03-26T04:19:51.950952+0000 mon.a (mon.0) 3612 : cluster [INF] pool '0e4b321e-b83b-43c2-ade1-c76622f394b4' no longer out of quota; removing NO_QUOTA flag 2024-03-26T04:19:52.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:52 smithi175 bash[21527]: cluster 2024-03-26T04:19:51.951519+0000 mon.a (mon.0) 3613 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-26T04:19:52.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:52 smithi175 bash[21527]: cluster 2024-03-26T04:19:51.961868+0000 mon.a (mon.0) 3614 : cluster [DBG] osdmap e775: 8 total, 8 up, 8 in 2024-03-26T04:19:52.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:52 smithi175 bash[21527]: audit 2024-03-26T04:19:52.330647+0000 mon.c (mon.2) 869 : audit [INF] from='client.? 172.21.15.175:0/2462648960' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-26T04:19:52.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:52 smithi175 bash[21527]: audit 2024-03-26T04:19:52.331249+0000 mon.a (mon.0) 3615 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-26T04:19:52.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:52 smithi190 bash[23392]: audit 2024-03-26T04:19:51.495655+0000 mgr.y (mgr.24431) 1316 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:19:52.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:52 smithi190 bash[23392]: cluster 2024-03-26T04:19:51.809196+0000 mgr.y (mgr.24431) 1317 : cluster [DBG] pgmap v1738: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:52.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:52 smithi190 bash[23392]: cluster 2024-03-26T04:19:51.950952+0000 mon.a (mon.0) 3612 : cluster [INF] pool '0e4b321e-b83b-43c2-ade1-c76622f394b4' no longer out of quota; removing NO_QUOTA flag 2024-03-26T04:19:52.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:52 smithi190 bash[23392]: cluster 2024-03-26T04:19:51.951519+0000 mon.a (mon.0) 3613 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-26T04:19:52.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:52 smithi190 bash[23392]: cluster 2024-03-26T04:19:51.961868+0000 mon.a (mon.0) 3614 : cluster [DBG] osdmap e775: 8 total, 8 up, 8 in 2024-03-26T04:19:52.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:52 smithi190 bash[23392]: audit 2024-03-26T04:19:52.330647+0000 mon.c (mon.2) 869 : audit [INF] from='client.? 172.21.15.175:0/2462648960' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-26T04:19:52.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:52 smithi190 bash[23392]: audit 2024-03-26T04:19:52.331249+0000 mon.a (mon.0) 3615 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-26T04:19:53.078 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:53.077+0000 7f3fb6297640 1 -- 172.21.15.175:0/2462648960 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "100"} v 0) -- 0x7f3fb00621e0 con 0x7f3fb0109080 2024-03-26T04:19:53.967 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:53.965+0000 7f3fadffb640 1 -- 172.21.15.175:0/2462648960 <== mon.2 v2:172.21.15.175:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 0e4b321e-b83b-43c2-ade1-c76622f394b4 v777) ==== 221+0+0 (secure 0 0 0) 0x7f3fa80fa160 con 0x7f3fb0109080 2024-03-26T04:19:53.967 INFO:tasks.workunit.client.0.smithi175.stderr:set-quota max_bytes = 100 for pool 0e4b321e-b83b-43c2-ade1-c76622f394b4 2024-03-26T04:19:53.971 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:53.969+0000 7f3fb6297640 1 -- 172.21.15.175:0/2462648960 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f3f9006e760 msgr2=0x7f3f90070c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:19:53.971 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:53.969+0000 7f3fb6297640 1 --2- 172.21.15.175:0/2462648960 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f3f9006e760 0x7f3f90070c20 secure :-1 s=READY pgs=3770 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009ab0 tx=0x7f3f980091e0 comp rx=0 tx=0).stop 2024-03-26T04:19:53.971 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:53.969+0000 7f3fb6297640 1 -- 172.21.15.175:0/2462648960 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fb0109080 msgr2=0x7f3fb01a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:19:53.971 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:53.969+0000 7f3fb6297640 1 --2- 172.21.15.175:0/2462648960 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fb0109080 0x7f3fb01a0bd0 secure :-1 s=READY pgs=3036 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8005cd0 tx=0x7f3fa8004660 comp rx=0 tx=0).stop 2024-03-26T04:19:53.971 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:53.973+0000 7f3fb6297640 1 -- 172.21.15.175:0/2462648960 shutdown_connections 2024-03-26T04:19:53.971 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:53.973+0000 7f3fb6297640 1 --2- 172.21.15.175:0/2462648960 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f3f9006e760 0x7f3f90070c20 unknown :-1 s=CLOSED pgs=3770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:53.972 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:53.973+0000 7f3fb6297640 1 --2- 172.21.15.175:0/2462648960 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fb0109080 0x7f3fb01a0bd0 unknown :-1 s=CLOSED pgs=3036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:53.972 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:53.973+0000 7f3fb6297640 1 --2- 172.21.15.175:0/2462648960 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fb0104900 0x7f3fb019c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:53.972 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:53.973+0000 7f3fb6297640 1 --2- 172.21.15.175:0/2462648960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3fb0103f50 0x7f3fb019c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:19:53.972 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:53.973+0000 7f3fb6297640 1 -- 172.21.15.175:0/2462648960 >> 172.21.15.175:0/2462648960 conn(0x7f3fb00ff7a0 msgr2=0x7f3fb00ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T04:19:53.972 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:53.973+0000 7f3fb6297640 1 -- 172.21.15.175:0/2462648960 shutdown_connections 2024-03-26T04:19:53.973 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:19:53.973+0000 7f3fb6297640 1 -- 172.21.15.175:0/2462648960 wait complete. 2024-03-26T04:19:53.992 INFO:tasks.workunit.client.0.smithi175.stderr:+ sleep 30 2024-03-26T04:19:54.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:53 smithi175 bash[29284]: audit 2024-03-26T04:19:52.958234+0000 mon.a (mon.0) 3616 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "100"}]': finished 2024-03-26T04:19:54.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:53 smithi175 bash[29284]: cluster 2024-03-26T04:19:52.965024+0000 mon.a (mon.0) 3617 : cluster [DBG] osdmap e776: 8 total, 8 up, 8 in 2024-03-26T04:19:54.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:53 smithi175 bash[29284]: audit 2024-03-26T04:19:53.083257+0000 mon.c (mon.2) 870 : audit [INF] from='client.? 172.21.15.175:0/2462648960' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-26T04:19:54.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:53 smithi175 bash[29284]: audit 2024-03-26T04:19:53.083948+0000 mon.a (mon.0) 3618 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-26T04:19:54.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:53 smithi175 bash[29284]: audit 2024-03-26T04:19:53.539783+0000 mon.a (mon.0) 3619 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:19:54.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:53 smithi175 bash[29284]: audit 2024-03-26T04:19:53.544835+0000 mon.c (mon.2) 871 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:19:54.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:53 smithi175 bash[29284]: cluster 2024-03-26T04:19:53.809937+0000 mgr.y (mgr.24431) 1318 : cluster [DBG] pgmap v1741: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:54.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:53 smithi175 bash[21527]: audit 2024-03-26T04:19:52.958234+0000 mon.a (mon.0) 3616 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "100"}]': finished 2024-03-26T04:19:54.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:53 smithi175 bash[21527]: cluster 2024-03-26T04:19:52.965024+0000 mon.a (mon.0) 3617 : cluster [DBG] osdmap e776: 8 total, 8 up, 8 in 2024-03-26T04:19:54.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:53 smithi175 bash[21527]: audit 2024-03-26T04:19:53.083257+0000 mon.c (mon.2) 870 : audit [INF] from='client.? 172.21.15.175:0/2462648960' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-26T04:19:54.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:53 smithi175 bash[21527]: audit 2024-03-26T04:19:53.083948+0000 mon.a (mon.0) 3618 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-26T04:19:54.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:53 smithi175 bash[21527]: audit 2024-03-26T04:19:53.539783+0000 mon.a (mon.0) 3619 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:19:54.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:53 smithi175 bash[21527]: audit 2024-03-26T04:19:53.544835+0000 mon.c (mon.2) 871 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:19:54.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:53 smithi175 bash[21527]: cluster 2024-03-26T04:19:53.809937+0000 mgr.y (mgr.24431) 1318 : cluster [DBG] pgmap v1741: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:54.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:53 smithi190 bash[23392]: audit 2024-03-26T04:19:52.958234+0000 mon.a (mon.0) 3616 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "100"}]': finished 2024-03-26T04:19:54.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:53 smithi190 bash[23392]: cluster 2024-03-26T04:19:52.965024+0000 mon.a (mon.0) 3617 : cluster [DBG] osdmap e776: 8 total, 8 up, 8 in 2024-03-26T04:19:54.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:53 smithi190 bash[23392]: audit 2024-03-26T04:19:53.083257+0000 mon.c (mon.2) 870 : audit [INF] from='client.? 172.21.15.175:0/2462648960' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-26T04:19:54.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:53 smithi190 bash[23392]: audit 2024-03-26T04:19:53.083948+0000 mon.a (mon.0) 3618 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-26T04:19:54.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:53 smithi190 bash[23392]: audit 2024-03-26T04:19:53.539783+0000 mon.a (mon.0) 3619 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:19:54.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:53 smithi190 bash[23392]: audit 2024-03-26T04:19:53.544835+0000 mon.c (mon.2) 871 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:19:54.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:53 smithi190 bash[23392]: cluster 2024-03-26T04:19:53.809937+0000 mgr.y (mgr.24431) 1318 : cluster [DBG] pgmap v1741: 176 pgs: 176 active+clean; 699 KiB data, 898 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:19:55.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:54 smithi175 bash[21527]: audit 2024-03-26T04:19:53.960944+0000 mon.a (mon.0) 3620 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "100"}]': finished 2024-03-26T04:19:55.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:54 smithi175 bash[21527]: cluster 2024-03-26T04:19:53.968303+0000 mon.a (mon.0) 3621 : cluster [DBG] osdmap e777: 8 total, 8 up, 8 in 2024-03-26T04:19:55.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:54 smithi175 bash[29284]: audit 2024-03-26T04:19:53.960944+0000 mon.a (mon.0) 3620 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "100"}]': finished 2024-03-26T04:19:55.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:54 smithi175 bash[29284]: cluster 2024-03-26T04:19:53.968303+0000 mon.a (mon.0) 3621 : cluster [DBG] osdmap e777: 8 total, 8 up, 8 in 2024-03-26T04:19:55.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:54 smithi190 bash[23392]: audit 2024-03-26T04:19:53.960944+0000 mon.a (mon.0) 3620 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "100"}]': finished 2024-03-26T04:19:55.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:54 smithi190 bash[23392]: cluster 2024-03-26T04:19:53.968303+0000 mon.a (mon.0) 3621 : cluster [DBG] osdmap e777: 8 total, 8 up, 8 in 2024-03-26T04:19:56.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:55 smithi175 bash[21527]: cluster 2024-03-26T04:19:55.811931+0000 mgr.y (mgr.24431) 1319 : cluster [DBG] pgmap v1743: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 973 B/s rd, 4.6 KiB/s wr, 1 op/s 2024-03-26T04:19:56.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:55 smithi175 bash[29284]: cluster 2024-03-26T04:19:55.811931+0000 mgr.y (mgr.24431) 1319 : cluster [DBG] pgmap v1743: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 973 B/s rd, 4.6 KiB/s wr, 1 op/s 2024-03-26T04:19:56.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:55 smithi190 bash[23392]: cluster 2024-03-26T04:19:55.811931+0000 mgr.y (mgr.24431) 1319 : cluster [DBG] pgmap v1743: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 973 B/s rd, 4.6 KiB/s wr, 1 op/s 2024-03-26T04:19:57.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:56 smithi175 bash[21527]: audit 2024-03-26T04:19:56.551248+0000 mon.c (mon.2) 872 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:19:57.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:56 smithi175 bash[21527]: cluster 2024-03-26T04:19:56.953501+0000 mon.a (mon.0) 3622 : cluster [WRN] pool '0e4b321e-b83b-43c2-ade1-c76622f394b4' is full (reached quota's max_bytes: 100 B) 2024-03-26T04:19:57.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:56 smithi175 bash[21527]: cluster 2024-03-26T04:19:56.953999+0000 mon.a (mon.0) 3623 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-26T04:19:57.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:56 smithi175 bash[21527]: cluster 2024-03-26T04:19:56.964692+0000 mon.a (mon.0) 3624 : cluster [DBG] osdmap e778: 8 total, 8 up, 8 in 2024-03-26T04:19:57.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:56 smithi175 bash[29284]: audit 2024-03-26T04:19:56.551248+0000 mon.c (mon.2) 872 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:19:57.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:56 smithi175 bash[29284]: cluster 2024-03-26T04:19:56.953501+0000 mon.a (mon.0) 3622 : cluster [WRN] pool '0e4b321e-b83b-43c2-ade1-c76622f394b4' is full (reached quota's max_bytes: 100 B) 2024-03-26T04:19:57.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:56 smithi175 bash[29284]: cluster 2024-03-26T04:19:56.953999+0000 mon.a (mon.0) 3623 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-26T04:19:57.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:56 smithi175 bash[29284]: cluster 2024-03-26T04:19:56.964692+0000 mon.a (mon.0) 3624 : cluster [DBG] osdmap e778: 8 total, 8 up, 8 in 2024-03-26T04:19:57.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:56 smithi190 bash[23392]: audit 2024-03-26T04:19:56.551248+0000 mon.c (mon.2) 872 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:19:57.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:56 smithi190 bash[23392]: cluster 2024-03-26T04:19:56.953501+0000 mon.a (mon.0) 3622 : cluster [WRN] pool '0e4b321e-b83b-43c2-ade1-c76622f394b4' is full (reached quota's max_bytes: 100 B) 2024-03-26T04:19:57.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:56 smithi190 bash[23392]: cluster 2024-03-26T04:19:56.953999+0000 mon.a (mon.0) 3623 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-26T04:19:57.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:56 smithi190 bash[23392]: cluster 2024-03-26T04:19:56.964692+0000 mon.a (mon.0) 3624 : cluster [DBG] osdmap e778: 8 total, 8 up, 8 in 2024-03-26T04:19:58.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:57 smithi175 bash[29284]: audit 2024-03-26T04:19:57.279349+0000 mon.c (mon.2) 873 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:19:58.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:57 smithi175 bash[29284]: audit 2024-03-26T04:19:57.281152+0000 mon.c (mon.2) 874 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:19:58.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:57 smithi175 bash[29284]: audit 2024-03-26T04:19:57.291731+0000 mon.a (mon.0) 3625 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:19:58.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:57 smithi175 bash[29284]: cluster 2024-03-26T04:19:57.812780+0000 mgr.y (mgr.24431) 1320 : cluster [DBG] pgmap v1745: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 874 B/s rd, 4.1 KiB/s wr, 1 op/s 2024-03-26T04:19:58.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:57 smithi175 bash[21527]: audit 2024-03-26T04:19:57.279349+0000 mon.c (mon.2) 873 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:19:58.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:57 smithi175 bash[21527]: audit 2024-03-26T04:19:57.281152+0000 mon.c (mon.2) 874 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:19:58.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:57 smithi175 bash[21527]: audit 2024-03-26T04:19:57.291731+0000 mon.a (mon.0) 3625 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:19:58.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:57 smithi175 bash[21527]: cluster 2024-03-26T04:19:57.812780+0000 mgr.y (mgr.24431) 1320 : cluster [DBG] pgmap v1745: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 874 B/s rd, 4.1 KiB/s wr, 1 op/s 2024-03-26T04:19:58.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:57 smithi190 bash[23392]: audit 2024-03-26T04:19:57.279349+0000 mon.c (mon.2) 873 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:19:58.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:57 smithi190 bash[23392]: audit 2024-03-26T04:19:57.281152+0000 mon.c (mon.2) 874 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:19:58.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:57 smithi190 bash[23392]: audit 2024-03-26T04:19:57.291731+0000 mon.a (mon.0) 3625 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:19:58.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:57 smithi190 bash[23392]: cluster 2024-03-26T04:19:57.812780+0000 mgr.y (mgr.24431) 1320 : cluster [DBG] pgmap v1745: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 874 B/s rd, 4.1 KiB/s wr, 1 op/s 2024-03-26T04:19:59.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:19:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:19:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:20:00.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:19:59 smithi175 bash[21527]: cluster 2024-03-26T04:19:59.814717+0000 mgr.y (mgr.24431) 1321 : cluster [DBG] pgmap v1746: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.5 KiB/s wr, 1 op/s 2024-03-26T04:20:00.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:19:59 smithi175 bash[29284]: cluster 2024-03-26T04:19:59.814717+0000 mgr.y (mgr.24431) 1321 : cluster [DBG] pgmap v1746: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.5 KiB/s wr, 1 op/s 2024-03-26T04:20:00.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:19:59 smithi190 bash[23392]: cluster 2024-03-26T04:19:59.814717+0000 mgr.y (mgr.24431) 1321 : cluster [DBG] pgmap v1746: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.5 KiB/s wr, 1 op/s 2024-03-26T04:20:01.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:00 smithi175 bash[29284]: cluster 2024-03-26T04:20:00.000173+0000 mon.a (mon.0) 3626 : cluster [WRN] Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled; 1 pool(s) full 2024-03-26T04:20:01.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:00 smithi175 bash[29284]: cluster 2024-03-26T04:20:00.000264+0000 mon.a (mon.0) 3627 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-03-26T04:20:01.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:00 smithi175 bash[29284]: cluster 2024-03-26T04:20:00.000305+0000 mon.a (mon.0) 3628 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-26T04:20:01.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:00 smithi175 bash[29284]: cluster 2024-03-26T04:20:00.000343+0000 mon.a (mon.0) 3629 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-26T04:20:01.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:00 smithi175 bash[29284]: cluster 2024-03-26T04:20:00.000392+0000 mon.a (mon.0) 3630 : cluster [WRN] [WRN] POOL_FULL: 1 pool(s) full 2024-03-26T04:20:01.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:00 smithi175 bash[29284]: cluster 2024-03-26T04:20:00.000434+0000 mon.a (mon.0) 3631 : cluster [WRN] pool '0e4b321e-b83b-43c2-ade1-c76622f394b4' is full (running out of quota) 2024-03-26T04:20:01.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:00 smithi175 bash[21527]: cluster 2024-03-26T04:20:00.000173+0000 mon.a (mon.0) 3626 : cluster [WRN] Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled; 1 pool(s) full 2024-03-26T04:20:01.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:00 smithi175 bash[21527]: cluster 2024-03-26T04:20:00.000264+0000 mon.a (mon.0) 3627 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-03-26T04:20:01.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:00 smithi175 bash[21527]: cluster 2024-03-26T04:20:00.000305+0000 mon.a (mon.0) 3628 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-26T04:20:01.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:00 smithi175 bash[21527]: cluster 2024-03-26T04:20:00.000343+0000 mon.a (mon.0) 3629 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-26T04:20:01.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:00 smithi175 bash[21527]: cluster 2024-03-26T04:20:00.000392+0000 mon.a (mon.0) 3630 : cluster [WRN] [WRN] POOL_FULL: 1 pool(s) full 2024-03-26T04:20:01.242 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:00 smithi175 bash[21527]: cluster 2024-03-26T04:20:00.000434+0000 mon.a (mon.0) 3631 : cluster [WRN] pool '0e4b321e-b83b-43c2-ade1-c76622f394b4' is full (running out of quota) 2024-03-26T04:20:01.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:00 smithi190 bash[23392]: cluster 2024-03-26T04:20:00.000173+0000 mon.a (mon.0) 3626 : cluster [WRN] Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled; 1 pool(s) full 2024-03-26T04:20:01.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:00 smithi190 bash[23392]: cluster 2024-03-26T04:20:00.000264+0000 mon.a (mon.0) 3627 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-03-26T04:20:01.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:00 smithi190 bash[23392]: cluster 2024-03-26T04:20:00.000305+0000 mon.a (mon.0) 3628 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-26T04:20:01.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:00 smithi190 bash[23392]: cluster 2024-03-26T04:20:00.000343+0000 mon.a (mon.0) 3629 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-26T04:20:01.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:00 smithi190 bash[23392]: cluster 2024-03-26T04:20:00.000392+0000 mon.a (mon.0) 3630 : cluster [WRN] [WRN] POOL_FULL: 1 pool(s) full 2024-03-26T04:20:01.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:00 smithi190 bash[23392]: cluster 2024-03-26T04:20:00.000434+0000 mon.a (mon.0) 3631 : cluster [WRN] pool '0e4b321e-b83b-43c2-ade1-c76622f394b4' is full (running out of quota) 2024-03-26T04:20:01.897 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:20:01 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:20:02.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:01 smithi175 bash[21527]: audit 2024-03-26T04:20:01.507087+0000 mgr.y (mgr.24431) 1322 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:20:02.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:01 smithi175 bash[21527]: cluster 2024-03-26T04:20:01.815670+0000 mgr.y (mgr.24431) 1323 : cluster [DBG] pgmap v1747: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-03-26T04:20:02.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:01 smithi175 bash[29284]: audit 2024-03-26T04:20:01.507087+0000 mgr.y (mgr.24431) 1322 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:20:02.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:01 smithi175 bash[29284]: cluster 2024-03-26T04:20:01.815670+0000 mgr.y (mgr.24431) 1323 : cluster [DBG] pgmap v1747: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-03-26T04:20:02.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:01 smithi190 bash[23392]: audit 2024-03-26T04:20:01.507087+0000 mgr.y (mgr.24431) 1322 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:20:02.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:01 smithi190 bash[23392]: cluster 2024-03-26T04:20:01.815670+0000 mgr.y (mgr.24431) 1323 : cluster [DBG] pgmap v1747: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-03-26T04:20:04.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:03 smithi175 bash[21527]: cluster 2024-03-26T04:20:03.816440+0000 mgr.y (mgr.24431) 1324 : cluster [DBG] pgmap v1748: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.4 KiB/s wr, 1 op/s 2024-03-26T04:20:04.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:03 smithi175 bash[29284]: cluster 2024-03-26T04:20:03.816440+0000 mgr.y (mgr.24431) 1324 : cluster [DBG] pgmap v1748: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.4 KiB/s wr, 1 op/s 2024-03-26T04:20:04.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:03 smithi190 bash[23392]: cluster 2024-03-26T04:20:03.816440+0000 mgr.y (mgr.24431) 1324 : cluster [DBG] pgmap v1748: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.4 KiB/s wr, 1 op/s 2024-03-26T04:20:06.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:05 smithi175 bash[29284]: cluster 2024-03-26T04:20:05.818410+0000 mgr.y (mgr.24431) 1325 : cluster [DBG] pgmap v1749: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T04:20:06.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:05 smithi175 bash[21527]: cluster 2024-03-26T04:20:05.818410+0000 mgr.y (mgr.24431) 1325 : cluster [DBG] pgmap v1749: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T04:20:06.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:05 smithi190 bash[23392]: cluster 2024-03-26T04:20:05.818410+0000 mgr.y (mgr.24431) 1325 : cluster [DBG] pgmap v1749: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T04:20:08.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:07 smithi175 bash[21527]: cluster 2024-03-26T04:20:07.819369+0000 mgr.y (mgr.24431) 1326 : cluster [DBG] pgmap v1750: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 942 B/s rd, 0 op/s 2024-03-26T04:20:08.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:07 smithi175 bash[29284]: cluster 2024-03-26T04:20:07.819369+0000 mgr.y (mgr.24431) 1326 : cluster [DBG] pgmap v1750: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 942 B/s rd, 0 op/s 2024-03-26T04:20:08.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:07 smithi190 bash[23392]: cluster 2024-03-26T04:20:07.819369+0000 mgr.y (mgr.24431) 1326 : cluster [DBG] pgmap v1750: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 942 B/s rd, 0 op/s 2024-03-26T04:20:09.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:20:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:20:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:20:09.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:09 smithi175 bash[21527]: audit 2024-03-26T04:20:08.585371+0000 mon.a (mon.0) 3632 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:20:09.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:09 smithi175 bash[21527]: audit 2024-03-26T04:20:08.590042+0000 mon.c (mon.2) 875 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:20:09.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:09 smithi175 bash[29284]: audit 2024-03-26T04:20:08.585371+0000 mon.a (mon.0) 3632 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:20:09.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:09 smithi175 bash[29284]: audit 2024-03-26T04:20:08.590042+0000 mon.c (mon.2) 875 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:20:09.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:09 smithi190 bash[23392]: audit 2024-03-26T04:20:08.585371+0000 mon.a (mon.0) 3632 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:20:09.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:09 smithi190 bash[23392]: audit 2024-03-26T04:20:08.590042+0000 mon.c (mon.2) 875 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:20:10.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:10 smithi175 bash[21527]: cluster 2024-03-26T04:20:09.821313+0000 mgr.y (mgr.24431) 1327 : cluster [DBG] pgmap v1751: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:20:10.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:10 smithi175 bash[29284]: cluster 2024-03-26T04:20:09.821313+0000 mgr.y (mgr.24431) 1327 : cluster [DBG] pgmap v1751: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:20:10.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:10 smithi190 bash[23392]: cluster 2024-03-26T04:20:09.821313+0000 mgr.y (mgr.24431) 1327 : cluster [DBG] pgmap v1751: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:20:11.894 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:20:11 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:20:12.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:11 smithi175 bash[29284]: audit 2024-03-26T04:20:11.518464+0000 mgr.y (mgr.24431) 1328 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:20:12.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:11 smithi175 bash[29284]: cluster 2024-03-26T04:20:11.822105+0000 mgr.y (mgr.24431) 1329 : cluster [DBG] pgmap v1752: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:20:12.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:11 smithi175 bash[21527]: audit 2024-03-26T04:20:11.518464+0000 mgr.y (mgr.24431) 1328 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:20:12.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:11 smithi175 bash[21527]: cluster 2024-03-26T04:20:11.822105+0000 mgr.y (mgr.24431) 1329 : cluster [DBG] pgmap v1752: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:20:12.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:11 smithi190 bash[23392]: audit 2024-03-26T04:20:11.518464+0000 mgr.y (mgr.24431) 1328 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:20:12.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:11 smithi190 bash[23392]: cluster 2024-03-26T04:20:11.822105+0000 mgr.y (mgr.24431) 1329 : cluster [DBG] pgmap v1752: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:20:14.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:13 smithi175 bash[21527]: cluster 2024-03-26T04:20:13.822897+0000 mgr.y (mgr.24431) 1330 : cluster [DBG] pgmap v1753: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:20:14.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:13 smithi175 bash[29284]: cluster 2024-03-26T04:20:13.822897+0000 mgr.y (mgr.24431) 1330 : cluster [DBG] pgmap v1753: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:20:14.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:13 smithi190 bash[23392]: cluster 2024-03-26T04:20:13.822897+0000 mgr.y (mgr.24431) 1330 : cluster [DBG] pgmap v1753: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:20:16.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:15 smithi175 bash[21527]: cluster 2024-03-26T04:20:15.824822+0000 mgr.y (mgr.24431) 1331 : cluster [DBG] pgmap v1754: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:20:16.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:15 smithi175 bash[29284]: cluster 2024-03-26T04:20:15.824822+0000 mgr.y (mgr.24431) 1331 : cluster [DBG] pgmap v1754: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:20:16.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:15 smithi190 bash[23392]: cluster 2024-03-26T04:20:15.824822+0000 mgr.y (mgr.24431) 1331 : cluster [DBG] pgmap v1754: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:20:18.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:17 smithi175 bash[29284]: cluster 2024-03-26T04:20:17.825602+0000 mgr.y (mgr.24431) 1332 : cluster [DBG] pgmap v1755: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:20:18.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:17 smithi175 bash[21527]: cluster 2024-03-26T04:20:17.825602+0000 mgr.y (mgr.24431) 1332 : cluster [DBG] pgmap v1755: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:20:18.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:17 smithi190 bash[23392]: cluster 2024-03-26T04:20:17.825602+0000 mgr.y (mgr.24431) 1332 : cluster [DBG] pgmap v1755: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:20:19.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:20:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:20:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:20:20.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:19 smithi175 bash[21527]: cluster 2024-03-26T04:20:19.827499+0000 mgr.y (mgr.24431) 1333 : cluster [DBG] pgmap v1756: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:20:20.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:19 smithi175 bash[29284]: cluster 2024-03-26T04:20:19.827499+0000 mgr.y (mgr.24431) 1333 : cluster [DBG] pgmap v1756: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:20:20.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:19 smithi190 bash[23392]: cluster 2024-03-26T04:20:19.827499+0000 mgr.y (mgr.24431) 1333 : cluster [DBG] pgmap v1756: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:20:21.899 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:20:21 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:20:22.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:21 smithi175 bash[21527]: audit 2024-03-26T04:20:21.529522+0000 mgr.y (mgr.24431) 1334 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:20:22.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:21 smithi175 bash[21527]: cluster 2024-03-26T04:20:21.828283+0000 mgr.y (mgr.24431) 1335 : cluster [DBG] pgmap v1757: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:20:22.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:21 smithi175 bash[29284]: audit 2024-03-26T04:20:21.529522+0000 mgr.y (mgr.24431) 1334 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:20:22.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:21 smithi175 bash[29284]: cluster 2024-03-26T04:20:21.828283+0000 mgr.y (mgr.24431) 1335 : cluster [DBG] pgmap v1757: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:20:22.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:21 smithi190 bash[23392]: audit 2024-03-26T04:20:21.529522+0000 mgr.y (mgr.24431) 1334 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:20:22.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:21 smithi190 bash[23392]: cluster 2024-03-26T04:20:21.828283+0000 mgr.y (mgr.24431) 1335 : cluster [DBG] pgmap v1757: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:20:23.995 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:23 smithi175 bash[21527]: audit 2024-03-26T04:20:23.622814+0000 mon.c (mon.2) 876 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:20:23.996 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:23 smithi175 bash[29284]: audit 2024-03-26T04:20:23.622814+0000 mon.c (mon.2) 876 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:20:23.996 INFO:tasks.workunit.client.0.smithi175.stderr:+ pid=136224 2024-03-26T04:20:23.997 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph osd pool set-quota 0e4b321e-b83b-43c2-ade1-c76622f394b4 max_bytes 0 2024-03-26T04:20:23.997 INFO:tasks.workunit.client.0.smithi175.stderr:+ rados -p 0e4b321e-b83b-43c2-ade1-c76622f394b4 put two /etc/passwd 2024-03-26T04:20:23.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:23 smithi190 bash[23392]: audit 2024-03-26T04:20:23.622814+0000 mon.c (mon.2) 876 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:20:24.125 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f9310bd1640 1 -- 172.21.15.175:0/3798325069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f930c10e480 msgr2=0x7f930c110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:20:24.126 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f9310bd1640 1 --2- 172.21.15.175:0/3798325069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f930c10e480 0x7f930c110870 secure :-1 s=READY pgs=2912 cs=0 l=1 rev1=1 crypto rx=0x7f9304009bf0 tx=0x7f930401caa0 comp rx=0 tx=0).stop 2024-03-26T04:20:24.126 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f9310bd1640 1 -- 172.21.15.175:0/3798325069 shutdown_connections 2024-03-26T04:20:24.126 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f9310bd1640 1 --2- 172.21.15.175:0/3798325069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f930c10e480 0x7f930c110870 unknown :-1 s=CLOSED pgs=2912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:24.126 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f9310bd1640 1 --2- 172.21.15.175:0/3798325069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f930c06a850 0x7f930c06acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:24.126 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f9310bd1640 1 --2- 172.21.15.175:0/3798325069 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f930c102ab0 0x7f930c06a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:24.126 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f9310bd1640 1 -- 172.21.15.175:0/3798325069 >> 172.21.15.175:0/3798325069 conn(0x7f930c077860 msgr2=0x7f930c077c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T04:20:24.126 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f9310bd1640 1 -- 172.21.15.175:0/3798325069 shutdown_connections 2024-03-26T04:20:24.126 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f9310bd1640 1 -- 172.21.15.175:0/3798325069 wait complete. 2024-03-26T04:20:24.126 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f9310bd1640 1 Processor -- start 2024-03-26T04:20:24.126 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f9310bd1640 1 -- start start 2024-03-26T04:20:24.126 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f9310bd1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f930c06a850 0x7f930c19c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:20:24.126 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f9310bd1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f930c102ab0 0x7f930c19c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:20:24.127 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f9310bd1640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f930c10e480 0x7f930c1a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:20:24.127 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f9310bd1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f930c115cc0 con 0x7f930c102ab0 2024-03-26T04:20:24.127 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f9310bd1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f930c115b40 con 0x7f930c06a850 2024-03-26T04:20:24.127 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f9310bd1640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f930c115e40 con 0x7f930c10e480 2024-03-26T04:20:24.127 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f9309d9b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f930c102ab0 0x7f930c19c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:20:24.127 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f930ad9d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f930c10e480 0x7f930c1a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:20:24.127 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f930ad9d640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f930c10e480 0x7f930c1a0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:40442/0 (socket says 172.21.15.175:40442) 2024-03-26T04:20:24.127 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f930ad9d640 1 -- 172.21.15.175:0/3212534508 learned_addr learned my addr 172.21.15.175:0/3212534508 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T04:20:24.127 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f9309d9b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f930c102ab0 0x7f930c19c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:48800/0 (socket says 172.21.15.175:48800) 2024-03-26T04:20:24.127 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f930a59c640 1 --2- 172.21.15.175:0/3212534508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f930c06a850 0x7f930c19c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:20:24.127 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f930ad9d640 1 -- 172.21.15.175:0/3212534508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f930c06a850 msgr2=0x7f930c19c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:20:24.127 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f930ad9d640 1 --2- 172.21.15.175:0/3212534508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f930c06a850 0x7f930c19c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:24.127 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f930ad9d640 1 -- 172.21.15.175:0/3212534508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f930c102ab0 msgr2=0x7f930c19c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:20:24.127 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f930ad9d640 1 --2- 172.21.15.175:0/3212534508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f930c102ab0 0x7f930c19c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:24.128 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f930ad9d640 1 -- 172.21.15.175:0/3212534508 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f930c1a1210 con 0x7f930c10e480 2024-03-26T04:20:24.128 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f9309d9b640 1 --2- 172.21.15.175:0/3212534508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f930c102ab0 0x7f930c19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T04:20:24.128 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f930ad9d640 1 --2- 172.21.15.175:0/3212534508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f930c10e480 0x7f930c1a0b30 secure :-1 s=READY pgs=3040 cs=0 l=1 rev1=1 crypto rx=0x7f93040b0040 tx=0x7f93040a5b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T04:20:24.128 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f92fb7fe640 1 -- 172.21.15.175:0/3212534508 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93040b2040 con 0x7f930c10e480 2024-03-26T04:20:24.128 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f9310bd1640 1 -- 172.21.15.175:0/3212534508 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f930c1a14a0 con 0x7f930c10e480 2024-03-26T04:20:24.128 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f92fb7fe640 1 -- 172.21.15.175:0/3212534508 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f93040ada70 con 0x7f930c10e480 2024-03-26T04:20:24.128 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f92fb7fe640 1 -- 172.21.15.175:0/3212534508 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93040047c0 con 0x7f930c10e480 2024-03-26T04:20:24.128 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f9310bd1640 1 -- 172.21.15.175:0/3212534508 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f930c1a18f0 con 0x7f930c10e480 2024-03-26T04:20:24.128 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.124+0000 7f9310bd1640 1 -- 172.21.15.175:0/3212534508 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f930c068e90 con 0x7f930c10e480 2024-03-26T04:20:24.129 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.128+0000 7f92fb7fe640 1 -- 172.21.15.175:0/3212534508 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7f93040bd020 con 0x7f930c10e480 2024-03-26T04:20:24.129 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.128+0000 7f92fb7fe640 1 --2- 172.21.15.175:0/3212534508 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f92f406e760 0x7f92f4070c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:20:24.129 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.128+0000 7f92fb7fe640 1 -- 172.21.15.175:0/3212534508 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(778..778 src has 1..778) ==== 9080+0+0 (secure 0 0 0) 0x7f9304129410 con 0x7f930c10e480 2024-03-26T04:20:24.129 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.128+0000 7f930a59c640 1 --2- 172.21.15.175:0/3212534508 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f92f406e760 0x7f92f4070c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:20:24.129 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.128+0000 7f92fb7fe640 1 -- 172.21.15.175:0/3212534508 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=779}) -- 0x7f92f407a1f0 con 0x7f930c10e480 2024-03-26T04:20:24.129 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.128+0000 7f930a59c640 1 --2- 172.21.15.175:0/3212534508 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f92f406e760 0x7f92f4070c20 secure :-1 s=READY pgs=3772 cs=0 l=1 rev1=1 crypto rx=0x7f92fc009ab0 tx=0x7f92fc009260 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T04:20:24.132 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.132+0000 7f92fb7fe640 1 -- 172.21.15.175:0/3212534508 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f93040c4020 con 0x7f930c10e480 2024-03-26T04:20:24.305 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.304+0000 7f9310bd1640 1 -- 172.21.15.175:0/3212534508 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"} v 0) -- 0x7f930c108a10 con 0x7f930c10e480 2024-03-26T04:20:24.688 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.688+0000 7f92fb7fe640 1 -- 172.21.15.175:0/3212534508 <== mon.2 v2:172.21.15.175:3301/0 7 ==== osd_map(779..779 src has 1..779) ==== 628+0+0 (secure 0 0 0) 0x7f930415dcb0 con 0x7f930c10e480 2024-03-26T04:20:24.688 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.688+0000 7f92fb7fe640 1 -- 172.21.15.175:0/3212534508 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=780}) -- 0x7f92f407ad60 con 0x7f930c10e480 2024-03-26T04:20:24.698 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.696+0000 7f92fb7fe640 1 -- 172.21.15.175:0/3212534508 <== mon.2 v2:172.21.15.175:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 0e4b321e-b83b-43c2-ade1-c76622f394b4 v779) ==== 217+0+0 (secure 0 0 0) 0x7f93040167b0 con 0x7f930c10e480 2024-03-26T04:20:24.805 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:24.804+0000 7f9310bd1640 1 -- 172.21.15.175:0/3212534508 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"} v 0) -- 0x7f930c108620 con 0x7f930c10e480 2024-03-26T04:20:24.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:24 smithi175 bash[21527]: cluster 2024-03-26T04:20:23.829053+0000 mgr.y (mgr.24431) 1336 : cluster [DBG] pgmap v1758: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:20:24.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:24 smithi175 bash[21527]: audit 2024-03-26T04:20:24.311119+0000 mon.c (mon.2) 877 : audit [INF] from='client.? 172.21.15.175:0/3212534508' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-26T04:20:24.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:24 smithi175 bash[21527]: audit 2024-03-26T04:20:24.311823+0000 mon.a (mon.0) 3633 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-26T04:20:24.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:24 smithi175 bash[29284]: cluster 2024-03-26T04:20:23.829053+0000 mgr.y (mgr.24431) 1336 : cluster [DBG] pgmap v1758: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:20:24.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:24 smithi175 bash[29284]: audit 2024-03-26T04:20:24.311119+0000 mon.c (mon.2) 877 : audit [INF] from='client.? 172.21.15.175:0/3212534508' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-26T04:20:24.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:24 smithi175 bash[29284]: audit 2024-03-26T04:20:24.311823+0000 mon.a (mon.0) 3633 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-26T04:20:24.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:24 smithi190 bash[23392]: cluster 2024-03-26T04:20:23.829053+0000 mgr.y (mgr.24431) 1336 : cluster [DBG] pgmap v1758: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:20:24.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:24 smithi190 bash[23392]: audit 2024-03-26T04:20:24.311119+0000 mon.c (mon.2) 877 : audit [INF] from='client.? 172.21.15.175:0/3212534508' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-26T04:20:24.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:24 smithi190 bash[23392]: audit 2024-03-26T04:20:24.311823+0000 mon.a (mon.0) 3633 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-26T04:20:25.690 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.688+0000 7f92fb7fe640 1 -- 172.21.15.175:0/3212534508 <== mon.2 v2:172.21.15.175:3301/0 9 ==== osd_map(780..780 src has 1..780) ==== 628+0+0 (secure 0 0 0) 0x7f93040f3750 con 0x7f930c10e480 2024-03-26T04:20:25.691 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.692+0000 7f92fb7fe640 1 -- 172.21.15.175:0/3212534508 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=781}) -- 0x7f92f407b150 con 0x7f930c10e480 2024-03-26T04:20:25.698 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.696+0000 7f92fb7fe640 1 -- 172.21.15.175:0/3212534508 <== mon.2 v2:172.21.15.175:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 0e4b321e-b83b-43c2-ade1-c76622f394b4 v780) ==== 217+0+0 (secure 0 0 0) 0x7f93040f5db0 con 0x7f930c10e480 2024-03-26T04:20:25.698 INFO:tasks.workunit.client.0.smithi175.stderr:set-quota max_bytes = 0 for pool 0e4b321e-b83b-43c2-ade1-c76622f394b4 2024-03-26T04:20:25.702 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.700+0000 7f9310bd1640 1 -- 172.21.15.175:0/3212534508 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f92f406e760 msgr2=0x7f92f4070c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:20:25.703 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.700+0000 7f9310bd1640 1 --2- 172.21.15.175:0/3212534508 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f92f406e760 0x7f92f4070c20 secure :-1 s=READY pgs=3772 cs=0 l=1 rev1=1 crypto rx=0x7f92fc009ab0 tx=0x7f92fc009260 comp rx=0 tx=0).stop 2024-03-26T04:20:25.703 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.700+0000 7f9310bd1640 1 -- 172.21.15.175:0/3212534508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f930c10e480 msgr2=0x7f930c1a0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:20:25.703 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.700+0000 7f9310bd1640 1 --2- 172.21.15.175:0/3212534508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f930c10e480 0x7f930c1a0b30 secure :-1 s=READY pgs=3040 cs=0 l=1 rev1=1 crypto rx=0x7f93040b0040 tx=0x7f93040a5b50 comp rx=0 tx=0).stop 2024-03-26T04:20:25.703 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.704+0000 7f9310bd1640 1 -- 172.21.15.175:0/3212534508 shutdown_connections 2024-03-26T04:20:25.703 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.704+0000 7f9310bd1640 1 --2- 172.21.15.175:0/3212534508 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f92f406e760 0x7f92f4070c20 unknown :-1 s=CLOSED pgs=3772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:25.703 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.704+0000 7f9310bd1640 1 --2- 172.21.15.175:0/3212534508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f930c10e480 0x7f930c1a0b30 unknown :-1 s=CLOSED pgs=3040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:25.703 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.704+0000 7f9310bd1640 1 --2- 172.21.15.175:0/3212534508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f930c102ab0 0x7f930c19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:25.703 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.704+0000 7f9310bd1640 1 --2- 172.21.15.175:0/3212534508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f930c06a850 0x7f930c19c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:25.703 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.704+0000 7f9310bd1640 1 -- 172.21.15.175:0/3212534508 >> 172.21.15.175:0/3212534508 conn(0x7f930c077860 msgr2=0x7f930c10f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T04:20:25.703 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.704+0000 7f9310bd1640 1 -- 172.21.15.175:0/3212534508 shutdown_connections 2024-03-26T04:20:25.703 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.704+0000 7f9310bd1640 1 -- 172.21.15.175:0/3212534508 wait complete. 2024-03-26T04:20:25.722 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph osd pool set-quota 0e4b321e-b83b-43c2-ade1-c76622f394b4 max_objects 0 2024-03-26T04:20:25.848 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c1b7ef640 1 -- 172.21.15.175:0/3024757266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c14102490 msgr2=0x7f5c1410e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:20:25.849 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c1b7ef640 1 --2- 172.21.15.175:0/3024757266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c14102490 0x7f5c1410e9a0 secure :-1 s=READY pgs=2913 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009c20 tx=0x7f5c0c01ca80 comp rx=0 tx=0).stop 2024-03-26T04:20:25.849 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c1b7ef640 1 -- 172.21.15.175:0/3024757266 shutdown_connections 2024-03-26T04:20:25.849 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c1b7ef640 1 --2- 172.21.15.175:0/3024757266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c14102490 0x7f5c1410e9a0 unknown :-1 s=CLOSED pgs=2913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:25.849 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c1b7ef640 1 --2- 172.21.15.175:0/3024757266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c14101ad0 0x7f5c14101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:25.849 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c1b7ef640 1 --2- 172.21.15.175:0/3024757266 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c14107b60 0x7f5c14107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:25.849 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c1b7ef640 1 -- 172.21.15.175:0/3024757266 >> 172.21.15.175:0/3024757266 conn(0x7f5c140fd620 msgr2=0x7f5c140ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T04:20:25.849 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c1b7ef640 1 -- 172.21.15.175:0/3024757266 shutdown_connections 2024-03-26T04:20:25.849 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c1b7ef640 1 -- 172.21.15.175:0/3024757266 wait complete. 2024-03-26T04:20:25.849 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c1b7ef640 1 Processor -- start 2024-03-26T04:20:25.849 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c1b7ef640 1 -- start start 2024-03-26T04:20:25.849 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c1b7ef640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c14101ad0 0x7f5c14073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:20:25.849 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c1b7ef640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c14102490 0x7f5c14074440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:20:25.849 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c1b7ef640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c14107b60 0x7f5c14074980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:20:25.850 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c1b7ef640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5c14113ac0 con 0x7f5c14107b60 2024-03-26T04:20:25.850 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c1b7ef640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5c14113940 con 0x7f5c14102490 2024-03-26T04:20:25.850 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c1b7ef640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5c14113c40 con 0x7f5c14101ad0 2024-03-26T04:20:25.850 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c19d8c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c14107b60 0x7f5c14074980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:20:25.850 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c1958b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c14101ad0 0x7f5c14073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:20:25.850 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c19d8c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c14107b60 0x7f5c14074980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:48822/0 (socket says 172.21.15.175:48822) 2024-03-26T04:20:25.850 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c19d8c640 1 -- 172.21.15.175:0/2681048664 learned_addr learned my addr 172.21.15.175:0/2681048664 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T04:20:25.850 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c19d8c640 1 -- 172.21.15.175:0/2681048664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c14101ad0 msgr2=0x7f5c14073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:20:25.850 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c19d8c640 1 --2- 172.21.15.175:0/2681048664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c14101ad0 0x7f5c14073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:25.850 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c18d8a640 1 --2- 172.21.15.175:0/2681048664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c14102490 0x7f5c14074440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:20:25.850 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c19d8c640 1 -- 172.21.15.175:0/2681048664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c14102490 msgr2=0x7f5c14074440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:20:25.850 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c19d8c640 1 --2- 172.21.15.175:0/2681048664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c14102490 0x7f5c14074440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:25.850 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c19d8c640 1 -- 172.21.15.175:0/2681048664 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c141a85a0 con 0x7f5c14107b60 2024-03-26T04:20:25.851 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c1958b640 1 --2- 172.21.15.175:0/2681048664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c14101ad0 0x7f5c14073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T04:20:25.851 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c19d8c640 1 --2- 172.21.15.175:0/2681048664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c14107b60 0x7f5c14074980 secure :-1 s=READY pgs=2914 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c0985a0 tx=0x7f5c0c0a5ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T04:20:25.851 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c0a7fc640 1 -- 172.21.15.175:0/2681048664 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c0b2040 con 0x7f5c14107b60 2024-03-26T04:20:25.851 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c1b7ef640 1 -- 172.21.15.175:0/2681048664 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c141a8830 con 0x7f5c14107b60 2024-03-26T04:20:25.851 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c0a7fc640 1 -- 172.21.15.175:0/2681048664 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f5c0c0bb070 con 0x7f5c14107b60 2024-03-26T04:20:25.851 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c0a7fc640 1 -- 172.21.15.175:0/2681048664 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c004780 con 0x7f5c14107b60 2024-03-26T04:20:25.851 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c1b7ef640 1 -- 172.21.15.175:0/2681048664 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c141a8d60 con 0x7f5c14107b60 2024-03-26T04:20:25.852 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c1b7ef640 1 -- 172.21.15.175:0/2681048664 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c14068a40 con 0x7f5c14107b60 2024-03-26T04:20:25.852 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.848+0000 7f5c0a7fc640 1 -- 172.21.15.175:0/2681048664 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7f5c0c0ad7f0 con 0x7f5c14107b60 2024-03-26T04:20:25.852 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.852+0000 7f5c0a7fc640 1 --2- 172.21.15.175:0/2681048664 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f5c0006e760 0x7f5c00070c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:20:25.852 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.852+0000 7f5c0a7fc640 1 -- 172.21.15.175:0/2681048664 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(780..780 src has 1..780) ==== 9080+0+0 (secure 0 0 0) 0x7f5c0c129d80 con 0x7f5c14107b60 2024-03-26T04:20:25.852 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.852+0000 7f5c1958b640 1 --2- 172.21.15.175:0/2681048664 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f5c0006e760 0x7f5c00070c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:20:25.852 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.852+0000 7f5c0a7fc640 1 -- 172.21.15.175:0/2681048664 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=781}) -- 0x7f5c0007a1f0 con 0x7f5c14107b60 2024-03-26T04:20:25.853 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.852+0000 7f5c1958b640 1 --2- 172.21.15.175:0/2681048664 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f5c0006e760 0x7f5c00070c20 secure :-1 s=READY pgs=3773 cs=0 l=1 rev1=1 crypto rx=0x7f5c10009f50 tx=0x7f5c10009260 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T04:20:25.855 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:25.856+0000 7f5c0a7fc640 1 -- 172.21.15.175:0/2681048664 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f5c0c016790 con 0x7f5c14107b60 2024-03-26T04:20:25.990 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:25 smithi175 bash[21527]: audit 2024-03-26T04:20:24.687192+0000 mon.a (mon.0) 3634 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]': finished 2024-03-26T04:20:25.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:25 smithi175 bash[21527]: cluster 2024-03-26T04:20:24.699082+0000 mon.a (mon.0) 3635 : cluster [DBG] osdmap e779: 8 total, 8 up, 8 in 2024-03-26T04:20:25.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:25 smithi175 bash[21527]: audit 2024-03-26T04:20:24.810837+0000 mon.c (mon.2) 878 : audit [INF] from='client.? 172.21.15.175:0/3212534508' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-26T04:20:25.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:25 smithi175 bash[21527]: audit 2024-03-26T04:20:24.811496+0000 mon.a (mon.0) 3636 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-26T04:20:25.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:25 smithi175 bash[29284]: audit 2024-03-26T04:20:24.687192+0000 mon.a (mon.0) 3634 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]': finished 2024-03-26T04:20:25.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:25 smithi175 bash[29284]: cluster 2024-03-26T04:20:24.699082+0000 mon.a (mon.0) 3635 : cluster [DBG] osdmap e779: 8 total, 8 up, 8 in 2024-03-26T04:20:25.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:25 smithi175 bash[29284]: audit 2024-03-26T04:20:24.810837+0000 mon.c (mon.2) 878 : audit [INF] from='client.? 172.21.15.175:0/3212534508' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-26T04:20:25.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:25 smithi175 bash[29284]: audit 2024-03-26T04:20:24.811496+0000 mon.a (mon.0) 3636 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-26T04:20:25.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:25 smithi190 bash[23392]: audit 2024-03-26T04:20:24.687192+0000 mon.a (mon.0) 3634 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]': finished 2024-03-26T04:20:25.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:25 smithi190 bash[23392]: cluster 2024-03-26T04:20:24.699082+0000 mon.a (mon.0) 3635 : cluster [DBG] osdmap e779: 8 total, 8 up, 8 in 2024-03-26T04:20:25.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:25 smithi190 bash[23392]: audit 2024-03-26T04:20:24.810837+0000 mon.c (mon.2) 878 : audit [INF] from='client.? 172.21.15.175:0/3212534508' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-26T04:20:25.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:25 smithi190 bash[23392]: audit 2024-03-26T04:20:24.811496+0000 mon.a (mon.0) 3636 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-26T04:20:26.029 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:26.028+0000 7f5c1b7ef640 1 -- 172.21.15.175:0/2681048664 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "0"} v 0) -- 0x7f5c141a8fc0 con 0x7f5c14107b60 2024-03-26T04:20:26.714 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:26.712+0000 7f5c0a7fc640 1 -- 172.21.15.175:0/2681048664 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 0e4b321e-b83b-43c2-ade1-c76622f394b4 v781) ==== 221+0+0 (secure 0 0 0) 0x7f5c0c0f6720 con 0x7f5c14107b60 2024-03-26T04:20:26.714 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:26.716+0000 7f5c0a7fc640 1 -- 172.21.15.175:0/2681048664 <== mon.0 v2:172.21.15.175:3300/0 8 ==== osd_map(781..781 src has 1..781) ==== 628+0+0 (secure 0 0 0) 0x7f5c0c0f40c0 con 0x7f5c14107b60 2024-03-26T04:20:26.715 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:26.716+0000 7f5c0a7fc640 1 -- 172.21.15.175:0/2681048664 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=782}) -- 0x7f5c0007b150 con 0x7f5c14107b60 2024-03-26T04:20:26.824 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:26.824+0000 7f5c1b7ef640 1 -- 172.21.15.175:0/2681048664 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "0"} v 0) -- 0x7f5c140621e0 con 0x7f5c14107b60 2024-03-26T04:20:26.960 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:26.960+0000 7f5c0a7fc640 1 -- 172.21.15.175:0/2681048664 <== mon.0 v2:172.21.15.175:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 0e4b321e-b83b-43c2-ade1-c76622f394b4 v782) ==== 221+0+0 (secure 0 0 0) 0x7f5c0c0fa3d0 con 0x7f5c14107b60 2024-03-26T04:20:26.960 INFO:tasks.workunit.client.0.smithi175.stderr:set-quota max_objects = 0 for pool 0e4b321e-b83b-43c2-ade1-c76622f394b4 2024-03-26T04:20:26.961 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:26.960+0000 7f5c0a7fc640 1 -- 172.21.15.175:0/2681048664 <== mon.0 v2:172.21.15.175:3300/0 10 ==== osd_map(782..782 src has 1..782) ==== 628+0+0 (secure 0 0 0) 0x7f5c0c127610 con 0x7f5c14107b60 2024-03-26T04:20:26.961 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:26.960+0000 7f5c0a7fc640 1 -- 172.21.15.175:0/2681048664 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=783}) -- 0x7f5c0007b240 con 0x7f5c14107b60 2024-03-26T04:20:26.964 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:26.964+0000 7f5c1b7ef640 1 -- 172.21.15.175:0/2681048664 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f5c0006e760 msgr2=0x7f5c00070c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:20:26.964 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:26.964+0000 7f5c1b7ef640 1 --2- 172.21.15.175:0/2681048664 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f5c0006e760 0x7f5c00070c20 secure :-1 s=READY pgs=3773 cs=0 l=1 rev1=1 crypto rx=0x7f5c10009f50 tx=0x7f5c10009260 comp rx=0 tx=0).stop 2024-03-26T04:20:26.964 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:26.964+0000 7f5c1b7ef640 1 -- 172.21.15.175:0/2681048664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c14107b60 msgr2=0x7f5c14074980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:20:26.964 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:26.964+0000 7f5c1b7ef640 1 --2- 172.21.15.175:0/2681048664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c14107b60 0x7f5c14074980 secure :-1 s=READY pgs=2914 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c0985a0 tx=0x7f5c0c0a5ab0 comp rx=0 tx=0).stop 2024-03-26T04:20:26.966 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:26.968+0000 7f5c1b7ef640 1 -- 172.21.15.175:0/2681048664 shutdown_connections 2024-03-26T04:20:26.967 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:26.968+0000 7f5c1b7ef640 1 --2- 172.21.15.175:0/2681048664 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f5c0006e760 0x7f5c00070c20 unknown :-1 s=CLOSED pgs=3773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:26.967 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:26.968+0000 7f5c1b7ef640 1 --2- 172.21.15.175:0/2681048664 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c14107b60 0x7f5c14074980 unknown :-1 s=CLOSED pgs=2914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:26.967 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:26.968+0000 7f5c1b7ef640 1 --2- 172.21.15.175:0/2681048664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c14102490 0x7f5c14074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:26.967 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:26.968+0000 7f5c1b7ef640 1 --2- 172.21.15.175:0/2681048664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c14101ad0 0x7f5c14073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:26.967 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:26.968+0000 7f5c1b7ef640 1 -- 172.21.15.175:0/2681048664 >> 172.21.15.175:0/2681048664 conn(0x7f5c140fd620 msgr2=0x7f5c140fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T04:20:26.967 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:26.968+0000 7f5c1b7ef640 1 -- 172.21.15.175:0/2681048664 shutdown_connections 2024-03-26T04:20:26.967 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:26.968+0000 7f5c1b7ef640 1 -- 172.21.15.175:0/2681048664 wait complete. 2024-03-26T04:20:26.990 INFO:tasks.workunit.client.0.smithi175.stderr:+ wait 136224 2024-03-26T04:20:26.990 INFO:tasks.workunit.client.0.smithi175.stderr:+ [ 0 -ne 0 ] 2024-03-26T04:20:26.990 INFO:tasks.workunit.client.0.smithi175.stderr:+ true 2024-03-26T04:20:26.990 INFO:tasks.workunit.client.0.smithi175.stderr:+ rados -p 0e4b321e-b83b-43c2-ade1-c76622f394b4 put three /etc/passwd 2024-03-26T04:20:26.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:26 smithi175 bash[29284]: audit 2024-03-26T04:20:25.689825+0000 mon.a (mon.0) 3637 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]': finished 2024-03-26T04:20:26.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:26 smithi175 bash[29284]: cluster 2024-03-26T04:20:25.697737+0000 mon.a (mon.0) 3638 : cluster [DBG] osdmap e780: 8 total, 8 up, 8 in 2024-03-26T04:20:26.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:26 smithi175 bash[29284]: cluster 2024-03-26T04:20:25.829828+0000 mgr.y (mgr.24431) 1337 : cluster [DBG] pgmap v1761: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:20:26.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:26 smithi175 bash[29284]: audit 2024-03-26T04:20:26.034784+0000 mon.a (mon.0) 3639 : audit [INF] from='client.? 172.21.15.175:0/2681048664' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "0"}]: dispatch 2024-03-26T04:20:26.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:26 smithi175 bash[21527]: audit 2024-03-26T04:20:25.689825+0000 mon.a (mon.0) 3637 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]': finished 2024-03-26T04:20:26.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:26 smithi175 bash[21527]: cluster 2024-03-26T04:20:25.697737+0000 mon.a (mon.0) 3638 : cluster [DBG] osdmap e780: 8 total, 8 up, 8 in 2024-03-26T04:20:26.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:26 smithi175 bash[21527]: cluster 2024-03-26T04:20:25.829828+0000 mgr.y (mgr.24431) 1337 : cluster [DBG] pgmap v1761: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:20:26.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:26 smithi175 bash[21527]: audit 2024-03-26T04:20:26.034784+0000 mon.a (mon.0) 3639 : audit [INF] from='client.? 172.21.15.175:0/2681048664' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "0"}]: dispatch 2024-03-26T04:20:26.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:26 smithi190 bash[23392]: audit 2024-03-26T04:20:25.689825+0000 mon.a (mon.0) 3637 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]': finished 2024-03-26T04:20:26.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:26 smithi190 bash[23392]: cluster 2024-03-26T04:20:25.697737+0000 mon.a (mon.0) 3638 : cluster [DBG] osdmap e780: 8 total, 8 up, 8 in 2024-03-26T04:20:26.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:26 smithi190 bash[23392]: cluster 2024-03-26T04:20:25.829828+0000 mgr.y (mgr.24431) 1337 : cluster [DBG] pgmap v1761: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:20:26.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:26 smithi190 bash[23392]: audit 2024-03-26T04:20:26.034784+0000 mon.a (mon.0) 3639 : audit [INF] from='client.? 172.21.15.175:0/2681048664' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "0"}]: dispatch 2024-03-26T04:20:27.031 INFO:tasks.workunit.client.0.smithi175.stderr:+ uuidgen 2024-03-26T04:20:27.031 INFO:tasks.workunit.client.0.smithi175.stderr:+ pp=721fce6c-d7d8-472d-8941-f0e24365dbcf 2024-03-26T04:20:27.032 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph osd pool create 721fce6c-d7d8-472d-8941-f0e24365dbcf 12 2024-03-26T04:20:27.153 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f94362cd640 1 -- 172.21.15.175:0/3714510196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94301011e0 msgr2=0x7f943010dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:20:27.153 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f94362cd640 1 --2- 172.21.15.175:0/3714510196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94301011e0 0x7f943010dea0 secure :-1 s=READY pgs=3041 cs=0 l=1 rev1=1 crypto rx=0x7f9418009930 tx=0x7f941801c7e0 comp rx=0 tx=0).stop 2024-03-26T04:20:27.153 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f94362cd640 1 -- 172.21.15.175:0/3714510196 shutdown_connections 2024-03-26T04:20:27.153 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f94362cd640 1 --2- 172.21.15.175:0/3714510196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94301011e0 0x7f943010dea0 unknown :-1 s=CLOSED pgs=3041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:27.153 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f94362cd640 1 --2- 172.21.15.175:0/3714510196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f943010e3e0 0x7f94301107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:27.153 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f94362cd640 1 --2- 172.21.15.175:0/3714510196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9430100830 0x7f9430100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:27.153 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f94362cd640 1 -- 172.21.15.175:0/3714510196 >> 172.21.15.175:0/3714510196 conn(0x7f94300fc510 msgr2=0x7f94300fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T04:20:27.154 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f94362cd640 1 -- 172.21.15.175:0/3714510196 shutdown_connections 2024-03-26T04:20:27.154 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f94362cd640 1 -- 172.21.15.175:0/3714510196 wait complete. 2024-03-26T04:20:27.154 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f94362cd640 1 Processor -- start 2024-03-26T04:20:27.154 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f94362cd640 1 -- start start 2024-03-26T04:20:27.154 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f94362cd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9430100830 0x7f943019c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:20:27.154 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f94362cd640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94301011e0 0x7f943019c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:20:27.154 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f94362cd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f943010e3e0 0x7f94301a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:20:27.154 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f94362cd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9430115d70 con 0x7f943010e3e0 2024-03-26T04:20:27.154 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f94362cd640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9430115bf0 con 0x7f9430100830 2024-03-26T04:20:27.154 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f94362cd640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9430115ef0 con 0x7f94301011e0 2024-03-26T04:20:27.154 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f94277fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94301011e0 0x7f943019c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:20:27.154 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f943486a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f943010e3e0 0x7f94301a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:20:27.154 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f943486a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f943010e3e0 0x7f94301a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:48880/0 (socket says 172.21.15.175:48880) 2024-03-26T04:20:27.154 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f943486a640 1 -- 172.21.15.175:0/2482660258 learned_addr learned my addr 172.21.15.175:0/2482660258 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T04:20:27.155 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f942ffff640 1 --2- 172.21.15.175:0/2482660258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9430100830 0x7f943019c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:20:27.155 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f94277fe640 1 -- 172.21.15.175:0/2482660258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9430100830 msgr2=0x7f943019c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:20:27.155 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f94277fe640 1 --2- 172.21.15.175:0/2482660258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9430100830 0x7f943019c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:27.155 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f94277fe640 1 -- 172.21.15.175:0/2482660258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f943010e3e0 msgr2=0x7f94301a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:20:27.155 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f94277fe640 1 --2- 172.21.15.175:0/2482660258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f943010e3e0 0x7f94301a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:27.155 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f94277fe640 1 -- 172.21.15.175:0/2482660258 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94301a1270 con 0x7f94301011e0 2024-03-26T04:20:27.155 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f943486a640 1 --2- 172.21.15.175:0/2482660258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f943010e3e0 0x7f94301a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T04:20:27.155 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f942ffff640 1 --2- 172.21.15.175:0/2482660258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9430100830 0x7f943019c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T04:20:27.155 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f94277fe640 1 --2- 172.21.15.175:0/2482660258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94301011e0 0x7f943019c800 secure :-1 s=READY pgs=3042 cs=0 l=1 rev1=1 crypto rx=0x7f9418009850 tx=0x7f94180ad6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T04:20:27.155 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f942dffb640 1 -- 172.21.15.175:0/2482660258 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94180b2040 con 0x7f94301011e0 2024-03-26T04:20:27.155 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f942dffb640 1 -- 172.21.15.175:0/2482660258 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f94180a5a60 con 0x7f94301011e0 2024-03-26T04:20:27.155 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f94362cd640 1 -- 172.21.15.175:0/2482660258 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94301a1500 con 0x7f94301011e0 2024-03-26T04:20:27.155 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f942dffb640 1 -- 172.21.15.175:0/2482660258 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94180053d0 con 0x7f94301011e0 2024-03-26T04:20:27.155 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f94362cd640 1 -- 172.21.15.175:0/2482660258 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f94301a1950 con 0x7f94301011e0 2024-03-26T04:20:27.157 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f94362cd640 1 -- 172.21.15.175:0/2482660258 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9430068e90 con 0x7f94301011e0 2024-03-26T04:20:27.158 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.152+0000 7f942dffb640 1 -- 172.21.15.175:0/2482660258 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7f94180bd060 con 0x7f94301011e0 2024-03-26T04:20:27.158 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.156+0000 7f942dffb640 1 --2- 172.21.15.175:0/2482660258 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f941c06e760 0x7f941c070c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:20:27.158 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.156+0000 7f942dffb640 1 -- 172.21.15.175:0/2482660258 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(782..782 src has 1..782) ==== 9080+0+0 (secure 0 0 0) 0x7f9418128ac0 con 0x7f94301011e0 2024-03-26T04:20:27.158 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.156+0000 7f942ffff640 1 --2- 172.21.15.175:0/2482660258 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f941c06e760 0x7f941c070c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:20:27.158 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.156+0000 7f942ffff640 1 --2- 172.21.15.175:0/2482660258 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f941c06e760 0x7f941c070c20 secure :-1 s=READY pgs=3775 cs=0 l=1 rev1=1 crypto rx=0x7f9420009b60 tx=0x7f94200091e0 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T04:20:27.159 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.160+0000 7f942dffb640 1 -- 172.21.15.175:0/2482660258 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f94180164f0 con 0x7f94301011e0 2024-03-26T04:20:27.332 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.332+0000 7f94362cd640 1 -- 172.21.15.175:0/2482660258 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "pg_num": 12} v 0) -- 0x7f94301a8f00 con 0x7f94301011e0 2024-03-26T04:20:27.974 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:27.972+0000 7f942dffb640 1 -- 172.21.15.175:0/2482660258 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "pg_num": 12}]=0 pool '721fce6c-d7d8-472d-8941-f0e24365dbcf' created v783) ==== 176+0+0 (secure 0 0 0) 0x7f94180f5460 con 0x7f94301011e0 2024-03-26T04:20:27.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:27 smithi175 bash[29284]: audit 2024-03-26T04:20:26.718130+0000 mon.a (mon.0) 3640 : audit [INF] from='client.? 172.21.15.175:0/2681048664' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "0"}]': finished 2024-03-26T04:20:27.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:27 smithi175 bash[29284]: cluster 2024-03-26T04:20:26.724946+0000 mon.a (mon.0) 3641 : cluster [DBG] osdmap e781: 8 total, 8 up, 8 in 2024-03-26T04:20:27.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:27 smithi175 bash[29284]: audit 2024-03-26T04:20:26.829332+0000 mon.a (mon.0) 3642 : audit [INF] from='client.? 172.21.15.175:0/2681048664' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "0"}]: dispatch 2024-03-26T04:20:27.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:27 smithi175 bash[29284]: cluster 2024-03-26T04:20:26.961897+0000 mon.a (mon.0) 3643 : cluster [INF] pool '0e4b321e-b83b-43c2-ade1-c76622f394b4' no longer out of quota; removing NO_QUOTA flag 2024-03-26T04:20:27.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:27 smithi175 bash[29284]: cluster 2024-03-26T04:20:26.962439+0000 mon.a (mon.0) 3644 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-26T04:20:27.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:27 smithi175 bash[29284]: audit 2024-03-26T04:20:26.964754+0000 mon.a (mon.0) 3645 : audit [INF] from='client.? 172.21.15.175:0/2681048664' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "0"}]': finished 2024-03-26T04:20:27.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:27 smithi175 bash[29284]: cluster 2024-03-26T04:20:26.971175+0000 mon.a (mon.0) 3646 : cluster [DBG] osdmap e782: 8 total, 8 up, 8 in 2024-03-26T04:20:27.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:27 smithi175 bash[29284]: audit 2024-03-26T04:20:27.337981+0000 mon.c (mon.2) 879 : audit [INF] from='client.? 172.21.15.175:0/2482660258' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "pg_num": 12}]: dispatch 2024-03-26T04:20:27.992 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:27 smithi175 bash[29284]: audit 2024-03-26T04:20:27.338621+0000 mon.a (mon.0) 3647 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "pg_num": 12}]: dispatch 2024-03-26T04:20:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:27 smithi175 bash[21527]: audit 2024-03-26T04:20:26.718130+0000 mon.a (mon.0) 3640 : audit [INF] from='client.? 172.21.15.175:0/2681048664' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "0"}]': finished 2024-03-26T04:20:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:27 smithi175 bash[21527]: cluster 2024-03-26T04:20:26.724946+0000 mon.a (mon.0) 3641 : cluster [DBG] osdmap e781: 8 total, 8 up, 8 in 2024-03-26T04:20:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:27 smithi175 bash[21527]: audit 2024-03-26T04:20:26.829332+0000 mon.a (mon.0) 3642 : audit [INF] from='client.? 172.21.15.175:0/2681048664' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "0"}]: dispatch 2024-03-26T04:20:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:27 smithi175 bash[21527]: cluster 2024-03-26T04:20:26.961897+0000 mon.a (mon.0) 3643 : cluster [INF] pool '0e4b321e-b83b-43c2-ade1-c76622f394b4' no longer out of quota; removing NO_QUOTA flag 2024-03-26T04:20:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:27 smithi175 bash[21527]: cluster 2024-03-26T04:20:26.962439+0000 mon.a (mon.0) 3644 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-26T04:20:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:27 smithi175 bash[21527]: audit 2024-03-26T04:20:26.964754+0000 mon.a (mon.0) 3645 : audit [INF] from='client.? 172.21.15.175:0/2681048664' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "0"}]': finished 2024-03-26T04:20:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:27 smithi175 bash[21527]: cluster 2024-03-26T04:20:26.971175+0000 mon.a (mon.0) 3646 : cluster [DBG] osdmap e782: 8 total, 8 up, 8 in 2024-03-26T04:20:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:27 smithi175 bash[21527]: audit 2024-03-26T04:20:27.337981+0000 mon.c (mon.2) 879 : audit [INF] from='client.? 172.21.15.175:0/2482660258' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "pg_num": 12}]: dispatch 2024-03-26T04:20:27.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:27 smithi175 bash[21527]: audit 2024-03-26T04:20:27.338621+0000 mon.a (mon.0) 3647 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "pg_num": 12}]: dispatch 2024-03-26T04:20:27.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:27 smithi190 bash[23392]: audit 2024-03-26T04:20:26.718130+0000 mon.a (mon.0) 3640 : audit [INF] from='client.? 172.21.15.175:0/2681048664' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "0"}]': finished 2024-03-26T04:20:27.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:27 smithi190 bash[23392]: cluster 2024-03-26T04:20:26.724946+0000 mon.a (mon.0) 3641 : cluster [DBG] osdmap e781: 8 total, 8 up, 8 in 2024-03-26T04:20:27.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:27 smithi190 bash[23392]: audit 2024-03-26T04:20:26.829332+0000 mon.a (mon.0) 3642 : audit [INF] from='client.? 172.21.15.175:0/2681048664' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "0"}]: dispatch 2024-03-26T04:20:27.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:27 smithi190 bash[23392]: cluster 2024-03-26T04:20:26.961897+0000 mon.a (mon.0) 3643 : cluster [INF] pool '0e4b321e-b83b-43c2-ade1-c76622f394b4' no longer out of quota; removing NO_QUOTA flag 2024-03-26T04:20:27.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:27 smithi190 bash[23392]: cluster 2024-03-26T04:20:26.962439+0000 mon.a (mon.0) 3644 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-26T04:20:27.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:27 smithi190 bash[23392]: audit 2024-03-26T04:20:26.964754+0000 mon.a (mon.0) 3645 : audit [INF] from='client.? 172.21.15.175:0/2681048664' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "0"}]': finished 2024-03-26T04:20:27.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:27 smithi190 bash[23392]: cluster 2024-03-26T04:20:26.971175+0000 mon.a (mon.0) 3646 : cluster [DBG] osdmap e782: 8 total, 8 up, 8 in 2024-03-26T04:20:27.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:27 smithi190 bash[23392]: audit 2024-03-26T04:20:27.337981+0000 mon.c (mon.2) 879 : audit [INF] from='client.? 172.21.15.175:0/2482660258' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "pg_num": 12}]: dispatch 2024-03-26T04:20:27.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:27 smithi190 bash[23392]: audit 2024-03-26T04:20:27.338621+0000 mon.a (mon.0) 3647 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "pg_num": 12}]: dispatch 2024-03-26T04:20:28.088 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.088+0000 7f94362cd640 1 -- 172.21.15.175:0/2482660258 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "pg_num": 12} v 0) -- 0x7f94300621a0 con 0x7f94301011e0 2024-03-26T04:20:28.089 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.088+0000 7f942dffb640 1 -- 172.21.15.175:0/2482660258 <== mon.2 v2:172.21.15.175:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "pg_num": 12}]=0 pool '721fce6c-d7d8-472d-8941-f0e24365dbcf' already exists v783) ==== 183+0+0 (secure 0 0 0) 0x7f9418115120 con 0x7f94301011e0 2024-03-26T04:20:28.089 INFO:tasks.workunit.client.0.smithi175.stderr:pool '721fce6c-d7d8-472d-8941-f0e24365dbcf' already exists 2024-03-26T04:20:28.092 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.092+0000 7f94362cd640 1 -- 172.21.15.175:0/2482660258 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f941c06e760 msgr2=0x7f941c070c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:20:28.092 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.092+0000 7f94362cd640 1 --2- 172.21.15.175:0/2482660258 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f941c06e760 0x7f941c070c20 secure :-1 s=READY pgs=3775 cs=0 l=1 rev1=1 crypto rx=0x7f9420009b60 tx=0x7f94200091e0 comp rx=0 tx=0).stop 2024-03-26T04:20:28.092 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.092+0000 7f94362cd640 1 -- 172.21.15.175:0/2482660258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94301011e0 msgr2=0x7f943019c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:20:28.092 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.092+0000 7f94362cd640 1 --2- 172.21.15.175:0/2482660258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94301011e0 0x7f943019c800 secure :-1 s=READY pgs=3042 cs=0 l=1 rev1=1 crypto rx=0x7f9418009850 tx=0x7f94180ad6f0 comp rx=0 tx=0).stop 2024-03-26T04:20:28.092 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.092+0000 7f94362cd640 1 -- 172.21.15.175:0/2482660258 shutdown_connections 2024-03-26T04:20:28.092 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.092+0000 7f94362cd640 1 --2- 172.21.15.175:0/2482660258 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f941c06e760 0x7f941c070c20 unknown :-1 s=CLOSED pgs=3775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:28.092 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.092+0000 7f94362cd640 1 --2- 172.21.15.175:0/2482660258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94301011e0 0x7f943019c800 unknown :-1 s=CLOSED pgs=3042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:28.092 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.092+0000 7f94362cd640 1 --2- 172.21.15.175:0/2482660258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f943010e3e0 0x7f94301a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:28.092 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.092+0000 7f94362cd640 1 --2- 172.21.15.175:0/2482660258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9430100830 0x7f943019c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:28.092 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.092+0000 7f94362cd640 1 -- 172.21.15.175:0/2482660258 >> 172.21.15.175:0/2482660258 conn(0x7f94300fc510 msgr2=0x7f943010e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T04:20:28.092 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.092+0000 7f94362cd640 1 -- 172.21.15.175:0/2482660258 shutdown_connections 2024-03-26T04:20:28.092 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.092+0000 7f94362cd640 1 -- 172.21.15.175:0/2482660258 wait complete. 2024-03-26T04:20:28.107 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph osd pool application enable 721fce6c-d7d8-472d-8941-f0e24365dbcf rados 2024-03-26T04:20:28.231 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.228+0000 7f14ee839640 1 -- 172.21.15.175:0/2579427055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14e8107b90 msgr2=0x7f14e8107f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:20:28.231 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.228+0000 7f14ee839640 1 --2- 172.21.15.175:0/2579427055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14e8107b90 0x7f14e8107f70 secure :-1 s=READY pgs=3043 cs=0 l=1 rev1=1 crypto rx=0x7f14d80098a0 tx=0x7f14d801c700 comp rx=0 tx=0).stop 2024-03-26T04:20:28.231 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.228+0000 7f14ee839640 1 -- 172.21.15.175:0/2579427055 shutdown_connections 2024-03-26T04:20:28.231 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.228+0000 7f14ee839640 1 --2- 172.21.15.175:0/2579427055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14e81024c0 0x7f14e810e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:28.231 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.228+0000 7f14ee839640 1 --2- 172.21.15.175:0/2579427055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14e8101b00 0x7f14e8101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:28.231 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.228+0000 7f14ee839640 1 --2- 172.21.15.175:0/2579427055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14e8107b90 0x7f14e8107f70 unknown :-1 s=CLOSED pgs=3043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:28.231 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.228+0000 7f14ee839640 1 -- 172.21.15.175:0/2579427055 >> 172.21.15.175:0/2579427055 conn(0x7f14e80fd5f0 msgr2=0x7f14e80ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T04:20:28.231 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.228+0000 7f14ee839640 1 -- 172.21.15.175:0/2579427055 shutdown_connections 2024-03-26T04:20:28.232 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.228+0000 7f14ee839640 1 -- 172.21.15.175:0/2579427055 wait complete. 2024-03-26T04:20:28.232 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.228+0000 7f14ee839640 1 Processor -- start 2024-03-26T04:20:28.232 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.228+0000 7f14ee839640 1 -- start start 2024-03-26T04:20:28.233 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.228+0000 7f14ee839640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14e8101b00 0x7f14e819a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:20:28.233 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.228+0000 7f14ee839640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14e81024c0 0x7f14e819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:20:28.233 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.228+0000 7f14ee839640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14e8107b90 0x7f14e819e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:20:28.233 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.228+0000 7f14ee839640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f14e8113af0 con 0x7f14e81024c0 2024-03-26T04:20:28.233 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.228+0000 7f14ee839640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f14e8113970 con 0x7f14e8101b00 2024-03-26T04:20:28.233 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.228+0000 7f14ee839640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f14e8113c70 con 0x7f14e8107b90 2024-03-26T04:20:28.233 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.228+0000 7f14e77fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14e81024c0 0x7f14e819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:20:28.233 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.228+0000 7f14e77fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14e81024c0 0x7f14e819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:48890/0 (socket says 172.21.15.175:48890) 2024-03-26T04:20:28.233 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.228+0000 7f14e77fe640 1 -- 172.21.15.175:0/1007571012 learned_addr learned my addr 172.21.15.175:0/1007571012 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T04:20:28.233 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.228+0000 7f14ecdd6640 1 --2- 172.21.15.175:0/1007571012 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14e8107b90 0x7f14e819e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:20:28.233 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.232+0000 7f14e7fff640 1 --2- 172.21.15.175:0/1007571012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14e8101b00 0x7f14e819a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:20:28.233 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.232+0000 7f14e77fe640 1 -- 172.21.15.175:0/1007571012 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14e8107b90 msgr2=0x7f14e819e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:20:28.234 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.232+0000 7f14e77fe640 1 --2- 172.21.15.175:0/1007571012 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14e8107b90 0x7f14e819e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:28.234 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.232+0000 7f14e77fe640 1 -- 172.21.15.175:0/1007571012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14e8101b00 msgr2=0x7f14e819a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:20:28.234 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.232+0000 7f14e77fe640 1 --2- 172.21.15.175:0/1007571012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14e8101b00 0x7f14e819a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:28.234 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.232+0000 7f14e77fe640 1 -- 172.21.15.175:0/1007571012 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e819f090 con 0x7f14e81024c0 2024-03-26T04:20:28.234 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.232+0000 7f14e77fe640 1 --2- 172.21.15.175:0/1007571012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14e81024c0 0x7f14e819a620 secure :-1 s=READY pgs=2918 cs=0 l=1 rev1=1 crypto rx=0x7f14dc00e9d0 tx=0x7f14dc00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T04:20:28.234 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.232+0000 7f14e57fa640 1 -- 172.21.15.175:0/1007571012 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc00cde0 con 0x7f14e81024c0 2024-03-26T04:20:28.234 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.232+0000 7f14ee839640 1 -- 172.21.15.175:0/1007571012 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14e819f380 con 0x7f14e81024c0 2024-03-26T04:20:28.234 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.232+0000 7f14e57fa640 1 -- 172.21.15.175:0/1007571012 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f14dc0044a0 con 0x7f14e81024c0 2024-03-26T04:20:28.235 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.232+0000 7f14e57fa640 1 -- 172.21.15.175:0/1007571012 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc010430 con 0x7f14e81024c0 2024-03-26T04:20:28.235 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.232+0000 7f14ee839640 1 -- 172.21.15.175:0/1007571012 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14e81a6c60 con 0x7f14e81024c0 2024-03-26T04:20:28.235 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.232+0000 7f14ee839640 1 -- 172.21.15.175:0/1007571012 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14e8068e90 con 0x7f14e81024c0 2024-03-26T04:20:28.236 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.232+0000 7f14e57fa640 1 -- 172.21.15.175:0/1007571012 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7f14dc003f40 con 0x7f14e81024c0 2024-03-26T04:20:28.236 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.232+0000 7f14e57fa640 1 --2- 172.21.15.175:0/1007571012 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f14d006e760 0x7f14d0070c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:20:28.236 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.232+0000 7f14e7fff640 1 --2- 172.21.15.175:0/1007571012 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f14d006e760 0x7f14d0070c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:20:28.236 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.232+0000 7f14e57fa640 1 -- 172.21.15.175:0/1007571012 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(783..783 src has 1..783) ==== 9455+0+0 (secure 0 0 0) 0x7f14dc08fc10 con 0x7f14e81024c0 2024-03-26T04:20:28.236 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.232+0000 7f14e7fff640 1 --2- 172.21.15.175:0/1007571012 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f14d006e760 0x7f14d0070c20 secure :-1 s=READY pgs=3776 cs=0 l=1 rev1=1 crypto rx=0x7f14d8098220 tx=0x7f14d8009530 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T04:20:28.238 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.240+0000 7f14e57fa640 1 -- 172.21.15.175:0/1007571012 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f14dc05c4c0 con 0x7f14e81024c0 2024-03-26T04:20:28.414 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.412+0000 7f14ee839640 1 -- 172.21.15.175:0/1007571012 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "app": "rados"} v 0) -- 0x7f14e819b440 con 0x7f14e81024c0 2024-03-26T04:20:28.966 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:28.964+0000 7f14e57fa640 1 -- 172.21.15.175:0/1007571012 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "app": "rados"}]=0 enabled application 'rados' on pool '721fce6c-d7d8-472d-8941-f0e24365dbcf' v784) ==== 213+0+0 (secure 0 0 0) 0x7f14e819b440 con 0x7f14e81024c0 2024-03-26T04:20:28.990 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:20:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:20:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:20:28.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:28 smithi175 bash[29284]: cluster 2024-03-26T04:20:27.830614+0000 mgr.y (mgr.24431) 1338 : cluster [DBG] pgmap v1764: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:20:28.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:28 smithi175 bash[29284]: audit 2024-03-26T04:20:27.967870+0000 mon.a (mon.0) 3648 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "pg_num": 12}]': finished 2024-03-26T04:20:28.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:28 smithi175 bash[29284]: cluster 2024-03-26T04:20:27.976194+0000 mon.a (mon.0) 3649 : cluster [DBG] osdmap e783: 8 total, 8 up, 8 in 2024-03-26T04:20:28.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:28 smithi175 bash[29284]: audit 2024-03-26T04:20:28.093309+0000 mon.c (mon.2) 880 : audit [INF] from='client.? 172.21.15.175:0/2482660258' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "pg_num": 12}]: dispatch 2024-03-26T04:20:28.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:28 smithi175 bash[29284]: audit 2024-03-26T04:20:28.093781+0000 mon.a (mon.0) 3650 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "pg_num": 12}]: dispatch 2024-03-26T04:20:28.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:28 smithi175 bash[29284]: audit 2024-03-26T04:20:28.419528+0000 mon.a (mon.0) 3651 : audit [INF] from='client.? 172.21.15.175:0/1007571012' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "app": "rados"}]: dispatch 2024-03-26T04:20:28.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:28 smithi175 bash[21527]: cluster 2024-03-26T04:20:27.830614+0000 mgr.y (mgr.24431) 1338 : cluster [DBG] pgmap v1764: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:20:28.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:28 smithi175 bash[21527]: audit 2024-03-26T04:20:27.967870+0000 mon.a (mon.0) 3648 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "pg_num": 12}]': finished 2024-03-26T04:20:28.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:28 smithi175 bash[21527]: cluster 2024-03-26T04:20:27.976194+0000 mon.a (mon.0) 3649 : cluster [DBG] osdmap e783: 8 total, 8 up, 8 in 2024-03-26T04:20:28.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:28 smithi175 bash[21527]: audit 2024-03-26T04:20:28.093309+0000 mon.c (mon.2) 880 : audit [INF] from='client.? 172.21.15.175:0/2482660258' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "pg_num": 12}]: dispatch 2024-03-26T04:20:28.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:28 smithi175 bash[21527]: audit 2024-03-26T04:20:28.093781+0000 mon.a (mon.0) 3650 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "pg_num": 12}]: dispatch 2024-03-26T04:20:28.992 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:28 smithi175 bash[21527]: audit 2024-03-26T04:20:28.419528+0000 mon.a (mon.0) 3651 : audit [INF] from='client.? 172.21.15.175:0/1007571012' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "app": "rados"}]: dispatch 2024-03-26T04:20:28.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:28 smithi190 bash[23392]: cluster 2024-03-26T04:20:27.830614+0000 mgr.y (mgr.24431) 1338 : cluster [DBG] pgmap v1764: 176 pgs: 176 active+clean; 723 KiB data, 899 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:20:28.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:28 smithi190 bash[23392]: audit 2024-03-26T04:20:27.967870+0000 mon.a (mon.0) 3648 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "pg_num": 12}]': finished 2024-03-26T04:20:28.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:28 smithi190 bash[23392]: cluster 2024-03-26T04:20:27.976194+0000 mon.a (mon.0) 3649 : cluster [DBG] osdmap e783: 8 total, 8 up, 8 in 2024-03-26T04:20:28.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:28 smithi190 bash[23392]: audit 2024-03-26T04:20:28.093309+0000 mon.c (mon.2) 880 : audit [INF] from='client.? 172.21.15.175:0/2482660258' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "pg_num": 12}]: dispatch 2024-03-26T04:20:28.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:28 smithi190 bash[23392]: audit 2024-03-26T04:20:28.093781+0000 mon.a (mon.0) 3650 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "pg_num": 12}]: dispatch 2024-03-26T04:20:28.998 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:28 smithi190 bash[23392]: audit 2024-03-26T04:20:28.419528+0000 mon.a (mon.0) 3651 : audit [INF] from='client.? 172.21.15.175:0/1007571012' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "app": "rados"}]: dispatch 2024-03-26T04:20:29.088 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:29.088+0000 7f14ee839640 1 -- 172.21.15.175:0/1007571012 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "app": "rados"} v 0) -- 0x7f14e80621e0 con 0x7f14e81024c0 2024-03-26T04:20:29.969 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:29.968+0000 7f14e57fa640 1 -- 172.21.15.175:0/1007571012 <== mon.0 v2:172.21.15.175:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "app": "rados"}]=0 enabled application 'rados' on pool '721fce6c-d7d8-472d-8941-f0e24365dbcf' v785) ==== 213+0+0 (secure 0 0 0) 0x7f14dc060170 con 0x7f14e81024c0 2024-03-26T04:20:29.969 INFO:tasks.workunit.client.0.smithi175.stderr:enabled application 'rados' on pool '721fce6c-d7d8-472d-8941-f0e24365dbcf' 2024-03-26T04:20:29.973 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:29.972+0000 7f14ee839640 1 -- 172.21.15.175:0/1007571012 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f14d006e760 msgr2=0x7f14d0070c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:20:29.973 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:29.972+0000 7f14ee839640 1 --2- 172.21.15.175:0/1007571012 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f14d006e760 0x7f14d0070c20 secure :-1 s=READY pgs=3776 cs=0 l=1 rev1=1 crypto rx=0x7f14d8098220 tx=0x7f14d8009530 comp rx=0 tx=0).stop 2024-03-26T04:20:29.973 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:29.972+0000 7f14ee839640 1 -- 172.21.15.175:0/1007571012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14e81024c0 msgr2=0x7f14e819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:20:29.973 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:29.972+0000 7f14ee839640 1 --2- 172.21.15.175:0/1007571012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14e81024c0 0x7f14e819a620 secure :-1 s=READY pgs=2918 cs=0 l=1 rev1=1 crypto rx=0x7f14dc00e9d0 tx=0x7f14dc00ee90 comp rx=0 tx=0).stop 2024-03-26T04:20:29.973 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:29.972+0000 7f14ee839640 1 -- 172.21.15.175:0/1007571012 shutdown_connections 2024-03-26T04:20:29.973 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:29.972+0000 7f14ee839640 1 --2- 172.21.15.175:0/1007571012 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f14d006e760 0x7f14d0070c20 unknown :-1 s=CLOSED pgs=3776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:29.973 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:29.972+0000 7f14ee839640 1 --2- 172.21.15.175:0/1007571012 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14e8107b90 0x7f14e819e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:29.973 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:29.972+0000 7f14ee839640 1 --2- 172.21.15.175:0/1007571012 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14e81024c0 0x7f14e819a620 unknown :-1 s=CLOSED pgs=2918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:29.973 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:29.972+0000 7f14ee839640 1 --2- 172.21.15.175:0/1007571012 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14e8101b00 0x7f14e819a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:29.974 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:29.972+0000 7f14ee839640 1 -- 172.21.15.175:0/1007571012 >> 172.21.15.175:0/1007571012 conn(0x7f14e80fd5f0 msgr2=0x7f14e81056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T04:20:29.974 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:29.972+0000 7f14ee839640 1 -- 172.21.15.175:0/1007571012 shutdown_connections 2024-03-26T04:20:29.974 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:29.976+0000 7f14ee839640 1 -- 172.21.15.175:0/1007571012 wait complete. 2024-03-26T04:20:29.998 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph osd pool set-quota 721fce6c-d7d8-472d-8941-f0e24365dbcf max_objects 10 2024-03-26T04:20:30.122 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f436a286640 1 -- 172.21.15.175:0/2514358303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f436410d480 msgr2=0x7f4364115000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:20:30.122 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f436a286640 1 --2- 172.21.15.175:0/2514358303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f436410d480 0x7f4364115000 secure :-1 s=READY pgs=3044 cs=0 l=1 rev1=1 crypto rx=0x7f435c009cb0 tx=0x7f435c01cca0 comp rx=0 tx=0).stop 2024-03-26T04:20:30.122 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f436a286640 1 -- 172.21.15.175:0/2514358303 shutdown_connections 2024-03-26T04:20:30.122 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f436a286640 1 --2- 172.21.15.175:0/2514358303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f436410d480 0x7f4364115000 unknown :-1 s=CLOSED pgs=3044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:30.122 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f436a286640 1 --2- 172.21.15.175:0/2514358303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4364108d50 0x7f436410cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:30.123 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f436a286640 1 --2- 172.21.15.175:0/2514358303 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43641083a0 0x7f4364108780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:30.123 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f436a286640 1 -- 172.21.15.175:0/2514358303 >> 172.21.15.175:0/2514358303 conn(0x7f43640779b0 msgr2=0x7f4364106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T04:20:30.123 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f436a286640 1 -- 172.21.15.175:0/2514358303 shutdown_connections 2024-03-26T04:20:30.123 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f436a286640 1 -- 172.21.15.175:0/2514358303 wait complete. 2024-03-26T04:20:30.123 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f436a286640 1 Processor -- start 2024-03-26T04:20:30.123 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f436a286640 1 -- start start 2024-03-26T04:20:30.123 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f436a286640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43641083a0 0x7f43641a1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:20:30.123 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f436a286640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4364108d50 0x7f43641a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:20:30.123 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f436a286640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f436410d480 0x7f43641a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:20:30.123 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f436a286640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f436411ad40 con 0x7f436410d480 2024-03-26T04:20:30.123 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f436a286640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f436411abc0 con 0x7f4364108d50 2024-03-26T04:20:30.123 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f436a286640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f436411aec0 con 0x7f43641083a0 2024-03-26T04:20:30.123 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f4363fff640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43641083a0 0x7f43641a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:20:30.123 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f4363fff640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43641083a0 0x7f43641a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:40556/0 (socket says 172.21.15.175:40556) 2024-03-26T04:20:30.124 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f4368823640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f436410d480 0x7f43641a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:20:30.124 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f4363fff640 1 -- 172.21.15.175:0/682882942 learned_addr learned my addr 172.21.15.175:0/682882942 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T04:20:30.124 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f4363fff640 1 -- 172.21.15.175:0/682882942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4364108d50 msgr2=0x7f43641a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:20:30.124 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f4363fff640 1 --2- 172.21.15.175:0/682882942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4364108d50 0x7f43641a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:30.124 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f4363fff640 1 -- 172.21.15.175:0/682882942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f436410d480 msgr2=0x7f43641a5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:20:30.124 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f4363fff640 1 --2- 172.21.15.175:0/682882942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f436410d480 0x7f43641a5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:30.124 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f4363fff640 1 -- 172.21.15.175:0/682882942 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43641a62f0 con 0x7f43641083a0 2024-03-26T04:20:30.125 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f4368823640 1 --2- 172.21.15.175:0/682882942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f436410d480 0x7f43641a5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T04:20:30.126 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f4363fff640 1 --2- 172.21.15.175:0/682882942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43641083a0 0x7f43641a1340 secure :-1 s=READY pgs=3045 cs=0 l=1 rev1=1 crypto rx=0x7f435400e9f0 tx=0x7f435400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T04:20:30.126 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f4361ffb640 1 -- 172.21.15.175:0/682882942 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435400cd80 con 0x7f43641083a0 2024-03-26T04:20:30.126 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f4361ffb640 1 -- 172.21.15.175:0/682882942 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f43540108b0 con 0x7f43641083a0 2024-03-26T04:20:30.126 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f436a286640 1 -- 172.21.15.175:0/682882942 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43641a65e0 con 0x7f43641083a0 2024-03-26T04:20:30.126 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f4361ffb640 1 -- 172.21.15.175:0/682882942 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4354018800 con 0x7f43641083a0 2024-03-26T04:20:30.126 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.120+0000 7f436a286640 1 -- 172.21.15.175:0/682882942 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f43641adec0 con 0x7f43641083a0 2024-03-26T04:20:30.127 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.124+0000 7f436a286640 1 -- 172.21.15.175:0/682882942 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4364068e90 con 0x7f43641083a0 2024-03-26T04:20:30.127 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.124+0000 7f4361ffb640 1 -- 172.21.15.175:0/682882942 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7f4354010430 con 0x7f43641083a0 2024-03-26T04:20:30.127 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.124+0000 7f4361ffb640 1 --2- 172.21.15.175:0/682882942 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f435006e760 0x7f4350070c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:20:30.127 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.124+0000 7f4361ffb640 1 -- 172.21.15.175:0/682882942 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(785..785 src has 1..785) ==== 9468+0+0 (secure 0 0 0) 0x7f435408fa30 con 0x7f43641083a0 2024-03-26T04:20:30.127 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.124+0000 7f435bfff640 1 --2- 172.21.15.175:0/682882942 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f435006e760 0x7f4350070c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:20:30.127 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.124+0000 7f435bfff640 1 --2- 172.21.15.175:0/682882942 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f435006e760 0x7f4350070c20 secure :-1 s=READY pgs=3777 cs=0 l=1 rev1=1 crypto rx=0x7f434c005d90 tx=0x7f434c005ca0 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T04:20:30.129 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.128+0000 7f4361ffb640 1 -- 172.21.15.175:0/682882942 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f435405ffc0 con 0x7f43641083a0 2024-03-26T04:20:30.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:29 smithi175 bash[21527]: audit 2024-03-26T04:20:28.970754+0000 mon.a (mon.0) 3652 : audit [INF] from='client.? 172.21.15.175:0/1007571012' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "app": "rados"}]': finished 2024-03-26T04:20:30.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:29 smithi175 bash[21527]: cluster 2024-03-26T04:20:28.980381+0000 mon.a (mon.0) 3653 : cluster [DBG] osdmap e784: 8 total, 8 up, 8 in 2024-03-26T04:20:30.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:29 smithi175 bash[21527]: audit 2024-03-26T04:20:29.093074+0000 mon.a (mon.0) 3654 : audit [INF] from='client.? 172.21.15.175:0/1007571012' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "app": "rados"}]: dispatch 2024-03-26T04:20:30.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:29 smithi175 bash[21527]: cluster 2024-03-26T04:20:29.832604+0000 mgr.y (mgr.24431) 1339 : cluster [DBG] pgmap v1767: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-03-26T04:20:30.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:29 smithi175 bash[29284]: audit 2024-03-26T04:20:28.970754+0000 mon.a (mon.0) 3652 : audit [INF] from='client.? 172.21.15.175:0/1007571012' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "app": "rados"}]': finished 2024-03-26T04:20:30.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:29 smithi175 bash[29284]: cluster 2024-03-26T04:20:28.980381+0000 mon.a (mon.0) 3653 : cluster [DBG] osdmap e784: 8 total, 8 up, 8 in 2024-03-26T04:20:30.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:29 smithi175 bash[29284]: audit 2024-03-26T04:20:29.093074+0000 mon.a (mon.0) 3654 : audit [INF] from='client.? 172.21.15.175:0/1007571012' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "app": "rados"}]: dispatch 2024-03-26T04:20:30.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:29 smithi175 bash[29284]: cluster 2024-03-26T04:20:29.832604+0000 mgr.y (mgr.24431) 1339 : cluster [DBG] pgmap v1767: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-03-26T04:20:30.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:29 smithi190 bash[23392]: audit 2024-03-26T04:20:28.970754+0000 mon.a (mon.0) 3652 : audit [INF] from='client.? 172.21.15.175:0/1007571012' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "app": "rados"}]': finished 2024-03-26T04:20:30.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:29 smithi190 bash[23392]: cluster 2024-03-26T04:20:28.980381+0000 mon.a (mon.0) 3653 : cluster [DBG] osdmap e784: 8 total, 8 up, 8 in 2024-03-26T04:20:30.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:29 smithi190 bash[23392]: audit 2024-03-26T04:20:29.093074+0000 mon.a (mon.0) 3654 : audit [INF] from='client.? 172.21.15.175:0/1007571012' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "app": "rados"}]: dispatch 2024-03-26T04:20:30.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:29 smithi190 bash[23392]: cluster 2024-03-26T04:20:29.832604+0000 mgr.y (mgr.24431) 1339 : cluster [DBG] pgmap v1767: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-03-26T04:20:30.302 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:30.300+0000 7f436a286640 1 -- 172.21.15.175:0/682882942 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "field": "max_objects", "val": "10"} v 0) -- 0x7f43641a26a0 con 0x7f43641083a0 2024-03-26T04:20:31.002 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:31.000+0000 7f4361ffb640 1 -- 172.21.15.175:0/682882942 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 721fce6c-d7d8-472d-8941-f0e24365dbcf v786) ==== 223+0+0 (secure 0 0 0) 0x7f43540647f0 con 0x7f43641083a0 2024-03-26T04:20:31.110 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:31.108+0000 7f436a286640 1 -- 172.21.15.175:0/682882942 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "field": "max_objects", "val": "10"} v 0) -- 0x7f43640621e0 con 0x7f43641083a0 2024-03-26T04:20:31.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:30 smithi175 bash[21527]: audit 2024-03-26T04:20:29.973771+0000 mon.a (mon.0) 3655 : audit [INF] from='client.? 172.21.15.175:0/1007571012' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "app": "rados"}]': finished 2024-03-26T04:20:31.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:30 smithi175 bash[21527]: cluster 2024-03-26T04:20:29.980666+0000 mon.a (mon.0) 3656 : cluster [DBG] osdmap e785: 8 total, 8 up, 8 in 2024-03-26T04:20:31.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:30 smithi175 bash[21527]: audit 2024-03-26T04:20:30.307281+0000 mon.c (mon.2) 881 : audit [INF] from='client.? 172.21.15.175:0/682882942' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "field": "max_objects", "val": "10"}]: dispatch 2024-03-26T04:20:31.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:30 smithi175 bash[21527]: audit 2024-03-26T04:20:30.307970+0000 mon.a (mon.0) 3657 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "field": "max_objects", "val": "10"}]: dispatch 2024-03-26T04:20:31.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:30 smithi175 bash[29284]: audit 2024-03-26T04:20:29.973771+0000 mon.a (mon.0) 3655 : audit [INF] from='client.? 172.21.15.175:0/1007571012' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "app": "rados"}]': finished 2024-03-26T04:20:31.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:30 smithi175 bash[29284]: cluster 2024-03-26T04:20:29.980666+0000 mon.a (mon.0) 3656 : cluster [DBG] osdmap e785: 8 total, 8 up, 8 in 2024-03-26T04:20:31.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:30 smithi175 bash[29284]: audit 2024-03-26T04:20:30.307281+0000 mon.c (mon.2) 881 : audit [INF] from='client.? 172.21.15.175:0/682882942' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "field": "max_objects", "val": "10"}]: dispatch 2024-03-26T04:20:31.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:30 smithi175 bash[29284]: audit 2024-03-26T04:20:30.307970+0000 mon.a (mon.0) 3657 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "field": "max_objects", "val": "10"}]: dispatch 2024-03-26T04:20:31.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:30 smithi190 bash[23392]: audit 2024-03-26T04:20:29.973771+0000 mon.a (mon.0) 3655 : audit [INF] from='client.? 172.21.15.175:0/1007571012' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "app": "rados"}]': finished 2024-03-26T04:20:31.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:30 smithi190 bash[23392]: cluster 2024-03-26T04:20:29.980666+0000 mon.a (mon.0) 3656 : cluster [DBG] osdmap e785: 8 total, 8 up, 8 in 2024-03-26T04:20:31.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:30 smithi190 bash[23392]: audit 2024-03-26T04:20:30.307281+0000 mon.c (mon.2) 881 : audit [INF] from='client.? 172.21.15.175:0/682882942' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "field": "max_objects", "val": "10"}]: dispatch 2024-03-26T04:20:31.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:30 smithi190 bash[23392]: audit 2024-03-26T04:20:30.307970+0000 mon.a (mon.0) 3657 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "field": "max_objects", "val": "10"}]: dispatch 2024-03-26T04:20:31.997 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:20:31 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:20:32.012 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:32.012+0000 7f4361ffb640 1 -- 172.21.15.175:0/682882942 <== mon.2 v2:172.21.15.175:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 721fce6c-d7d8-472d-8941-f0e24365dbcf v787) ==== 223+0+0 (secure 0 0 0) 0x7f4354010d40 con 0x7f43641083a0 2024-03-26T04:20:32.013 INFO:tasks.workunit.client.0.smithi175.stderr:set-quota max_objects = 10 for pool 721fce6c-d7d8-472d-8941-f0e24365dbcf 2024-03-26T04:20:32.016 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:32.016+0000 7f436a286640 1 -- 172.21.15.175:0/682882942 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f435006e760 msgr2=0x7f4350070c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:20:32.017 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:32.016+0000 7f436a286640 1 --2- 172.21.15.175:0/682882942 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f435006e760 0x7f4350070c20 secure :-1 s=READY pgs=3777 cs=0 l=1 rev1=1 crypto rx=0x7f434c005d90 tx=0x7f434c005ca0 comp rx=0 tx=0).stop 2024-03-26T04:20:32.017 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:32.016+0000 7f436a286640 1 -- 172.21.15.175:0/682882942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43641083a0 msgr2=0x7f43641a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:20:32.017 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:32.016+0000 7f436a286640 1 --2- 172.21.15.175:0/682882942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43641083a0 0x7f43641a1340 secure :-1 s=READY pgs=3045 cs=0 l=1 rev1=1 crypto rx=0x7f435400e9f0 tx=0x7f435400eeb0 comp rx=0 tx=0).stop 2024-03-26T04:20:32.017 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:32.016+0000 7f436a286640 1 -- 172.21.15.175:0/682882942 shutdown_connections 2024-03-26T04:20:32.017 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:32.016+0000 7f436a286640 1 --2- 172.21.15.175:0/682882942 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f435006e760 0x7f4350070c20 unknown :-1 s=CLOSED pgs=3777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:32.018 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:32.016+0000 7f436a286640 1 --2- 172.21.15.175:0/682882942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f436410d480 0x7f43641a5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:32.018 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:32.016+0000 7f436a286640 1 --2- 172.21.15.175:0/682882942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4364108d50 0x7f43641a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:32.018 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:32.016+0000 7f436a286640 1 --2- 172.21.15.175:0/682882942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43641083a0 0x7f43641a1340 unknown :-1 s=CLOSED pgs=3045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:20:32.018 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:32.016+0000 7f436a286640 1 -- 172.21.15.175:0/682882942 >> 172.21.15.175:0/682882942 conn(0x7f43640779b0 msgr2=0x7f4364077d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T04:20:32.018 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:32.016+0000 7f436a286640 1 -- 172.21.15.175:0/682882942 shutdown_connections 2024-03-26T04:20:32.018 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:20:32.016+0000 7f436a286640 1 -- 172.21.15.175:0/682882942 wait complete. 2024-03-26T04:20:32.035 INFO:tasks.workunit.client.0.smithi175.stderr:+ sleep 30 2024-03-26T04:20:32.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:32 smithi175 bash[21527]: audit 2024-03-26T04:20:30.997819+0000 mon.a (mon.0) 3658 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "field": "max_objects", "val": "10"}]': finished 2024-03-26T04:20:32.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:32 smithi175 bash[21527]: cluster 2024-03-26T04:20:31.005465+0000 mon.a (mon.0) 3659 : cluster [DBG] osdmap e786: 8 total, 8 up, 8 in 2024-03-26T04:20:32.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:32 smithi175 bash[21527]: audit 2024-03-26T04:20:31.115323+0000 mon.c (mon.2) 882 : audit [INF] from='client.? 172.21.15.175:0/682882942' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "field": "max_objects", "val": "10"}]: dispatch 2024-03-26T04:20:32.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:32 smithi175 bash[21527]: audit 2024-03-26T04:20:31.116003+0000 mon.a (mon.0) 3660 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "field": "max_objects", "val": "10"}]: dispatch 2024-03-26T04:20:32.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:32 smithi175 bash[21527]: audit 2024-03-26T04:20:31.532165+0000 mgr.y (mgr.24431) 1340 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:20:32.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:32 smithi175 bash[21527]: cluster 2024-03-26T04:20:31.833353+0000 mgr.y (mgr.24431) 1341 : cluster [DBG] pgmap v1770: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-03-26T04:20:32.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:32 smithi175 bash[29284]: audit 2024-03-26T04:20:30.997819+0000 mon.a (mon.0) 3658 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "field": "max_objects", "val": "10"}]': finished 2024-03-26T04:20:32.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:32 smithi175 bash[29284]: cluster 2024-03-26T04:20:31.005465+0000 mon.a (mon.0) 3659 : cluster [DBG] osdmap e786: 8 total, 8 up, 8 in 2024-03-26T04:20:32.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:32 smithi175 bash[29284]: audit 2024-03-26T04:20:31.115323+0000 mon.c (mon.2) 882 : audit [INF] from='client.? 172.21.15.175:0/682882942' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "field": "max_objects", "val": "10"}]: dispatch 2024-03-26T04:20:32.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:32 smithi175 bash[29284]: audit 2024-03-26T04:20:31.116003+0000 mon.a (mon.0) 3660 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "field": "max_objects", "val": "10"}]: dispatch 2024-03-26T04:20:32.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:32 smithi175 bash[29284]: audit 2024-03-26T04:20:31.532165+0000 mgr.y (mgr.24431) 1340 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:20:32.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:32 smithi175 bash[29284]: cluster 2024-03-26T04:20:31.833353+0000 mgr.y (mgr.24431) 1341 : cluster [DBG] pgmap v1770: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-03-26T04:20:32.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:32 smithi190 bash[23392]: audit 2024-03-26T04:20:30.997819+0000 mon.a (mon.0) 3658 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "field": "max_objects", "val": "10"}]': finished 2024-03-26T04:20:32.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:32 smithi190 bash[23392]: cluster 2024-03-26T04:20:31.005465+0000 mon.a (mon.0) 3659 : cluster [DBG] osdmap e786: 8 total, 8 up, 8 in 2024-03-26T04:20:32.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:32 smithi190 bash[23392]: audit 2024-03-26T04:20:31.115323+0000 mon.c (mon.2) 882 : audit [INF] from='client.? 172.21.15.175:0/682882942' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "field": "max_objects", "val": "10"}]: dispatch 2024-03-26T04:20:32.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:32 smithi190 bash[23392]: audit 2024-03-26T04:20:31.116003+0000 mon.a (mon.0) 3660 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "field": "max_objects", "val": "10"}]: dispatch 2024-03-26T04:20:32.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:32 smithi190 bash[23392]: audit 2024-03-26T04:20:31.532165+0000 mgr.y (mgr.24431) 1340 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:20:32.498 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:32 smithi190 bash[23392]: cluster 2024-03-26T04:20:31.833353+0000 mgr.y (mgr.24431) 1341 : cluster [DBG] pgmap v1770: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-03-26T04:20:33.490 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:33 smithi175 bash[29284]: audit 2024-03-26T04:20:32.007102+0000 mon.a (mon.0) 3661 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "field": "max_objects", "val": "10"}]': finished 2024-03-26T04:20:33.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:33 smithi175 bash[29284]: cluster 2024-03-26T04:20:32.016623+0000 mon.a (mon.0) 3662 : cluster [DBG] osdmap e787: 8 total, 8 up, 8 in 2024-03-26T04:20:33.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:33 smithi175 bash[21527]: audit 2024-03-26T04:20:32.007102+0000 mon.a (mon.0) 3661 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "field": "max_objects", "val": "10"}]': finished 2024-03-26T04:20:33.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:33 smithi175 bash[21527]: cluster 2024-03-26T04:20:32.016623+0000 mon.a (mon.0) 3662 : cluster [DBG] osdmap e787: 8 total, 8 up, 8 in 2024-03-26T04:20:33.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:33 smithi190 bash[23392]: audit 2024-03-26T04:20:32.007102+0000 mon.a (mon.0) 3661 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "field": "max_objects", "val": "10"}]': finished 2024-03-26T04:20:33.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:33 smithi190 bash[23392]: cluster 2024-03-26T04:20:32.016623+0000 mon.a (mon.0) 3662 : cluster [DBG] osdmap e787: 8 total, 8 up, 8 in 2024-03-26T04:20:34.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:34 smithi175 bash[21527]: cluster 2024-03-26T04:20:33.834160+0000 mgr.y (mgr.24431) 1342 : cluster [DBG] pgmap v1772: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 4.9 KiB/s wr, 1 op/s 2024-03-26T04:20:34.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:34 smithi175 bash[29284]: cluster 2024-03-26T04:20:33.834160+0000 mgr.y (mgr.24431) 1342 : cluster [DBG] pgmap v1772: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 4.9 KiB/s wr, 1 op/s 2024-03-26T04:20:34.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:34 smithi190 bash[23392]: cluster 2024-03-26T04:20:33.834160+0000 mgr.y (mgr.24431) 1342 : cluster [DBG] pgmap v1772: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 4.9 KiB/s wr, 1 op/s 2024-03-26T04:20:36.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:35 smithi175 bash[29284]: cluster 2024-03-26T04:20:35.836288+0000 mgr.y (mgr.24431) 1343 : cluster [DBG] pgmap v1773: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:20:36.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:35 smithi175 bash[21527]: cluster 2024-03-26T04:20:35.836288+0000 mgr.y (mgr.24431) 1343 : cluster [DBG] pgmap v1773: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:20:36.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:35 smithi190 bash[23392]: cluster 2024-03-26T04:20:35.836288+0000 mgr.y (mgr.24431) 1343 : cluster [DBG] pgmap v1773: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:20:38.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:37 smithi175 bash[21527]: cluster 2024-03-26T04:20:37.837141+0000 mgr.y (mgr.24431) 1344 : cluster [DBG] pgmap v1774: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 651 B/s rd, 0 op/s 2024-03-26T04:20:38.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:37 smithi175 bash[29284]: cluster 2024-03-26T04:20:37.837141+0000 mgr.y (mgr.24431) 1344 : cluster [DBG] pgmap v1774: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 651 B/s rd, 0 op/s 2024-03-26T04:20:38.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:37 smithi190 bash[23392]: cluster 2024-03-26T04:20:37.837141+0000 mgr.y (mgr.24431) 1344 : cluster [DBG] pgmap v1774: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 651 B/s rd, 0 op/s 2024-03-26T04:20:39.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:20:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:20:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:20:39.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:39 smithi175 bash[21527]: audit 2024-03-26T04:20:38.663389+0000 mon.a (mon.0) 3663 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:20:39.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:39 smithi175 bash[21527]: audit 2024-03-26T04:20:38.666535+0000 mon.c (mon.2) 883 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:20:39.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:39 smithi175 bash[29284]: audit 2024-03-26T04:20:38.663389+0000 mon.a (mon.0) 3663 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:20:39.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:39 smithi175 bash[29284]: audit 2024-03-26T04:20:38.666535+0000 mon.c (mon.2) 883 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:20:39.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:39 smithi190 bash[23392]: audit 2024-03-26T04:20:38.663389+0000 mon.a (mon.0) 3663 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:20:39.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:39 smithi190 bash[23392]: audit 2024-03-26T04:20:38.666535+0000 mon.c (mon.2) 883 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:20:40.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:40 smithi175 bash[21527]: cluster 2024-03-26T04:20:39.839106+0000 mgr.y (mgr.24431) 1345 : cluster [DBG] pgmap v1775: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-26T04:20:40.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:40 smithi175 bash[29284]: cluster 2024-03-26T04:20:39.839106+0000 mgr.y (mgr.24431) 1345 : cluster [DBG] pgmap v1775: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-26T04:20:40.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:40 smithi190 bash[23392]: cluster 2024-03-26T04:20:39.839106+0000 mgr.y (mgr.24431) 1345 : cluster [DBG] pgmap v1775: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-26T04:20:41.904 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:20:41 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:20:42.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:41 smithi175 bash[21527]: audit 2024-03-26T04:20:41.543666+0000 mgr.y (mgr.24431) 1346 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:20:42.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:41 smithi175 bash[21527]: cluster 2024-03-26T04:20:41.839957+0000 mgr.y (mgr.24431) 1347 : cluster [DBG] pgmap v1776: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T04:20:42.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:41 smithi175 bash[29284]: audit 2024-03-26T04:20:41.543666+0000 mgr.y (mgr.24431) 1346 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:20:42.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:41 smithi175 bash[29284]: cluster 2024-03-26T04:20:41.839957+0000 mgr.y (mgr.24431) 1347 : cluster [DBG] pgmap v1776: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T04:20:42.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:41 smithi190 bash[23392]: audit 2024-03-26T04:20:41.543666+0000 mgr.y (mgr.24431) 1346 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:20:42.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:41 smithi190 bash[23392]: cluster 2024-03-26T04:20:41.839957+0000 mgr.y (mgr.24431) 1347 : cluster [DBG] pgmap v1776: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T04:20:44.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:43 smithi175 bash[21527]: cluster 2024-03-26T04:20:43.840754+0000 mgr.y (mgr.24431) 1348 : cluster [DBG] pgmap v1777: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 865 B/s rd, 0 op/s 2024-03-26T04:20:44.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:43 smithi175 bash[29284]: cluster 2024-03-26T04:20:43.840754+0000 mgr.y (mgr.24431) 1348 : cluster [DBG] pgmap v1777: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 865 B/s rd, 0 op/s 2024-03-26T04:20:44.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:43 smithi190 bash[23392]: cluster 2024-03-26T04:20:43.840754+0000 mgr.y (mgr.24431) 1348 : cluster [DBG] pgmap v1777: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 865 B/s rd, 0 op/s 2024-03-26T04:20:46.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:45 smithi175 bash[21527]: cluster 2024-03-26T04:20:45.842759+0000 mgr.y (mgr.24431) 1349 : cluster [DBG] pgmap v1778: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:20:46.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:45 smithi175 bash[29284]: cluster 2024-03-26T04:20:45.842759+0000 mgr.y (mgr.24431) 1349 : cluster [DBG] pgmap v1778: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:20:46.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:45 smithi190 bash[23392]: cluster 2024-03-26T04:20:45.842759+0000 mgr.y (mgr.24431) 1349 : cluster [DBG] pgmap v1778: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:20:48.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:47 smithi175 bash[21527]: cluster 2024-03-26T04:20:47.843568+0000 mgr.y (mgr.24431) 1350 : cluster [DBG] pgmap v1779: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:20:48.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:47 smithi175 bash[29284]: cluster 2024-03-26T04:20:47.843568+0000 mgr.y (mgr.24431) 1350 : cluster [DBG] pgmap v1779: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:20:48.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:47 smithi190 bash[23392]: cluster 2024-03-26T04:20:47.843568+0000 mgr.y (mgr.24431) 1350 : cluster [DBG] pgmap v1779: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:20:49.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:20:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:20:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:20:50.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:49 smithi175 bash[21527]: cluster 2024-03-26T04:20:49.845504+0000 mgr.y (mgr.24431) 1351 : cluster [DBG] pgmap v1780: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:20:50.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:49 smithi175 bash[29284]: cluster 2024-03-26T04:20:49.845504+0000 mgr.y (mgr.24431) 1351 : cluster [DBG] pgmap v1780: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:20:50.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:49 smithi190 bash[23392]: cluster 2024-03-26T04:20:49.845504+0000 mgr.y (mgr.24431) 1351 : cluster [DBG] pgmap v1780: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:20:51.905 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:20:51 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:20:52.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:51 smithi175 bash[29284]: audit 2024-03-26T04:20:51.553380+0000 mgr.y (mgr.24431) 1352 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:20:52.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:51 smithi175 bash[29284]: cluster 2024-03-26T04:20:51.846296+0000 mgr.y (mgr.24431) 1353 : cluster [DBG] pgmap v1781: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:20:52.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:51 smithi175 bash[21527]: audit 2024-03-26T04:20:51.553380+0000 mgr.y (mgr.24431) 1352 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:20:52.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:51 smithi175 bash[21527]: cluster 2024-03-26T04:20:51.846296+0000 mgr.y (mgr.24431) 1353 : cluster [DBG] pgmap v1781: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:20:52.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:51 smithi190 bash[23392]: audit 2024-03-26T04:20:51.553380+0000 mgr.y (mgr.24431) 1352 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:20:52.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:51 smithi190 bash[23392]: cluster 2024-03-26T04:20:51.846296+0000 mgr.y (mgr.24431) 1353 : cluster [DBG] pgmap v1781: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:20:54.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:53 smithi175 bash[29284]: audit 2024-03-26T04:20:53.700135+0000 mon.c (mon.2) 884 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:20:54.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:53 smithi175 bash[21527]: audit 2024-03-26T04:20:53.700135+0000 mon.c (mon.2) 884 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:20:54.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:53 smithi190 bash[23392]: audit 2024-03-26T04:20:53.700135+0000 mon.c (mon.2) 884 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:20:55.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:54 smithi175 bash[21527]: cluster 2024-03-26T04:20:53.847069+0000 mgr.y (mgr.24431) 1354 : cluster [DBG] pgmap v1782: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:20:55.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:54 smithi175 bash[29284]: cluster 2024-03-26T04:20:53.847069+0000 mgr.y (mgr.24431) 1354 : cluster [DBG] pgmap v1782: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:20:55.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:54 smithi190 bash[23392]: cluster 2024-03-26T04:20:53.847069+0000 mgr.y (mgr.24431) 1354 : cluster [DBG] pgmap v1782: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:20:56.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:55 smithi175 bash[29284]: cluster 2024-03-26T04:20:55.849067+0000 mgr.y (mgr.24431) 1355 : cluster [DBG] pgmap v1783: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:20:56.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:55 smithi175 bash[21527]: cluster 2024-03-26T04:20:55.849067+0000 mgr.y (mgr.24431) 1355 : cluster [DBG] pgmap v1783: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:20:56.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:55 smithi190 bash[23392]: cluster 2024-03-26T04:20:55.849067+0000 mgr.y (mgr.24431) 1355 : cluster [DBG] pgmap v1783: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:20:57.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:57 smithi175 bash[21527]: audit 2024-03-26T04:20:57.384692+0000 mon.c (mon.2) 885 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:20:57.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:57 smithi175 bash[29284]: audit 2024-03-26T04:20:57.384692+0000 mon.c (mon.2) 885 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:20:57.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:57 smithi190 bash[23392]: audit 2024-03-26T04:20:57.384692+0000 mon.c (mon.2) 885 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:20:58.740 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:58 smithi175 bash[21527]: cluster 2024-03-26T04:20:57.849609+0000 mgr.y (mgr.24431) 1356 : cluster [DBG] pgmap v1784: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:20:58.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:58 smithi175 bash[21527]: audit 2024-03-26T04:20:58.181919+0000 mon.c (mon.2) 886 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:20:58.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:58 smithi175 bash[21527]: audit 2024-03-26T04:20:58.183748+0000 mon.c (mon.2) 887 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:20:58.741 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:58 smithi175 bash[21527]: audit 2024-03-26T04:20:58.194359+0000 mon.a (mon.0) 3664 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:20:58.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:58 smithi175 bash[29284]: cluster 2024-03-26T04:20:57.849609+0000 mgr.y (mgr.24431) 1356 : cluster [DBG] pgmap v1784: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:20:58.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:58 smithi175 bash[29284]: audit 2024-03-26T04:20:58.181919+0000 mon.c (mon.2) 886 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:20:58.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:58 smithi175 bash[29284]: audit 2024-03-26T04:20:58.183748+0000 mon.c (mon.2) 887 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:20:58.741 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:58 smithi175 bash[29284]: audit 2024-03-26T04:20:58.194359+0000 mon.a (mon.0) 3664 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:20:58.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:58 smithi190 bash[23392]: cluster 2024-03-26T04:20:57.849609+0000 mgr.y (mgr.24431) 1356 : cluster [DBG] pgmap v1784: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:20:58.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:58 smithi190 bash[23392]: audit 2024-03-26T04:20:58.181919+0000 mon.c (mon.2) 886 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:20:58.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:58 smithi190 bash[23392]: audit 2024-03-26T04:20:58.183748+0000 mon.c (mon.2) 887 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:20:58.747 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:58 smithi190 bash[23392]: audit 2024-03-26T04:20:58.194359+0000 mon.a (mon.0) 3664 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:20:59.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:20:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:20:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:21:00.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:20:59 smithi175 bash[21527]: cluster 2024-03-26T04:20:59.851504+0000 mgr.y (mgr.24431) 1357 : cluster [DBG] pgmap v1785: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:21:00.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:20:59 smithi175 bash[29284]: cluster 2024-03-26T04:20:59.851504+0000 mgr.y (mgr.24431) 1357 : cluster [DBG] pgmap v1785: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:21:00.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:20:59 smithi190 bash[23392]: cluster 2024-03-26T04:20:59.851504+0000 mgr.y (mgr.24431) 1357 : cluster [DBG] pgmap v1785: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:21:01.912 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:21:01 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:21:02.037 INFO:tasks.workunit.client.0.smithi175.stderr:+ seq 1 10 2024-03-26T04:21:02.038 INFO:tasks.workunit.client.0.smithi175.stderr:+ rados -p 721fce6c-d7d8-472d-8941-f0e24365dbcf put obj1 /etc/passwd 2024-03-26T04:21:02.086 INFO:tasks.workunit.client.0.smithi175.stderr:+ rados -p 721fce6c-d7d8-472d-8941-f0e24365dbcf put obj2 /etc/passwd 2024-03-26T04:21:02.127 INFO:tasks.workunit.client.0.smithi175.stderr:+ rados -p 721fce6c-d7d8-472d-8941-f0e24365dbcf put obj3 /etc/passwd 2024-03-26T04:21:02.165 INFO:tasks.workunit.client.0.smithi175.stderr:+ rados -p 721fce6c-d7d8-472d-8941-f0e24365dbcf put obj4 /etc/passwd 2024-03-26T04:21:02.204 INFO:tasks.workunit.client.0.smithi175.stderr:+ rados -p 721fce6c-d7d8-472d-8941-f0e24365dbcf put obj5 /etc/passwd 2024-03-26T04:21:02.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:01 smithi175 bash[21527]: audit 2024-03-26T04:21:01.561597+0000 mgr.y (mgr.24431) 1358 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:21:02.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:01 smithi175 bash[21527]: cluster 2024-03-26T04:21:01.852465+0000 mgr.y (mgr.24431) 1359 : cluster [DBG] pgmap v1786: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:21:02.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:01 smithi175 bash[29284]: audit 2024-03-26T04:21:01.561597+0000 mgr.y (mgr.24431) 1358 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:21:02.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:01 smithi175 bash[29284]: cluster 2024-03-26T04:21:01.852465+0000 mgr.y (mgr.24431) 1359 : cluster [DBG] pgmap v1786: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:21:02.243 INFO:tasks.workunit.client.0.smithi175.stderr:+ rados -p 721fce6c-d7d8-472d-8941-f0e24365dbcf put obj6 /etc/passwd 2024-03-26T04:21:02.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:01 smithi190 bash[23392]: audit 2024-03-26T04:21:01.561597+0000 mgr.y (mgr.24431) 1358 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:21:02.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:01 smithi190 bash[23392]: cluster 2024-03-26T04:21:01.852465+0000 mgr.y (mgr.24431) 1359 : cluster [DBG] pgmap v1786: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:21:02.281 INFO:tasks.workunit.client.0.smithi175.stderr:+ rados -p 721fce6c-d7d8-472d-8941-f0e24365dbcf put obj7 /etc/passwd 2024-03-26T04:21:02.318 INFO:tasks.workunit.client.0.smithi175.stderr:+ rados -p 721fce6c-d7d8-472d-8941-f0e24365dbcf put obj8 /etc/passwd 2024-03-26T04:21:02.356 INFO:tasks.workunit.client.0.smithi175.stderr:+ rados -p 721fce6c-d7d8-472d-8941-f0e24365dbcf put obj9 /etc/passwd 2024-03-26T04:21:02.395 INFO:tasks.workunit.client.0.smithi175.stderr:+ rados -p 721fce6c-d7d8-472d-8941-f0e24365dbcf put obj10 /etc/passwd 2024-03-26T04:21:02.434 INFO:tasks.workunit.client.0.smithi175.stderr:+ sleep 30 2024-03-26T04:21:04.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:03 smithi175 bash[21527]: cluster 2024-03-26T04:21:03.853298+0000 mgr.y (mgr.24431) 1360 : cluster [DBG] pgmap v1787: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:21:04.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:03 smithi175 bash[29284]: cluster 2024-03-26T04:21:03.853298+0000 mgr.y (mgr.24431) 1360 : cluster [DBG] pgmap v1787: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:21:04.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:03 smithi190 bash[23392]: cluster 2024-03-26T04:21:03.853298+0000 mgr.y (mgr.24431) 1360 : cluster [DBG] pgmap v1787: 188 pgs: 188 active+clean; 746 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:21:06.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:05 smithi175 bash[21527]: cluster 2024-03-26T04:21:05.855309+0000 mgr.y (mgr.24431) 1361 : cluster [DBG] pgmap v1788: 188 pgs: 188 active+clean; 863 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-03-26T04:21:06.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:05 smithi175 bash[29284]: cluster 2024-03-26T04:21:05.855309+0000 mgr.y (mgr.24431) 1361 : cluster [DBG] pgmap v1788: 188 pgs: 188 active+clean; 863 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-03-26T04:21:06.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:05 smithi190 bash[23392]: cluster 2024-03-26T04:21:05.855309+0000 mgr.y (mgr.24431) 1361 : cluster [DBG] pgmap v1788: 188 pgs: 188 active+clean; 863 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-03-26T04:21:07.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:07 smithi175 bash[21527]: cluster 2024-03-26T04:21:06.972034+0000 mon.a (mon.0) 3665 : cluster [WRN] pool '721fce6c-d7d8-472d-8941-f0e24365dbcf' is full (reached quota's max_objects: 10) 2024-03-26T04:21:07.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:07 smithi175 bash[21527]: cluster 2024-03-26T04:21:06.972562+0000 mon.a (mon.0) 3666 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-26T04:21:07.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:07 smithi175 bash[21527]: cluster 2024-03-26T04:21:06.983436+0000 mon.a (mon.0) 3667 : cluster [DBG] osdmap e788: 8 total, 8 up, 8 in 2024-03-26T04:21:07.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:07 smithi175 bash[29284]: cluster 2024-03-26T04:21:06.972034+0000 mon.a (mon.0) 3665 : cluster [WRN] pool '721fce6c-d7d8-472d-8941-f0e24365dbcf' is full (reached quota's max_objects: 10) 2024-03-26T04:21:07.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:07 smithi175 bash[29284]: cluster 2024-03-26T04:21:06.972562+0000 mon.a (mon.0) 3666 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-26T04:21:07.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:07 smithi175 bash[29284]: cluster 2024-03-26T04:21:06.983436+0000 mon.a (mon.0) 3667 : cluster [DBG] osdmap e788: 8 total, 8 up, 8 in 2024-03-26T04:21:07.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:07 smithi190 bash[23392]: cluster 2024-03-26T04:21:06.972034+0000 mon.a (mon.0) 3665 : cluster [WRN] pool '721fce6c-d7d8-472d-8941-f0e24365dbcf' is full (reached quota's max_objects: 10) 2024-03-26T04:21:07.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:07 smithi190 bash[23392]: cluster 2024-03-26T04:21:06.972562+0000 mon.a (mon.0) 3666 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-26T04:21:07.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:07 smithi190 bash[23392]: cluster 2024-03-26T04:21:06.983436+0000 mon.a (mon.0) 3667 : cluster [DBG] osdmap e788: 8 total, 8 up, 8 in 2024-03-26T04:21:09.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:21:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:21:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:21:09.990 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:09 smithi175 bash[29284]: cluster 2024-03-26T04:21:07.856188+0000 mgr.y (mgr.24431) 1362 : cluster [DBG] pgmap v1790: 188 pgs: 188 active+clean; 863 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-26T04:21:09.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:09 smithi175 bash[29284]: audit 2024-03-26T04:21:08.740374+0000 mon.a (mon.0) 3668 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:21:09.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:09 smithi175 bash[29284]: audit 2024-03-26T04:21:08.743293+0000 mon.c (mon.2) 888 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:21:09.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:09 smithi175 bash[21527]: cluster 2024-03-26T04:21:07.856188+0000 mgr.y (mgr.24431) 1362 : cluster [DBG] pgmap v1790: 188 pgs: 188 active+clean; 863 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-26T04:21:09.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:09 smithi175 bash[21527]: audit 2024-03-26T04:21:08.740374+0000 mon.a (mon.0) 3668 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:21:09.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:09 smithi175 bash[21527]: audit 2024-03-26T04:21:08.743293+0000 mon.c (mon.2) 888 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:21:09.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:09 smithi190 bash[23392]: cluster 2024-03-26T04:21:07.856188+0000 mgr.y (mgr.24431) 1362 : cluster [DBG] pgmap v1790: 188 pgs: 188 active+clean; 863 KiB data, 900 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-26T04:21:09.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:09 smithi190 bash[23392]: audit 2024-03-26T04:21:08.740374+0000 mon.a (mon.0) 3668 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:21:09.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:09 smithi190 bash[23392]: audit 2024-03-26T04:21:08.743293+0000 mon.c (mon.2) 888 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:21:11.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:11 smithi175 bash[29284]: cluster 2024-03-26T04:21:09.858229+0000 mgr.y (mgr.24431) 1363 : cluster [DBG] pgmap v1791: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-26T04:21:11.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:11 smithi175 bash[21527]: cluster 2024-03-26T04:21:09.858229+0000 mgr.y (mgr.24431) 1363 : cluster [DBG] pgmap v1791: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-26T04:21:11.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:11 smithi190 bash[23392]: cluster 2024-03-26T04:21:09.858229+0000 mgr.y (mgr.24431) 1363 : cluster [DBG] pgmap v1791: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-26T04:21:11.997 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:21:11 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:21:12.991 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:12 smithi175 bash[29284]: audit 2024-03-26T04:21:11.565998+0000 mgr.y (mgr.24431) 1364 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:21:12.991 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:12 smithi175 bash[21527]: audit 2024-03-26T04:21:11.565998+0000 mgr.y (mgr.24431) 1364 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:21:12.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:12 smithi190 bash[23392]: audit 2024-03-26T04:21:11.565998+0000 mgr.y (mgr.24431) 1364 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:21:14.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:13 smithi175 bash[29284]: cluster 2024-03-26T04:21:11.859025+0000 mgr.y (mgr.24431) 1365 : cluster [DBG] pgmap v1792: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-26T04:21:14.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:13 smithi175 bash[21527]: cluster 2024-03-26T04:21:11.859025+0000 mgr.y (mgr.24431) 1365 : cluster [DBG] pgmap v1792: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-26T04:21:14.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:13 smithi190 bash[23392]: cluster 2024-03-26T04:21:11.859025+0000 mgr.y (mgr.24431) 1365 : cluster [DBG] pgmap v1792: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-26T04:21:16.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:15 smithi175 bash[29284]: cluster 2024-03-26T04:21:13.859884+0000 mgr.y (mgr.24431) 1366 : cluster [DBG] pgmap v1793: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-26T04:21:16.243 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:15 smithi175 bash[21527]: cluster 2024-03-26T04:21:13.859884+0000 mgr.y (mgr.24431) 1366 : cluster [DBG] pgmap v1793: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-26T04:21:16.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:15 smithi190 bash[23392]: cluster 2024-03-26T04:21:13.859884+0000 mgr.y (mgr.24431) 1366 : cluster [DBG] pgmap v1793: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-26T04:21:18.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:17 smithi175 bash[21527]: cluster 2024-03-26T04:21:15.861870+0000 mgr.y (mgr.24431) 1367 : cluster [DBG] pgmap v1794: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T04:21:18.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:17 smithi175 bash[29284]: cluster 2024-03-26T04:21:15.861870+0000 mgr.y (mgr.24431) 1367 : cluster [DBG] pgmap v1794: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T04:21:18.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:17 smithi190 bash[23392]: cluster 2024-03-26T04:21:15.861870+0000 mgr.y (mgr.24431) 1367 : cluster [DBG] pgmap v1794: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-26T04:21:19.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:21:18 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:21:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:21:20.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:19 smithi175 bash[29284]: cluster 2024-03-26T04:21:17.862651+0000 mgr.y (mgr.24431) 1368 : cluster [DBG] pgmap v1795: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 940 B/s rd, 0 op/s 2024-03-26T04:21:20.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:19 smithi175 bash[21527]: cluster 2024-03-26T04:21:17.862651+0000 mgr.y (mgr.24431) 1368 : cluster [DBG] pgmap v1795: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 940 B/s rd, 0 op/s 2024-03-26T04:21:20.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:19 smithi190 bash[23392]: cluster 2024-03-26T04:21:17.862651+0000 mgr.y (mgr.24431) 1368 : cluster [DBG] pgmap v1795: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 940 B/s rd, 0 op/s 2024-03-26T04:21:21.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:21 smithi190 bash[23392]: cluster 2024-03-26T04:21:19.864652+0000 mgr.y (mgr.24431) 1369 : cluster [DBG] pgmap v1796: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:21:21.997 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:21:21 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:21:22.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:21 smithi175 bash[29284]: cluster 2024-03-26T04:21:19.864652+0000 mgr.y (mgr.24431) 1369 : cluster [DBG] pgmap v1796: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:21:22.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:21 smithi175 bash[21527]: cluster 2024-03-26T04:21:19.864652+0000 mgr.y (mgr.24431) 1369 : cluster [DBG] pgmap v1796: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:21:23.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:22 smithi175 bash[21527]: audit 2024-03-26T04:21:21.569263+0000 mgr.y (mgr.24431) 1370 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:21:23.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:22 smithi175 bash[29284]: audit 2024-03-26T04:21:21.569263+0000 mgr.y (mgr.24431) 1370 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:21:23.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:22 smithi190 bash[23392]: audit 2024-03-26T04:21:21.569263+0000 mgr.y (mgr.24431) 1370 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:21:24.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:23 smithi175 bash[21527]: cluster 2024-03-26T04:21:21.865435+0000 mgr.y (mgr.24431) 1371 : cluster [DBG] pgmap v1797: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:21:24.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:23 smithi175 bash[21527]: audit 2024-03-26T04:21:23.776843+0000 mon.c (mon.2) 889 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:21:24.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:23 smithi175 bash[29284]: cluster 2024-03-26T04:21:21.865435+0000 mgr.y (mgr.24431) 1371 : cluster [DBG] pgmap v1797: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:21:24.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:23 smithi175 bash[29284]: audit 2024-03-26T04:21:23.776843+0000 mon.c (mon.2) 889 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:21:24.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:23 smithi190 bash[23392]: cluster 2024-03-26T04:21:21.865435+0000 mgr.y (mgr.24431) 1371 : cluster [DBG] pgmap v1797: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:21:24.248 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:23 smithi190 bash[23392]: audit 2024-03-26T04:21:23.776843+0000 mon.c (mon.2) 889 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:21:26.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:25 smithi175 bash[21527]: cluster 2024-03-26T04:21:23.866193+0000 mgr.y (mgr.24431) 1372 : cluster [DBG] pgmap v1798: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:21:26.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:25 smithi175 bash[29284]: cluster 2024-03-26T04:21:23.866193+0000 mgr.y (mgr.24431) 1372 : cluster [DBG] pgmap v1798: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:21:26.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:25 smithi190 bash[23392]: cluster 2024-03-26T04:21:23.866193+0000 mgr.y (mgr.24431) 1372 : cluster [DBG] pgmap v1798: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:21:28.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:27 smithi175 bash[21527]: cluster 2024-03-26T04:21:25.868162+0000 mgr.y (mgr.24431) 1373 : cluster [DBG] pgmap v1799: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:21:28.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:27 smithi175 bash[29284]: cluster 2024-03-26T04:21:25.868162+0000 mgr.y (mgr.24431) 1373 : cluster [DBG] pgmap v1799: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:21:28.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:27 smithi190 bash[23392]: cluster 2024-03-26T04:21:25.868162+0000 mgr.y (mgr.24431) 1373 : cluster [DBG] pgmap v1799: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:21:29.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:21:28 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:21:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:21:30.240 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:29 smithi175 bash[21527]: cluster 2024-03-26T04:21:27.868905+0000 mgr.y (mgr.24431) 1374 : cluster [DBG] pgmap v1800: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:21:30.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:29 smithi175 bash[29284]: cluster 2024-03-26T04:21:27.868905+0000 mgr.y (mgr.24431) 1374 : cluster [DBG] pgmap v1800: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:21:30.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:29 smithi190 bash[23392]: cluster 2024-03-26T04:21:27.868905+0000 mgr.y (mgr.24431) 1374 : cluster [DBG] pgmap v1800: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:21:31.997 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:31 smithi190 bash[23392]: cluster 2024-03-26T04:21:29.870876+0000 mgr.y (mgr.24431) 1375 : cluster [DBG] pgmap v1801: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:21:31.997 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:21:31 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:21:32.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:31 smithi175 bash[29284]: cluster 2024-03-26T04:21:29.870876+0000 mgr.y (mgr.24431) 1375 : cluster [DBG] pgmap v1801: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:21:32.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:31 smithi175 bash[21527]: cluster 2024-03-26T04:21:29.870876+0000 mgr.y (mgr.24431) 1375 : cluster [DBG] pgmap v1801: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:21:32.435 INFO:tasks.workunit.client.0.smithi175.stderr:+ rados -p 0e4b321e-b83b-43c2-ade1-c76622f394b4 put threemore /etc/passwd 2024-03-26T04:21:32.482 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph osd pool set-quota 0e4b321e-b83b-43c2-ade1-c76622f394b4 max_bytes 0 2024-03-26T04:21:32.603 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.599+0000 7fe7d963e640 1 -- 172.21.15.175:0/2476655351 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7d4101ae0 msgr2=0x7fe7d4101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:21:32.604 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.599+0000 7fe7d963e640 1 --2- 172.21.15.175:0/2476655351 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7d4101ae0 0x7fe7d4101f60 secure :-1 s=READY pgs=3059 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc009ce0 tx=0x7fe7bc01cc80 comp rx=0 tx=0).stop 2024-03-26T04:21:32.604 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.599+0000 7fe7d963e640 1 -- 172.21.15.175:0/2476655351 shutdown_connections 2024-03-26T04:21:32.604 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.599+0000 7fe7d963e640 1 --2- 172.21.15.175:0/2476655351 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7d4101ae0 0x7fe7d4101f60 unknown :-1 s=CLOSED pgs=3059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:21:32.604 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.599+0000 7fe7d963e640 1 --2- 172.21.15.175:0/2476655351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d41024a0 0x7fe7d410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:21:32.604 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.599+0000 7fe7d963e640 1 --2- 172.21.15.175:0/2476655351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7d4107b70 0x7fe7d4107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:21:32.604 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.599+0000 7fe7d963e640 1 -- 172.21.15.175:0/2476655351 >> 172.21.15.175:0/2476655351 conn(0x7fe7d40fd630 msgr2=0x7fe7d40ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T04:21:32.604 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.599+0000 7fe7d963e640 1 -- 172.21.15.175:0/2476655351 shutdown_connections 2024-03-26T04:21:32.604 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.599+0000 7fe7d963e640 1 -- 172.21.15.175:0/2476655351 wait complete. 2024-03-26T04:21:32.604 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.599+0000 7fe7d963e640 1 Processor -- start 2024-03-26T04:21:32.604 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.599+0000 7fe7d963e640 1 -- start start 2024-03-26T04:21:32.604 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.599+0000 7fe7d963e640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7d4101ae0 0x7fe7d419a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:21:32.604 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.599+0000 7fe7d963e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7d41024a0 0x7fe7d419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:21:32.605 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.599+0000 7fe7d963e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d4107b70 0x7fe7d419ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:21:32.605 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.599+0000 7fe7d963e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d4113b20 con 0x7fe7d4107b70 2024-03-26T04:21:32.605 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.599+0000 7fe7d963e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d41139a0 con 0x7fe7d41024a0 2024-03-26T04:21:32.605 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.599+0000 7fe7d963e640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe7d4113ca0 con 0x7fe7d4101ae0 2024-03-26T04:21:32.605 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.599+0000 7fe7d2ffd640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7d4101ae0 0x7fe7d419a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:21:32.605 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.599+0000 7fe7d37fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d4107b70 0x7fe7d419ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:21:32.605 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.599+0000 7fe7d2ffd640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7d4101ae0 0x7fe7d419a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:41186/0 (socket says 172.21.15.175:41186) 2024-03-26T04:21:32.605 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.599+0000 7fe7d2ffd640 1 -- 172.21.15.175:0/4181574722 learned_addr learned my addr 172.21.15.175:0/4181574722 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T04:21:32.605 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.599+0000 7fe7d37fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d4107b70 0x7fe7d419ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55568/0 (socket says 172.21.15.175:55568) 2024-03-26T04:21:32.605 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.599+0000 7fe7d27fc640 1 --2- 172.21.15.175:0/4181574722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7d41024a0 0x7fe7d419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:21:32.605 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.599+0000 7fe7d2ffd640 1 -- 172.21.15.175:0/4181574722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7d41024a0 msgr2=0x7fe7d419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:21:32.605 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.599+0000 7fe7d2ffd640 1 --2- 172.21.15.175:0/4181574722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7d41024a0 0x7fe7d419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:21:32.605 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.599+0000 7fe7d2ffd640 1 -- 172.21.15.175:0/4181574722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d4107b70 msgr2=0x7fe7d419ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:21:32.605 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.599+0000 7fe7d2ffd640 1 --2- 172.21.15.175:0/4181574722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d4107b70 0x7fe7d419ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:21:32.606 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.599+0000 7fe7d2ffd640 1 -- 172.21.15.175:0/4181574722 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d419f100 con 0x7fe7d4101ae0 2024-03-26T04:21:32.606 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.603+0000 7fe7d37fe640 1 --2- 172.21.15.175:0/4181574722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d4107b70 0x7fe7d419ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T04:21:32.606 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.603+0000 7fe7d2ffd640 1 --2- 172.21.15.175:0/4181574722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7d4101ae0 0x7fe7d419a150 secure :-1 s=READY pgs=3060 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00c970 tx=0x7fe7cc00ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T04:21:32.606 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.603+0000 7fe7cbfff640 1 -- 172.21.15.175:0/4181574722 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc007bb0 con 0x7fe7d4101ae0 2024-03-26T04:21:32.606 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.603+0000 7fe7cbfff640 1 -- 172.21.15.175:0/4181574722 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fe7cc007d50 con 0x7fe7d4101ae0 2024-03-26T04:21:32.606 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.603+0000 7fe7d963e640 1 -- 172.21.15.175:0/4181574722 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d419f3f0 con 0x7fe7d4101ae0 2024-03-26T04:21:32.606 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.603+0000 7fe7cbfff640 1 -- 172.21.15.175:0/4181574722 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc004e40 con 0x7fe7d4101ae0 2024-03-26T04:21:32.606 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.603+0000 7fe7d963e640 1 -- 172.21.15.175:0/4181574722 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d41a6cd0 con 0x7fe7d4101ae0 2024-03-26T04:21:32.606 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.603+0000 7fe7d963e640 1 -- 172.21.15.175:0/4181574722 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7d4068e90 con 0x7fe7d4101ae0 2024-03-26T04:21:32.607 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.603+0000 7fe7cbfff640 1 -- 172.21.15.175:0/4181574722 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7fe7cc005000 con 0x7fe7d4101ae0 2024-03-26T04:21:32.607 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.603+0000 7fe7cbfff640 1 --2- 172.21.15.175:0/4181574722 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fe7c006e760 0x7fe7c0070c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:21:32.607 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.603+0000 7fe7cbfff640 1 -- 172.21.15.175:0/4181574722 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(788..788 src has 1..788) ==== 9468+0+0 (secure 0 0 0) 0x7fe7cc08f810 con 0x7fe7d4101ae0 2024-03-26T04:21:32.607 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.603+0000 7fe7d27fc640 1 --2- 172.21.15.175:0/4181574722 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fe7c006e760 0x7fe7c0070c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:21:32.607 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.603+0000 7fe7cbfff640 1 -- 172.21.15.175:0/4181574722 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=789}) -- 0x7fe7c007aa90 con 0x7fe7d4101ae0 2024-03-26T04:21:32.607 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.603+0000 7fe7d27fc640 1 --2- 172.21.15.175:0/4181574722 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fe7c006e760 0x7fe7c0070c20 secure :-1 s=READY pgs=3789 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc009cb0 tx=0x7fe7bc019a20 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T04:21:32.609 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.607+0000 7fe7cbfff640 1 -- 172.21.15.175:0/4181574722 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7fe7cc014030 con 0x7fe7d4101ae0 2024-03-26T04:21:32.776 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.775+0000 7fe7d963e640 1 -- 172.21.15.175:0/4181574722 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"} v 0) -- 0x7fe7d41a6f00 con 0x7fe7d4101ae0 2024-03-26T04:21:32.807 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.807+0000 7fe7cbfff640 1 -- 172.21.15.175:0/4181574722 <== mon.2 v2:172.21.15.175:3301/0 7 ==== osd_map(789..789 src has 1..789) ==== 628+0+0 (secure 0 0 0) 0x7fe7cc08cfd0 con 0x7fe7d4101ae0 2024-03-26T04:21:32.808 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.807+0000 7fe7cbfff640 1 -- 172.21.15.175:0/4181574722 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=790}) -- 0x7fe7c007b670 con 0x7fe7d4101ae0 2024-03-26T04:21:32.815 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.815+0000 7fe7cbfff640 1 -- 172.21.15.175:0/4181574722 <== mon.2 v2:172.21.15.175:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 0e4b321e-b83b-43c2-ade1-c76622f394b4 v789) ==== 217+0+0 (secure 0 0 0) 0x7fe7cc05fde0 con 0x7fe7d4101ae0 2024-03-26T04:21:32.916 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:32.915+0000 7fe7d963e640 1 -- 172.21.15.175:0/4181574722 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"} v 0) -- 0x7fe7d4062220 con 0x7fe7d4101ae0 2024-03-26T04:21:33.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:32 smithi175 bash[21527]: audit 2024-03-26T04:21:31.576408+0000 mgr.y (mgr.24431) 1376 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:21:33.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:32 smithi175 bash[21527]: audit 2024-03-26T04:21:32.781052+0000 mon.c (mon.2) 890 : audit [INF] from='client.? 172.21.15.175:0/4181574722' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-26T04:21:33.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:32 smithi175 bash[21527]: audit 2024-03-26T04:21:32.781623+0000 mon.a (mon.0) 3669 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-26T04:21:33.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:32 smithi175 bash[29284]: audit 2024-03-26T04:21:31.576408+0000 mgr.y (mgr.24431) 1376 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:21:33.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:32 smithi175 bash[29284]: audit 2024-03-26T04:21:32.781052+0000 mon.c (mon.2) 890 : audit [INF] from='client.? 172.21.15.175:0/4181574722' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-26T04:21:33.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:32 smithi175 bash[29284]: audit 2024-03-26T04:21:32.781623+0000 mon.a (mon.0) 3669 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-26T04:21:33.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:32 smithi190 bash[23392]: audit 2024-03-26T04:21:31.576408+0000 mgr.y (mgr.24431) 1376 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:21:33.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:32 smithi190 bash[23392]: audit 2024-03-26T04:21:32.781052+0000 mon.c (mon.2) 890 : audit [INF] from='client.? 172.21.15.175:0/4181574722' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-26T04:21:33.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:32 smithi190 bash[23392]: audit 2024-03-26T04:21:32.781623+0000 mon.a (mon.0) 3669 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-26T04:21:33.831 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.831+0000 7fe7cbfff640 1 -- 172.21.15.175:0/4181574722 <== mon.2 v2:172.21.15.175:3301/0 9 ==== osd_map(790..790 src has 1..790) ==== 628+0+0 (secure 0 0 0) 0x7fe7cc0046d0 con 0x7fe7d4101ae0 2024-03-26T04:21:33.831 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.831+0000 7fe7cbfff640 1 -- 172.21.15.175:0/4181574722 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=791}) -- 0x7fe7c007bcb0 con 0x7fe7d4101ae0 2024-03-26T04:21:33.839 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.839+0000 7fe7cbfff640 1 -- 172.21.15.175:0/4181574722 <== mon.2 v2:172.21.15.175:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 0e4b321e-b83b-43c2-ade1-c76622f394b4 v790) ==== 217+0+0 (secure 0 0 0) 0x7fe7cc064610 con 0x7fe7d4101ae0 2024-03-26T04:21:33.839 INFO:tasks.workunit.client.0.smithi175.stderr:set-quota max_bytes = 0 for pool 0e4b321e-b83b-43c2-ade1-c76622f394b4 2024-03-26T04:21:33.843 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.843+0000 7fe7d963e640 1 -- 172.21.15.175:0/4181574722 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fe7c006e760 msgr2=0x7fe7c0070c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:21:33.844 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.843+0000 7fe7d963e640 1 --2- 172.21.15.175:0/4181574722 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fe7c006e760 0x7fe7c0070c20 secure :-1 s=READY pgs=3789 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc009cb0 tx=0x7fe7bc019a20 comp rx=0 tx=0).stop 2024-03-26T04:21:33.844 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.843+0000 7fe7d963e640 1 -- 172.21.15.175:0/4181574722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7d4101ae0 msgr2=0x7fe7d419a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:21:33.844 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.843+0000 7fe7d963e640 1 --2- 172.21.15.175:0/4181574722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7d4101ae0 0x7fe7d419a150 secure :-1 s=READY pgs=3060 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00c970 tx=0x7fe7cc00ce30 comp rx=0 tx=0).stop 2024-03-26T04:21:33.846 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.843+0000 7fe7d963e640 1 -- 172.21.15.175:0/4181574722 shutdown_connections 2024-03-26T04:21:33.846 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.843+0000 7fe7d963e640 1 --2- 172.21.15.175:0/4181574722 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fe7c006e760 0x7fe7c0070c20 unknown :-1 s=CLOSED pgs=3789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:21:33.846 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.843+0000 7fe7d963e640 1 --2- 172.21.15.175:0/4181574722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7d4107b70 0x7fe7d419ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:21:33.846 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.843+0000 7fe7d963e640 1 --2- 172.21.15.175:0/4181574722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7d41024a0 0x7fe7d419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:21:33.846 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.843+0000 7fe7d963e640 1 --2- 172.21.15.175:0/4181574722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7d4101ae0 0x7fe7d419a150 unknown :-1 s=CLOSED pgs=3060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:21:33.846 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.843+0000 7fe7d963e640 1 -- 172.21.15.175:0/4181574722 >> 172.21.15.175:0/4181574722 conn(0x7fe7d40fd630 msgr2=0x7fe7d40fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T04:21:33.846 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.843+0000 7fe7d963e640 1 -- 172.21.15.175:0/4181574722 shutdown_connections 2024-03-26T04:21:33.847 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.843+0000 7fe7d963e640 1 -- 172.21.15.175:0/4181574722 wait complete. 2024-03-26T04:21:33.864 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph osd pool set-quota 0e4b321e-b83b-43c2-ade1-c76622f394b4 max_objects 0 2024-03-26T04:21:33.987 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.983+0000 7f9197342640 1 -- 172.21.15.175:0/42017655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9190103f30 msgr2=0x7f9190104310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:21:33.987 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.983+0000 7f9197342640 1 --2- 172.21.15.175:0/42017655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9190103f30 0x7f9190104310 secure :-1 s=READY pgs=3061 cs=0 l=1 rev1=1 crypto rx=0x7f918c009cb0 tx=0x7f918c01cca0 comp rx=0 tx=0).stop 2024-03-26T04:21:33.988 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.983+0000 7f9197342640 1 -- 172.21.15.175:0/42017655 shutdown_connections 2024-03-26T04:21:33.988 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.983+0000 7f9197342640 1 --2- 172.21.15.175:0/42017655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9190109060 0x7f9190110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:21:33.988 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.983+0000 7f9197342640 1 --2- 172.21.15.175:0/42017655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91901048e0 0x7f9190108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:21:33.988 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.983+0000 7f9197342640 1 --2- 172.21.15.175:0/42017655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9190103f30 0x7f9190104310 unknown :-1 s=CLOSED pgs=3061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:21:33.988 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.983+0000 7f9197342640 1 -- 172.21.15.175:0/42017655 >> 172.21.15.175:0/42017655 conn(0x7f91900ff780 msgr2=0x7f9190101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T04:21:33.988 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.983+0000 7f9197342640 1 -- 172.21.15.175:0/42017655 shutdown_connections 2024-03-26T04:21:33.988 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.983+0000 7f9197342640 1 -- 172.21.15.175:0/42017655 wait complete. 2024-03-26T04:21:33.988 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.983+0000 7f9197342640 1 Processor -- start 2024-03-26T04:21:33.988 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.983+0000 7f9197342640 1 -- start start 2024-03-26T04:21:33.988 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.983+0000 7f9197342640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9190103f30 0x7f919019c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:21:33.988 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.983+0000 7f9197342640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91901048e0 0x7f919019c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:21:33.988 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.983+0000 7f9197342640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9190109060 0x7f91901a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:21:33.989 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.983+0000 7f9197342640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9190115d90 con 0x7f9190109060 2024-03-26T04:21:33.989 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.983+0000 7f9197342640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9190115c10 con 0x7f91901048e0 2024-03-26T04:21:33.989 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.983+0000 7f9197342640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9190115f10 con 0x7f9190103f30 2024-03-26T04:21:33.989 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.983+0000 7f91958df640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9190109060 0x7f91901a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:21:33.989 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.983+0000 7f91950de640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9190103f30 0x7f919019c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:21:33.989 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.983+0000 7f91958df640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9190109060 0x7f91901a0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55586/0 (socket says 172.21.15.175:55586) 2024-03-26T04:21:33.989 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.983+0000 7f91958df640 1 -- 172.21.15.175:0/4270377391 learned_addr learned my addr 172.21.15.175:0/4270377391 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T04:21:33.989 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.983+0000 7f91958df640 1 -- 172.21.15.175:0/4270377391 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9190103f30 msgr2=0x7f919019c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:21:33.989 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.983+0000 7f91958df640 1 --2- 172.21.15.175:0/4270377391 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9190103f30 0x7f919019c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:21:33.989 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.983+0000 7f91958df640 1 -- 172.21.15.175:0/4270377391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91901048e0 msgr2=0x7f919019c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:21:33.989 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.983+0000 7f91948dd640 1 --2- 172.21.15.175:0/4270377391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91901048e0 0x7f919019c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:21:33.989 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.983+0000 7f91958df640 1 --2- 172.21.15.175:0/4270377391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91901048e0 0x7f919019c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:21:33.989 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.983+0000 7f91958df640 1 -- 172.21.15.175:0/4270377391 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91901a1360 con 0x7f9190109060 2024-03-26T04:21:33.990 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.983+0000 7f91958df640 1 --2- 172.21.15.175:0/4270377391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9190109060 0x7f91901a0c80 secure :-1 s=READY pgs=2939 cs=0 l=1 rev1=1 crypto rx=0x7f918800d750 tx=0x7f918800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T04:21:33.990 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.983+0000 7f91867fc640 1 -- 172.21.15.175:0/4270377391 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188017070 con 0x7f9190109060 2024-03-26T04:21:33.990 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.987+0000 7f9197342640 1 -- 172.21.15.175:0/4270377391 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91901a1650 con 0x7f9190109060 2024-03-26T04:21:33.990 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.987+0000 7f91867fc640 1 -- 172.21.15.175:0/4270377391 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f91880054a0 con 0x7f9190109060 2024-03-26T04:21:33.990 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.987+0000 7f91867fc640 1 -- 172.21.15.175:0/4270377391 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188004bc0 con 0x7f9190109060 2024-03-26T04:21:33.990 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.987+0000 7f9197342640 1 -- 172.21.15.175:0/4270377391 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91901a8f30 con 0x7f9190109060 2024-03-26T04:21:33.990 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.987+0000 7f9197342640 1 -- 172.21.15.175:0/4270377391 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9190068e90 con 0x7f9190109060 2024-03-26T04:21:33.991 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.987+0000 7f91867fc640 1 -- 172.21.15.175:0/4270377391 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7f9188004d60 con 0x7f9190109060 2024-03-26T04:21:33.991 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.987+0000 7f91867fc640 1 --2- 172.21.15.175:0/4270377391 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f917c06e760 0x7f917c070c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:21:33.991 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.987+0000 7f91867fc640 1 -- 172.21.15.175:0/4270377391 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(790..790 src has 1..790) ==== 9468+0+0 (secure 0 0 0) 0x7f918808f0d0 con 0x7f9190109060 2024-03-26T04:21:33.992 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.987+0000 7f91950de640 1 --2- 172.21.15.175:0/4270377391 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f917c06e760 0x7f917c070c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:21:33.992 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.987+0000 7f91867fc640 1 -- 172.21.15.175:0/4270377391 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=791}) -- 0x7f917c07aa10 con 0x7f9190109060 2024-03-26T04:21:33.992 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.987+0000 7f91950de640 1 --2- 172.21.15.175:0/4270377391 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f917c06e760 0x7f917c070c20 secure :-1 s=READY pgs=3790 cs=0 l=1 rev1=1 crypto rx=0x7f918c01d180 tx=0x7f918c0a7040 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T04:21:33.993 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:33.991+0000 7f91867fc640 1 -- 172.21.15.175:0/4270377391 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f918805f5a0 con 0x7f9190109060 2024-03-26T04:21:34.167 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:34.167+0000 7f9197342640 1 -- 172.21.15.175:0/4270377391 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "0"} v 0) -- 0x7f91901a9100 con 0x7f9190109060 2024-03-26T04:21:34.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:33 smithi175 bash[21527]: cluster 2024-03-26T04:21:31.871756+0000 mgr.y (mgr.24431) 1377 : cluster [DBG] pgmap v1802: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:21:34.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:33 smithi175 bash[21527]: audit 2024-03-26T04:21:32.807724+0000 mon.a (mon.0) 3670 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]': finished 2024-03-26T04:21:34.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:33 smithi175 bash[21527]: cluster 2024-03-26T04:21:32.810924+0000 mon.a (mon.0) 3671 : cluster [DBG] osdmap e789: 8 total, 8 up, 8 in 2024-03-26T04:21:34.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:33 smithi175 bash[21527]: audit 2024-03-26T04:21:32.921138+0000 mon.c (mon.2) 891 : audit [INF] from='client.? 172.21.15.175:0/4181574722' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-26T04:21:34.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:33 smithi175 bash[21527]: audit 2024-03-26T04:21:32.921869+0000 mon.a (mon.0) 3672 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-26T04:21:34.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:33 smithi175 bash[29284]: cluster 2024-03-26T04:21:31.871756+0000 mgr.y (mgr.24431) 1377 : cluster [DBG] pgmap v1802: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:21:34.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:33 smithi175 bash[29284]: audit 2024-03-26T04:21:32.807724+0000 mon.a (mon.0) 3670 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]': finished 2024-03-26T04:21:34.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:33 smithi175 bash[29284]: cluster 2024-03-26T04:21:32.810924+0000 mon.a (mon.0) 3671 : cluster [DBG] osdmap e789: 8 total, 8 up, 8 in 2024-03-26T04:21:34.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:33 smithi175 bash[29284]: audit 2024-03-26T04:21:32.921138+0000 mon.c (mon.2) 891 : audit [INF] from='client.? 172.21.15.175:0/4181574722' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-26T04:21:34.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:33 smithi175 bash[29284]: audit 2024-03-26T04:21:32.921869+0000 mon.a (mon.0) 3672 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-26T04:21:34.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:33 smithi190 bash[23392]: cluster 2024-03-26T04:21:31.871756+0000 mgr.y (mgr.24431) 1377 : cluster [DBG] pgmap v1802: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:21:34.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:33 smithi190 bash[23392]: audit 2024-03-26T04:21:32.807724+0000 mon.a (mon.0) 3670 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]': finished 2024-03-26T04:21:34.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:33 smithi190 bash[23392]: cluster 2024-03-26T04:21:32.810924+0000 mon.a (mon.0) 3671 : cluster [DBG] osdmap e789: 8 total, 8 up, 8 in 2024-03-26T04:21:34.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:33 smithi190 bash[23392]: audit 2024-03-26T04:21:32.921138+0000 mon.c (mon.2) 891 : audit [INF] from='client.? 172.21.15.175:0/4181574722' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-26T04:21:34.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:33 smithi190 bash[23392]: audit 2024-03-26T04:21:32.921869+0000 mon.a (mon.0) 3672 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-26T04:21:34.841 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:34.839+0000 7f91867fc640 1 -- 172.21.15.175:0/4270377391 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 0e4b321e-b83b-43c2-ade1-c76622f394b4 v791) ==== 221+0+0 (secure 0 0 0) 0x7f9188063dd0 con 0x7f9190109060 2024-03-26T04:21:34.842 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:34.839+0000 7f91867fc640 1 -- 172.21.15.175:0/4270377391 <== mon.0 v2:172.21.15.175:3300/0 8 ==== osd_map(791..791 src has 1..791) ==== 628+0+0 (secure 0 0 0) 0x7f91880c3cb0 con 0x7f9190109060 2024-03-26T04:21:34.842 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:34.839+0000 7f91867fc640 1 -- 172.21.15.175:0/4270377391 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=792}) -- 0x7f917c07b950 con 0x7f9190109060 2024-03-26T04:21:34.953 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:34.951+0000 7f9197342640 1 -- 172.21.15.175:0/4270377391 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "0"} v 0) -- 0x7f91901a9350 con 0x7f9190109060 2024-03-26T04:21:35.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:34 smithi175 bash[21527]: audit 2024-03-26T04:21:33.834051+0000 mon.a (mon.0) 3673 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]': finished 2024-03-26T04:21:35.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:34 smithi175 bash[21527]: cluster 2024-03-26T04:21:33.842236+0000 mon.a (mon.0) 3674 : cluster [DBG] osdmap e790: 8 total, 8 up, 8 in 2024-03-26T04:21:35.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:34 smithi175 bash[21527]: audit 2024-03-26T04:21:34.172723+0000 mon.a (mon.0) 3675 : audit [INF] from='client.? 172.21.15.175:0/4270377391' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "0"}]: dispatch 2024-03-26T04:21:35.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:34 smithi175 bash[29284]: audit 2024-03-26T04:21:33.834051+0000 mon.a (mon.0) 3673 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]': finished 2024-03-26T04:21:35.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:34 smithi175 bash[29284]: cluster 2024-03-26T04:21:33.842236+0000 mon.a (mon.0) 3674 : cluster [DBG] osdmap e790: 8 total, 8 up, 8 in 2024-03-26T04:21:35.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:34 smithi175 bash[29284]: audit 2024-03-26T04:21:34.172723+0000 mon.a (mon.0) 3675 : audit [INF] from='client.? 172.21.15.175:0/4270377391' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "0"}]: dispatch 2024-03-26T04:21:35.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:34 smithi190 bash[23392]: audit 2024-03-26T04:21:33.834051+0000 mon.a (mon.0) 3673 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_bytes", "val": "0"}]': finished 2024-03-26T04:21:35.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:34 smithi190 bash[23392]: cluster 2024-03-26T04:21:33.842236+0000 mon.a (mon.0) 3674 : cluster [DBG] osdmap e790: 8 total, 8 up, 8 in 2024-03-26T04:21:35.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:34 smithi190 bash[23392]: audit 2024-03-26T04:21:34.172723+0000 mon.a (mon.0) 3675 : audit [INF] from='client.? 172.21.15.175:0/4270377391' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "0"}]: dispatch 2024-03-26T04:21:35.852 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:35.851+0000 7f91867fc640 1 -- 172.21.15.175:0/4270377391 <== mon.0 v2:172.21.15.175:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 0e4b321e-b83b-43c2-ade1-c76622f394b4 v792) ==== 221+0+0 (secure 0 0 0) 0x7f9188059340 con 0x7f9190109060 2024-03-26T04:21:35.852 INFO:tasks.workunit.client.0.smithi175.stderr:set-quota max_objects = 0 for pool 0e4b321e-b83b-43c2-ade1-c76622f394b4 2024-03-26T04:21:35.856 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:35.855+0000 7f91867fc640 1 -- 172.21.15.175:0/4270377391 <== mon.0 v2:172.21.15.175:3300/0 10 ==== osd_map(792..792 src has 1..792) ==== 628+0+0 (secure 0 0 0) 0x7f918808c7e0 con 0x7f9190109060 2024-03-26T04:21:35.856 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:35.855+0000 7f9197342640 1 -- 172.21.15.175:0/4270377391 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f917c06e760 msgr2=0x7f917c070c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:21:35.856 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:35.855+0000 7f9197342640 1 --2- 172.21.15.175:0/4270377391 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f917c06e760 0x7f917c070c20 secure :-1 s=READY pgs=3790 cs=0 l=1 rev1=1 crypto rx=0x7f918c01d180 tx=0x7f918c0a7040 comp rx=0 tx=0).stop 2024-03-26T04:21:35.856 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:35.855+0000 7f9197342640 1 -- 172.21.15.175:0/4270377391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9190109060 msgr2=0x7f91901a0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:21:35.856 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:35.855+0000 7f9197342640 1 --2- 172.21.15.175:0/4270377391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9190109060 0x7f91901a0c80 secure :-1 s=READY pgs=2939 cs=0 l=1 rev1=1 crypto rx=0x7f918800d750 tx=0x7f918800dc10 comp rx=0 tx=0).stop 2024-03-26T04:21:35.857 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:35.855+0000 7f9197342640 1 -- 172.21.15.175:0/4270377391 shutdown_connections 2024-03-26T04:21:35.857 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:35.855+0000 7f9197342640 1 --2- 172.21.15.175:0/4270377391 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f917c06e760 0x7f917c070c20 unknown :-1 s=CLOSED pgs=3790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:21:35.857 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:35.855+0000 7f9197342640 1 --2- 172.21.15.175:0/4270377391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9190109060 0x7f91901a0c80 unknown :-1 s=CLOSED pgs=2939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:21:35.857 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:35.855+0000 7f9197342640 1 --2- 172.21.15.175:0/4270377391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91901048e0 0x7f919019c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:21:35.857 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:35.855+0000 7f9197342640 1 --2- 172.21.15.175:0/4270377391 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9190103f30 0x7f919019c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:21:35.857 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:35.855+0000 7f9197342640 1 -- 172.21.15.175:0/4270377391 >> 172.21.15.175:0/4270377391 conn(0x7f91900ff780 msgr2=0x7f91900ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T04:21:35.857 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:35.855+0000 7f9197342640 1 -- 172.21.15.175:0/4270377391 shutdown_connections 2024-03-26T04:21:35.857 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:21:35.855+0000 7f9197342640 1 -- 172.21.15.175:0/4270377391 wait complete. 2024-03-26T04:21:35.877 INFO:tasks.workunit.client.0.smithi175.stderr:+ sleep 30 2024-03-26T04:21:36.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:35 smithi175 bash[21527]: cluster 2024-03-26T04:21:33.872223+0000 mgr.y (mgr.24431) 1378 : cluster [DBG] pgmap v1805: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-26T04:21:36.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:35 smithi175 bash[21527]: audit 2024-03-26T04:21:34.845289+0000 mon.a (mon.0) 3676 : audit [INF] from='client.? 172.21.15.175:0/4270377391' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "0"}]': finished 2024-03-26T04:21:36.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:35 smithi175 bash[21527]: cluster 2024-03-26T04:21:34.855339+0000 mon.a (mon.0) 3677 : cluster [DBG] osdmap e791: 8 total, 8 up, 8 in 2024-03-26T04:21:36.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:35 smithi175 bash[21527]: audit 2024-03-26T04:21:34.958912+0000 mon.a (mon.0) 3678 : audit [INF] from='client.? 172.21.15.175:0/4270377391' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "0"}]: dispatch 2024-03-26T04:21:36.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:35 smithi175 bash[29284]: cluster 2024-03-26T04:21:33.872223+0000 mgr.y (mgr.24431) 1378 : cluster [DBG] pgmap v1805: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-26T04:21:36.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:35 smithi175 bash[29284]: audit 2024-03-26T04:21:34.845289+0000 mon.a (mon.0) 3676 : audit [INF] from='client.? 172.21.15.175:0/4270377391' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "0"}]': finished 2024-03-26T04:21:36.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:35 smithi175 bash[29284]: cluster 2024-03-26T04:21:34.855339+0000 mon.a (mon.0) 3677 : cluster [DBG] osdmap e791: 8 total, 8 up, 8 in 2024-03-26T04:21:36.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:35 smithi175 bash[29284]: audit 2024-03-26T04:21:34.958912+0000 mon.a (mon.0) 3678 : audit [INF] from='client.? 172.21.15.175:0/4270377391' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "0"}]: dispatch 2024-03-26T04:21:36.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:35 smithi190 bash[23392]: cluster 2024-03-26T04:21:33.872223+0000 mgr.y (mgr.24431) 1378 : cluster [DBG] pgmap v1805: 188 pgs: 188 active+clean; 863 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-26T04:21:36.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:35 smithi190 bash[23392]: audit 2024-03-26T04:21:34.845289+0000 mon.a (mon.0) 3676 : audit [INF] from='client.? 172.21.15.175:0/4270377391' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "0"}]': finished 2024-03-26T04:21:36.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:35 smithi190 bash[23392]: cluster 2024-03-26T04:21:34.855339+0000 mon.a (mon.0) 3677 : cluster [DBG] osdmap e791: 8 total, 8 up, 8 in 2024-03-26T04:21:36.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:35 smithi190 bash[23392]: audit 2024-03-26T04:21:34.958912+0000 mon.a (mon.0) 3678 : audit [INF] from='client.? 172.21.15.175:0/4270377391' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "0"}]: dispatch 2024-03-26T04:21:37.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:36 smithi175 bash[21527]: audit 2024-03-26T04:21:35.856127+0000 mon.a (mon.0) 3679 : audit [INF] from='client.? 172.21.15.175:0/4270377391' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "0"}]': finished 2024-03-26T04:21:37.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:36 smithi175 bash[21527]: cluster 2024-03-26T04:21:35.861387+0000 mon.a (mon.0) 3680 : cluster [DBG] osdmap e792: 8 total, 8 up, 8 in 2024-03-26T04:21:37.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:36 smithi175 bash[29284]: audit 2024-03-26T04:21:35.856127+0000 mon.a (mon.0) 3679 : audit [INF] from='client.? 172.21.15.175:0/4270377391' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "0"}]': finished 2024-03-26T04:21:37.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:36 smithi175 bash[29284]: cluster 2024-03-26T04:21:35.861387+0000 mon.a (mon.0) 3680 : cluster [DBG] osdmap e792: 8 total, 8 up, 8 in 2024-03-26T04:21:37.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:36 smithi190 bash[23392]: audit 2024-03-26T04:21:35.856127+0000 mon.a (mon.0) 3679 : audit [INF] from='client.? 172.21.15.175:0/4270377391' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "field": "max_objects", "val": "0"}]': finished 2024-03-26T04:21:37.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:36 smithi190 bash[23392]: cluster 2024-03-26T04:21:35.861387+0000 mon.a (mon.0) 3680 : cluster [DBG] osdmap e792: 8 total, 8 up, 8 in 2024-03-26T04:21:38.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:37 smithi175 bash[29284]: cluster 2024-03-26T04:21:35.872825+0000 mgr.y (mgr.24431) 1379 : cluster [DBG] pgmap v1808: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-03-26T04:21:38.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:37 smithi175 bash[21527]: cluster 2024-03-26T04:21:35.872825+0000 mgr.y (mgr.24431) 1379 : cluster [DBG] pgmap v1808: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-03-26T04:21:38.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:37 smithi190 bash[23392]: cluster 2024-03-26T04:21:35.872825+0000 mgr.y (mgr.24431) 1379 : cluster [DBG] pgmap v1808: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-03-26T04:21:39.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:21:38 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:21:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:21:39.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:38 smithi175 bash[29284]: cluster 2024-03-26T04:21:37.873766+0000 mgr.y (mgr.24431) 1380 : cluster [DBG] pgmap v1809: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1011 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-03-26T04:21:39.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:38 smithi175 bash[29284]: audit 2024-03-26T04:21:38.814585+0000 mon.c (mon.2) 892 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:21:39.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:38 smithi175 bash[21527]: cluster 2024-03-26T04:21:37.873766+0000 mgr.y (mgr.24431) 1380 : cluster [DBG] pgmap v1809: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1011 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-03-26T04:21:39.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:38 smithi175 bash[21527]: audit 2024-03-26T04:21:38.814585+0000 mon.c (mon.2) 892 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:21:39.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:38 smithi190 bash[23392]: cluster 2024-03-26T04:21:37.873766+0000 mgr.y (mgr.24431) 1380 : cluster [DBG] pgmap v1809: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1011 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-03-26T04:21:39.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:38 smithi190 bash[23392]: audit 2024-03-26T04:21:38.814585+0000 mon.c (mon.2) 892 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:21:41.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:40 smithi175 bash[21527]: cluster 2024-03-26T04:21:39.875814+0000 mgr.y (mgr.24431) 1381 : cluster [DBG] pgmap v1810: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-03-26T04:21:41.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:40 smithi175 bash[29284]: cluster 2024-03-26T04:21:39.875814+0000 mgr.y (mgr.24431) 1381 : cluster [DBG] pgmap v1810: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-03-26T04:21:41.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:40 smithi190 bash[23392]: cluster 2024-03-26T04:21:39.875814+0000 mgr.y (mgr.24431) 1381 : cluster [DBG] pgmap v1810: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-03-26T04:21:41.890 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:21:41 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:21:42.240 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:41 smithi175 bash[29284]: audit 2024-03-26T04:21:41.584932+0000 mgr.y (mgr.24431) 1382 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:21:42.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:41 smithi175 bash[21527]: audit 2024-03-26T04:21:41.584932+0000 mgr.y (mgr.24431) 1382 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:21:42.249 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:41 smithi190 bash[23392]: audit 2024-03-26T04:21:41.584932+0000 mgr.y (mgr.24431) 1382 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:21:43.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:42 smithi175 bash[21527]: cluster 2024-03-26T04:21:41.876623+0000 mgr.y (mgr.24431) 1383 : cluster [DBG] pgmap v1811: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-03-26T04:21:43.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:42 smithi175 bash[29284]: cluster 2024-03-26T04:21:41.876623+0000 mgr.y (mgr.24431) 1383 : cluster [DBG] pgmap v1811: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-03-26T04:21:43.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:42 smithi190 bash[23392]: cluster 2024-03-26T04:21:41.876623+0000 mgr.y (mgr.24431) 1383 : cluster [DBG] pgmap v1811: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-03-26T04:21:45.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:44 smithi175 bash[21527]: cluster 2024-03-26T04:21:43.877426+0000 mgr.y (mgr.24431) 1384 : cluster [DBG] pgmap v1812: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 567 B/s rd, 0 op/s 2024-03-26T04:21:45.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:44 smithi175 bash[29284]: cluster 2024-03-26T04:21:43.877426+0000 mgr.y (mgr.24431) 1384 : cluster [DBG] pgmap v1812: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 567 B/s rd, 0 op/s 2024-03-26T04:21:45.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:44 smithi190 bash[23392]: cluster 2024-03-26T04:21:43.877426+0000 mgr.y (mgr.24431) 1384 : cluster [DBG] pgmap v1812: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 567 B/s rd, 0 op/s 2024-03-26T04:21:47.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:46 smithi175 bash[21527]: cluster 2024-03-26T04:21:45.879371+0000 mgr.y (mgr.24431) 1385 : cluster [DBG] pgmap v1813: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1022 B/s rd, 0 op/s 2024-03-26T04:21:47.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:46 smithi175 bash[29284]: cluster 2024-03-26T04:21:45.879371+0000 mgr.y (mgr.24431) 1385 : cluster [DBG] pgmap v1813: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1022 B/s rd, 0 op/s 2024-03-26T04:21:47.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:46 smithi190 bash[23392]: cluster 2024-03-26T04:21:45.879371+0000 mgr.y (mgr.24431) 1385 : cluster [DBG] pgmap v1813: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1022 B/s rd, 0 op/s 2024-03-26T04:21:49.240 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:21:48 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:21:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:21:49.241 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:48 smithi175 bash[29284]: cluster 2024-03-26T04:21:47.880214+0000 mgr.y (mgr.24431) 1386 : cluster [DBG] pgmap v1814: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:21:49.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:48 smithi175 bash[21527]: cluster 2024-03-26T04:21:47.880214+0000 mgr.y (mgr.24431) 1386 : cluster [DBG] pgmap v1814: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:21:49.247 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:48 smithi190 bash[23392]: cluster 2024-03-26T04:21:47.880214+0000 mgr.y (mgr.24431) 1386 : cluster [DBG] pgmap v1814: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:21:51.250 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:50 smithi190 bash[23392]: cluster 2024-03-26T04:21:49.882121+0000 mgr.y (mgr.24431) 1387 : cluster [DBG] pgmap v1815: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:21:51.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:50 smithi175 bash[21527]: cluster 2024-03-26T04:21:49.882121+0000 mgr.y (mgr.24431) 1387 : cluster [DBG] pgmap v1815: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:21:51.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:50 smithi175 bash[29284]: cluster 2024-03-26T04:21:49.882121+0000 mgr.y (mgr.24431) 1387 : cluster [DBG] pgmap v1815: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:21:51.997 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:21:51 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:21:52.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:52 smithi175 bash[21527]: audit 2024-03-26T04:21:51.592888+0000 mgr.y (mgr.24431) 1388 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:21:52.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:52 smithi175 bash[29284]: audit 2024-03-26T04:21:51.592888+0000 mgr.y (mgr.24431) 1388 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:21:52.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:52 smithi190 bash[23392]: audit 2024-03-26T04:21:51.592888+0000 mgr.y (mgr.24431) 1388 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:21:53.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:53 smithi175 bash[29284]: cluster 2024-03-26T04:21:51.882884+0000 mgr.y (mgr.24431) 1389 : cluster [DBG] pgmap v1816: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:21:53.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:53 smithi175 bash[21527]: cluster 2024-03-26T04:21:51.882884+0000 mgr.y (mgr.24431) 1389 : cluster [DBG] pgmap v1816: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:21:53.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:53 smithi190 bash[23392]: cluster 2024-03-26T04:21:51.882884+0000 mgr.y (mgr.24431) 1389 : cluster [DBG] pgmap v1816: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:21:54.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:54 smithi175 bash[29284]: audit 2024-03-26T04:21:53.849102+0000 mon.c (mon.2) 893 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:21:54.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:54 smithi175 bash[21527]: audit 2024-03-26T04:21:53.849102+0000 mon.c (mon.2) 893 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:21:54.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:54 smithi190 bash[23392]: audit 2024-03-26T04:21:53.849102+0000 mon.c (mon.2) 893 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-26T04:21:55.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:55 smithi175 bash[29284]: cluster 2024-03-26T04:21:53.883788+0000 mgr.y (mgr.24431) 1390 : cluster [DBG] pgmap v1817: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:21:55.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:55 smithi175 bash[21527]: cluster 2024-03-26T04:21:53.883788+0000 mgr.y (mgr.24431) 1390 : cluster [DBG] pgmap v1817: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:21:55.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:55 smithi190 bash[23392]: cluster 2024-03-26T04:21:53.883788+0000 mgr.y (mgr.24431) 1390 : cluster [DBG] pgmap v1817: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:21:57.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:57 smithi175 bash[29284]: cluster 2024-03-26T04:21:55.885807+0000 mgr.y (mgr.24431) 1391 : cluster [DBG] pgmap v1818: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:21:57.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:57 smithi175 bash[21527]: cluster 2024-03-26T04:21:55.885807+0000 mgr.y (mgr.24431) 1391 : cluster [DBG] pgmap v1818: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:21:57.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:57 smithi190 bash[23392]: cluster 2024-03-26T04:21:55.885807+0000 mgr.y (mgr.24431) 1391 : cluster [DBG] pgmap v1818: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:21:59.241 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:21:58 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:21:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:21:59.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:59 smithi175 bash[21527]: cluster 2024-03-26T04:21:57.886597+0000 mgr.y (mgr.24431) 1392 : cluster [DBG] pgmap v1819: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:21:59.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:59 smithi175 bash[21527]: audit 2024-03-26T04:21:58.288180+0000 mon.c (mon.2) 894 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:21:59.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:59 smithi175 bash[21527]: audit 2024-03-26T04:21:59.018014+0000 mon.c (mon.2) 895 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:21:59.241 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:21:59 smithi175 bash[21527]: audit 2024-03-26T04:21:59.019879+0000 mon.c (mon.2) 896 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:21:59.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:59 smithi175 bash[29284]: cluster 2024-03-26T04:21:57.886597+0000 mgr.y (mgr.24431) 1392 : cluster [DBG] pgmap v1819: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:21:59.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:59 smithi175 bash[29284]: audit 2024-03-26T04:21:58.288180+0000 mon.c (mon.2) 894 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:21:59.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:59 smithi175 bash[29284]: audit 2024-03-26T04:21:59.018014+0000 mon.c (mon.2) 895 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:21:59.242 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:21:59 smithi175 bash[29284]: audit 2024-03-26T04:21:59.019879+0000 mon.c (mon.2) 896 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:21:59.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:59 smithi190 bash[23392]: cluster 2024-03-26T04:21:57.886597+0000 mgr.y (mgr.24431) 1392 : cluster [DBG] pgmap v1819: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:21:59.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:59 smithi190 bash[23392]: audit 2024-03-26T04:21:58.288180+0000 mon.c (mon.2) 894 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T04:21:59.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:59 smithi190 bash[23392]: audit 2024-03-26T04:21:59.018014+0000 mon.c (mon.2) 895 : audit [DBG] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T04:21:59.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:21:59 smithi190 bash[23392]: audit 2024-03-26T04:21:59.019879+0000 mon.c (mon.2) 896 : audit [INF] from='mgr.24431 172.21.15.175:0/1025540336' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T04:22:00.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:22:00 smithi175 bash[21527]: audit 2024-03-26T04:21:59.036443+0000 mon.a (mon.0) 3681 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:22:00.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:22:00 smithi175 bash[29284]: audit 2024-03-26T04:21:59.036443+0000 mon.a (mon.0) 3681 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:22:00.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:22:00 smithi190 bash[23392]: audit 2024-03-26T04:21:59.036443+0000 mon.a (mon.0) 3681 : audit [INF] from='mgr.24431 ' entity='mgr.y' 2024-03-26T04:22:01.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:22:01 smithi175 bash[21527]: cluster 2024-03-26T04:21:59.888569+0000 mgr.y (mgr.24431) 1393 : cluster [DBG] pgmap v1820: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:22:01.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:22:01 smithi175 bash[29284]: cluster 2024-03-26T04:21:59.888569+0000 mgr.y (mgr.24431) 1393 : cluster [DBG] pgmap v1820: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:22:01.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:22:01 smithi190 bash[23392]: cluster 2024-03-26T04:21:59.888569+0000 mgr.y (mgr.24431) 1393 : cluster [DBG] pgmap v1820: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:22:01.997 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:22:01 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:22:02.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:22:02 smithi175 bash[21527]: audit 2024-03-26T04:22:01.603076+0000 mgr.y (mgr.24431) 1394 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:22:02.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:22:02 smithi175 bash[29284]: audit 2024-03-26T04:22:01.603076+0000 mgr.y (mgr.24431) 1394 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:22:02.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:22:02 smithi190 bash[23392]: audit 2024-03-26T04:22:01.603076+0000 mgr.y (mgr.24431) 1394 : audit [DBG] from='client.24391 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-26T04:22:03.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:22:03 smithi175 bash[29284]: cluster 2024-03-26T04:22:01.889378+0000 mgr.y (mgr.24431) 1395 : cluster [DBG] pgmap v1821: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:22:03.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:22:03 smithi175 bash[21527]: cluster 2024-03-26T04:22:01.889378+0000 mgr.y (mgr.24431) 1395 : cluster [DBG] pgmap v1821: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:22:03.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:22:03 smithi190 bash[23392]: cluster 2024-03-26T04:22:01.889378+0000 mgr.y (mgr.24431) 1395 : cluster [DBG] pgmap v1821: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:22:05.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:22:05 smithi175 bash[29284]: cluster 2024-03-26T04:22:03.890163+0000 mgr.y (mgr.24431) 1396 : cluster [DBG] pgmap v1822: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:22:05.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:22:05 smithi175 bash[21527]: cluster 2024-03-26T04:22:03.890163+0000 mgr.y (mgr.24431) 1396 : cluster [DBG] pgmap v1822: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:22:05.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:22:05 smithi190 bash[23392]: cluster 2024-03-26T04:22:03.890163+0000 mgr.y (mgr.24431) 1396 : cluster [DBG] pgmap v1822: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-26T04:22:05.878 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph osd pool delete 0e4b321e-b83b-43c2-ade1-c76622f394b4 0e4b321e-b83b-43c2-ade1-c76622f394b4 --yes-i-really-really-mean-it 2024-03-26T04:22:06.012 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.010+0000 7fd945291640 1 -- 172.21.15.175:0/2286967678 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9401024c0 msgr2=0x7fd94010e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:22:06.012 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.010+0000 7fd945291640 1 --2- 172.21.15.175:0/2286967678 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9401024c0 0x7fd94010e9d0 secure :-1 s=READY pgs=3062 cs=0 l=1 rev1=1 crypto rx=0x7fd938009ce0 tx=0x7fd93801cc80 comp rx=0 tx=0).stop 2024-03-26T04:22:06.012 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.010+0000 7fd945291640 1 -- 172.21.15.175:0/2286967678 shutdown_connections 2024-03-26T04:22:06.012 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.010+0000 7fd945291640 1 --2- 172.21.15.175:0/2286967678 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9401024c0 0x7fd94010e9d0 unknown :-1 s=CLOSED pgs=3062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:22:06.012 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.010+0000 7fd945291640 1 --2- 172.21.15.175:0/2286967678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd940101b00 0x7fd940101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:22:06.012 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.010+0000 7fd945291640 1 --2- 172.21.15.175:0/2286967678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd940107b90 0x7fd940107f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:22:06.012 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.010+0000 7fd945291640 1 -- 172.21.15.175:0/2286967678 >> 172.21.15.175:0/2286967678 conn(0x7fd9400fd610 msgr2=0x7fd9400ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T04:22:06.013 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.010+0000 7fd945291640 1 -- 172.21.15.175:0/2286967678 shutdown_connections 2024-03-26T04:22:06.013 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.010+0000 7fd945291640 1 -- 172.21.15.175:0/2286967678 wait complete. 2024-03-26T04:22:06.013 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.010+0000 7fd945291640 1 Processor -- start 2024-03-26T04:22:06.013 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.010+0000 7fd945291640 1 -- start start 2024-03-26T04:22:06.013 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.014+0000 7fd945291640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd940101b00 0x7fd940073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:22:06.013 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.014+0000 7fd945291640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9401024c0 0x7fd940074440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:22:06.013 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.014+0000 7fd945291640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd940107b90 0x7fd940074980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:22:06.013 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.014+0000 7fd945291640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd940113b30 con 0x7fd940107b90 2024-03-26T04:22:06.013 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.014+0000 7fd945291640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd9401139b0 con 0x7fd940101b00 2024-03-26T04:22:06.013 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.014+0000 7fd945291640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd940113cb0 con 0x7fd9401024c0 2024-03-26T04:22:06.013 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.014+0000 7fd93f7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd940107b90 0x7fd940074980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:22:06.013 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.014+0000 7fd9367fc640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9401024c0 0x7fd940074440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:22:06.013 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.014+0000 7fd93f7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd940107b90 0x7fd940074980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:48904/0 (socket says 172.21.15.175:48904) 2024-03-26T04:22:06.013 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.014+0000 7fd93f7fe640 1 -- 172.21.15.175:0/1875093517 learned_addr learned my addr 172.21.15.175:0/1875093517 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T04:22:06.014 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.014+0000 7fd93effd640 1 --2- 172.21.15.175:0/1875093517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd940101b00 0x7fd940073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:22:06.014 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.014+0000 7fd93f7fe640 1 -- 172.21.15.175:0/1875093517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9401024c0 msgr2=0x7fd940074440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:22:06.014 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.014+0000 7fd93f7fe640 1 --2- 172.21.15.175:0/1875093517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9401024c0 0x7fd940074440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:22:06.014 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.014+0000 7fd93f7fe640 1 -- 172.21.15.175:0/1875093517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd940101b00 msgr2=0x7fd940073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:22:06.014 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.014+0000 7fd93f7fe640 1 --2- 172.21.15.175:0/1875093517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd940101b00 0x7fd940073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:22:06.014 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.014+0000 7fd93f7fe640 1 -- 172.21.15.175:0/1875093517 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9401a8580 con 0x7fd940107b90 2024-03-26T04:22:06.014 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.014+0000 7fd93effd640 1 --2- 172.21.15.175:0/1875093517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd940101b00 0x7fd940073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T04:22:06.014 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.014+0000 7fd93f7fe640 1 --2- 172.21.15.175:0/1875093517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd940107b90 0x7fd940074980 secure :-1 s=READY pgs=2940 cs=0 l=1 rev1=1 crypto rx=0x7fd9380987b0 tx=0x7fd938004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T04:22:06.014 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.014+0000 7fd93cff9640 1 -- 172.21.15.175:0/1875093517 <== mon.0 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9380b2040 con 0x7fd940107b90 2024-03-26T04:22:06.014 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.014+0000 7fd945291640 1 -- 172.21.15.175:0/1875093517 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9401a8810 con 0x7fd940107b90 2024-03-26T04:22:06.014 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.014+0000 7fd93cff9640 1 -- 172.21.15.175:0/1875093517 <== mon.0 v2:172.21.15.175:3300/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7fd938004850 con 0x7fd940107b90 2024-03-26T04:22:06.014 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.014+0000 7fd93cff9640 1 -- 172.21.15.175:0/1875093517 <== mon.0 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9380adab0 con 0x7fd940107b90 2024-03-26T04:22:06.014 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.014+0000 7fd945291640 1 -- 172.21.15.175:0/1875093517 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9401a8d40 con 0x7fd940107b90 2024-03-26T04:22:06.015 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.014+0000 7fd945291640 1 -- 172.21.15.175:0/1875093517 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd940068e90 con 0x7fd940107b90 2024-03-26T04:22:06.016 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.014+0000 7fd93cff9640 1 -- 172.21.15.175:0/1875093517 <== mon.0 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7fd938004a50 con 0x7fd940107b90 2024-03-26T04:22:06.016 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.014+0000 7fd93cff9640 1 --2- 172.21.15.175:0/1875093517 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fd92c06e760 0x7fd92c070c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:22:06.016 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.014+0000 7fd93cff9640 1 -- 172.21.15.175:0/1875093517 <== mon.0 v2:172.21.15.175:3300/0 5 ==== osd_map(792..792 src has 1..792) ==== 9468+0+0 (secure 0 0 0) 0x7fd93812a5b0 con 0x7fd940107b90 2024-03-26T04:22:06.016 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.014+0000 7fd93effd640 1 --2- 172.21.15.175:0/1875093517 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fd92c06e760 0x7fd92c070c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:22:06.016 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.014+0000 7fd93cff9640 1 -- 172.21.15.175:0/1875093517 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=793}) -- 0x7fd92c07aa70 con 0x7fd940107b90 2024-03-26T04:22:06.016 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.014+0000 7fd93effd640 1 --2- 172.21.15.175:0/1875093517 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fd92c06e760 0x7fd92c070c20 secure :-1 s=READY pgs=3791 cs=0 l=1 rev1=1 crypto rx=0x7fd930009ae0 tx=0x7fd9300091e0 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T04:22:06.018 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.018+0000 7fd93cff9640 1 -- 172.21.15.175:0/1875093517 <== mon.0 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7fd938016990 con 0x7fd940107b90 2024-03-26T04:22:06.190 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:06.190+0000 7fd945291640 1 -- 172.21.15.175:0/1875093517 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "pool2": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd940075430 con 0x7fd940107b90 2024-03-26T04:22:07.069 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.070+0000 7fd93cff9640 1 -- 172.21.15.175:0/1875093517 <== mon.0 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "pool2": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "yes_i_really_really_mean_it": true}]=0 pool '0e4b321e-b83b-43c2-ade1-c76622f394b4' removed v793) ==== 248+0+0 (secure 0 0 0) 0x7fd940075430 con 0x7fd940107b90 2024-03-26T04:22:07.071 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.074+0000 7fd93cff9640 1 -- 172.21.15.175:0/1875093517 <== mon.0 v2:172.21.15.175:3300/0 8 ==== osd_map(793..793 src has 1..793) ==== 296+0+0 (secure 0 0 0) 0x7fd9380f47f0 con 0x7fd940107b90 2024-03-26T04:22:07.071 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.074+0000 7fd93cff9640 1 -- 172.21.15.175:0/1875093517 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=794}) -- 0x7fd92c07b7c0 con 0x7fd940107b90 2024-03-26T04:22:07.181 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.182+0000 7fd945291640 1 -- 172.21.15.175:0/1875093517 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "pool2": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd940062210 con 0x7fd940107b90 2024-03-26T04:22:07.182 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.182+0000 7fd93cff9640 1 -- 172.21.15.175:0/1875093517 <== mon.0 v2:172.21.15.175:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "pool2": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "yes_i_really_really_mean_it": true}]=0 pool '0e4b321e-b83b-43c2-ade1-c76622f394b4' does not exist v793) ==== 255+0+0 (secure 0 0 0) 0x7fd9380f6e50 con 0x7fd940107b90 2024-03-26T04:22:07.182 INFO:tasks.workunit.client.0.smithi175.stderr:pool '0e4b321e-b83b-43c2-ade1-c76622f394b4' does not exist 2024-03-26T04:22:07.184 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.186+0000 7fd945291640 1 -- 172.21.15.175:0/1875093517 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fd92c06e760 msgr2=0x7fd92c070c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:22:07.184 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.186+0000 7fd945291640 1 --2- 172.21.15.175:0/1875093517 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fd92c06e760 0x7fd92c070c20 secure :-1 s=READY pgs=3791 cs=0 l=1 rev1=1 crypto rx=0x7fd930009ae0 tx=0x7fd9300091e0 comp rx=0 tx=0).stop 2024-03-26T04:22:07.184 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.186+0000 7fd945291640 1 -- 172.21.15.175:0/1875093517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd940107b90 msgr2=0x7fd940074980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:22:07.184 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.186+0000 7fd945291640 1 --2- 172.21.15.175:0/1875093517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd940107b90 0x7fd940074980 secure :-1 s=READY pgs=2940 cs=0 l=1 rev1=1 crypto rx=0x7fd9380987b0 tx=0x7fd938004660 comp rx=0 tx=0).stop 2024-03-26T04:22:07.184 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.186+0000 7fd945291640 1 -- 172.21.15.175:0/1875093517 shutdown_connections 2024-03-26T04:22:07.184 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.186+0000 7fd945291640 1 --2- 172.21.15.175:0/1875093517 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7fd92c06e760 0x7fd92c070c20 unknown :-1 s=CLOSED pgs=3791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:22:07.184 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.186+0000 7fd945291640 1 --2- 172.21.15.175:0/1875093517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9401024c0 0x7fd940074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:22:07.184 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.186+0000 7fd945291640 1 --2- 172.21.15.175:0/1875093517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd940107b90 0x7fd940074980 unknown :-1 s=CLOSED pgs=2940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:22:07.184 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.186+0000 7fd945291640 1 --2- 172.21.15.175:0/1875093517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd940101b00 0x7fd940073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:22:07.185 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.186+0000 7fd945291640 1 -- 172.21.15.175:0/1875093517 >> 172.21.15.175:0/1875093517 conn(0x7fd9400fd610 msgr2=0x7fd9401056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T04:22:07.185 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.186+0000 7fd945291640 1 -- 172.21.15.175:0/1875093517 shutdown_connections 2024-03-26T04:22:07.185 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.186+0000 7fd945291640 1 -- 172.21.15.175:0/1875093517 wait complete. 2024-03-26T04:22:07.199 INFO:tasks.workunit.client.0.smithi175.stderr:+ ceph osd pool delete 721fce6c-d7d8-472d-8941-f0e24365dbcf 721fce6c-d7d8-472d-8941-f0e24365dbcf --yes-i-really-really-mean-it 2024-03-26T04:22:07.311 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.310+0000 7f76c5f2b640 1 -- 172.21.15.175:0/1621223354 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76c0107c80 msgr2=0x7f76c0108060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:22:07.311 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.310+0000 7f76c5f2b640 1 --2- 172.21.15.175:0/1621223354 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76c0107c80 0x7f76c0108060 secure :-1 s=READY pgs=3063 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009ce0 tx=0x7f76b801cc80 comp rx=0 tx=0).stop 2024-03-26T04:22:07.311 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.310+0000 7f76c5f2b640 1 -- 172.21.15.175:0/1621223354 shutdown_connections 2024-03-26T04:22:07.311 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.310+0000 7f76c5f2b640 1 --2- 172.21.15.175:0/1621223354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76c006ac40 0x7f76c010e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:22:07.312 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.310+0000 7f76c5f2b640 1 --2- 172.21.15.175:0/1621223354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76c006a2a0 0x7f76c006a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:22:07.312 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.310+0000 7f76c5f2b640 1 --2- 172.21.15.175:0/1621223354 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76c0107c80 0x7f76c0108060 unknown :-1 s=CLOSED pgs=3063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:22:07.312 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.310+0000 7f76c5f2b640 1 -- 172.21.15.175:0/1621223354 >> 172.21.15.175:0/1621223354 conn(0x7f76c0077860 msgr2=0x7f76c0077c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T04:22:07.312 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.310+0000 7f76c5f2b640 1 -- 172.21.15.175:0/1621223354 shutdown_connections 2024-03-26T04:22:07.312 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.310+0000 7f76c5f2b640 1 -- 172.21.15.175:0/1621223354 wait complete. 2024-03-26T04:22:07.312 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.310+0000 7f76c5f2b640 1 Processor -- start 2024-03-26T04:22:07.312 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.310+0000 7f76c5f2b640 1 -- start start 2024-03-26T04:22:07.312 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.310+0000 7f76c5f2b640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76c006a2a0 0x7f76c0100db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:22:07.312 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.310+0000 7f76c5f2b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76c006ac40 0x7f76c01012f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:22:07.312 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.310+0000 7f76c5f2b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76c0107c80 0x7f76c0101830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:22:07.312 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.310+0000 7f76c5f2b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f76c0113b00 con 0x7f76c0107c80 2024-03-26T04:22:07.312 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.310+0000 7f76c5f2b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f76c0113980 con 0x7f76c006ac40 2024-03-26T04:22:07.313 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.310+0000 7f76c5f2b640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f76c0113c80 con 0x7f76c006a2a0 2024-03-26T04:22:07.313 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.310+0000 7f76bf7fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76c006a2a0 0x7f76c0100db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:22:07.313 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.310+0000 7f76bffff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76c0107c80 0x7f76c0101830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:22:07.313 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.310+0000 7f76bf7fe640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76c006a2a0 0x7f76c0100db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:57484/0 (socket says 172.21.15.175:57484) 2024-03-26T04:22:07.313 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.310+0000 7f76bf7fe640 1 -- 172.21.15.175:0/3135423106 learned_addr learned my addr 172.21.15.175:0/3135423106 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-03-26T04:22:07.313 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.310+0000 7f76bf7fe640 1 -- 172.21.15.175:0/3135423106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76c006ac40 msgr2=0x7f76c01012f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:22:07.313 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.310+0000 7f76bf7fe640 1 --2- 172.21.15.175:0/3135423106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76c006ac40 0x7f76c01012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:22:07.313 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.310+0000 7f76bf7fe640 1 -- 172.21.15.175:0/3135423106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76c0107c80 msgr2=0x7f76c0101830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:22:07.313 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.310+0000 7f76bf7fe640 1 --2- 172.21.15.175:0/3135423106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76c0107c80 0x7f76c0101830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:22:07.313 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.310+0000 7f76bf7fe640 1 -- 172.21.15.175:0/3135423106 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c01a85b0 con 0x7f76c006a2a0 2024-03-26T04:22:07.313 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.310+0000 7f76bf7fe640 1 --2- 172.21.15.175:0/3135423106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76c006a2a0 0x7f76c0100db0 secure :-1 s=READY pgs=3064 cs=0 l=1 rev1=1 crypto rx=0x7f76b801d160 tx=0x7f76b8004570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T04:22:07.313 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.314+0000 7f76bcff9640 1 -- 172.21.15.175:0/3135423106 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b80b2040 con 0x7f76c006a2a0 2024-03-26T04:22:07.313 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.314+0000 7f76bcff9640 1 -- 172.21.15.175:0/3135423106 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(24 keys) ==== 1042+0+0 (secure 0 0 0) 0x7f76b8004760 con 0x7f76c006a2a0 2024-03-26T04:22:07.313 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.314+0000 7f76c5f2b640 1 -- 172.21.15.175:0/3135423106 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c01a8840 con 0x7f76c006a2a0 2024-03-26T04:22:07.313 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.314+0000 7f76bcff9640 1 -- 172.21.15.175:0/3135423106 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b80adb30 con 0x7f76c006a2a0 2024-03-26T04:22:07.314 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.314+0000 7f76c5f2b640 1 -- 172.21.15.175:0/3135423106 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c01a8cd0 con 0x7f76c006a2a0 2024-03-26T04:22:07.314 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.314+0000 7f76c5f2b640 1 -- 172.21.15.175:0/3135423106 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76c0068e90 con 0x7f76c006a2a0 2024-03-26T04:22:07.315 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.314+0000 7f76bcff9640 1 -- 172.21.15.175:0/3135423106 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 21) ==== 92533+0+0 (secure 0 0 0) 0x7f76b8004a50 con 0x7f76c006a2a0 2024-03-26T04:22:07.315 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.314+0000 7f76bcff9640 1 --2- 172.21.15.175:0/3135423106 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f76ac06e760 0x7f76ac070c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T04:22:07.315 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.314+0000 7f76bcff9640 1 -- 172.21.15.175:0/3135423106 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(793..793 src has 1..793) ==== 9080+0+0 (secure 0 0 0) 0x7f76b812a050 con 0x7f76c006a2a0 2024-03-26T04:22:07.315 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.314+0000 7f76beffd640 1 --2- 172.21.15.175:0/3135423106 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f76ac06e760 0x7f76ac070c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T04:22:07.315 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.314+0000 7f76bcff9640 1 -- 172.21.15.175:0/3135423106 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=794}) -- 0x7f76ac07a250 con 0x7f76c006a2a0 2024-03-26T04:22:07.315 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.314+0000 7f76beffd640 1 --2- 172.21.15.175:0/3135423106 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f76ac06e760 0x7f76ac070c20 secure :-1 s=READY pgs=3792 cs=0 l=1 rev1=1 crypto rx=0x7f76c0102840 tx=0x7f76a8005c00 comp rx=0 tx=0).ready entity=mgr.24431 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T04:22:07.317 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.318+0000 7f76bcff9640 1 -- 172.21.15.175:0/3135423106 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187543 (secure 0 0 0) 0x7f76b8016990 con 0x7f76c006a2a0 2024-03-26T04:22:07.479 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:07.482+0000 7f76c5f2b640 1 -- 172.21.15.175:0/3135423106 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "pool2": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "yes_i_really_really_mean_it": true} v 0) -- 0x7f76c01a8f30 con 0x7f76c006a2a0 2024-03-26T04:22:07.490 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:22:07 smithi175 bash[21527]: cluster 2024-03-26T04:22:05.891812+0000 mgr.y (mgr.24431) 1397 : cluster [DBG] pgmap v1823: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:22:07.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:22:07 smithi175 bash[21527]: audit 2024-03-26T04:22:06.195419+0000 mon.a (mon.0) 3682 : audit [INF] from='client.? 172.21.15.175:0/1875093517' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "pool2": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T04:22:07.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:22:07 smithi175 bash[29284]: cluster 2024-03-26T04:22:05.891812+0000 mgr.y (mgr.24431) 1397 : cluster [DBG] pgmap v1823: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:22:07.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:22:07 smithi175 bash[29284]: audit 2024-03-26T04:22:06.195419+0000 mon.a (mon.0) 3682 : audit [INF] from='client.? 172.21.15.175:0/1875093517' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "pool2": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T04:22:07.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:22:07 smithi190 bash[23392]: cluster 2024-03-26T04:22:05.891812+0000 mgr.y (mgr.24431) 1397 : cluster [DBG] pgmap v1823: 188 pgs: 188 active+clean; 875 KiB data, 901 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-26T04:22:07.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:22:07 smithi190 bash[23392]: audit 2024-03-26T04:22:06.195419+0000 mon.a (mon.0) 3682 : audit [INF] from='client.? 172.21.15.175:0/1875093517' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "pool2": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T04:22:08.080 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:08.082+0000 7f76bcff9640 1 -- 172.21.15.175:0/3135423106 <== mon.2 v2:172.21.15.175:3301/0 7 ==== osd_map(794..794 src has 1..794) ==== 296+0+0 (secure 0 0 0) 0x7f76b80ad440 con 0x7f76c006a2a0 2024-03-26T04:22:08.080 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:08.082+0000 7f76bcff9640 1 -- 172.21.15.175:0/3135423106 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=795}) -- 0x7f76ac07ab10 con 0x7f76c006a2a0 2024-03-26T04:22:08.089 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:08.090+0000 7f76bcff9640 1 -- 172.21.15.175:0/3135423106 <== mon.2 v2:172.21.15.175:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "pool2": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "yes_i_really_really_mean_it": true}]=0 pool '721fce6c-d7d8-472d-8941-f0e24365dbcf' removed v794) ==== 248+0+0 (secure 0 0 0) 0x7f76b80f6a70 con 0x7f76c006a2a0 2024-03-26T04:22:08.199 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:08.202+0000 7f76c5f2b640 1 -- 172.21.15.175:0/3135423106 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "pool2": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "yes_i_really_really_mean_it": true} v 0) -- 0x7f76c0062210 con 0x7f76c006a2a0 2024-03-26T04:22:08.201 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:08.202+0000 7f76bcff9640 1 -- 172.21.15.175:0/3135423106 <== mon.2 v2:172.21.15.175:3301/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "pool2": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "yes_i_really_really_mean_it": true}]=0 pool '721fce6c-d7d8-472d-8941-f0e24365dbcf' does not exist v794) ==== 255+0+0 (secure 0 0 0) 0x7f76b810f030 con 0x7f76c006a2a0 2024-03-26T04:22:08.201 INFO:tasks.workunit.client.0.smithi175.stderr:pool '721fce6c-d7d8-472d-8941-f0e24365dbcf' does not exist 2024-03-26T04:22:08.204 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:08.206+0000 7f76c5f2b640 1 -- 172.21.15.175:0/3135423106 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f76ac06e760 msgr2=0x7f76ac070c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:22:08.204 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:08.206+0000 7f76c5f2b640 1 --2- 172.21.15.175:0/3135423106 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f76ac06e760 0x7f76ac070c20 secure :-1 s=READY pgs=3792 cs=0 l=1 rev1=1 crypto rx=0x7f76c0102840 tx=0x7f76a8005c00 comp rx=0 tx=0).stop 2024-03-26T04:22:08.204 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:08.206+0000 7f76c5f2b640 1 -- 172.21.15.175:0/3135423106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76c006a2a0 msgr2=0x7f76c0100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T04:22:08.204 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:08.206+0000 7f76c5f2b640 1 --2- 172.21.15.175:0/3135423106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76c006a2a0 0x7f76c0100db0 secure :-1 s=READY pgs=3064 cs=0 l=1 rev1=1 crypto rx=0x7f76b801d160 tx=0x7f76b8004570 comp rx=0 tx=0).stop 2024-03-26T04:22:08.204 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:08.206+0000 7f76c5f2b640 1 -- 172.21.15.175:0/3135423106 shutdown_connections 2024-03-26T04:22:08.204 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:08.206+0000 7f76c5f2b640 1 --2- 172.21.15.175:0/3135423106 >> [v2:172.21.15.175:6800/1140903233,v1:172.21.15.175:6801/1140903233] conn(0x7f76ac06e760 0x7f76ac070c20 unknown :-1 s=CLOSED pgs=3792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:22:08.204 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:08.206+0000 7f76c5f2b640 1 --2- 172.21.15.175:0/3135423106 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76c0107c80 0x7f76c0101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:22:08.204 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:08.206+0000 7f76c5f2b640 1 --2- 172.21.15.175:0/3135423106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76c006ac40 0x7f76c01012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:22:08.204 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:08.206+0000 7f76c5f2b640 1 --2- 172.21.15.175:0/3135423106 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76c006a2a0 0x7f76c0100db0 unknown :-1 s=CLOSED pgs=3064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T04:22:08.204 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:08.206+0000 7f76c5f2b640 1 -- 172.21.15.175:0/3135423106 >> 172.21.15.175:0/3135423106 conn(0x7f76c0077860 msgr2=0x7f76c0105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T04:22:08.204 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:08.206+0000 7f76c5f2b640 1 -- 172.21.15.175:0/3135423106 shutdown_connections 2024-03-26T04:22:08.204 INFO:tasks.workunit.client.0.smithi175.stderr:2024-03-26T04:22:08.206+0000 7f76c5f2b640 1 -- 172.21.15.175:0/3135423106 wait complete. 2024-03-26T04:22:08.219 INFO:tasks.workunit.client.0.smithi175.stdout:OK 2024-03-26T04:22:08.220 INFO:tasks.workunit.client.0.smithi175.stderr:+ echo OK 2024-03-26T04:22:08.220 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-26T04:22:08.220 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-26T04:22:08.230 INFO:tasks.workunit:Stopping ['rados/test.sh', 'rados/test_pool_quota.sh'] on client.0... 2024-03-26T04:22:08.231 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-26T04:22:08.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:22:08 smithi175 bash[21527]: audit 2024-03-26T04:22:07.072964+0000 mon.a (mon.0) 3683 : audit [INF] from='client.? 172.21.15.175:0/1875093517' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "pool2": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T04:22:08.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:22:08 smithi175 bash[21527]: cluster 2024-03-26T04:22:07.080535+0000 mon.a (mon.0) 3684 : cluster [DBG] osdmap e793: 8 total, 8 up, 8 in 2024-03-26T04:22:08.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:22:08 smithi175 bash[21527]: audit 2024-03-26T04:22:07.185871+0000 mon.a (mon.0) 3685 : audit [INF] from='client.? 172.21.15.175:0/1875093517' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "pool2": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T04:22:08.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:22:08 smithi175 bash[21527]: audit 2024-03-26T04:22:07.484670+0000 mon.c (mon.2) 897 : audit [INF] from='client.? 172.21.15.175:0/3135423106' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "pool2": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T04:22:08.491 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:22:08 smithi175 bash[21527]: audit 2024-03-26T04:22:07.485420+0000 mon.a (mon.0) 3686 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "pool2": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T04:22:08.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:22:08 smithi175 bash[29284]: audit 2024-03-26T04:22:07.072964+0000 mon.a (mon.0) 3683 : audit [INF] from='client.? 172.21.15.175:0/1875093517' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "pool2": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T04:22:08.491 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:22:08 smithi175 bash[29284]: cluster 2024-03-26T04:22:07.080535+0000 mon.a (mon.0) 3684 : cluster [DBG] osdmap e793: 8 total, 8 up, 8 in 2024-03-26T04:22:08.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:22:08 smithi175 bash[29284]: audit 2024-03-26T04:22:07.185871+0000 mon.a (mon.0) 3685 : audit [INF] from='client.? 172.21.15.175:0/1875093517' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "pool2": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T04:22:08.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:22:08 smithi175 bash[29284]: audit 2024-03-26T04:22:07.484670+0000 mon.c (mon.2) 897 : audit [INF] from='client.? 172.21.15.175:0/3135423106' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "pool2": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T04:22:08.492 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:22:08 smithi175 bash[29284]: audit 2024-03-26T04:22:07.485420+0000 mon.a (mon.0) 3686 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "pool2": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T04:22:08.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:22:08 smithi190 bash[23392]: audit 2024-03-26T04:22:07.072964+0000 mon.a (mon.0) 3683 : audit [INF] from='client.? 172.21.15.175:0/1875093517' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "pool2": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T04:22:08.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:22:08 smithi190 bash[23392]: cluster 2024-03-26T04:22:07.080535+0000 mon.a (mon.0) 3684 : cluster [DBG] osdmap e793: 8 total, 8 up, 8 in 2024-03-26T04:22:08.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:22:08 smithi190 bash[23392]: audit 2024-03-26T04:22:07.185871+0000 mon.a (mon.0) 3685 : audit [INF] from='client.? 172.21.15.175:0/1875093517' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "pool2": "0e4b321e-b83b-43c2-ade1-c76622f394b4", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T04:22:08.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:22:08 smithi190 bash[23392]: audit 2024-03-26T04:22:07.484670+0000 mon.c (mon.2) 897 : audit [INF] from='client.? 172.21.15.175:0/3135423106' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "pool2": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T04:22:08.497 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:22:08 smithi190 bash[23392]: audit 2024-03-26T04:22:07.485420+0000 mon.a (mon.0) 3686 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "pool2": "721fce6c-d7d8-472d-8941-f0e24365dbcf", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T04:22:08.552 DEBUG:teuthology.parallel:result is None 2024-03-26T04:22:08.552 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-26T04:22:08.564 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-26T04:22:08.564 DEBUG:teuthology.orchestra.run.smithi175:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-03-26T04:22:08.619 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-03-26T04:22:08.619 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-26T04:22:08.628 INFO:tasks.cephadm:Teardown begin 2024-03-26T04:22:08.629 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T04:22:08.674 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T04:22:08.689 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-26T04:22:08.689 DEBUG:teuthology.orchestra.run.smithi175:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-26T04:22:08.723 DEBUG:teuthology.orchestra.run.smithi190:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-26T04:22:08.736 INFO:tasks.cephadm:Stopping all daemons... 2024-03-26T04:22:08.736 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-26T04:22:08.736 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-762e930c-eb21-11ee-95d0-87774f69a715@mon.a 2024-03-26T04:22:08.945 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:22:08 smithi175 systemd[1]: Stopping Ceph mon.a for 762e930c-eb21-11ee-95d0-87774f69a715... 2024-03-26T04:22:08.946 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:22:08 smithi175 bash[21783]: ::ffff:172.21.15.190 - - [26/Mar/2024:04:22:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-26T04:22:09.032 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:22:08 smithi175 bash[21527]: debug 2024-03-26T04:22:08.946+0000 7f5552ee0700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-03-26T04:22:09.032 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:22:08 smithi175 bash[21527]: debug 2024-03-26T04:22:08.946+0000 7f5552ee0700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-26T04:22:10.381 INFO:journalctl@ceph.mon.a.smithi175.stdout:Mar 26 04:22:10 smithi175 bash[136765]: ceph-762e930c-eb21-11ee-95d0-87774f69a715-mon-a 2024-03-26T04:22:10.395 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@mon.a.service' 2024-03-26T04:22:10.421 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T04:22:10.421 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-26T04:22:10.421 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-03-26T04:22:10.421 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-762e930c-eb21-11ee-95d0-87774f69a715@mon.c 2024-03-26T04:22:10.721 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:22:10 smithi175 bash[21783]: [26/Mar/2024:04:22:10] ENGINE Bus STOPPING 2024-03-26T04:22:10.721 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:22:10 smithi175 bash[21783]: [26/Mar/2024:04:22:10] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-26T04:22:10.721 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:22:10 smithi175 bash[21783]: [26/Mar/2024:04:22:10] ENGINE Bus STOPPED 2024-03-26T04:22:10.721 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:22:10 smithi175 bash[21783]: [26/Mar/2024:04:22:10] ENGINE Bus STARTING 2024-03-26T04:22:10.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:22:10 smithi175 systemd[1]: Stopping Ceph mon.c for 762e930c-eb21-11ee-95d0-87774f69a715... 2024-03-26T04:22:10.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:22:10 smithi175 bash[29284]: debug 2024-03-26T04:22:10.590+0000 7fd732cc2700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-03-26T04:22:10.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:22:10 smithi175 bash[29284]: debug 2024-03-26T04:22:10.590+0000 7fd732cc2700 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-03-26T04:22:10.991 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:22:10 smithi175 bash[21783]: [26/Mar/2024:04:22:10] ENGINE Serving on http://:::9283 2024-03-26T04:22:10.991 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:22:10 smithi175 bash[21783]: [26/Mar/2024:04:22:10] ENGINE Bus STARTED 2024-03-26T04:22:11.718 INFO:journalctl@ceph.mon.c.smithi175.stdout:Mar 26 04:22:11 smithi175 bash[136875]: ceph-762e930c-eb21-11ee-95d0-87774f69a715-mon-c 2024-03-26T04:22:11.721 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@mon.c.service' 2024-03-26T04:22:11.744 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T04:22:11.744 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-03-26T04:22:11.744 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-03-26T04:22:11.744 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-762e930c-eb21-11ee-95d0-87774f69a715@mon.b 2024-03-26T04:22:11.902 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:22:11 smithi190 systemd[1]: Stopping Ceph mon.b for 762e930c-eb21-11ee-95d0-87774f69a715... 2024-03-26T04:22:11.902 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:22:11 smithi190 bash[47836]: debug there is no tcmu-runner data available 2024-03-26T04:22:12.089 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:22:11 smithi190 bash[23392]: debug 2024-03-26T04:22:11.900+0000 7f71c160f700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-03-26T04:22:12.089 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:22:11 smithi190 bash[23392]: debug 2024-03-26T04:22:11.900+0000 7f71c160f700 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-03-26T04:22:13.111 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@mon.b.service' 2024-03-26T04:22:13.119 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:22:12 smithi190 bash[56576]: ceph-762e930c-eb21-11ee-95d0-87774f69a715-mon-b 2024-03-26T04:22:13.119 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:22:13 smithi190 bash[56645]: Error response from daemon: No such container: ceph-762e930c-eb21-11ee-95d0-87774f69a715-mon-b 2024-03-26T04:22:13.119 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:22:13 smithi190 systemd[1]: ceph-762e930c-eb21-11ee-95d0-87774f69a715@mon.b.service: Deactivated successfully. 2024-03-26T04:22:13.119 INFO:journalctl@ceph.mon.b.smithi190.stdout:Mar 26 04:22:13 smithi190 systemd[1]: Stopped Ceph mon.b for 762e930c-eb21-11ee-95d0-87774f69a715. 2024-03-26T04:22:13.146 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T04:22:13.146 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-03-26T04:22:13.147 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-26T04:22:13.147 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-762e930c-eb21-11ee-95d0-87774f69a715@mgr.y 2024-03-26T04:22:13.490 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:22:13 smithi175 systemd[1]: Stopping Ceph mgr.y for 762e930c-eb21-11ee-95d0-87774f69a715... 2024-03-26T04:22:13.491 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:22:13 smithi175 bash[21783]: debug 2024-03-26T04:22:13.294+0000 7f6f4d4db700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-03-26T04:22:13.491 INFO:journalctl@ceph.mgr.y.smithi175.stdout:Mar 26 04:22:13 smithi175 bash[21783]: debug 2024-03-26T04:22:13.294+0000 7f6f4d4db700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-26T04:22:14.236 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@mgr.y.service' 2024-03-26T04:22:14.262 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T04:22:14.262 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-26T04:22:14.262 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-26T04:22:14.262 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-762e930c-eb21-11ee-95d0-87774f69a715@mgr.x 2024-03-26T04:22:14.695 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 04:22:14 smithi190 systemd[1]: Stopping Ceph mgr.x for 762e930c-eb21-11ee-95d0-87774f69a715... 2024-03-26T04:22:15.342 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@mgr.x.service' 2024-03-26T04:22:15.350 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 04:22:15 smithi190 bash[56685]: ceph-762e930c-eb21-11ee-95d0-87774f69a715-mgr-x 2024-03-26T04:22:15.350 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 04:22:15 smithi190 systemd[1]: ceph-762e930c-eb21-11ee-95d0-87774f69a715@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-03-26T04:22:15.350 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 04:22:15 smithi190 bash[56745]: Error response from daemon: No such container: ceph-762e930c-eb21-11ee-95d0-87774f69a715-mgr-x 2024-03-26T04:22:15.350 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 04:22:15 smithi190 systemd[1]: ceph-762e930c-eb21-11ee-95d0-87774f69a715@mgr.x.service: Failed with result 'exit-code'. 2024-03-26T04:22:15.350 INFO:journalctl@ceph.mgr.x.smithi190.stdout:Mar 26 04:22:15 smithi190 systemd[1]: Stopped Ceph mgr.x for 762e930c-eb21-11ee-95d0-87774f69a715. 2024-03-26T04:22:15.364 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T04:22:15.364 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-26T04:22:15.364 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-26T04:22:15.364 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-762e930c-eb21-11ee-95d0-87774f69a715@osd.0 2024-03-26T04:22:15.741 INFO:journalctl@ceph.osd.0.smithi175.stdout:Mar 26 04:22:15 smithi175 systemd[1]: Stopping Ceph osd.0 for 762e930c-eb21-11ee-95d0-87774f69a715... 2024-03-26T04:22:15.741 INFO:journalctl@ceph.osd.0.smithi175.stdout:Mar 26 04:22:15 smithi175 bash[32304]: debug 2024-03-26T04:22:15.522+0000 7ff3cf366700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-03-26T04:22:15.741 INFO:journalctl@ceph.osd.0.smithi175.stdout:Mar 26 04:22:15 smithi175 bash[32304]: debug 2024-03-26T04:22:15.522+0000 7ff3cf366700 -1 osd.0 794 *** Got signal Terminated *** 2024-03-26T04:22:15.741 INFO:journalctl@ceph.osd.0.smithi175.stdout:Mar 26 04:22:15 smithi175 bash[32304]: debug 2024-03-26T04:22:15.522+0000 7ff3cf366700 -1 osd.0 794 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T04:22:21.978 INFO:journalctl@ceph.osd.0.smithi175.stdout:Mar 26 04:22:21 smithi175 bash[137085]: ceph-762e930c-eb21-11ee-95d0-87774f69a715-osd-0 2024-03-26T04:22:21.979 INFO:journalctl@ceph.osd.0.smithi175.stdout:Mar 26 04:22:21 smithi175 bash[137149]: Error response from daemon: No such container: ceph-762e930c-eb21-11ee-95d0-87774f69a715-osd-0 2024-03-26T04:22:24.094 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@osd.0.service' 2024-03-26T04:22:24.122 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T04:22:24.122 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-26T04:22:24.122 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-26T04:22:24.122 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-762e930c-eb21-11ee-95d0-87774f69a715@osd.1 2024-03-26T04:22:24.491 INFO:journalctl@ceph.osd.1.smithi175.stdout:Mar 26 04:22:24 smithi175 systemd[1]: Stopping Ceph osd.1 for 762e930c-eb21-11ee-95d0-87774f69a715... 2024-03-26T04:22:24.491 INFO:journalctl@ceph.osd.1.smithi175.stdout:Mar 26 04:22:24 smithi175 bash[37791]: debug 2024-03-26T04:22:24.314+0000 7ffa57d38700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-03-26T04:22:24.491 INFO:journalctl@ceph.osd.1.smithi175.stdout:Mar 26 04:22:24 smithi175 bash[37791]: debug 2024-03-26T04:22:24.314+0000 7ffa57d38700 -1 osd.1 794 *** Got signal Terminated *** 2024-03-26T04:22:24.491 INFO:journalctl@ceph.osd.1.smithi175.stdout:Mar 26 04:22:24 smithi175 bash[37791]: debug 2024-03-26T04:22:24.314+0000 7ffa57d38700 -1 osd.1 794 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T04:22:29.747 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 04:22:29 smithi190 bash[51574]: ts=2024-03-26T04:22:29.471Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.175:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.175:8765: connect: connection refused" 2024-03-26T04:22:29.747 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 04:22:29 smithi190 bash[51574]: ts=2024-03-26T04:22:29.473Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.175:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.175:8765: connect: connection refused" 2024-03-26T04:22:29.747 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 04:22:29 smithi190 bash[51574]: ts=2024-03-26T04:22:29.475Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.175:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.175:8765: connect: connection refused" 2024-03-26T04:22:29.748 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 04:22:29 smithi190 bash[51574]: ts=2024-03-26T04:22:29.477Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.175:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.175:8765: connect: connection refused" 2024-03-26T04:22:29.748 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 04:22:29 smithi190 bash[51574]: ts=2024-03-26T04:22:29.478Z caller=refresh.go:99 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.175:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.175:8765: connect: connection refused" 2024-03-26T04:22:30.835 INFO:journalctl@ceph.osd.1.smithi175.stdout:Mar 26 04:22:30 smithi175 bash[137284]: ceph-762e930c-eb21-11ee-95d0-87774f69a715-osd-1 2024-03-26T04:22:30.835 INFO:journalctl@ceph.osd.1.smithi175.stdout:Mar 26 04:22:30 smithi175 bash[137347]: Error response from daemon: No such container: ceph-762e930c-eb21-11ee-95d0-87774f69a715-osd-1 2024-03-26T04:22:32.794 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@osd.1.service' 2024-03-26T04:22:32.822 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T04:22:32.822 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-26T04:22:32.822 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-26T04:22:32.822 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-762e930c-eb21-11ee-95d0-87774f69a715@osd.2 2024-03-26T04:22:33.241 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 26 04:22:32 smithi175 systemd[1]: Stopping Ceph osd.2 for 762e930c-eb21-11ee-95d0-87774f69a715... 2024-03-26T04:22:33.241 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 26 04:22:33 smithi175 bash[43199]: debug 2024-03-26T04:22:33.034+0000 7f96c5c59700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-03-26T04:22:33.241 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 26 04:22:33 smithi175 bash[43199]: debug 2024-03-26T04:22:33.034+0000 7f96c5c59700 -1 osd.2 794 *** Got signal Terminated *** 2024-03-26T04:22:33.241 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 26 04:22:33 smithi175 bash[43199]: debug 2024-03-26T04:22:33.034+0000 7f96c5c59700 -1 osd.2 794 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T04:22:39.263 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 26 04:22:39 smithi175 bash[137481]: ceph-762e930c-eb21-11ee-95d0-87774f69a715-osd-2 2024-03-26T04:22:39.264 INFO:journalctl@ceph.osd.2.smithi175.stdout:Mar 26 04:22:39 smithi175 bash[137548]: Error response from daemon: No such container: ceph-762e930c-eb21-11ee-95d0-87774f69a715-osd-2 2024-03-26T04:22:41.259 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@osd.2.service' 2024-03-26T04:22:41.286 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T04:22:41.286 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-26T04:22:41.286 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-26T04:22:41.286 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-762e930c-eb21-11ee-95d0-87774f69a715@osd.3 2024-03-26T04:22:41.741 INFO:journalctl@ceph.osd.3.smithi175.stdout:Mar 26 04:22:41 smithi175 systemd[1]: Stopping Ceph osd.3 for 762e930c-eb21-11ee-95d0-87774f69a715... 2024-03-26T04:22:41.741 INFO:journalctl@ceph.osd.3.smithi175.stdout:Mar 26 04:22:41 smithi175 bash[48970]: debug 2024-03-26T04:22:41.462+0000 7fb0199a8700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-03-26T04:22:41.741 INFO:journalctl@ceph.osd.3.smithi175.stdout:Mar 26 04:22:41 smithi175 bash[48970]: debug 2024-03-26T04:22:41.462+0000 7fb0199a8700 -1 osd.3 794 *** Got signal Terminated *** 2024-03-26T04:22:41.741 INFO:journalctl@ceph.osd.3.smithi175.stdout:Mar 26 04:22:41 smithi175 bash[48970]: debug 2024-03-26T04:22:41.462+0000 7fb0199a8700 -1 osd.3 794 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T04:22:42.714 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:22:42 smithi190 bash[43655]: debug 2024-03-26T04:22:42.403+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:22:42.997 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 04:22:42 smithi190 bash[26939]: debug 2024-03-26T04:22:42.711+0000 7f21fbd70700 -1 osd.4 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.680046+0000 front 2024-03-26T04:22:17.680446+0000 (oldest deadline 2024-03-26T04:22:41.779852+0000) 2024-03-26T04:22:43.710 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:22:43 smithi190 bash[43655]: debug 2024-03-26T04:22:43.371+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:22:43.711 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:22:43 smithi190 bash[32238]: debug 2024-03-26T04:22:43.435+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.276965+0000 front 2024-03-26T04:22:17.277165+0000 (oldest deadline 2024-03-26T04:22:42.576762+0000) 2024-03-26T04:22:43.997 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 04:22:43 smithi190 bash[26939]: debug 2024-03-26T04:22:43.707+0000 7f21fbd70700 -1 osd.4 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.680046+0000 front 2024-03-26T04:22:17.680446+0000 (oldest deadline 2024-03-26T04:22:41.779852+0000) 2024-03-26T04:22:44.668 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:22:44 smithi190 bash[43655]: debug 2024-03-26T04:22:44.399+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:22:44.668 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:22:44 smithi190 bash[32238]: debug 2024-03-26T04:22:44.431+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.276965+0000 front 2024-03-26T04:22:17.277165+0000 (oldest deadline 2024-03-26T04:22:42.576762+0000) 2024-03-26T04:22:44.991 INFO:journalctl@ceph.osd.3.smithi175.stdout:Mar 26 04:22:44 smithi175 bash[48970]: debug 2024-03-26T04:22:44.574+0000 7fb015f9c700 -1 osd.3 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.267521+0000 front 2024-03-26T04:22:18.267464+0000 (oldest deadline 2024-03-26T04:22:44.167462+0000) 2024-03-26T04:22:44.996 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 04:22:44 smithi190 bash[26939]: debug 2024-03-26T04:22:44.667+0000 7f21fbd70700 -1 osd.4 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.680046+0000 front 2024-03-26T04:22:17.680446+0000 (oldest deadline 2024-03-26T04:22:41.779852+0000) 2024-03-26T04:22:45.671 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:22:45 smithi190 bash[32238]: debug 2024-03-26T04:22:45.399+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.276965+0000 front 2024-03-26T04:22:17.277165+0000 (oldest deadline 2024-03-26T04:22:42.576762+0000) 2024-03-26T04:22:45.672 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:22:45 smithi190 bash[43655]: debug 2024-03-26T04:22:45.399+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:22:45.990 INFO:journalctl@ceph.osd.3.smithi175.stdout:Mar 26 04:22:45 smithi175 bash[48970]: debug 2024-03-26T04:22:45.538+0000 7fb015f9c700 -1 osd.3 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.267521+0000 front 2024-03-26T04:22:18.267464+0000 (oldest deadline 2024-03-26T04:22:44.167462+0000) 2024-03-26T04:22:45.996 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 04:22:45 smithi190 bash[26939]: debug 2024-03-26T04:22:45.667+0000 7f21fbd70700 -1 osd.4 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.680046+0000 front 2024-03-26T04:22:17.680446+0000 (oldest deadline 2024-03-26T04:22:41.779852+0000) 2024-03-26T04:22:45.997 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:22:45 smithi190 bash[37968]: debug 2024-03-26T04:22:45.879+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:20.030724+0000 front 2024-03-26T04:22:20.030946+0000 (oldest deadline 2024-03-26T04:22:45.330384+0000) 2024-03-26T04:22:46.747 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:22:46 smithi190 bash[43655]: debug 2024-03-26T04:22:46.435+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:22:46.748 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 04:22:46 smithi190 bash[26939]: debug 2024-03-26T04:22:46.643+0000 7f21fbd70700 -1 osd.4 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.680046+0000 front 2024-03-26T04:22:17.680446+0000 (oldest deadline 2024-03-26T04:22:41.779852+0000) 2024-03-26T04:22:46.748 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:22:46 smithi190 bash[32238]: debug 2024-03-26T04:22:46.427+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.276965+0000 front 2024-03-26T04:22:17.277165+0000 (oldest deadline 2024-03-26T04:22:42.576762+0000) 2024-03-26T04:22:47.247 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:22:46 smithi190 bash[37968]: debug 2024-03-26T04:22:46.891+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:20.030724+0000 front 2024-03-26T04:22:20.030946+0000 (oldest deadline 2024-03-26T04:22:45.330384+0000) 2024-03-26T04:22:47.734 INFO:journalctl@ceph.osd.3.smithi175.stdout:Mar 26 04:22:47 smithi175 bash[137683]: ceph-762e930c-eb21-11ee-95d0-87774f69a715-osd-3 2024-03-26T04:22:47.735 INFO:journalctl@ceph.osd.3.smithi175.stdout:Mar 26 04:22:47 smithi175 bash[137746]: Error response from daemon: No such container: ceph-762e930c-eb21-11ee-95d0-87774f69a715-osd-3 2024-03-26T04:22:47.747 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 04:22:47 smithi190 bash[26939]: debug 2024-03-26T04:22:47.619+0000 7f21fbd70700 -1 osd.4 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.680046+0000 front 2024-03-26T04:22:17.680446+0000 (oldest deadline 2024-03-26T04:22:41.779852+0000) 2024-03-26T04:22:47.747 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:22:47 smithi190 bash[32238]: debug 2024-03-26T04:22:47.439+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.276965+0000 front 2024-03-26T04:22:17.277165+0000 (oldest deadline 2024-03-26T04:22:42.576762+0000) 2024-03-26T04:22:47.747 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:22:47 smithi190 bash[43655]: debug 2024-03-26T04:22:47.399+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:22:48.247 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:22:47 smithi190 bash[37968]: debug 2024-03-26T04:22:47.927+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:20.030724+0000 front 2024-03-26T04:22:20.030946+0000 (oldest deadline 2024-03-26T04:22:45.330384+0000) 2024-03-26T04:22:48.747 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 04:22:48 smithi190 bash[26939]: debug 2024-03-26T04:22:48.591+0000 7f21fbd70700 -1 osd.4 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.680046+0000 front 2024-03-26T04:22:17.680446+0000 (oldest deadline 2024-03-26T04:22:41.779852+0000) 2024-03-26T04:22:48.747 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:22:48 smithi190 bash[32238]: debug 2024-03-26T04:22:48.411+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.276965+0000 front 2024-03-26T04:22:17.277165+0000 (oldest deadline 2024-03-26T04:22:42.576762+0000) 2024-03-26T04:22:48.747 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:22:48 smithi190 bash[43655]: debug 2024-03-26T04:22:48.387+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:22:49.247 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:22:48 smithi190 bash[37968]: debug 2024-03-26T04:22:48.875+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:20.030724+0000 front 2024-03-26T04:22:20.030946+0000 (oldest deadline 2024-03-26T04:22:45.330384+0000) 2024-03-26T04:22:49.525 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@osd.3.service' 2024-03-26T04:22:49.552 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T04:22:49.552 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-26T04:22:49.552 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-26T04:22:49.553 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-762e930c-eb21-11ee-95d0-87774f69a715@osd.4 2024-03-26T04:22:49.747 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:22:49 smithi190 bash[43655]: debug 2024-03-26T04:22:49.363+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:22:49.747 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 04:22:49 smithi190 systemd[1]: Stopping Ceph osd.4 for 762e930c-eb21-11ee-95d0-87774f69a715... 2024-03-26T04:22:49.747 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 04:22:49 smithi190 bash[26939]: debug 2024-03-26T04:22:49.579+0000 7f21fbd70700 -1 osd.4 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.680046+0000 front 2024-03-26T04:22:17.680446+0000 (oldest deadline 2024-03-26T04:22:41.779852+0000) 2024-03-26T04:22:49.747 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:22:49 smithi190 bash[32238]: debug 2024-03-26T04:22:49.387+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.276965+0000 front 2024-03-26T04:22:17.277165+0000 (oldest deadline 2024-03-26T04:22:42.576762+0000) 2024-03-26T04:22:49.844 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 04:22:49 smithi190 bash[26939]: debug 2024-03-26T04:22:49.771+0000 7f21ff77c700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-03-26T04:22:49.844 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 04:22:49 smithi190 bash[26939]: debug 2024-03-26T04:22:49.771+0000 7f21ff77c700 -1 osd.4 794 *** Got signal Terminated *** 2024-03-26T04:22:49.844 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 04:22:49 smithi190 bash[26939]: debug 2024-03-26T04:22:49.771+0000 7f21ff77c700 -1 osd.4 794 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T04:22:50.247 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:22:49 smithi190 bash[37968]: debug 2024-03-26T04:22:49.843+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:20.030724+0000 front 2024-03-26T04:22:20.030946+0000 (oldest deadline 2024-03-26T04:22:45.330384+0000) 2024-03-26T04:22:50.591 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:22:50 smithi190 bash[32238]: debug 2024-03-26T04:22:50.335+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.276965+0000 front 2024-03-26T04:22:17.277165+0000 (oldest deadline 2024-03-26T04:22:42.576762+0000) 2024-03-26T04:22:50.591 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:22:50 smithi190 bash[43655]: debug 2024-03-26T04:22:50.399+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:22:50.884 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 04:22:50 smithi190 bash[26939]: debug 2024-03-26T04:22:50.591+0000 7f21fbd70700 -1 osd.4 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.680046+0000 front 2024-03-26T04:22:17.680446+0000 (oldest deadline 2024-03-26T04:22:41.779852+0000) 2024-03-26T04:22:51.246 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:22:50 smithi190 bash[37968]: debug 2024-03-26T04:22:50.883+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:20.030724+0000 front 2024-03-26T04:22:20.030946+0000 (oldest deadline 2024-03-26T04:22:45.330384+0000) 2024-03-26T04:22:51.630 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:22:51 smithi190 bash[32238]: debug 2024-03-26T04:22:51.311+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.276965+0000 front 2024-03-26T04:22:17.277165+0000 (oldest deadline 2024-03-26T04:22:42.576762+0000) 2024-03-26T04:22:51.630 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:22:51 smithi190 bash[32238]: debug 2024-03-26T04:22:51.311+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:24.878269+0000 front 2024-03-26T04:22:24.878385+0000 (oldest deadline 2024-03-26T04:22:50.778302+0000) 2024-03-26T04:22:51.631 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:22:51 smithi190 bash[43655]: debug 2024-03-26T04:22:51.415+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:22:51.883 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 04:22:51 smithi190 bash[26939]: debug 2024-03-26T04:22:51.627+0000 7f21fbd70700 -1 osd.4 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.680046+0000 front 2024-03-26T04:22:17.680446+0000 (oldest deadline 2024-03-26T04:22:41.779852+0000) 2024-03-26T04:22:51.884 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 04:22:51 smithi190 bash[26939]: debug 2024-03-26T04:22:51.627+0000 7f21fbd70700 -1 osd.4 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:23.481240+0000 front 2024-03-26T04:22:23.481288+0000 (oldest deadline 2024-03-26T04:22:50.982350+0000) 2024-03-26T04:22:52.247 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:22:51 smithi190 bash[37968]: debug 2024-03-26T04:22:51.883+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:20.030724+0000 front 2024-03-26T04:22:20.030946+0000 (oldest deadline 2024-03-26T04:22:45.330384+0000) 2024-03-26T04:22:52.677 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:22:52 smithi190 bash[43655]: debug 2024-03-26T04:22:52.375+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:22:52.677 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:22:52 smithi190 bash[32238]: debug 2024-03-26T04:22:52.335+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.276965+0000 front 2024-03-26T04:22:17.277165+0000 (oldest deadline 2024-03-26T04:22:42.576762+0000) 2024-03-26T04:22:52.677 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:22:52 smithi190 bash[32238]: debug 2024-03-26T04:22:52.335+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:24.878269+0000 front 2024-03-26T04:22:24.878385+0000 (oldest deadline 2024-03-26T04:22:50.778302+0000) 2024-03-26T04:22:52.997 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:22:52 smithi190 bash[37968]: debug 2024-03-26T04:22:52.855+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:20.030724+0000 front 2024-03-26T04:22:20.030946+0000 (oldest deadline 2024-03-26T04:22:45.330384+0000) 2024-03-26T04:22:52.997 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 04:22:52 smithi190 bash[26939]: debug 2024-03-26T04:22:52.679+0000 7f21fbd70700 -1 osd.4 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.680046+0000 front 2024-03-26T04:22:17.680446+0000 (oldest deadline 2024-03-26T04:22:41.779852+0000) 2024-03-26T04:22:52.997 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 04:22:52 smithi190 bash[26939]: debug 2024-03-26T04:22:52.679+0000 7f21fbd70700 -1 osd.4 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:23.481240+0000 front 2024-03-26T04:22:23.481288+0000 (oldest deadline 2024-03-26T04:22:50.982350+0000) 2024-03-26T04:22:53.694 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:22:53 smithi190 bash[32238]: debug 2024-03-26T04:22:53.287+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.276965+0000 front 2024-03-26T04:22:17.277165+0000 (oldest deadline 2024-03-26T04:22:42.576762+0000) 2024-03-26T04:22:53.694 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:22:53 smithi190 bash[32238]: debug 2024-03-26T04:22:53.287+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:24.878269+0000 front 2024-03-26T04:22:24.878385+0000 (oldest deadline 2024-03-26T04:22:50.778302+0000) 2024-03-26T04:22:53.694 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:22:53 smithi190 bash[43655]: debug 2024-03-26T04:22:53.331+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:22:53.694 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:22:53 smithi190 bash[43655]: debug 2024-03-26T04:22:53.331+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:28.014352+0000 front 2024-03-26T04:22:28.014229+0000 (oldest deadline 2024-03-26T04:22:53.314009+0000) 2024-03-26T04:22:53.997 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:22:53 smithi190 bash[37968]: debug 2024-03-26T04:22:53.815+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:20.030724+0000 front 2024-03-26T04:22:20.030946+0000 (oldest deadline 2024-03-26T04:22:45.330384+0000) 2024-03-26T04:22:53.997 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 04:22:53 smithi190 bash[26939]: debug 2024-03-26T04:22:53.695+0000 7f21fbd70700 -1 osd.4 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.680046+0000 front 2024-03-26T04:22:17.680446+0000 (oldest deadline 2024-03-26T04:22:41.779852+0000) 2024-03-26T04:22:53.997 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 04:22:53 smithi190 bash[26939]: debug 2024-03-26T04:22:53.695+0000 7f21fbd70700 -1 osd.4 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:23.481240+0000 front 2024-03-26T04:22:23.481288+0000 (oldest deadline 2024-03-26T04:22:50.982350+0000) 2024-03-26T04:22:54.707 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:22:54 smithi190 bash[32238]: debug 2024-03-26T04:22:54.311+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.276965+0000 front 2024-03-26T04:22:17.277165+0000 (oldest deadline 2024-03-26T04:22:42.576762+0000) 2024-03-26T04:22:54.707 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:22:54 smithi190 bash[32238]: debug 2024-03-26T04:22:54.311+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:24.878269+0000 front 2024-03-26T04:22:24.878385+0000 (oldest deadline 2024-03-26T04:22:50.778302+0000) 2024-03-26T04:22:54.707 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:22:54 smithi190 bash[43655]: debug 2024-03-26T04:22:54.351+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:22:54.707 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:22:54 smithi190 bash[43655]: debug 2024-03-26T04:22:54.351+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:28.014352+0000 front 2024-03-26T04:22:28.014229+0000 (oldest deadline 2024-03-26T04:22:53.314009+0000) 2024-03-26T04:22:54.997 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 04:22:54 smithi190 bash[26939]: debug 2024-03-26T04:22:54.707+0000 7f21fbd70700 -1 osd.4 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.680046+0000 front 2024-03-26T04:22:17.680446+0000 (oldest deadline 2024-03-26T04:22:41.779852+0000) 2024-03-26T04:22:54.997 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 04:22:54 smithi190 bash[26939]: debug 2024-03-26T04:22:54.707+0000 7f21fbd70700 -1 osd.4 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:23.481240+0000 front 2024-03-26T04:22:23.481288+0000 (oldest deadline 2024-03-26T04:22:50.982350+0000) 2024-03-26T04:22:54.997 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:22:54 smithi190 bash[37968]: debug 2024-03-26T04:22:54.855+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:20.030724+0000 front 2024-03-26T04:22:20.030946+0000 (oldest deadline 2024-03-26T04:22:45.330384+0000) 2024-03-26T04:22:55.521 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:22:55 smithi190 bash[32238]: debug 2024-03-26T04:22:55.323+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.276965+0000 front 2024-03-26T04:22:17.277165+0000 (oldest deadline 2024-03-26T04:22:42.576762+0000) 2024-03-26T04:22:55.521 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:22:55 smithi190 bash[32238]: debug 2024-03-26T04:22:55.323+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:24.878269+0000 front 2024-03-26T04:22:24.878385+0000 (oldest deadline 2024-03-26T04:22:50.778302+0000) 2024-03-26T04:22:55.521 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:22:55 smithi190 bash[43655]: debug 2024-03-26T04:22:55.319+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:22:55.521 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:22:55 smithi190 bash[43655]: debug 2024-03-26T04:22:55.319+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:28.014352+0000 front 2024-03-26T04:22:28.014229+0000 (oldest deadline 2024-03-26T04:22:53.314009+0000) 2024-03-26T04:22:56.135 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 04:22:55 smithi190 bash[56787]: ceph-762e930c-eb21-11ee-95d0-87774f69a715-osd-4 2024-03-26T04:22:56.135 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:22:55 smithi190 bash[37968]: debug 2024-03-26T04:22:55.815+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:20.030724+0000 front 2024-03-26T04:22:20.030946+0000 (oldest deadline 2024-03-26T04:22:45.330384+0000) 2024-03-26T04:22:56.135 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:22:55 smithi190 bash[37968]: debug 2024-03-26T04:22:55.815+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:27.632134+0000 front 2024-03-26T04:22:27.632209+0000 (oldest deadline 2024-03-26T04:22:55.238195+0000) 2024-03-26T04:22:56.497 INFO:journalctl@ceph.osd.4.smithi190.stdout:Mar 26 04:22:56 smithi190 bash[56849]: ceph-762e930c-eb21-11ee-95d0-87774f69a715-osd-4 2024-03-26T04:22:56.497 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:22:56 smithi190 bash[32238]: debug 2024-03-26T04:22:56.339+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.276965+0000 front 2024-03-26T04:22:17.277165+0000 (oldest deadline 2024-03-26T04:22:42.576762+0000) 2024-03-26T04:22:56.497 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:22:56 smithi190 bash[32238]: debug 2024-03-26T04:22:56.339+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:24.878269+0000 front 2024-03-26T04:22:24.878385+0000 (oldest deadline 2024-03-26T04:22:50.778302+0000) 2024-03-26T04:22:56.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:22:56 smithi190 bash[43655]: debug 2024-03-26T04:22:56.315+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:22:56.498 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:22:56 smithi190 bash[43655]: debug 2024-03-26T04:22:56.315+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:28.014352+0000 front 2024-03-26T04:22:28.014229+0000 (oldest deadline 2024-03-26T04:22:53.314009+0000) 2024-03-26T04:22:57.209 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:22:56 smithi190 bash[37968]: debug 2024-03-26T04:22:56.827+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:20.030724+0000 front 2024-03-26T04:22:20.030946+0000 (oldest deadline 2024-03-26T04:22:45.330384+0000) 2024-03-26T04:22:57.210 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:22:56 smithi190 bash[37968]: debug 2024-03-26T04:22:56.827+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:27.632134+0000 front 2024-03-26T04:22:27.632209+0000 (oldest deadline 2024-03-26T04:22:55.238195+0000) 2024-03-26T04:22:57.497 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:22:57 smithi190 bash[32238]: debug 2024-03-26T04:22:57.355+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.276965+0000 front 2024-03-26T04:22:17.277165+0000 (oldest deadline 2024-03-26T04:22:42.576762+0000) 2024-03-26T04:22:57.497 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:22:57 smithi190 bash[32238]: debug 2024-03-26T04:22:57.355+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:24.878269+0000 front 2024-03-26T04:22:24.878385+0000 (oldest deadline 2024-03-26T04:22:50.778302+0000) 2024-03-26T04:22:57.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:22:57 smithi190 bash[43655]: debug 2024-03-26T04:22:57.291+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:22:57.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:22:57 smithi190 bash[43655]: debug 2024-03-26T04:22:57.291+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:28.014352+0000 front 2024-03-26T04:22:28.014229+0000 (oldest deadline 2024-03-26T04:22:53.314009+0000) 2024-03-26T04:22:57.997 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:22:57 smithi190 bash[37968]: debug 2024-03-26T04:22:57.819+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:20.030724+0000 front 2024-03-26T04:22:20.030946+0000 (oldest deadline 2024-03-26T04:22:45.330384+0000) 2024-03-26T04:22:57.997 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:22:57 smithi190 bash[37968]: debug 2024-03-26T04:22:57.819+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:27.632134+0000 front 2024-03-26T04:22:27.632209+0000 (oldest deadline 2024-03-26T04:22:55.238195+0000) 2024-03-26T04:22:58.319 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@osd.4.service' 2024-03-26T04:22:58.348 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T04:22:58.348 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-26T04:22:58.348 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-26T04:22:58.348 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-762e930c-eb21-11ee-95d0-87774f69a715@osd.5 2024-03-26T04:22:58.534 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:22:58 smithi190 bash[43655]: debug 2024-03-26T04:22:58.250+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:22:58.534 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:22:58 smithi190 bash[43655]: debug 2024-03-26T04:22:58.250+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:28.014352+0000 front 2024-03-26T04:22:28.014229+0000 (oldest deadline 2024-03-26T04:22:53.314009+0000) 2024-03-26T04:22:58.534 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:22:58 smithi190 bash[32238]: debug 2024-03-26T04:22:58.370+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.276965+0000 front 2024-03-26T04:22:17.277165+0000 (oldest deadline 2024-03-26T04:22:42.576762+0000) 2024-03-26T04:22:58.534 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:22:58 smithi190 bash[32238]: debug 2024-03-26T04:22:58.370+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:24.878269+0000 front 2024-03-26T04:22:24.878385+0000 (oldest deadline 2024-03-26T04:22:50.778302+0000) 2024-03-26T04:22:58.534 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:22:58 smithi190 systemd[1]: Stopping Ceph osd.5 for 762e930c-eb21-11ee-95d0-87774f69a715... 2024-03-26T04:22:58.747 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:22:58 smithi190 bash[32238]: debug 2024-03-26T04:22:58.530+0000 7feeb3c2d700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-03-26T04:22:58.747 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:22:58 smithi190 bash[32238]: debug 2024-03-26T04:22:58.530+0000 7feeb3c2d700 -1 osd.5 794 *** Got signal Terminated *** 2024-03-26T04:22:58.747 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:22:58 smithi190 bash[32238]: debug 2024-03-26T04:22:58.530+0000 7feeb3c2d700 -1 osd.5 794 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T04:22:59.247 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:22:58 smithi190 bash[37968]: debug 2024-03-26T04:22:58.814+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:20.030724+0000 front 2024-03-26T04:22:20.030946+0000 (oldest deadline 2024-03-26T04:22:45.330384+0000) 2024-03-26T04:22:59.247 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:22:58 smithi190 bash[37968]: debug 2024-03-26T04:22:58.814+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:27.632134+0000 front 2024-03-26T04:22:27.632209+0000 (oldest deadline 2024-03-26T04:22:55.238195+0000) 2024-03-26T04:22:59.747 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:22:59 smithi190 bash[32238]: debug 2024-03-26T04:22:59.382+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.276965+0000 front 2024-03-26T04:22:17.277165+0000 (oldest deadline 2024-03-26T04:22:42.576762+0000) 2024-03-26T04:22:59.747 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:22:59 smithi190 bash[32238]: debug 2024-03-26T04:22:59.382+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:24.878269+0000 front 2024-03-26T04:22:24.878385+0000 (oldest deadline 2024-03-26T04:22:50.778302+0000) 2024-03-26T04:22:59.747 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:22:59 smithi190 bash[43655]: debug 2024-03-26T04:22:59.262+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:22:59.747 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:22:59 smithi190 bash[43655]: debug 2024-03-26T04:22:59.262+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:28.014352+0000 front 2024-03-26T04:22:28.014229+0000 (oldest deadline 2024-03-26T04:22:53.314009+0000) 2024-03-26T04:23:00.237 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:22:59 smithi190 bash[37968]: debug 2024-03-26T04:22:59.846+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:20.030724+0000 front 2024-03-26T04:22:20.030946+0000 (oldest deadline 2024-03-26T04:22:45.330384+0000) 2024-03-26T04:23:00.237 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:22:59 smithi190 bash[37968]: debug 2024-03-26T04:22:59.846+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:27.632134+0000 front 2024-03-26T04:22:27.632209+0000 (oldest deadline 2024-03-26T04:22:55.238195+0000) 2024-03-26T04:23:00.497 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:23:00 smithi190 bash[32238]: debug 2024-03-26T04:23:00.402+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.276965+0000 front 2024-03-26T04:22:17.277165+0000 (oldest deadline 2024-03-26T04:22:42.576762+0000) 2024-03-26T04:23:00.497 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:23:00 smithi190 bash[32238]: debug 2024-03-26T04:23:00.402+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:24.878269+0000 front 2024-03-26T04:22:24.878385+0000 (oldest deadline 2024-03-26T04:22:50.778302+0000) 2024-03-26T04:23:00.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:00 smithi190 bash[43655]: debug 2024-03-26T04:23:00.238+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:23:00.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:00 smithi190 bash[43655]: debug 2024-03-26T04:23:00.238+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:28.014352+0000 front 2024-03-26T04:22:28.014229+0000 (oldest deadline 2024-03-26T04:22:53.314009+0000) 2024-03-26T04:23:00.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:00 smithi190 bash[43655]: debug 2024-03-26T04:23:00.238+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:34.415638+0000 front 2024-03-26T04:22:34.415939+0000 (oldest deadline 2024-03-26T04:22:59.715238+0000) 2024-03-26T04:23:01.246 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:00 smithi190 bash[37968]: debug 2024-03-26T04:23:00.890+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:20.030724+0000 front 2024-03-26T04:22:20.030946+0000 (oldest deadline 2024-03-26T04:22:45.330384+0000) 2024-03-26T04:23:01.247 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:00 smithi190 bash[37968]: debug 2024-03-26T04:23:00.890+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:27.632134+0000 front 2024-03-26T04:22:27.632209+0000 (oldest deadline 2024-03-26T04:22:55.238195+0000) 2024-03-26T04:23:01.747 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:23:01 smithi190 bash[32238]: debug 2024-03-26T04:23:01.354+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.276965+0000 front 2024-03-26T04:22:17.277165+0000 (oldest deadline 2024-03-26T04:22:42.576762+0000) 2024-03-26T04:23:01.747 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:23:01 smithi190 bash[32238]: debug 2024-03-26T04:23:01.354+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:24.878269+0000 front 2024-03-26T04:22:24.878385+0000 (oldest deadline 2024-03-26T04:22:50.778302+0000) 2024-03-26T04:23:01.747 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:23:01 smithi190 bash[32238]: debug 2024-03-26T04:23:01.354+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:35.980571+0000 front 2024-03-26T04:22:35.980530+0000 (oldest deadline 2024-03-26T04:23:01.280318+0000) 2024-03-26T04:23:01.747 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:01 smithi190 bash[43655]: debug 2024-03-26T04:23:01.286+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:23:01.747 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:01 smithi190 bash[43655]: debug 2024-03-26T04:23:01.286+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:28.014352+0000 front 2024-03-26T04:22:28.014229+0000 (oldest deadline 2024-03-26T04:22:53.314009+0000) 2024-03-26T04:23:01.747 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:01 smithi190 bash[43655]: debug 2024-03-26T04:23:01.286+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:34.415638+0000 front 2024-03-26T04:22:34.415939+0000 (oldest deadline 2024-03-26T04:22:59.715238+0000) 2024-03-26T04:23:02.247 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:01 smithi190 bash[37968]: debug 2024-03-26T04:23:01.938+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:20.030724+0000 front 2024-03-26T04:22:20.030946+0000 (oldest deadline 2024-03-26T04:22:45.330384+0000) 2024-03-26T04:23:02.247 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:01 smithi190 bash[37968]: debug 2024-03-26T04:23:01.938+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:27.632134+0000 front 2024-03-26T04:22:27.632209+0000 (oldest deadline 2024-03-26T04:22:55.238195+0000) 2024-03-26T04:23:02.247 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:01 smithi190 bash[37968]: debug 2024-03-26T04:23:01.938+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:35.239813+0000 front 2024-03-26T04:22:35.239458+0000 (oldest deadline 2024-03-26T04:23:01.138987+0000) 2024-03-26T04:23:02.747 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:23:02 smithi190 bash[32238]: debug 2024-03-26T04:23:02.314+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.276965+0000 front 2024-03-26T04:22:17.277165+0000 (oldest deadline 2024-03-26T04:22:42.576762+0000) 2024-03-26T04:23:02.747 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:23:02 smithi190 bash[32238]: debug 2024-03-26T04:23:02.314+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:24.878269+0000 front 2024-03-26T04:22:24.878385+0000 (oldest deadline 2024-03-26T04:22:50.778302+0000) 2024-03-26T04:23:02.747 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:23:02 smithi190 bash[32238]: debug 2024-03-26T04:23:02.314+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:35.980571+0000 front 2024-03-26T04:22:35.980530+0000 (oldest deadline 2024-03-26T04:23:01.280318+0000) 2024-03-26T04:23:02.747 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:02 smithi190 bash[43655]: debug 2024-03-26T04:23:02.258+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:23:02.747 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:02 smithi190 bash[43655]: debug 2024-03-26T04:23:02.258+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:28.014352+0000 front 2024-03-26T04:22:28.014229+0000 (oldest deadline 2024-03-26T04:22:53.314009+0000) 2024-03-26T04:23:02.747 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:02 smithi190 bash[43655]: debug 2024-03-26T04:23:02.258+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:34.415638+0000 front 2024-03-26T04:22:34.415939+0000 (oldest deadline 2024-03-26T04:22:59.715238+0000) 2024-03-26T04:23:03.213 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:02 smithi190 bash[37968]: debug 2024-03-26T04:23:02.930+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:20.030724+0000 front 2024-03-26T04:22:20.030946+0000 (oldest deadline 2024-03-26T04:22:45.330384+0000) 2024-03-26T04:23:03.213 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:02 smithi190 bash[37968]: debug 2024-03-26T04:23:02.930+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:27.632134+0000 front 2024-03-26T04:22:27.632209+0000 (oldest deadline 2024-03-26T04:22:55.238195+0000) 2024-03-26T04:23:03.213 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:02 smithi190 bash[37968]: debug 2024-03-26T04:23:02.930+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:35.239813+0000 front 2024-03-26T04:22:35.239458+0000 (oldest deadline 2024-03-26T04:23:01.138987+0000) 2024-03-26T04:23:03.497 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:23:03 smithi190 bash[32238]: debug 2024-03-26T04:23:03.290+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:17.276965+0000 front 2024-03-26T04:22:17.277165+0000 (oldest deadline 2024-03-26T04:22:42.576762+0000) 2024-03-26T04:23:03.497 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:23:03 smithi190 bash[32238]: debug 2024-03-26T04:23:03.290+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:24.878269+0000 front 2024-03-26T04:22:24.878385+0000 (oldest deadline 2024-03-26T04:22:50.778302+0000) 2024-03-26T04:23:03.497 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:23:03 smithi190 bash[32238]: debug 2024-03-26T04:23:03.290+0000 7feeac219700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:35.980571+0000 front 2024-03-26T04:22:35.980530+0000 (oldest deadline 2024-03-26T04:23:01.280318+0000) 2024-03-26T04:23:03.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:03 smithi190 bash[43655]: debug 2024-03-26T04:23:03.214+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:23:03.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:03 smithi190 bash[43655]: debug 2024-03-26T04:23:03.214+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:28.014352+0000 front 2024-03-26T04:22:28.014229+0000 (oldest deadline 2024-03-26T04:22:53.314009+0000) 2024-03-26T04:23:03.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:03 smithi190 bash[43655]: debug 2024-03-26T04:23:03.214+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:34.415638+0000 front 2024-03-26T04:22:34.415939+0000 (oldest deadline 2024-03-26T04:22:59.715238+0000) 2024-03-26T04:23:04.177 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:03 smithi190 bash[37968]: debug 2024-03-26T04:23:03.946+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:20.030724+0000 front 2024-03-26T04:22:20.030946+0000 (oldest deadline 2024-03-26T04:22:45.330384+0000) 2024-03-26T04:23:04.177 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:03 smithi190 bash[37968]: debug 2024-03-26T04:23:03.946+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:27.632134+0000 front 2024-03-26T04:22:27.632209+0000 (oldest deadline 2024-03-26T04:22:55.238195+0000) 2024-03-26T04:23:04.177 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:03 smithi190 bash[37968]: debug 2024-03-26T04:23:03.946+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:35.239813+0000 front 2024-03-26T04:22:35.239458+0000 (oldest deadline 2024-03-26T04:23:01.138987+0000) 2024-03-26T04:23:04.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:04 smithi190 bash[43655]: debug 2024-03-26T04:23:04.178+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:23:04.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:04 smithi190 bash[43655]: debug 2024-03-26T04:23:04.182+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:28.014352+0000 front 2024-03-26T04:22:28.014229+0000 (oldest deadline 2024-03-26T04:22:53.314009+0000) 2024-03-26T04:23:04.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:04 smithi190 bash[43655]: debug 2024-03-26T04:23:04.182+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:34.415638+0000 front 2024-03-26T04:22:34.415939+0000 (oldest deadline 2024-03-26T04:22:59.715238+0000) 2024-03-26T04:23:04.859 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:23:04 smithi190 bash[56982]: ceph-762e930c-eb21-11ee-95d0-87774f69a715-osd-5 2024-03-26T04:23:04.859 INFO:journalctl@ceph.osd.5.smithi190.stdout:Mar 26 04:23:04 smithi190 bash[57046]: Error response from daemon: No such container: ceph-762e930c-eb21-11ee-95d0-87774f69a715-osd-5 2024-03-26T04:23:05.174 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:04 smithi190 bash[37968]: debug 2024-03-26T04:23:04.906+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:20.030724+0000 front 2024-03-26T04:22:20.030946+0000 (oldest deadline 2024-03-26T04:22:45.330384+0000) 2024-03-26T04:23:05.174 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:04 smithi190 bash[37968]: debug 2024-03-26T04:23:04.906+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:27.632134+0000 front 2024-03-26T04:22:27.632209+0000 (oldest deadline 2024-03-26T04:22:55.238195+0000) 2024-03-26T04:23:05.174 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:04 smithi190 bash[37968]: debug 2024-03-26T04:23:04.906+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:35.239813+0000 front 2024-03-26T04:22:35.239458+0000 (oldest deadline 2024-03-26T04:23:01.138987+0000) 2024-03-26T04:23:05.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:05 smithi190 bash[43655]: debug 2024-03-26T04:23:05.174+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:23:05.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:05 smithi190 bash[43655]: debug 2024-03-26T04:23:05.174+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:28.014352+0000 front 2024-03-26T04:22:28.014229+0000 (oldest deadline 2024-03-26T04:22:53.314009+0000) 2024-03-26T04:23:05.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:05 smithi190 bash[43655]: debug 2024-03-26T04:23:05.174+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:34.415638+0000 front 2024-03-26T04:22:34.415939+0000 (oldest deadline 2024-03-26T04:22:59.715238+0000) 2024-03-26T04:23:06.246 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:05 smithi190 bash[37968]: debug 2024-03-26T04:23:05.942+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:20.030724+0000 front 2024-03-26T04:22:20.030946+0000 (oldest deadline 2024-03-26T04:22:45.330384+0000) 2024-03-26T04:23:06.247 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:05 smithi190 bash[37968]: debug 2024-03-26T04:23:05.942+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:27.632134+0000 front 2024-03-26T04:22:27.632209+0000 (oldest deadline 2024-03-26T04:22:55.238195+0000) 2024-03-26T04:23:06.247 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:05 smithi190 bash[37968]: debug 2024-03-26T04:23:05.942+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:35.239813+0000 front 2024-03-26T04:22:35.239458+0000 (oldest deadline 2024-03-26T04:23:01.138987+0000) 2024-03-26T04:23:06.247 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:06 smithi190 bash[43655]: debug 2024-03-26T04:23:06.174+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:23:06.247 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:06 smithi190 bash[43655]: debug 2024-03-26T04:23:06.174+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:28.014352+0000 front 2024-03-26T04:22:28.014229+0000 (oldest deadline 2024-03-26T04:22:53.314009+0000) 2024-03-26T04:23:06.247 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:06 smithi190 bash[43655]: debug 2024-03-26T04:23:06.174+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:34.415638+0000 front 2024-03-26T04:22:34.415939+0000 (oldest deadline 2024-03-26T04:22:59.715238+0000) 2024-03-26T04:23:07.119 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@osd.5.service' 2024-03-26T04:23:07.149 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T04:23:07.149 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-26T04:23:07.149 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-26T04:23:07.149 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-762e930c-eb21-11ee-95d0-87774f69a715@osd.6 2024-03-26T04:23:07.246 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:06 smithi190 bash[37968]: debug 2024-03-26T04:23:06.990+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:20.030724+0000 front 2024-03-26T04:22:20.030946+0000 (oldest deadline 2024-03-26T04:22:45.330384+0000) 2024-03-26T04:23:07.246 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:06 smithi190 bash[37968]: debug 2024-03-26T04:23:06.990+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:27.632134+0000 front 2024-03-26T04:22:27.632209+0000 (oldest deadline 2024-03-26T04:22:55.238195+0000) 2024-03-26T04:23:07.247 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:06 smithi190 bash[37968]: debug 2024-03-26T04:23:06.990+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:35.239813+0000 front 2024-03-26T04:22:35.239458+0000 (oldest deadline 2024-03-26T04:23:01.138987+0000) 2024-03-26T04:23:07.247 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:07 smithi190 systemd[1]: Stopping Ceph osd.6 for 762e930c-eb21-11ee-95d0-87774f69a715... 2024-03-26T04:23:07.247 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:07 smithi190 bash[43655]: debug 2024-03-26T04:23:07.198+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:23:07.247 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:07 smithi190 bash[43655]: debug 2024-03-26T04:23:07.198+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:28.014352+0000 front 2024-03-26T04:22:28.014229+0000 (oldest deadline 2024-03-26T04:22:53.314009+0000) 2024-03-26T04:23:07.247 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:07 smithi190 bash[43655]: debug 2024-03-26T04:23:07.198+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:34.415638+0000 front 2024-03-26T04:22:34.415939+0000 (oldest deadline 2024-03-26T04:22:59.715238+0000) 2024-03-26T04:23:07.497 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:07 smithi190 bash[37968]: debug 2024-03-26T04:23:07.314+0000 7f216646d700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-03-26T04:23:07.498 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:07 smithi190 bash[37968]: debug 2024-03-26T04:23:07.314+0000 7f216646d700 -1 osd.6 794 *** Got signal Terminated *** 2024-03-26T04:23:07.498 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:07 smithi190 bash[37968]: debug 2024-03-26T04:23:07.314+0000 7f216646d700 -1 osd.6 794 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T04:23:08.247 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:08 smithi190 bash[43655]: debug 2024-03-26T04:23:08.178+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:23:08.247 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:08 smithi190 bash[43655]: debug 2024-03-26T04:23:08.178+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:28.014352+0000 front 2024-03-26T04:22:28.014229+0000 (oldest deadline 2024-03-26T04:22:53.314009+0000) 2024-03-26T04:23:08.247 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:08 smithi190 bash[43655]: debug 2024-03-26T04:23:08.178+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:34.415638+0000 front 2024-03-26T04:22:34.415939+0000 (oldest deadline 2024-03-26T04:22:59.715238+0000) 2024-03-26T04:23:08.247 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:07 smithi190 bash[37968]: debug 2024-03-26T04:23:07.942+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:20.030724+0000 front 2024-03-26T04:22:20.030946+0000 (oldest deadline 2024-03-26T04:22:45.330384+0000) 2024-03-26T04:23:08.247 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:07 smithi190 bash[37968]: debug 2024-03-26T04:23:07.942+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:27.632134+0000 front 2024-03-26T04:22:27.632209+0000 (oldest deadline 2024-03-26T04:22:55.238195+0000) 2024-03-26T04:23:08.247 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:07 smithi190 bash[37968]: debug 2024-03-26T04:23:07.942+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:35.239813+0000 front 2024-03-26T04:22:35.239458+0000 (oldest deadline 2024-03-26T04:23:01.138987+0000) 2024-03-26T04:23:09.247 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:08 smithi190 bash[37968]: debug 2024-03-26T04:23:08.934+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:20.030724+0000 front 2024-03-26T04:22:20.030946+0000 (oldest deadline 2024-03-26T04:22:45.330384+0000) 2024-03-26T04:23:09.247 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:08 smithi190 bash[37968]: debug 2024-03-26T04:23:08.934+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:27.632134+0000 front 2024-03-26T04:22:27.632209+0000 (oldest deadline 2024-03-26T04:22:55.238195+0000) 2024-03-26T04:23:09.247 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:08 smithi190 bash[37968]: debug 2024-03-26T04:23:08.934+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:35.239813+0000 front 2024-03-26T04:22:35.239458+0000 (oldest deadline 2024-03-26T04:23:01.138987+0000) 2024-03-26T04:23:09.247 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:09 smithi190 bash[43655]: debug 2024-03-26T04:23:09.166+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:23:09.247 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:09 smithi190 bash[43655]: debug 2024-03-26T04:23:09.166+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:28.014352+0000 front 2024-03-26T04:22:28.014229+0000 (oldest deadline 2024-03-26T04:22:53.314009+0000) 2024-03-26T04:23:09.247 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:09 smithi190 bash[43655]: debug 2024-03-26T04:23:09.166+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:34.415638+0000 front 2024-03-26T04:22:34.415939+0000 (oldest deadline 2024-03-26T04:22:59.715238+0000) 2024-03-26T04:23:09.247 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:09 smithi190 bash[43655]: debug 2024-03-26T04:23:09.166+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6830 osd.3 since back 2024-03-26T04:22:42.517456+0000 front 2024-03-26T04:22:42.517393+0000 (oldest deadline 2024-03-26T04:23:08.318342+0000) 2024-03-26T04:23:10.247 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:09 smithi190 bash[37968]: debug 2024-03-26T04:23:09.890+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:20.030724+0000 front 2024-03-26T04:22:20.030946+0000 (oldest deadline 2024-03-26T04:22:45.330384+0000) 2024-03-26T04:23:10.247 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:09 smithi190 bash[37968]: debug 2024-03-26T04:23:09.890+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:27.632134+0000 front 2024-03-26T04:22:27.632209+0000 (oldest deadline 2024-03-26T04:22:55.238195+0000) 2024-03-26T04:23:10.247 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:09 smithi190 bash[37968]: debug 2024-03-26T04:23:09.890+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:35.239813+0000 front 2024-03-26T04:22:35.239458+0000 (oldest deadline 2024-03-26T04:23:01.138987+0000) 2024-03-26T04:23:10.247 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:09 smithi190 bash[37968]: debug 2024-03-26T04:23:09.890+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6830 osd.3 since back 2024-03-26T04:22:46.341090+0000 front 2024-03-26T04:22:46.340935+0000 (oldest deadline 2024-03-26T04:23:09.240832+0000) 2024-03-26T04:23:10.247 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:10 smithi190 bash[43655]: debug 2024-03-26T04:23:10.126+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:23:10.247 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:10 smithi190 bash[43655]: debug 2024-03-26T04:23:10.126+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:28.014352+0000 front 2024-03-26T04:22:28.014229+0000 (oldest deadline 2024-03-26T04:22:53.314009+0000) 2024-03-26T04:23:10.247 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:10 smithi190 bash[43655]: debug 2024-03-26T04:23:10.126+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:34.415638+0000 front 2024-03-26T04:22:34.415939+0000 (oldest deadline 2024-03-26T04:22:59.715238+0000) 2024-03-26T04:23:10.248 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:10 smithi190 bash[43655]: debug 2024-03-26T04:23:10.126+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6830 osd.3 since back 2024-03-26T04:22:42.517456+0000 front 2024-03-26T04:22:42.517393+0000 (oldest deadline 2024-03-26T04:23:08.318342+0000) 2024-03-26T04:23:11.133 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:10 smithi190 bash[37968]: debug 2024-03-26T04:23:10.850+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:20.030724+0000 front 2024-03-26T04:22:20.030946+0000 (oldest deadline 2024-03-26T04:22:45.330384+0000) 2024-03-26T04:23:11.133 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:10 smithi190 bash[37968]: debug 2024-03-26T04:23:10.850+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:27.632134+0000 front 2024-03-26T04:22:27.632209+0000 (oldest deadline 2024-03-26T04:22:55.238195+0000) 2024-03-26T04:23:11.133 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:10 smithi190 bash[37968]: debug 2024-03-26T04:23:10.850+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:35.239813+0000 front 2024-03-26T04:22:35.239458+0000 (oldest deadline 2024-03-26T04:23:01.138987+0000) 2024-03-26T04:23:11.133 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:10 smithi190 bash[37968]: debug 2024-03-26T04:23:10.850+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6830 osd.3 since back 2024-03-26T04:22:46.341090+0000 front 2024-03-26T04:22:46.340935+0000 (oldest deadline 2024-03-26T04:23:09.240832+0000) 2024-03-26T04:23:11.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:11 smithi190 bash[43655]: debug 2024-03-26T04:23:11.134+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:23:11.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:11 smithi190 bash[43655]: debug 2024-03-26T04:23:11.134+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:28.014352+0000 front 2024-03-26T04:22:28.014229+0000 (oldest deadline 2024-03-26T04:22:53.314009+0000) 2024-03-26T04:23:11.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:11 smithi190 bash[43655]: debug 2024-03-26T04:23:11.134+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:34.415638+0000 front 2024-03-26T04:22:34.415939+0000 (oldest deadline 2024-03-26T04:22:59.715238+0000) 2024-03-26T04:23:11.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:11 smithi190 bash[43655]: debug 2024-03-26T04:23:11.134+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6830 osd.3 since back 2024-03-26T04:22:42.517456+0000 front 2024-03-26T04:22:42.517393+0000 (oldest deadline 2024-03-26T04:23:08.318342+0000) 2024-03-26T04:23:12.160 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:11 smithi190 bash[37968]: debug 2024-03-26T04:23:11.890+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:20.030724+0000 front 2024-03-26T04:22:20.030946+0000 (oldest deadline 2024-03-26T04:22:45.330384+0000) 2024-03-26T04:23:12.160 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:11 smithi190 bash[37968]: debug 2024-03-26T04:23:11.890+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:27.632134+0000 front 2024-03-26T04:22:27.632209+0000 (oldest deadline 2024-03-26T04:22:55.238195+0000) 2024-03-26T04:23:12.160 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:11 smithi190 bash[37968]: debug 2024-03-26T04:23:11.890+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:35.239813+0000 front 2024-03-26T04:22:35.239458+0000 (oldest deadline 2024-03-26T04:23:01.138987+0000) 2024-03-26T04:23:12.160 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:11 smithi190 bash[37968]: debug 2024-03-26T04:23:11.890+0000 7f2162a61700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.175:6830 osd.3 since back 2024-03-26T04:22:46.341090+0000 front 2024-03-26T04:22:46.340935+0000 (oldest deadline 2024-03-26T04:23:09.240832+0000) 2024-03-26T04:23:12.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:12 smithi190 bash[43655]: debug 2024-03-26T04:23:12.162+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:23:12.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:12 smithi190 bash[43655]: debug 2024-03-26T04:23:12.162+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:28.014352+0000 front 2024-03-26T04:22:28.014229+0000 (oldest deadline 2024-03-26T04:22:53.314009+0000) 2024-03-26T04:23:12.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:12 smithi190 bash[43655]: debug 2024-03-26T04:23:12.162+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:34.415638+0000 front 2024-03-26T04:22:34.415939+0000 (oldest deadline 2024-03-26T04:22:59.715238+0000) 2024-03-26T04:23:12.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:12 smithi190 bash[43655]: debug 2024-03-26T04:23:12.162+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6830 osd.3 since back 2024-03-26T04:22:42.517456+0000 front 2024-03-26T04:22:42.517393+0000 (oldest deadline 2024-03-26T04:23:08.318342+0000) 2024-03-26T04:23:13.482 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:13 smithi190 bash[43655]: debug 2024-03-26T04:23:13.134+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:23:13.482 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:13 smithi190 bash[43655]: debug 2024-03-26T04:23:13.134+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:28.014352+0000 front 2024-03-26T04:22:28.014229+0000 (oldest deadline 2024-03-26T04:22:53.314009+0000) 2024-03-26T04:23:13.482 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:13 smithi190 bash[43655]: debug 2024-03-26T04:23:13.134+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:34.415638+0000 front 2024-03-26T04:22:34.415939+0000 (oldest deadline 2024-03-26T04:22:59.715238+0000) 2024-03-26T04:23:13.482 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:13 smithi190 bash[43655]: debug 2024-03-26T04:23:13.134+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6830 osd.3 since back 2024-03-26T04:22:42.517456+0000 front 2024-03-26T04:22:42.517393+0000 (oldest deadline 2024-03-26T04:23:08.318342+0000) 2024-03-26T04:23:13.746 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:13 smithi190 bash[57182]: ceph-762e930c-eb21-11ee-95d0-87774f69a715-osd-6 2024-03-26T04:23:13.746 INFO:journalctl@ceph.osd.6.smithi190.stdout:Mar 26 04:23:13 smithi190 bash[57246]: Error response from daemon: No such container: ceph-762e930c-eb21-11ee-95d0-87774f69a715-osd-6 2024-03-26T04:23:14.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:14 smithi190 bash[43655]: debug 2024-03-26T04:23:14.150+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:23:14.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:14 smithi190 bash[43655]: debug 2024-03-26T04:23:14.150+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:28.014352+0000 front 2024-03-26T04:22:28.014229+0000 (oldest deadline 2024-03-26T04:22:53.314009+0000) 2024-03-26T04:23:14.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:14 smithi190 bash[43655]: debug 2024-03-26T04:23:14.150+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:34.415638+0000 front 2024-03-26T04:22:34.415939+0000 (oldest deadline 2024-03-26T04:22:59.715238+0000) 2024-03-26T04:23:14.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:14 smithi190 bash[43655]: debug 2024-03-26T04:23:14.150+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6830 osd.3 since back 2024-03-26T04:22:42.517456+0000 front 2024-03-26T04:22:42.517393+0000 (oldest deadline 2024-03-26T04:23:08.318342+0000) 2024-03-26T04:23:15.309 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:15 smithi190 bash[43655]: debug 2024-03-26T04:23:15.186+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:23:15.309 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:15 smithi190 bash[43655]: debug 2024-03-26T04:23:15.186+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:28.014352+0000 front 2024-03-26T04:22:28.014229+0000 (oldest deadline 2024-03-26T04:22:53.314009+0000) 2024-03-26T04:23:15.309 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:15 smithi190 bash[43655]: debug 2024-03-26T04:23:15.186+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:34.415638+0000 front 2024-03-26T04:22:34.415939+0000 (oldest deadline 2024-03-26T04:22:59.715238+0000) 2024-03-26T04:23:15.309 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:15 smithi190 bash[43655]: debug 2024-03-26T04:23:15.186+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6830 osd.3 since back 2024-03-26T04:22:42.517456+0000 front 2024-03-26T04:22:42.517393+0000 (oldest deadline 2024-03-26T04:23:08.318342+0000) 2024-03-26T04:23:15.760 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@osd.6.service' 2024-03-26T04:23:15.786 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T04:23:15.787 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-26T04:23:15.787 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-26T04:23:15.787 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-762e930c-eb21-11ee-95d0-87774f69a715@osd.7 2024-03-26T04:23:16.234 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:15 smithi190 systemd[1]: Stopping Ceph osd.7 for 762e930c-eb21-11ee-95d0-87774f69a715... 2024-03-26T04:23:16.235 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:15 smithi190 bash[43655]: debug 2024-03-26T04:23:15.966+0000 7fefb0db2700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-03-26T04:23:16.235 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:15 smithi190 bash[43655]: debug 2024-03-26T04:23:15.966+0000 7fefb0db2700 -1 osd.7 794 *** Got signal Terminated *** 2024-03-26T04:23:16.235 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:15 smithi190 bash[43655]: debug 2024-03-26T04:23:15.966+0000 7fefb0db2700 -1 osd.7 794 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T04:23:16.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:16 smithi190 bash[43655]: debug 2024-03-26T04:23:16.234+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:23:16.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:16 smithi190 bash[43655]: debug 2024-03-26T04:23:16.234+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:28.014352+0000 front 2024-03-26T04:22:28.014229+0000 (oldest deadline 2024-03-26T04:22:53.314009+0000) 2024-03-26T04:23:16.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:16 smithi190 bash[43655]: debug 2024-03-26T04:23:16.234+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:34.415638+0000 front 2024-03-26T04:22:34.415939+0000 (oldest deadline 2024-03-26T04:22:59.715238+0000) 2024-03-26T04:23:16.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:16 smithi190 bash[43655]: debug 2024-03-26T04:23:16.234+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6830 osd.3 since back 2024-03-26T04:22:42.517456+0000 front 2024-03-26T04:22:42.517393+0000 (oldest deadline 2024-03-26T04:23:08.318342+0000) 2024-03-26T04:23:17.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:17 smithi190 bash[43655]: debug 2024-03-26T04:23:17.202+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:23:17.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:17 smithi190 bash[43655]: debug 2024-03-26T04:23:17.202+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:28.014352+0000 front 2024-03-26T04:22:28.014229+0000 (oldest deadline 2024-03-26T04:22:53.314009+0000) 2024-03-26T04:23:17.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:17 smithi190 bash[43655]: debug 2024-03-26T04:23:17.202+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:34.415638+0000 front 2024-03-26T04:22:34.415939+0000 (oldest deadline 2024-03-26T04:22:59.715238+0000) 2024-03-26T04:23:17.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:17 smithi190 bash[43655]: debug 2024-03-26T04:23:17.202+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6830 osd.3 since back 2024-03-26T04:22:42.517456+0000 front 2024-03-26T04:22:42.517393+0000 (oldest deadline 2024-03-26T04:23:08.318342+0000) 2024-03-26T04:23:17.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:17 smithi190 bash[43655]: debug 2024-03-26T04:23:17.202+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.190:6804 osd.4 since back 2024-03-26T04:22:54.219751+0000 front 2024-03-26T04:22:54.219816+0000 (oldest deadline 2024-03-26T04:23:16.519632+0000) 2024-03-26T04:23:18.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:18 smithi190 bash[43655]: debug 2024-03-26T04:23:18.158+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:23:18.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:18 smithi190 bash[43655]: debug 2024-03-26T04:23:18.158+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:28.014352+0000 front 2024-03-26T04:22:28.014229+0000 (oldest deadline 2024-03-26T04:22:53.314009+0000) 2024-03-26T04:23:18.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:18 smithi190 bash[43655]: debug 2024-03-26T04:23:18.158+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:34.415638+0000 front 2024-03-26T04:22:34.415939+0000 (oldest deadline 2024-03-26T04:22:59.715238+0000) 2024-03-26T04:23:18.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:18 smithi190 bash[43655]: debug 2024-03-26T04:23:18.158+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6830 osd.3 since back 2024-03-26T04:22:42.517456+0000 front 2024-03-26T04:22:42.517393+0000 (oldest deadline 2024-03-26T04:23:08.318342+0000) 2024-03-26T04:23:18.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:18 smithi190 bash[43655]: debug 2024-03-26T04:23:18.158+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.190:6804 osd.4 since back 2024-03-26T04:22:54.219751+0000 front 2024-03-26T04:22:54.219816+0000 (oldest deadline 2024-03-26T04:23:16.519632+0000) 2024-03-26T04:23:19.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:19 smithi190 bash[43655]: debug 2024-03-26T04:23:19.186+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:23:19.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:19 smithi190 bash[43655]: debug 2024-03-26T04:23:19.186+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:28.014352+0000 front 2024-03-26T04:22:28.014229+0000 (oldest deadline 2024-03-26T04:22:53.314009+0000) 2024-03-26T04:23:19.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:19 smithi190 bash[43655]: debug 2024-03-26T04:23:19.186+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:34.415638+0000 front 2024-03-26T04:22:34.415939+0000 (oldest deadline 2024-03-26T04:22:59.715238+0000) 2024-03-26T04:23:19.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:19 smithi190 bash[43655]: debug 2024-03-26T04:23:19.186+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6830 osd.3 since back 2024-03-26T04:22:42.517456+0000 front 2024-03-26T04:22:42.517393+0000 (oldest deadline 2024-03-26T04:23:08.318342+0000) 2024-03-26T04:23:19.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:19 smithi190 bash[43655]: debug 2024-03-26T04:23:19.186+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.190:6804 osd.4 since back 2024-03-26T04:22:54.219751+0000 front 2024-03-26T04:22:54.219816+0000 (oldest deadline 2024-03-26T04:23:16.519632+0000) 2024-03-26T04:23:20.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:20 smithi190 bash[43655]: debug 2024-03-26T04:23:20.198+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6806 osd.0 since back 2024-03-26T04:22:18.211490+0000 front 2024-03-26T04:22:18.211627+0000 (oldest deadline 2024-03-26T04:22:41.711100+0000) 2024-03-26T04:23:20.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:20 smithi190 bash[43655]: debug 2024-03-26T04:23:20.198+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6814 osd.1 since back 2024-03-26T04:22:28.014352+0000 front 2024-03-26T04:22:28.014229+0000 (oldest deadline 2024-03-26T04:22:53.314009+0000) 2024-03-26T04:23:20.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:20 smithi190 bash[43655]: debug 2024-03-26T04:23:20.198+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6822 osd.2 since back 2024-03-26T04:22:34.415638+0000 front 2024-03-26T04:22:34.415939+0000 (oldest deadline 2024-03-26T04:22:59.715238+0000) 2024-03-26T04:23:20.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:20 smithi190 bash[43655]: debug 2024-03-26T04:23:20.198+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.175:6830 osd.3 since back 2024-03-26T04:22:42.517456+0000 front 2024-03-26T04:22:42.517393+0000 (oldest deadline 2024-03-26T04:23:08.318342+0000) 2024-03-26T04:23:20.497 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:20 smithi190 bash[43655]: debug 2024-03-26T04:23:20.198+0000 7fefad3a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.190:6804 osd.4 since back 2024-03-26T04:22:54.219751+0000 front 2024-03-26T04:22:54.219816+0000 (oldest deadline 2024-03-26T04:23:16.519632+0000) 2024-03-26T04:23:22.233 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:21 smithi190 bash[57379]: ceph-762e930c-eb21-11ee-95d0-87774f69a715-osd-7 2024-03-26T04:23:22.233 INFO:journalctl@ceph.osd.7.smithi190.stdout:Mar 26 04:23:22 smithi190 bash[57443]: Error response from daemon: No such container: ceph-762e930c-eb21-11ee-95d0-87774f69a715-osd-7 2024-03-26T04:23:24.177 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@osd.7.service' 2024-03-26T04:23:24.205 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T04:23:24.205 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-26T04:23:24.205 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-03-26T04:23:24.206 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-762e930c-eb21-11ee-95d0-87774f69a715@rgw.foo.a 2024-03-26T04:23:24.491 INFO:journalctl@ceph.rgw.foo.a.smithi175.stdout:Mar 26 04:23:24 smithi175 systemd[1]: Stopping Ceph rgw.foo.a for 762e930c-eb21-11ee-95d0-87774f69a715... 2024-03-26T04:23:24.491 INFO:journalctl@ceph.rgw.foo.a.smithi175.stdout:Mar 26 04:23:24 smithi175 bash[53366]: debug 2024-03-26T04:23:24.357+0000 7f5a29fff700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-03-26T04:23:24.491 INFO:journalctl@ceph.rgw.foo.a.smithi175.stdout:Mar 26 04:23:24 smithi175 bash[53366]: debug 2024-03-26T04:23:24.357+0000 7f5a38885ac0 -1 shutting down 2024-03-26T04:23:29.747 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 04:23:29 smithi190 bash[51574]: ts=2024-03-26T04:23:29.471Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.175:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.175:8765: connect: connection refused" 2024-03-26T04:23:29.747 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 04:23:29 smithi190 bash[51574]: ts=2024-03-26T04:23:29.473Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.175:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.175:8765: connect: connection refused" 2024-03-26T04:23:29.747 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 04:23:29 smithi190 bash[51574]: ts=2024-03-26T04:23:29.475Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.175:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.175:8765: connect: connection refused" 2024-03-26T04:23:29.747 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 04:23:29 smithi190 bash[51574]: ts=2024-03-26T04:23:29.478Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.175:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.175:8765: connect: connection refused" 2024-03-26T04:23:29.747 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 04:23:29 smithi190 bash[51574]: ts=2024-03-26T04:23:29.479Z caller=refresh.go:99 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.175:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.175:8765: connect: connection refused" 2024-03-26T04:23:34.991 INFO:journalctl@ceph.rgw.foo.a.smithi175.stdout:Mar 26 04:23:34 smithi175 bash[137883]: ceph-762e930c-eb21-11ee-95d0-87774f69a715-rgw-foo-a 2024-03-26T04:23:35.149 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@rgw.foo.a.service' 2024-03-26T04:23:35.172 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T04:23:35.173 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-03-26T04:23:35.173 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-03-26T04:23:35.173 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-762e930c-eb21-11ee-95d0-87774f69a715@prometheus.a 2024-03-26T04:23:35.497 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 04:23:35 smithi190 systemd[1]: Stopping Ceph prometheus.a for 762e930c-eb21-11ee-95d0-87774f69a715... 2024-03-26T04:23:35.497 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 04:23:35 smithi190 bash[51574]: ts=2024-03-26T04:23:35.357Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-03-26T04:23:35.497 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 04:23:35 smithi190 bash[51574]: ts=2024-03-26T04:23:35.357Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-03-26T04:23:35.497 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 04:23:35 smithi190 bash[51574]: ts=2024-03-26T04:23:35.357Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-03-26T04:23:35.497 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 04:23:35 smithi190 bash[51574]: ts=2024-03-26T04:23:35.357Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-03-26T04:23:35.497 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 04:23:35 smithi190 bash[51574]: ts=2024-03-26T04:23:35.357Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-03-26T04:23:35.497 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 04:23:35 smithi190 bash[51574]: ts=2024-03-26T04:23:35.357Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-03-26T04:23:35.497 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 04:23:35 smithi190 bash[51574]: ts=2024-03-26T04:23:35.357Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-03-26T04:23:35.498 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 04:23:35 smithi190 bash[51574]: ts=2024-03-26T04:23:35.357Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-03-26T04:23:35.498 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 04:23:35 smithi190 bash[51574]: ts=2024-03-26T04:23:35.357Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-03-26T04:23:35.498 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 04:23:35 smithi190 bash[51574]: ts=2024-03-26T04:23:35.359Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-03-26T04:23:35.498 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 04:23:35 smithi190 bash[51574]: ts=2024-03-26T04:23:35.359Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-03-26T04:23:35.498 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 04:23:35 smithi190 bash[51574]: ts=2024-03-26T04:23:35.359Z caller=main.go:1147 level=info msg="See you next time!" 2024-03-26T04:23:36.238 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 04:23:35 smithi190 bash[57576]: ceph-762e930c-eb21-11ee-95d0-87774f69a715-prometheus-a 2024-03-26T04:23:36.238 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 04:23:36 smithi190 bash[57639]: Error response from daemon: No such container: ceph-762e930c-eb21-11ee-95d0-87774f69a715-prometheus-a 2024-03-26T04:23:36.238 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 04:23:36 smithi190 systemd[1]: ceph-762e930c-eb21-11ee-95d0-87774f69a715@prometheus.a.service: Deactivated successfully. 2024-03-26T04:23:36.238 INFO:journalctl@ceph.prometheus.a.smithi190.stdout:Mar 26 04:23:36 smithi190 systemd[1]: Stopped Ceph prometheus.a for 762e930c-eb21-11ee-95d0-87774f69a715. 2024-03-26T04:23:36.239 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-762e930c-eb21-11ee-95d0-87774f69a715@prometheus.a.service' 2024-03-26T04:23:36.261 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T04:23:36.261 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-03-26T04:23:36.262 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm rm-cluster --fsid 762e930c-eb21-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-26T04:23:36.430 INFO:teuthology.orchestra.run.smithi175.stdout:Deleting cluster with fsid: 762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T04:23:41.356 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 04:23:41 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:41.357 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 04:23:41 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:41.656 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 04:23:41 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:41.656 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 04:23:41 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:41.656 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 04:23:41 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:41.656 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 04:23:41 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:41.915 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 04:23:41 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:41.916 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 04:23:41 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:42.185 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 04:23:42 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:42.185 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 04:23:42 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:42.490 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 04:23:42 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:42.491 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 04:23:42 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:42.808 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 04:23:42 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:42.808 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 04:23:42 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:42.808 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 04:23:42 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:42.808 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 04:23:42 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:43.175 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 04:23:42 smithi175 systemd[1]: Stopping Ceph node-exporter.a for 762e930c-eb21-11ee-95d0-87774f69a715... 2024-03-26T04:23:43.745 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 04:23:43 smithi175 bash[138291]: ceph-762e930c-eb21-11ee-95d0-87774f69a715-node-exporter-a 2024-03-26T04:23:43.745 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 04:23:43 smithi175 systemd[1]: ceph-762e930c-eb21-11ee-95d0-87774f69a715@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-03-26T04:23:44.020 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 04:23:43 smithi175 bash[138354]: Error response from daemon: No such container: ceph-762e930c-eb21-11ee-95d0-87774f69a715-node-exporter-a 2024-03-26T04:23:44.021 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 04:23:43 smithi175 systemd[1]: ceph-762e930c-eb21-11ee-95d0-87774f69a715@node-exporter.a.service: Failed with result 'exit-code'. 2024-03-26T04:23:44.021 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 04:23:43 smithi175 systemd[1]: Stopped Ceph node-exporter.a for 762e930c-eb21-11ee-95d0-87774f69a715. 2024-03-26T04:23:44.021 INFO:journalctl@ceph.node-exporter.a.smithi175.stdout:Mar 26 04:23:43 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:44.021 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 04:23:43 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:44.357 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 04:23:44 smithi175 systemd[1]: Stopping Ceph alertmanager.a for 762e930c-eb21-11ee-95d0-87774f69a715... 2024-03-26T04:23:44.358 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 04:23:44 smithi175 bash[60142]: ts=2024-03-26T04:23:44.145Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-03-26T04:23:44.991 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 04:23:44 smithi175 bash[138423]: ceph-762e930c-eb21-11ee-95d0-87774f69a715-alertmanager-a 2024-03-26T04:23:44.991 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 04:23:44 smithi175 bash[138483]: Error response from daemon: No such container: ceph-762e930c-eb21-11ee-95d0-87774f69a715-alertmanager-a 2024-03-26T04:23:44.991 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 04:23:44 smithi175 systemd[1]: ceph-762e930c-eb21-11ee-95d0-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-03-26T04:23:44.991 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 04:23:44 smithi175 systemd[1]: Stopped Ceph alertmanager.a for 762e930c-eb21-11ee-95d0-87774f69a715. 2024-03-26T04:23:45.316 INFO:journalctl@ceph.alertmanager.a.smithi175.stdout:Mar 26 04:23:45 smithi175 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:45.596 DEBUG:teuthology.orchestra.run.smithi190:> sudo cephadm rm-cluster --fsid 762e930c-eb21-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-26T04:23:45.765 INFO:teuthology.orchestra.run.smithi190.stdout:Deleting cluster with fsid: 762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T04:23:50.661 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:23:50 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:50.662 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 04:23:50 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:50.662 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 04:23:50 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:50.928 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:23:50 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:50.929 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 04:23:50 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:50.929 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 04:23:50 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:51.246 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:23:51 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:51.247 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 04:23:51 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:51.247 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 04:23:51 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:51.539 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 04:23:51 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:51.539 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 04:23:51 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:51.540 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:23:51 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:51.790 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 04:23:51 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:51.790 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:23:51 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:51.790 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 04:23:51 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:52.041 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 04:23:51 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:52.041 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 04:23:51 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:52.042 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:23:51 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:23:52.042 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:23:51 smithi190 systemd[1]: Stopping Ceph iscsi.iscsi.a for 762e930c-eb21-11ee-95d0-87774f69a715... 2024-03-26T04:23:52.496 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:23:52 smithi190 bash[47836]: debug Shutdown received 2024-03-26T04:24:02.996 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:24:02 smithi190 bash[57913]: ceph-762e930c-eb21-11ee-95d0-87774f69a715-iscsi-iscsi-a 2024-03-26T04:24:02.996 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:24:02 smithi190 systemd[1]: ceph-762e930c-eb21-11ee-95d0-87774f69a715@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-03-26T04:24:02.996 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:24:02 smithi190 bash[57981]: Error response from daemon: No such container: ceph-762e930c-eb21-11ee-95d0-87774f69a715-iscsi-iscsi-a 2024-03-26T04:24:03.363 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 04:24:03 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:24:03.364 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:24:03 smithi190 systemd[1]: ceph-762e930c-eb21-11ee-95d0-87774f69a715@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-03-26T04:24:03.364 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:24:03 smithi190 systemd[1]: Stopped Ceph iscsi.iscsi.a for 762e930c-eb21-11ee-95d0-87774f69a715. 2024-03-26T04:24:03.364 INFO:journalctl@ceph.iscsi.iscsi.a.smithi190.stdout:Mar 26 04:24:03 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:24:03.364 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 04:24:03 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:24:04.466 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 04:24:04 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:24:04.466 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 04:24:04 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:24:04.746 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 04:24:04 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:24:04.746 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 04:24:04 smithi190 systemd[1]: Stopping Ceph node-exporter.b for 762e930c-eb21-11ee-95d0-87774f69a715... 2024-03-26T04:24:04.746 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 04:24:04 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:24:05.741 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 04:24:05 smithi190 bash[58180]: ceph-762e930c-eb21-11ee-95d0-87774f69a715-node-exporter-b 2024-03-26T04:24:05.742 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 04:24:05 smithi190 systemd[1]: ceph-762e930c-eb21-11ee-95d0-87774f69a715@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-03-26T04:24:05.742 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 04:24:05 smithi190 bash[58238]: Error response from daemon: No such container: ceph-762e930c-eb21-11ee-95d0-87774f69a715-node-exporter-b 2024-03-26T04:24:05.742 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 04:24:05 smithi190 systemd[1]: ceph-762e930c-eb21-11ee-95d0-87774f69a715@node-exporter.b.service: Failed with result 'exit-code'. 2024-03-26T04:24:05.742 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 04:24:05 smithi190 systemd[1]: Stopped Ceph node-exporter.b for 762e930c-eb21-11ee-95d0-87774f69a715. 2024-03-26T04:24:05.996 INFO:journalctl@ceph.node-exporter.b.smithi190.stdout:Mar 26 04:24:05 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:24:05.996 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 04:24:05 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:24:05.996 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 04:24:05 smithi190 systemd[1]: Stopping Ceph grafana.a for 762e930c-eb21-11ee-95d0-87774f69a715... 2024-03-26T04:24:05.997 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 04:24:05 smithi190 bash[50860]: logger=server t=2024-03-26T04:24:05.972644954Z level=info msg="Shutdown started" reason="System signal: terminated" 2024-03-26T04:24:05.997 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 04:24:05 smithi190 bash[50860]: logger=ticker t=2024-03-26T04:24:05.972707713Z level=info msg=stopped last_tick=2024-03-26T04:24:00Z 2024-03-26T04:24:05.997 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 04:24:05 smithi190 bash[50860]: logger=tracing t=2024-03-26T04:24:05.972702413Z level=info msg="Closing tracing" 2024-03-26T04:24:06.703 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 04:24:06 smithi190 bash[58305]: ceph-762e930c-eb21-11ee-95d0-87774f69a715-grafana-a 2024-03-26T04:24:06.703 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 04:24:06 smithi190 bash[58365]: Error response from daemon: No such container: ceph-762e930c-eb21-11ee-95d0-87774f69a715-grafana-a 2024-03-26T04:24:06.704 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 04:24:06 smithi190 systemd[1]: ceph-762e930c-eb21-11ee-95d0-87774f69a715@grafana.a.service: Deactivated successfully. 2024-03-26T04:24:06.954 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 04:24:06 smithi190 systemd[1]: Stopped Ceph grafana.a for 762e930c-eb21-11ee-95d0-87774f69a715. 2024-03-26T04:24:06.954 INFO:journalctl@ceph.grafana.a.smithi190.stdout:Mar 26 04:24:06 smithi190 systemd[1]: /etc/systemd/system/ceph-762e930c-eb21-11ee-95d0-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-26T04:24:07.453 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T04:24:07.466 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T04:24:07.478 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-26T04:24:07.480 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/adking-2024-03-26_01:50:53-orch:cephadm-wip-adk-testing-2024-03-25-1628-distro-default-smithi/7622440/remote/smithi175/crash 2024-03-26T04:24:07.481 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/crash -- . 2024-03-26T04:24:07.522 INFO:teuthology.orchestra.run.smithi175.stderr:tar: /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-26T04:24:07.522 INFO:teuthology.orchestra.run.smithi175.stderr:tar: Error is not recoverable: exiting now 2024-03-26T04:24:07.524 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/adking-2024-03-26_01:50:53-orch:cephadm-wip-adk-testing-2024-03-25-1628-distro-default-smithi/7622440/remote/smithi190/crash 2024-03-26T04:24:07.525 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/crash -- . 2024-03-26T04:24:07.535 INFO:teuthology.orchestra.run.smithi190.stderr:tar: /var/lib/ceph/762e930c-eb21-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-26T04:24:07.535 INFO:teuthology.orchestra.run.smithi190.stderr:tar: Error is not recoverable: exiting now 2024-03-26T04:24:07.537 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-26T04:24:07.537 DEBUG:teuthology.orchestra.run.smithi175:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v MON_DOWN | egrep -v 'mons down, quorum' | egrep -v 'is down \(out of quorum\)' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-03-26T04:24:07.583 INFO:teuthology.orchestra.run.smithi175.stdout:2024-03-26T03:50:00.000143+0000 mon.a (mon.0) 1382 : cluster [WRN] Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 5 pool(s) do not have an application enabled 2024-03-26T04:24:07.583 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-26T04:24:07.583 DEBUG:teuthology.orchestra.run.smithi175:> sudo egrep '\[SEC\]' /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v MON_DOWN | egrep -v 'mons down, quorum' | egrep -v 'is down \(out of quorum\)' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-03-26T04:24:07.641 DEBUG:teuthology.orchestra.run.smithi175:> sudo egrep '\[ERR\]' /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v MON_DOWN | egrep -v 'mons down, quorum' | egrep -v 'is down \(out of quorum\)' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-03-26T04:24:07.698 DEBUG:teuthology.orchestra.run.smithi175:> sudo egrep '\[WRN\]' /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v MON_DOWN | egrep -v 'mons down, quorum' | egrep -v 'is down \(out of quorum\)' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-03-26T04:24:07.753 INFO:teuthology.orchestra.run.smithi175.stdout:2024-03-26T03:50:00.000143+0000 mon.a (mon.0) 1382 : cluster [WRN] Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 5 pool(s) do not have an application enabled 2024-03-26T04:24:07.753 INFO:tasks.cephadm:Compressing logs... 2024-03-26T04:24:07.753 DEBUG:teuthology.orchestra.run.smithi175:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-26T04:24:07.797 DEBUG:teuthology.orchestra.run.smithi190:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-26T04:24:07.804 INFO:teuthology.orchestra.run.smithi175.stderr:find: '/var/log/rbd-target-api': No such file or directory 2024-03-26T04:24:07.811 INFO:teuthology.orchestra.run.smithi190.stderr:find: '/var/log/rbd-target-api': No such file or directory 2024-03-26T04:24:07.817 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-26T04:24:07.818 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-mgr.y.log 2024-03-26T04:24:07.818 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-osd.0.log 2024-03-26T04:24:07.818 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-osd.2.log 2024-03-26T04:24:07.821 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-mon.a.log 2024-03-26T04:24:07.821 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-26T04:24:07.821 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-osd.3.log 2024-03-26T04:24:07.821 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-osd.1.log 2024-03-26T04:24:07.821 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-osd.3.log: 88.8% -- replaced with /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-26T04:24:07.821 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph.log 2024-03-26T04:24:07.822 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-26T04:24:07.828 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-osd.1.log: /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-26T04:24:07.828 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-mon.c.log 2024-03-26T04:24:07.831 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-26T04:24:07.831 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-osd.7.log 2024-03-26T04:24:07.832 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-osd.6.log 2024-03-26T04:24:07.832 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-26T04:24:07.833 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-osd.5.log 2024-03-26T04:24:07.834 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-volume.log: 93.4% -- replaced with /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-26T04:24:07.834 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-26T04:24:07.835 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/tcmu-runner.log 2024-03-26T04:24:07.835 INFO:teuthology.orchestra.run.smithi190.stderr: 81.1% -- replaced with /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-26T04:24:07.836 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-mgr.x.log 2024-03-26T04:24:07.836 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/tcmu-runner.log: 67.8% -- replaced with /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/tcmu-runner.log.gz 2024-03-26T04:24:07.836 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph.log 2024-03-26T04:24:07.836 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-26T04:24:07.836 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-mgr.x.log: /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-26T04:24:07.836 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-mon.b.log 2024-03-26T04:24:07.842 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph.audit.log: 88.1% -- replaced with /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-26T04:24:07.842 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-osd.4.log 2024-03-26T04:24:07.844 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-mon.c.log: 91.2% -- replaced with /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-26T04:24:07.844 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-client.rgw.foo.a.log 2024-03-26T04:24:07.848 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-mon.b.log: 90.9% 92.3% -- replaced with /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph.audit.log.gz 2024-03-26T04:24:07.848 INFO:teuthology.orchestra.run.smithi190.stderr: -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-26T04:24:07.850 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-osd.4.log: 91.1% -- replaced with /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-26T04:24:07.854 INFO:teuthology.orchestra.run.smithi190.stderr: 92.9% -- replaced with /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-mgr.x.log.gz 2024-03-26T04:24:07.865 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph.audit.log: /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-client.rgw.foo.a.log: 95.2% -- replaced with /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph.audit.log.gz 2024-03-26T04:24:07.927 INFO:teuthology.orchestra.run.smithi175.stderr: 94.5% -- replaced with /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-client.rgw.foo.a.log.gz 2024-03-26T04:24:08.425 INFO:teuthology.orchestra.run.smithi175.stderr: 91.2% -- replaced with /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-mgr.y.log.gz 2024-03-26T04:24:08.902 INFO:teuthology.orchestra.run.smithi190.stderr: 92.9% -- replaced with /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-mon.b.log.gz 2024-03-26T04:24:09.022 INFO:teuthology.orchestra.run.smithi175.stderr: 92.7% -- replaced with /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-mon.c.log.gz 2024-03-26T04:24:10.596 INFO:teuthology.orchestra.run.smithi175.stderr: 91.9% -- replaced with /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-mon.a.log.gz 2024-03-26T04:24:15.397 INFO:teuthology.orchestra.run.smithi190.stderr: 93.8% -- replaced with /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-osd.6.log.gz 2024-03-26T04:24:16.002 INFO:teuthology.orchestra.run.smithi175.stderr: 93.9% -- replaced with /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-osd.1.log.gz 2024-03-26T04:24:16.111 INFO:teuthology.orchestra.run.smithi175.stderr: 93.8% -- replaced with /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-osd.2.log.gz 2024-03-26T04:24:16.128 INFO:teuthology.orchestra.run.smithi190.stderr: 93.9% -- replaced with /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-osd.7.log.gz 2024-03-26T04:24:16.344 INFO:teuthology.orchestra.run.smithi190.stderr: 93.9% -- replaced with /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-osd.4.log.gz 2024-03-26T04:24:16.469 INFO:teuthology.orchestra.run.smithi175.stderr: 94.0% -- replaced with /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-osd.0.log.gz 2024-03-26T04:24:16.684 INFO:teuthology.orchestra.run.smithi190.stderr: 93.9% -- replaced with /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-osd.5.log.gz 2024-03-26T04:24:16.685 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-03-26T04:24:16.685 INFO:teuthology.orchestra.run.smithi190.stderr:real 0m8.884s 2024-03-26T04:24:16.685 INFO:teuthology.orchestra.run.smithi190.stderr:user 0m33.220s 2024-03-26T04:24:16.685 INFO:teuthology.orchestra.run.smithi190.stderr:sys 0m1.143s 2024-03-26T04:24:16.917 INFO:teuthology.orchestra.run.smithi175.stderr: 93.8% -- replaced with /var/log/ceph/762e930c-eb21-11ee-95d0-87774f69a715/ceph-osd.3.log.gz 2024-03-26T04:24:16.919 INFO:teuthology.orchestra.run.smithi175.stderr: 2024-03-26T04:24:16.920 INFO:teuthology.orchestra.run.smithi175.stderr:real 0m9.121s 2024-03-26T04:24:16.920 INFO:teuthology.orchestra.run.smithi175.stderr:user 0m37.552s 2024-03-26T04:24:16.920 INFO:teuthology.orchestra.run.smithi175.stderr:sys 0m1.381s 2024-03-26T04:24:16.920 INFO:tasks.cephadm:Archiving logs... 2024-03-26T04:24:16.920 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/log/ceph to /home/teuthworker/archive/adking-2024-03-26_01:50:53-orch:cephadm-wip-adk-testing-2024-03-25-1628-distro-default-smithi/7622440/remote/smithi175/log 2024-03-26T04:24:16.921 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-26T04:24:21.295 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/log/ceph to /home/teuthworker/archive/adking-2024-03-26_01:50:53-orch:cephadm-wip-adk-testing-2024-03-25-1628-distro-default-smithi/7622440/remote/smithi190/log 2024-03-26T04:24:21.296 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-26T04:24:25.123 INFO:tasks.cephadm:Removing cluster... 2024-03-26T04:24:25.124 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm rm-cluster --fsid 762e930c-eb21-11ee-95d0-87774f69a715 --force 2024-03-26T04:24:25.295 INFO:teuthology.orchestra.run.smithi175.stdout:Deleting cluster with fsid: 762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T04:24:26.681 DEBUG:teuthology.orchestra.run.smithi190:> sudo cephadm rm-cluster --fsid 762e930c-eb21-11ee-95d0-87774f69a715 --force 2024-03-26T04:24:26.849 INFO:teuthology.orchestra.run.smithi190.stdout:Deleting cluster with fsid: 762e930c-eb21-11ee-95d0-87774f69a715 2024-03-26T04:24:28.233 INFO:tasks.cephadm:Teardown complete 2024-03-26T04:24:28.234 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-26T04:24:28.244 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-26T04:24:28.245 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-26T04:24:28.247 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-26T04:24:28.301 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-26T04:24:28.309 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-26T04:24:28.309 DEBUG:teuthology.orchestra.run.smithi175:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T04:24:28.312 DEBUG:teuthology.orchestra.run.smithi190:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T04:24:28.321 INFO:teuthology.orchestra.run.smithi175.stdout: remote refid st t when poll reach delay offset jitter 2024-03-26T04:24:28.322 INFO:teuthology.orchestra.run.smithi175.stdout:============================================================================== 2024-03-26T04:24:28.322 INFO:teuthology.orchestra.run.smithi175.stdout:*hv01.front.sepi 44.4.53.2 2 u 20 128 377 0.148 -4.143 0.554 2024-03-26T04:24:28.322 INFO:teuthology.orchestra.run.smithi175.stdout:+hv02.front.sepi 63.231.80.2 3 u 21 128 377 0.180 -2.692 0.736 2024-03-26T04:24:28.322 INFO:teuthology.orchestra.run.smithi175.stdout:+hv03.front.sepi 74.6.168.72 3 u 77 128 377 0.161 -3.544 0.798 2024-03-26T04:24:28.322 INFO:teuthology.orchestra.run.smithi175.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-03-26T04:24:28.323 INFO:teuthology.orchestra.run.smithi190.stdout: remote refid st t when poll reach delay offset jitter 2024-03-26T04:24:28.324 INFO:teuthology.orchestra.run.smithi190.stdout:============================================================================== 2024-03-26T04:24:28.324 INFO:teuthology.orchestra.run.smithi190.stdout:*hv01.front.sepi 44.4.53.2 2 u 107 128 377 0.096 -1.686 0.629 2024-03-26T04:24:28.324 INFO:teuthology.orchestra.run.smithi190.stdout:+hv02.front.sepi 63.231.80.2 3 u 16 128 377 0.092 -0.887 0.831 2024-03-26T04:24:28.324 INFO:teuthology.orchestra.run.smithi190.stdout:+hv03.front.sepi 74.6.168.72 3 u 23 128 377 0.116 -1.739 0.862 2024-03-26T04:24:28.324 INFO:teuthology.orchestra.run.smithi190.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-03-26T04:24:28.324 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-26T04:24:28.333 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-26T04:24:28.334 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-26T04:24:28.344 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-26T04:24:28.358 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-26T04:24:28.372 INFO:teuthology.task.internal:Duration was 3666.067499 seconds 2024-03-26T04:24:28.372 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-26T04:24:28.383 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-26T04:24:28.383 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-26T04:24:28.386 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-26T04:24:28.420 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-26T04:24:28.420 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi175.front.sepia.ceph.com 2024-03-26T04:24:28.421 DEBUG:teuthology.orchestra.run.smithi175:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-26T04:24:28.473 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi190.front.sepia.ceph.com 2024-03-26T04:24:28.473 DEBUG:teuthology.orchestra.run.smithi190:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-26T04:24:28.483 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-26T04:24:28.483 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-26T04:24:28.516 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-26T04:24:28.589 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-26T04:24:28.589 DEBUG:teuthology.orchestra.run.smithi175:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-26T04:24:28.591 DEBUG:teuthology.orchestra.run.smithi190:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-26T04:24:28.807 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-26T04:24:28.816 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-26T04:24:28.816 DEBUG:teuthology.orchestra.run.smithi175:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-26T04:24:28.829 DEBUG:teuthology.orchestra.run.smithi190:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-26T04:24:28.843 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-26T04:24:28.871 DEBUG:teuthology.orchestra.run.smithi175:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-26T04:24:28.873 DEBUG:teuthology.orchestra.run.smithi190:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-26T04:24:28.884 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = core 2024-03-26T04:24:28.897 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = core 2024-03-26T04:24:28.921 DEBUG:teuthology.orchestra.run.smithi175:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-26T04:24:28.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T04:24:28.943 DEBUG:teuthology.orchestra.run.smithi190:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-26T04:24:28.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T04:24:28.967 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-26T04:24:28.977 INFO:teuthology.task.internal:Transferring archived files... 2024-03-26T04:24:28.977 DEBUG:teuthology.misc:Transferring archived files from smithi175:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/adking-2024-03-26_01:50:53-orch:cephadm-wip-adk-testing-2024-03-25-1628-distro-default-smithi/7622440/remote/smithi175 2024-03-26T04:24:28.978 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-26T04:24:29.036 DEBUG:teuthology.misc:Transferring archived files from smithi190:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/adking-2024-03-26_01:50:53-orch:cephadm-wip-adk-testing-2024-03-25-1628-distro-default-smithi/7622440/remote/smithi190 2024-03-26T04:24:29.037 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-26T04:24:29.064 INFO:teuthology.task.internal:Removing archive directory... 2024-03-26T04:24:29.064 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-26T04:24:29.067 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-26T04:24:29.104 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-26T04:24:29.119 INFO:teuthology.task.internal:Not uploading archives. 2024-03-26T04:24:29.119 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-26T04:24:29.131 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-26T04:24:29.131 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-26T04:24:29.133 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-26T04:24:29.138 INFO:teuthology.orchestra.run.smithi175.stdout: 658624 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 26 04:24 /home/ubuntu/cephtest 2024-03-26T04:24:29.151 INFO:teuthology.orchestra.run.smithi190.stdout: 658617 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 26 04:24 /home/ubuntu/cephtest 2024-03-26T04:24:29.153 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-26T04:24:29.166 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-26T04:24:29.190 INFO:teuthology.nuke:Checking targets against current locks 2024-03-26T04:24:29.209 DEBUG:teuthology.nuke:shortname: smithi175 2024-03-26T04:24:29.210 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-26T04:24:29.230 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi175.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2024-03-26_01:50:53-orch:cephadm-wip-adk-testing-2024-03-25-1628-distro-default-smithi/7622440', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 03:15:02.068639', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLS3vMmR6LQz7OTue9eOIL2OPIHfMsTq81cV/Hd9LKHcdbqE0SCDqNH4t5UP7TRj/iqpCxaQcXJBnrU9FdGf+lU='} 2024-03-26T04:24:29.233 DEBUG:teuthology.nuke:shortname: smithi190 2024-03-26T04:24:29.234 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-26T04:24:29.251 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2024-03-26_01:50:53-orch:cephadm-wip-adk-testing-2024-03-25-1628-distro-default-smithi/7622440', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 03:15:02.067232', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM4zMdKdSS3SArQxMk+GuqerWQeHcu1JcHhWEy+x3TQqdrbK7Bd5P0UyT+uh7l7rHeoPKRAMF2tf8fODpQTstsA='} 2024-03-26T04:24:29.284 INFO:teuthology.orchestra.console.smithi175:Power off 2024-03-26T04:24:29.284 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-26T04:24:29.308 DEBUG:teuthology.orchestra.console.smithi175:power off output: Chassis Power Control: Down/Off 2024-03-26T04:24:29.308 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T04:24:29.324 INFO:teuthology.orchestra.console.smithi190:Power off 2024-03-26T04:24:29.325 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-26T04:24:29.335 DEBUG:teuthology.orchestra.console.smithi175:check power output: Chassis Power is on 2024-03-26T04:24:29.350 DEBUG:teuthology.orchestra.console.smithi190:power off output: Chassis Power Control: Down/Off 2024-03-26T04:24:29.350 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T04:24:29.374 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on 2024-03-26T04:24:33.336 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T04:24:33.375 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T04:24:33.446 DEBUG:teuthology.orchestra.console.smithi175:check power output: Chassis Power is on 2024-03-26T04:24:33.485 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on 2024-03-26T04:24:37.450 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T04:24:37.486 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T04:24:37.560 DEBUG:teuthology.orchestra.console.smithi175:check power output: Chassis Power is on 2024-03-26T04:24:37.596 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on 2024-03-26T04:24:41.562 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T04:24:41.597 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T04:24:41.672 DEBUG:teuthology.orchestra.console.smithi175:check power output: Chassis Power is off 2024-03-26T04:24:41.706 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is off 2024-03-26T04:24:41.774 INFO:teuthology.orchestra.console.smithi175:Power off completed 2024-03-26T04:24:41.808 INFO:teuthology.orchestra.console.smithi190:Power off completed 2024-03-26T04:24:41.912 INFO:teuthology.run:Summary data: description: orch:cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} duration: 3666.0674991607666 failure_reason: '"2024-03-26T03:50:00.000143+0000 mon.a (mon.0) 1382 : cluster [WRN] Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 5 pool(s) do not have an application enabled" in cluster log' flavor: default owner: scheduled_adking@teuthology success: false 2024-03-26T04:24:41.912 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T04:24:42.003 INFO:teuthology.run:FAIL